
Closed
Posted
I need a Scrapy-based spider that reliably collects complete product information across three sources I specify: traditional e-commerce sites, larger multi-vendor marketplaces, and a small set of social-media storefront pages. For every product it reaches, the crawler must capture exactly these fields and nothing less: • Product name • Price (in the currency displayed) • Full product description The code should be clean Python 3, using Scrapy pipelines for any post-processing and exporting the results in both CSV and JSON. Please structure it so I can add or swap domains later with minimal edits—ideally via a single config or a [login to view URL] file. Acceptance criteria 1. One command deploys the crawl for each domain without errors. 2. Output files include all three fields for at least 95 % of product URLs in a provided test list. 3. Your README explains setup, required Python packages, and how to extend the spider to new platforms. I can run quick tests and give you prompt feedback, so feel free to suggest milestones that break the work into manageable chunks.
Project ID: 40661965
36 proposals
Remote project
Active 1 day ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
36 freelancers are bidding on average ₹2,528 INR/hour for this job

As experts in web scraping and data extraction, our team at BN-Droids Digital Services is equipped with the skills and knowledge required for your Product Data Scraper Build project. With over 5 years of experience under our belt, we have developed automated data extraction solutions that can handle millions of records daily. We have successfully created and maintained databases containing upwards of 20 million business and retail records across multiple regions. In regards to web scraping, e-commerce product data scraping is an area where we've demonstrated great proficiency. Extracting product details like name, price, and description aligns seamlessly with our capabilities. Using Scrapy - a versatile framework that allows thorough data collection, we ensure clean Python code which simplifies post-processing and facilitates easy CSV and JSON export. Additionally, my full-stack development background adds value to the job as I can structure your code to allow future domain expansion with little effort on your part. With my unique skill set in mind, I assure you that working together on this project will lead to a top-notch solution that meets your needs. Let's collaborate and turn your ideas into actionable data!
₹2,500 INR in 40 days
7.0
7.0

Have over 18 years of experience in data mining/ Web scrapping/ Scraping Bots/ Chrome/Opera Extensions I have done it all. Tell us your source and we will put it in excel for you, Or we can even give you filtered results as per your requirement, In the format you want. You can also ask for data into a particular format - Excel, Json, Mysql, Databases, XMLs, you name them. Further Can help you with integrating it with ur databases, Can create json outputs. We are not only good with scraping but also with the tools that u may need after that. We can help you build you softwares round the data we have 99% Data Accuracy. We have Duplicate finder. etc., We can help with Statistics on the data We can help with creating Api's front the data We can create Softwares to manage that data We can build Sites round the data
₹2,500 INR in 40 days
6.9
6.9

Hello, I’ll build a clean domain-configurable architecture, use Scrapy pipelines for processing, export both CSV/JSON, and provide a clear README covering setup, dependencies, deployment, and adding new domains with minimal code changes. With 10+ years of Python, web scraping, Scrapy and automation experience, I can develop and test this reliably while keeping the code maintainable and scalable. Let’s chat to review the target domains and agree on milestones. thank you Regards Gaurav Garg
₹2,500 INR in 40 days
6.7
6.7

★★★ TOP 1% IN FREE LANCER WORLD ★★★ ★★★ 20+ Year Experience in IBD being CMD★★★ ★★★ 200+ Country Satisfied Clientele ★★★ ★Linkedin★ ★Data Entry★ ★Business Plans★★★ ★★★ Operational Strategic planner Customer Support 24*7★★★ ★★★Excel/Word Operation★★★ ★★★Chat Support★★★ ★★★Calling Support★★★ ★★★Business Plans / Marketing Strategy ★★★ * Digital Marketing★★★ ★★★Social Media Marketing ★★★ ★★★Internet Marketing ★★★ ★★★Any type of Data Projects★★★ ★★★★★★ Regards, ★★★CMD★★★ ★★★PVSYS GROUP (INDIA)★★★ ★★★IF YOU THINK THEN I CAN★★★
₹2,500 INR in 40 days
6.2
6.2

I can build a clean Python 3 Scrapy spider with configurable domains, reusable parsing logic, pipelines, and CSV/JSON exports for your e-commerce, marketplace, and storefront sources. I’ll target the 95%+ field-completeness requirement and include a clear README for setup, testing, and adding new domains.
₹2,500 INR in 40 days
5.6
5.6

As an experienced technologist, specifically in automation and data processing, I understand the requirements of your Product Data Scraper Build project better than most. I specialize in using Python's Scrapy for clean and efficient web scraping, utilizing it to build agile systems that can successfully scrape from different websites of your choosing, while retaining a high level of accuracy. My in-depth knowledge of Scrapy pipelines ensures smooth post-processing and I am familiar with exporting data to both CSV and JSON – perfectly aligning with your project expectations. What sets me apart is my commitment to sustainable solutions. I will structure the code in a way that enables you to easily add or replace domains with minimal effort. Leveraging this flexibility for future proofing is crucial, and I assure you that my process will meet your requirements fully.
₹2,500 INR in 10 days
5.3
5.3

You need a reusable Scrapy crawler that can handle different storefront structures without turning each new domain into a rewrite. I’ve built Python data-processing and automation workflows where reliable extraction, validation, and clean exports were core requirements. I’d structure this around shared product items, domain-specific selectors/config, and Scrapy pipelines for normalization so product name, displayed price, and full description remain consistent across e-commerce sites, marketplaces, and supported storefront pages. Each source would run through a clear command, export to both CSV and JSON, and log missing fields or blocked pages instead of silently producing incomplete rows. I’ll keep selectors and domain settings isolated so adding a new site later requires minimal code changes, and include setup plus extension instructions in the README. Can you share the three target domains and whether any of them require login or JavaScript rendering?
₹2,500 INR in 40 days
4.9
4.9

Hi! This is an excellent, well-defined brief. I appreciate the clarity on the stack (Scrapy/Python 3), the specific data fields, and the focus on extensibility through a configuration file. A key point to clarify is the nature of the "social-media storefront pages." These are often the most challenging due to dynamic JavaScript and anti-bot measures. Have you identified the specific platforms? For these, we might need to integrate a browser automation tool like Playwright, which Scrapy supports well. This would be crucial for hitting your 95% data capture goal. My approach is to build a core Scrapy project using Items to define the strict data structure you need. I'll leverage Scrapy Pipelines for data validation, cleaning, and exporting to both CSV and JSON. As you requested, I'll manage target domains and their specific selectors in a separate, easy-to-edit configuration file to make future additions straightforward. I am confident in building a robust scraper that meets all your acceptance criteria, including the detailed README. The 95% success rate is achievable, and my proposed approach for handling the trickier sites will be key to that. Happy to discuss the specific target sites when you're ready. Philip O.
₹2,500 INR in 7 days
4.3
4.3

As an experienced and dedicated Python developer with a strong background in web scraping and automation, I am confident that I am the right person for the task of building your Scrapy-based product data scraper. With over 17 years working in the IT industry, my team and I have successfully completed numerous projects similar to your requirements, using clean Python 3 Architecture and Job Scripting techniques. We also have extensive skills in SQL Server database performance optimization and web development, which will come in handy when dealing with large data sets. My expertise in utilizing Scrapy pipelines for post-processing of the scraped data will ensure a smooth and efficient process throughout - from data collection to exportation into CSV and JSON formats. I am also well-versed in creating thorough README files that ease the setup for users with minimal programming experience while offering them clear instructions on how to extend the spider for new platforms or domains. Offering cost-effective solutions whilst ensuring 100% client satisfaction is a principle I adhere to diligently. Moreover, employing me means getting prompt feedback as well as deliverables being split into manageable milestones. My working hours are flexible to provide you with excellent support throughout the entire project duration no matter the time zone difference.
₹2,500 INR in 40 days
4.5
4.5

With my extensive and versatile 5+ years of full-stack development experience, I'm well-equipped to handle your Product Data Scraper build. My Python skills are rock-solid and I specialize in clean and efficient code, which is exactly what your project demands. Not only have I worked with similar data scraping projects before, but I've built sleek and robust Scrappy spiders enabling customers to extract critical information from multiple sources- be it traditional e-commerce platforms or multi-vendor marketplaces. As a full-stack developer, backend and web app systems are within my domain. My proficiency in Python along with Scrapy pipelines guarantees that I can deliver clean results in both CSV and JSON formats as required by you. Extending the spider to new platforms or domains won't be an issue due to my knack for modular design - I can write a configurational file allowing seamless swapping or addition of new domains with minimal edits. Moreover, my fintech backend experience from i2c Inc aids me in handling large-scale projects, delivering high-throughput production-ready solutions. As a freelancer, my work ethic is uncompromising with zero late deliveries and a 4.9★ rating amongst my satisfied customers spanning across the globe. Collaborating with me means a promise of quality work, timely delivery and an added advantage of prompt feedback as I strive for customer satisfaction at every step of the way. Let's get this scraper up and running!
₹2,500 INR in 40 days
4.2
4.2

Hi, I can build a clean Scrapy-based product data extraction system in Python 3 that collects product name, price, and full description from your specified sources. My approach will be to create a modular crawler structure with separate spiders/configuration so new domains can be added easily without major code changes. I’ll use Scrapy pipelines for cleaning, validation, and exporting results into both CSV and JSON formats. I’m comfortable with: * Python 3 and Scrapy * Web data extraction * Custom spiders * Item pipelines * Data validation and cleaning * CSV/JSON exports * Configurable crawler architecture * Documentation and deployment setup Deliverables: * Working Scrapy project * Domain-specific spiders * Configurable settings/domains file * CSV and JSON output * Data processing pipelines * Error handling * README with setup instructions * Guide for adding new websites I’ll focus on reliable extraction, maintainable code structure, and clean outputs while respecting source accessibility requirements. Best regards Ankit
₹2,500 INR in 40 days
3.3
3.3

With over 5 years of experience in Python and Software Architecture, I have the skillset needed to develop a top-notch Scrapy-based spider for collecting product information in various domains. My understanding of Scrapy pipelines and exporting data in CSV and JSON formats is extensive, ensuring your thorough satisfaction with the project outcome. What sets me apart is my ability to anticipate and plan ahead. I understand the importance of flexibility in digital projects and I'm committed to building a solution that allows you to make updates or additions to domains seamlessly. Additionally, my commitment to clean, well-documented code ensures that future modifications can be made with minimal effort, saving you time and resources. Lastly, as a part of Designer Bros, we don't just complete projects - we build partnerships. I value your trust and will work hard to deserve it by providing quality support and timely assistance throughout our association. Let's transform your digital dreams into reality together!
₹2,500 INR in 40 days
3.0
3.0

Hello, I can build the **Python 3 Scrapy spider** with clean, reusable domain configuration and support for e-commerce, marketplaces, and social storefronts. It will include: * Product name, price, and full description * CSV + JSON exports * Scrapy pipelines for cleaning * Easy domain/platform configuration * Robust extraction and error handling * README with setup and extension instructions * Testing against your URL list to target **95%+ complete records** I can start with one domain, validate the structure, then expand to the remaining sources and deliver the complete project.
₹2,500 INR in 40 days
1.8
1.8

As an experienced web and app developer with a specialty in web scraping and software architecture, I believe I am the perfect candidate to undertake your Product Data Scraper Build. Over the past decade, I've not only designed numerous high-quality and user-friendly digital solutions but have also gained invaluable experience in using Scrapy, adeptly integrating it into Python pipes for post-processing data just like you require. I'm no stranger to creating clean code that is easily flexible, allowing for effortless addition or swapping of domains—something crucial in your project. Additionally, my familiarity with both JSON and CSV would mean exporting the gathered data in your preferred format will be a quick and seamless process. Alongside providing you with a complete product information such as name, price, and description from diverse e-commerce sites, multi-vendor marketplaces, and social media stores. With 150+ successfully delivered mobile apps and 200+ web applications under my belt, I know what it takes to meet stringent project requirements like capturing fields for at least 95% of product URLs within stipulated criterion. Expect clear communication throughout the project, comprehensive documentation for easy setup of the spider even post-implementation, on-time delivery of completed milestones, all backed with long-term client support. Let's turn your vision into a tangible reality; let's work together!
₹2,500 INR in 40 days
0.0
0.0

As someone well-versed in API development, I have a deep understanding of information retrieval from large sources - a skill that's most pertinent to your product data scraping project. I have expert familiarity with Scrapy and Python 3, and have utilized these tools to build robust and efficient data crawlers for a variety of e-commerce and social media platforms in the past. The spider I'll create will guarantee 95% success rate in capturing the exact fields you require, be it product names, prices, or full descriptions. In addition to my technical skills, I take immense pride in ensuring my code is clean, scalable and well-documented. This means you will have no trouble extending the spider to new platforms as required - something your project explicitly demands. But my expertise extends far beyond coding. With over 10 years of experience across more than 500 projects, I know how to structure work for effective milestones that are easy to manage; a skill I encourage you to leverage. Regards AKif A
₹2,500 INR in 40 days
0.0
0.0

Hello, I understand that your primary goal is to create a robust Scrapy-based spider that efficiently gathers complete product information from various sources. To achieve this, I would focus on developing a flexible and scalable crawler, ensuring it captures the required fields accurately while allowing for easy updates to the domains. I will structure the code for seamless integration, utilizing Scrapy pipelines for post-processing. Throughout the project, I'll maintain open communication, providing updates and seeking your feedback at key milestones to ensure we stay aligned with your vision. To clarify, do you have specific e-commerce sites and marketplaces in mind, or should I conduct some research to identify suitable targets? Additionally, what is your timeline for testing and deployment? Regards, Soné Joubert
₹2,500 INR in 14 days
0.0
0.0

Hi, I can build this as a clean Python/Scrapy project with separate spiders/configuration for each source and reusable pipelines for validation and CSV/JSON export. For each product I’ll capture the required name, displayed price and full description, with structured logging and clear handling for missing/invalid records. I would keep domain-specific selectors isolated so additional sites can be added later without changing the common extraction/export logic. I’ll also include configuration for the source domains, requirements/dependencies, a README with setup and execution instructions, and validation against the supplied test URLs. Before implementation I would quickly inspect the three target sources to determine whether plain Scrapy requests are sufficient or whether any dynamic rendering is required. Estimated initial delivery: 7 days after receiving the target domains/test URLs. Thanks, Sasanka
₹2,500 INR in 40 days
0.0
0.0

With your project requiring extensive knowledge in Python and web scraping, my 5+ years of practical development experience places me in a great position to deliver results beyond your expectations. Throughout my career, I have consistently built scalable and performance-driven applications aligned with clean coding practices, all hallmarks of the code you need for this scrapping spider. One aspect that differentiates me is my ability to adapt and learn new technologies swiftly. While Scrapy is not unfamiliar to me, if there are any specific Python packages or APIs I am unfamiliar with, count on me to learn them rapidly to provide seamless integration. Furthermore, as a client-oriented developer, I appreciate clear communication and setting achievable milestones. Rest assured, you'll receive prompt feedback and will always be within reach throughout this project. Lastly, a well-structured README has been an unshakable habit of mine. With concise explanations on setup requirements, necessary Python packages, and extendibility tips for new platforms, your future maintenance tasks will be as smooth as possible. Keeping these factors in mind alongside my demonstrated capabilities mirrored through my clean, scalable architecture designs; let's make your product data scraper vision a reality!
₹2,500 INR in 40 days
0.0
0.0

Dear Client, We at Resonite Technologies are excited to submit our proposal for your Product Data Scraper project. With a proven track record in web scraping and a skilled team well-versed in Python and Scrapy, we are confident in our ability to deliver a robust solution tailored to your requirements. Our approach will ensure that the spider reliably collects complete product information, capturing the required fields: Product name, Price, and Full product description, from the specified sources. We will implement clean Python 3 code with Scrapy pipelines for efficient post-processing and provide output in both CSV and JSON formats. The architecture will allow for easy addition or swapping of domains through a single configuration file. We commit to meeting your acceptance criteria, including seamless deployment and high output accuracy. Additionally, our comprehensive README will guide you through setup, necessary packages, and extending the spider to new platforms. We appreciate the opportunity to collaborate and are ready to discuss milestones to ensure a smooth workflow. Best regards, Karthik B Resonite Technologies
₹3,500 INR in 40 days
0.0
0.0

I can build this as a clean, modular Python/Scrapy project with separate domain-specific spiders, shared processing through Scrapy pipelines, and CSV + JSON output for product name, displayed price, and full description. I’ll structure the project so individual domains can be updated or added without rewriting the entire scraper, and include a README covering setup, dependencies, running each spider, and extending it to additional sources. Before starting, I’d like to review the three target domains/test URLs. This will let me confirm whether each source can be handled directly with Scrapy or whether any JavaScript-rendered marketplace/social storefront requires a different extraction approach. I’ll test against your supplied URLs and target the stated 95%+ field completeness requirement before delivery. I’m also happy to break the work into milestones by source so you can test each part as it is completed.
₹2,500 INR in 10 days
0.0
0.0

Ahmedabad, India
Member since Aug 21, 2026
₹250000-500000 INR
₹600-700 INR
£10-15 GBP / hour
₹12500-37500 INR
€12-18 EUR / hour
₹12500-37500 INR
$250-750 USD
$15-25 USD / hour
₹750-1250 INR / hour
$25-50 USD / hour
₹75000-150000 INR
$3000-5000 USD
$30-250 USD
$250-750 USD
$100 USD
₹12500-37500 INR
€30-250 EUR
₹600-1500 INR
₹1500-12500 INR
$30-250 USD