Hi uumarkhalid31, I noticed your profile and would like to offer you my project. We can discuss any details over chat.
I would need a amazon crawler.
I want to scrape amazon and want to avoid being blocked. I know it's not 100% possible. So the scraper should contain a proxy function (I have a paid proxy provider) and different user agents/headers. And the crawler should be able to do two different things. One is to scrape the whole amazon and on just by typing in a keyword and than it checks all search results. I know for the first one I would need MUCH power.
The scraper should scrape some things from the product page like:
- Title of the product
- Sold by
- Fullfilled by Amazon or not
- Amazon best sellers rank
- Average Customer review (how much is the star rating and the number of ratings)
and some other informations on the same page.
Now the tricky part:
The script should check how much the stock count is (how much they have left in the inventory). Don't know whether you know how to check but on the product site when you click "add to cart" than "cart" and then when you change the "Quantitiy" to 999 it show how many are on stock.
So basically the crawler has to folllow this ways to find out the stock count.
All should than be written in the Database.
Are you able to to this?
Thanks in advance