Clone the site:
[url removed, login to view]
We need to save ALL the product information to the MySQL DB.
Script is written by PHP.
The product image PATH is stored in the Database first.
The crawlwer should run to grap the TEXT first.
then grap the IMAGE (query from the local db), then save it specific folder as we stated.
The site have BOTH Japanese and English version.
We need BOTH, Text is stored in UTF-8 in Mysql DB 5.x version.
UI is needed, it will allow us to crawl AT LEAST 4 ways:
1. Grap products category by category (allow to select ALL category to grap)
2. Grap product by entering the FULL URL
3. If a category is selected to grap, and hang in meanwhile, it can RESUME for last stopping point.
4. If a product is exist, DELETE the old one, INSERT a NEW ONE.
More detail will be given, just ask if you really want to do this.
We need EFFICIENT code, low memory and your script should NOT have timeout problem. (it is the min. technique you need to have, if you can't handle this, we are sorry we won't choose you)