Develop an Extractor and Load Tool to scrape site content
Bidders must answer the following 3 questions
[url removed, login to view] experience in developing a spider, extracting data, transforming data and then loading into new website
2. Proven Expert in JAVA or PHP
3. Sample site that data was extracted and loaded to s separate site.
An application that will spider specific website and extract content that follows specific 5 business rules. The data should be extracted and presented in a readable format such as excel for us to review, adjust. An additional column should be added that will check that all mandatory data was found and set the flag to complete record. There should be multiple extract files based on each category (12).
A load tool that will take the extracted format from scraping and post to our web form to insert new data. This tool should be able to run multiple times and handle errors such as record already exists or report success or errors.
DETAILED Spec will be supplied to top 3 bidders who answer the initial questions.