1 / SQL SERVER - Reading and storing data
2 / Python - To modify and fix the Crawler
3 / Python libraries for modify and fix the Crawler
1 / Correct 16 Crawler that was developed 6 months ago and that stopped working, because some pages were modified.
2 / Integrate all modules in a single solution
3 / The system will read in a table the information that you will need to search on the websites. Once locating it will store in another three tables. The search table will have the fields:
[BL] - Information you will need to seek
[SOURCE] - The site to search
[DATE_CRAWLERS] - From the date the information was found
[WRITE] - Located
NUMBER_OF_ATTEMPTS] - Number of attempts (only 5 attempts)
Once located, the crawler will store in the SQL database.
But information, I can detail by chat or send more detailed documentation