I need a script that can be run on a linux system that is set by cron to run daily and update a database. The script needs to crawl several websites and extract certain data from the pages and place the data in a local mysql db.
The sites to be crawled must be login into in order to get to the pages that need to be examined and I do have valid login credentials.