I want a web crawler to be made that will
- Scan a URL of choice - (URL will be provided by me)
- It Should take multiple URLs as input and read all of them
- After crawling thru all of the HTML content, the Crawler will give a condensed view of key words used in the page.
- it will also reference the read content against a select set of key words that It will read from a .txt file and provide a summary of the amount of times a word appears
- As a sample - open any news site - read all the contents and give a summary of key words the crawler finds there. - End Objective is to provide a 1 page summary of all the sites crawled.
I have vast expierence of web scraping and large data analysis mostly text analysis so this task suits me well I'll deliver the end product in required time, Message for further discussions of project