Develop a web-mining tool to content mine web data (Freelancer only lists Web Scraping as a skill) based on an advanced search function. Create a script that content mines from numerous web services and conducts multiple searches at any one time.
Setup database (Can be SQLite, MSSQL, etc.). Allocate data output (files (BLOB), text, graphics, tables, charts etc.) to numerous variables. Store the data in a database.
Create a function to view mined web content and database management.
You should have a good understanding of:
- Web mining techniques (advanced search techniques etc...)
- Script writing to access numerous web services (search engines, social media sites etc...) at any one time.
- Previous experience sorting data (text/files (BLOB or similar) /graphics/tables/charts etc.) into databases
Further information will be provided.