I need a keyword research scraper that scrapes keywords from search engines.
If you go to [url removed, login to view] and in the search field start typing, it will start suggesting keywords in a drop-down box. These are the keywords in this drop-down box I need to scrape. [url removed, login to view], [url removed, login to view], [url removed, login to view] also do the same thing. I need keywords scraped from these search engines. Delete any duplicates and export them into a txt or excel csv document. I don't need any identifiable information that says if the keywords are from google or yahoo etc.. (I prefer not to have this).
Here's more information on the keywords I need scraped incase you're not clear -
If you go to [url removed, login to view]
In the search field, start typing, for example: type the letter, "p".
Do you see how the keywords automatically start completing in that dropdown window this is called a search engine "suggest" feature. So, now type more letters like pi, then pizz, then pizza, see how the search engine suggest feature gives you suggestions based on what you're typing as to other relevant keywords that match what you're typing.
I need a "deep level" feature up to 10. What this does is, for example if you type "pizza" you only get 10 keywords in that drop-down box from the search engines. Now, let's say if I set the "deep level" to 2. It will take those keywords from the drop-down box, scrape them and put them back into the search field to get more related search terms. If the "deep level" is 10. Then it keeps going up to 10 times, scraping, and putting the kewyords back in the search field to get more similar keywords.
Your program should work in a way that won't create a problem with search engines doing this, for example: google shouldn't be able to detect your program as a bot or something and ban the IP the program is using or give any problems during the scraping, etc...So you should delay the search queries/pings with your program. Also maybe an option to use a proxy.
Another feature the program needs is a filter with the option to enter a "negative" keyword. So, for example. If I type the keyword, "free" in the negative field. Then any keyword that is scraped from the search engines with the word "free" in it. It is automatically omitted (not scraped). Here's an example of how the "filter" negative option works - [url removed, login to view] (click on where it says, "filter my results").
I should be able to upload negative keywords into the software from a text document.
I should be able to upload keywords that need to be search from the scraper in a text document.
I would like the GUI to be simple and clean with an icon. I don't want the GUI to look like it's a program created in 1990. The GUI is important to me to be simple and clean, so if you can send me screenshots of previous work you've done (doesn't have to be related to this) that would be helpful. Or maybe you can provide source code if you can do this project but, not an expert in graphic design then I can find a graphic designer to do the GUI. Let me know if you prefer this option.
When you contact me, let me know what kind of scraping projects you have done before and what language you will code in. Also, the time frame is important to me. I need this project done very quickly.
I would like this program to be a software application and not web-based. No exception.
Program should work on all current Windows platforms. 2000/XP/Vista/Win 7 etc..