URL Scraper and Processor

I need a tool uses a local database to create and manage campaigns. I need to be able to add new campaigns and select campaigns that have already been created. The tool must be able to accept and use proxies (both ip:port and user:password:ip:port format for private and public proxies).

Test added proxies to see if they're working with Google, and highlight the ones that aren't working in red. A one-click option to delete all failed proxies.

Once a campaign is selected or created, I need the tool to:

1. Accept input for an unlimited amount of keywords (1-10,000 keywords)

2. Go to google and search for each keyword, and scrape the top 1000 results

3. Move to the next keyword, and scrape the results, moving down the list of all keywords. Each time it runs a new search the tool should move to the next proxy.

4. Keep track of the keywords that have been successfully scraped and the keywords that failed scraping - Show me which keywords failed and which succeeded, plus allow me to "retry failed keywords"

5. Return all results and save them in the database for the current campaign.

6. After finishing, I need to be able to view all results in a table in the software

7. Click a button that will go and check the PR (Page Rank) of each URL (not domain, but specific URL) from the scraped results, and sort them by highest PR to lowest

8. Click another button that will filter/delete all pages below PR 5

9. Take this list and run a link-check function, where the software visits each page in the URL list created above and extracts all of the links that each of these URLs points to.

10. Check the status of each of these links that it found to see which ones return "no such host" errors (not 404 - page can't be found -- I'm looking for sites that are no longer live at all).

11. Save all of these "no such host" results to database and clear everything else, and show the results in a table in the software

12. Click a button to clean up these URLs - Trim them to root, remove subdomains, and delete "http://" and "https://" and "www." so that all that remains is a list of "no such host" domains in the following format:

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]


13. Click a button that will run up to 3000-5000 domains through [url removed, login to view]'s bulk checker ([url removed, login to view]), and return results stating which ones are available for purchase and which ones are not.

14. Export the "available domains" to a separate area where the software can run them through SEOMoz free API to check the Domain Authority and Page Authority for the domain, and return the data in a table.

15. Automatically save as it goes so that if the software crashes it can pick up where it left off by clicking "Start" or "Resume," Plus a save button to save work when finished.

16. A separate area where I can add domains to wishlist (clicking a "+" image next to each domain in the table from step 14), where the available domain + seomoz data is listed in a table).

Steps 1-8 are basically ScrapeBox functions - If you've used the ScrapeBox software this will all make sense to you.

Steps 9-11 are basically Xenu Linksleuth type features (Xenu is free and you can try it out to see what I mean -- Xenu returns error 12007 for the type of results I'm looking for)

Also, once the tool hits the end of the proxy list, it needs to cycle back up and go through the proxies again. A random delay between each search on google for 20-60 seconds needs to be included. Multithreading is a must to speed up the process (for all steps). I need to be able to plug in my SEOMoz account data (Member ID and secret key). An option to set the number of threads in the settings area (where proxies can be added, and where SEOMoz account info can be added).

Please message me with any further questions!

Thanks for your time... Also I will need to add more to this tool in the very near future, and would be happy to pay much more to add some more features after the basic tool is created.

Also, on step 12 after removing subdomains, http://, www., etc, I need the tool to automatically remove duplicate URLs.

Taidot: C# -ohjelmointi, C++ -ohjelmointi, MySQL, PHP, Windows Desktop

Näytä lisää: seomoz, allow scraping, google click, what to name your resume, top down programming, table checker, sort function c, should my resume be one page, root info, resume names, pr resume, programming test questions, programming resume, private proxy search, page plus programming, off by one errors, near c programming, multithreading programming, moving a domain to a new host, lowest domain purchase, list of available domain names, key programming tool, database processor, c sort function, c programming random number

Tietoa työnantajasta:
( 1 arvostelu ) San Marcos, United States

Projektin tunnus: #4229876

Myönnetty käyttäjälle:


kindly check my PMB

$300 USD 3 päivässä
(144 Arvostelua)

13 freelanceria on tarjonnut keskimäärin %project_bid_stats_avg_sub_26% %project_currencyDetails_sign_sub_27% tähän työhön


I can help in your project, please check PMB and our ratings/reviews to get idea of our experience. Please let me know if you have any queries.

$199 USD 5 päivässä
(226 arvostelua)

Professional solutions

$1500 USD 13 päivässä
(101 arvostelua)

Hi sir, please check PM, thx Kimi.

$250 USD 5 päivässä
(123 arvostelua)

I can help you

$150 USD 10 päivässä
(27 arvostelua)

We can create this scraper for you within few days. Check the message and contact us. SI Team - Scraping Experts!

$900 USD 18 päivässä
(16 arvostelua)

Hi, I have experience with automated bots doing searches on Google w/o proxies. I'll be happy to help you out. Cheers.

$250 USD 7 päivässä
(16 arvostelua)

I have been working with bots and creating bots for google for a very long time, i have also sent you a message proves this.

$250 USD 30 päivässä
(18 arvostelua)

Automation expert here. I can easily write such a bot/[url removed, login to view] check your PMB.

$250 USD 6 päivässä
(11 arvostelua)

Hello, It can be done high quality.

$400 USD 10 päivässä
(2 arvostelua)

Hello. I can work on this project and provide also a custom gui.

$60 USD 6 päivässä
(2 arvostelua)

please check my pm

$80 USD 5 päivässä
(0 arvostelua)

Hello Sir, I can help you in this. Thanks

$200 USD 2 päivässä
(0 arvostelua)

######################## Your Search ENDS Here!. Get it RIGHT the first time. ########################

$2000 USD 30 päivässä
(0 arvostelua)

Hi There, We are a software house based in the United States and have many years of experience and expertise with web and client development. Some of our clients include: Thorpe Park, Staffordshire County Council an Lisää

$380 USD 10 päivässä
(0 arvostelua)