We have an email scraper. It does NOT scrape the website. It scrapes URLS uploaded by user.
Process is VERY slow.
Our solution is 2 fold.
1. We need multi-threading that uses proxies.
2. System will open up one thread per proxy to scrape a URL.
What we have:
I have the open source code for you to plug in.
We need the python or php coded for multi-threading.
System is current slow with PHP.