Convert MySQL and epoll based web crawler to use asynchronous MySQL calls to speed up web crawler. -- 2

See my question on stackoverflow for more details about what this requires: [login to view URL]

I have a working epoll based web crawling architecture which consumes over 3 Gbps of bandwidth on my dedicated server. When I add MySQL code to keep the state of the frontier and to save crawled information the web crawler slows down. Feedback to my question suggests I need to use asynchronous calls to MySQL so that the web crawler can continue to create new connections while waiting for query results.

I need someone with experience of asynchronous MySQL programming in C to fix this for me.

Taidot: C-ohjelmointi, Linux, MySQL

Näytä lisää: freelance work for beginner programmers, freelance coding jobs, peopleperhour for beginners, web crawler resume, freelance programming jobs for beginners, freelance coding jobs online, freelance programming jobs for students, entry level freelance programming jobs, convert table based layout css, convert windows based application web based application, convert window based application web based application, steps convert table based oscommerce div based oscommerce, producr commonly use outbound calls, convert javascript ebay use, convert table based templates, convert asp based rss feed script, convert net app use net, convert flash based website html website, based total number calls telemarketing pricing, convert windows based application web application net

Tietoa työnantajasta:
( 0 arvostelua ) London, United Kingdom

Projektin tunnus: #25829787

4 freelanceria on tarjonnut keskimäärin 205£ tähän työhön


I am very proficient in c and c++. I have 18 years c++ developing experience now, and have worked for more than 10 years. My work is online game developing, and mainly focus on server side, using c++ under Linux enviro Lisää

£150 GBP 2 päivässä
(134 arvostelua)

Dear Hiring Manager! I'm very confident of your project. I can complete your project in 1 day. Do you like python? 1. My scraping works and experience I have 9 years of experience in web scraping and scraped so many m Lisää

£169 GBP 1 päivässä
(4 arvostelua)

I CAN COMPLETE YOUR PROJECT EASILY Hello, I am programmer with enough experiences on C/C++ for 13 years. C/C++ is nice programming language and I am majoring on it for long years so I have confident to complete your pr Lisää

£200 GBP 7 päivässä
(1 arvostelu)

This problem is not easy. My solution 1: MySQL uses connection pooling to improve efficiency. (The number of connections is still limited, if you wait for processing, then it will affect the speed of the crawler). 2: W Lisää

£300 GBP 7 päivässä
(1 arvostelu)