1.6 mio public employee records
... scraped from a US federal website.\n\nThis job required some lateral thinking and reverse engineering as this website was heavily using ajax / javascript.\n\nStandard approaches would have taken about one week for scraping. My alternative approach scraped those 1.6 million records in little more than 1 hour.
About Me
Software engineer (with almost 20 years experience) helping companies to extract/scrape, transform and analyse complex data from the internet and APIs into high quality structured datasets - even if the data source is difficult to access due to technology used or aggressive protection. Responsible and dedicated professional with attention to detail, delivers on time and within budget. Prefered technology Python and R as these are the most powerful tools in the area of data scraping and data mining (used by big companies like Google, etc) Using these technologies I can deliver on most scraping jobs within a few days or even hours. Results are statistically checked for quality (missing values etc). Also fluent in Javascript, jQuery, Node.JS, Ruby (on Rails), PHP and several other technologies to reverse engineer scraping target websites (if needed) as well as for creating and maintaining custom build scrapers.
NEW FREELANCER!
