a web crawler for data collection

1. Get full list of companies from data sources like Glassdoor, or Linkedin

2. Crawl the products page from each of the companies, if the page exists

3. Process the crawled data

4. Save the crawled data into a NoSql database, like MongoDB

5. Must be multi-processing, efficient and effective

6. Easy to stop and start to run

7. Can be written in Java or Python. Only these two languages allowed.

8. Able to use proxy IPs to prevent blocking

Taidot: Java, NoSQL Couch & Mongo, Rinnakkaisprosessointi, Python, Web Crawling

Näytä lisää: need web crawler data, web hosting data collection, web crawler data mining java, web research data collection, java web crawler data mining, web crawler data extraction, web scraping data collection website, web design data collection, data collection services java, web excel data collection, web crawler data database, web based data collection excel, list prices web robot data collection, web crawler data sql server, web crawler data extractor, web form data collection, j2me data collection struct java

Tietoa työnantajasta:
( 25 arvostelua ) Herndon, United States

Projektin tunnus: #14939218

25 freelanceria on tarjonnut keskimäärin 184 $ tähän työhön


Already designed and developed webcrawler . Contact me I am interested. Relevant Skills and Experience EXPERT Proposed Milestones $500 USD - Default

500 $ USD 3 päivässä
(107 arvostelua)
155 $ USD 3 päivässä
(44 arvostelua)

Hi, I am FURKAN a CERTIFIED Java, NoSQL Couch & Mongo, Parallel Processing, Python, Web Crawling, WEB SCRAPPING EXPERT . I would love to provide you Quality data by getting a list of companies from LinkedIn and th Lisää

30 $ USD 1 päivässä
(31 arvostelua)
555 $ USD 10 päivässä
(27 arvostelua)

Hi I am very,very interested in your project I have 6 years of web scraper development experience such this projects. And I have good skills with this program languages - C/C++, C#, java, php,, python, VB.NET Lisää

155 $ USD 2 päivässä
(26 arvostelua)

Hi, I read your requirement and understand it very well. I am able to do it. I am scraping data by using tools which are developed by using "Java Technology". Relevant Skills and Experience Experience: I have comple Lisää

250 $ USD 3 päivässä
(2 arvostelua)
200 $ USD 3 päivässä
(2 arvostelua)
155 $ USD 3 päivässä
(10 arvostelua)
155 $ USD 3 päivässä
(1 arvostelu)

I am an IITK graduate and I have 9 years of experience in software development. I have 100% completion rate and I have finished all the projects with the highest level of customer satisfaction. Relevant Skills and Exp Lisää

155 $ USD 3 päivässä
(18 arvostelua)

I'm a professional Python developer. Also I have a lot of experience in web scraping. I read the description and it looks pretty straight forward. please contact me to discuss more. Relevant Skills and Experience Pyth Lisää

140 $ USD 5 päivässä
(3 arvostelua)

A proposal has not yet been provided

155 $ USD 3 päivässä
(1 arvostelu)

I’m a Data Analyst at I currently build statistical model to improve the sales performance and predict sales using Python, Machine Learning and other Data Science Tools. Relevant Skills and Experience 3 Lisää

83 $ USD 15 päivässä
(0 arvostelua)

A proposal has not yet been provided

93 $ USD 3 päivässä
(0 arvostelua)

Hello , this project is quite interesting me , i have a lot of experience in the web development, i would like to make a python app for your need. Can we chat a bit before? Relevant Skills and Experience Web engineer Lisää

166 $ USD 3 päivässä
(0 arvostelua)

Hello, I've reviewed your complete job description, and I fulfill all the qualifications required for this project. I am expert developer, i can perfectly develop a web crawler in any language which you can use to g Lisää

30 $ USD 3 päivässä
(0 arvostelua)

I am perfect of this. I made bots to scrap the data. I had made bot for my company which is running over alibaba and other big b2b portals and directories. So, I know about the security majors etc. Relevant Skills and Lisää

222 $ USD 7 päivässä
(0 arvostelua)
133 $ USD 3 päivässä
(0 arvostelua)

I am very experienced in crawling. Relevant Skills and Experience java Proposed Milestones $155 USD - milestone

155 $ USD 3 päivässä
(0 arvostelua)

Hey there , off topic , are you I tested I a full USA business database (updated ) includes emails phone numbers location names and more ..

111 $ USD 1 päivässä
(0 arvostelua)