Build me a crawling tool

- We need you to build us a crawling tool from scratch which follow these requirements:

--- Requirements ---

- The tool should be used Scapy Python and MongoDB.

- It should dockerized into docker-compose separated into two containers: web and db.

- It should be easier to extend if we gonna crawl the new site.

Such as we will have folderA will crawl site A, folderB will crawl site B...etc

- It should be easier for configure the database and the site we gonna crawling.

- When the structure of codebase is done, we will review it and you need to send us a documentation to show:

+ How can we setup crawler for new site ?

+ Which file/folder need to be created

+ Which command need to be executed to crawl all the sites which were configured in the codebase.

- We will send you an example site, so you will do an example for us when the structure of codebase is done.

- I will send you invitation to github repository then you will work on it.

---- Note ---

- Please read it carefully before you ask us or bid with the right price.

- Only release payment if the codebase is good enough and easy for us to crawling the site.

- Please mark prefix 'DDPythonMongo' when you bid. Otherwise, I'll skip the bid due to automatically bid or bot.

Taidot: Python, Tietojen kaavinta verkosta, MongoDB, Elasticsearch

Näytä lisää: read mail php file, read data excel file, build msn chat bot msn messenger, read values txt file html, company will read book audio file, read delete file folder php code, read lines text file, php read complex xml file, read check text file, tool program read hotmail, read skype dbb file, read header flv file, read line text file net, read simple xml file, java read hebrew utf file, read certain part file, bot crawling flash website scraping data xml file, discord bot read text file

Tietoa työnantajasta:
( 2 arvostelua ) Ho Chi Minh, Vietnam

Projektin tunnus: #21358106

27 freelanceria on tarjonnut keskimäärin 172$ tähän työhön


Hello Respect sir I saw your description, I can't stand the excitement of seeing the project you post on... I bet one's shirt on I have the ability to complete your project perfetly because I have experience with your Lisää

$140 USD 7 päivässä
(89 arvostelua)

DDPythonMongo Hi, Client!! @$@$@$PLEASE CONTACT ME. I CAN DO IT WONDERFULLY.$@$@$@ ##% Your idea is attracted me to your world.%## I am Python expert. i have various skills. @example@ python selenium and beautiful Lisää

$140 USD 7 päivässä
(47 arvostelua)

Hello, I have gone through your job posting and become very much interested to work with you. I am an expert in this field. I have already completed several projects like this. For evidence you can see my profile. Pl Lisää

$220 USD 4 päivässä
(35 arvostelua)

'DDPythonMongo' Hello sir, Please take a look at my reviews and portfolio. I have completed web/data scraping jobs many times. I am interested in your project as well. I would like to discuss details via pm. I loo Lisää

$400 USD 5 päivässä
(45 arvostelua)

Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHON Lisää

$222 USD 3 päivässä
(62 arvostelua)

Hi there. Thank you for your posting. My name is YK Sun and I am a cloud computing engineer. While working with several companies, I have been familiar those things such as elasticsearch, kibana, fluentd, etc. I used e Lisää

$300 USD 2 päivässä
(11 arvostelua)

Hello, How are you? I'm very interested in your job post.I believe my skills would be ideal for your project. I am Python Expert and I have rich experience with Web Scraping and I can help you perfectly. And then I am Lisää

$140 USD 5 päivässä
(6 arvostelua)

DDPythonMongo. I can create the crawler with python, scrapy and mongo; dockerized with docker-compose, that can be managed from the database (depending on the type and content of sites being crawled and scaled). Please Lisää

$200 USD 7 päivässä
(13 arvostelua)

DDPythonMongo. Hello I have read your description carefully to get a clear understanding of what you want and what I should achieve. Now I want to say that I am really interested in your task and I can do it successful Lisää

$140 USD 7 päivässä
(3 arvostelua)

Hi.Dear.. Credit is my motto. I am a freelancer working in 24/7 service. I can scrape anything.I know all the angles of website scraping. I have much experience for 10+ years such as avoiding recaptcha, using multi thr Lisää

$100 USD 3 päivässä
(5 arvostelua)

Hello "DDPythonMongo'" We are familiar with creating crawlers with python scrapy and selenium as well meanwhile we have experienced with IP rotation,User agent rotat Lisää

$140 USD 7 päivässä
(4 arvostelua)

Hello, I have majored in web scraping by using scrapy, selenium, python, etc. Also, I am familiar with MongoDB and Docker. I can help you to build your crawling tool perfectly. Ready to start. Thanks.

$200 USD 3 päivässä
(6 arvostelua)

DDPythonMongo Dear, employer. OK, NO PROBLEM! I am a Web Scraping expert. After reading over your description this looks like a perfect fit for my skill sets. I estimate to complete your project within the time-line a Lisää

$150 USD 7 päivässä
(5 arvostelua)

DDPythonMongo Hi I have checked your requirement and understand it well. I am a full stack developer with enough experience and skills in Web. I have best experiences in Web Developing such as PHP, Wordpres, Woocomerce Lisää

$140 USD 5 päivässä
(2 arvostelua)

Hi , I am a mathematician. I use scrapy to crawl websites. I use python scripts to shell websites and xpath tofind textual data. I test the python commands in shell using python jupyter before transposing it to code in Lisää

$100 USD 7 päivässä
(1 arvostelu)

Hello Sir! I'm skilled in web scraping with Scrapy framework, and scraping automation with Selenium and beautifulsoup. If you need to get information off the web in an automatic fashion, I can write you a special scrip Lisää

$140 USD 7 päivässä
(2 arvostelua)

DDPythonMongo Hello, I have been developing reptiles for 2 years. I am qualified for this job and look forward to working with you.

$140 USD 7 päivässä
(0 arvostelua)

Hi, Thanks for posting !!! I have 12 years of experience in Python development. I have checked The requirement and understand that as well. I have created Website SIMILAR to Build me a crawling tool. Do you w Lisää

$140 USD 7 päivässä
(0 arvostelua)

Hi, nice to meet you. I am senior scraping expert. I am interesting in your project. If you hire me, I will give you satisfaction. Please contact me and discuss more details. Thank you for reading my proposal. Regards.

$200 USD 7 päivässä
(0 arvostelua)

Dear Hiring Manager, I read your job description and I am confident that I can exceed your expectations. I am a professional programmer on PHP+MySQL+WordPress. Please have a look at my recent work for WordPress : 1. Lisää

$140 USD 7 päivässä
(0 arvostelua)