Suljettu

Asynchronous Web Scraping and Data Aggregator

OVERVIEW

The objective of this project is to aggregate data from various websites related to real estate listings and property info. The results should be aggregated into a single CSV which can be easily viewed in Excel.

REQUIREMENTS

1. The program should be configurable to limit results based on price, property type, and city including its surrounding area

2. Individual websites should have ability to be enabled/disabled in the configuration settings

3. Datapoints which are scraped must be normalized and/or converted to the same units of measurement so they can be easily and meaningfully sorted in Excel

4. Furthermore, the program must be able to send asynchronous requests to the websites to speed up execution time

5. All code must be thoroughly commented and easy to understand for future modifications and debugging

6. Code must also be robust to continue working even if minor website layout changes occur

7. Program must also produce meaningful logs to debug and troubleshoot should problems arise in the future

8. Upon completion of the project, an instructional video (screen recording with captions and/or audio narration) and/or detailed instruction manual should be provided to demonstrate the key features and functionality of how the program works as well as the code’s structure

Please see attached document for detailed project description including datapoints and sources to scrape.

Please answer the following questions when submitting a proposal:

1.) Do you have any questions about the job description?

2.) Do you have suggestions to make this project run successfully? How do you plan on improving the execution speed when scrapping thousands of listings from several websites?

3.) What past project or job have you had that is most like this one and why?

4.) Which of the required job skills do you feel you are strongest at?

5.) What programming language, libraries, and frameworks do you plan on using for this project and why do you choose them VS available alternatives?

Taidot: BeautifulSoup, Python, Scrapy, Selenium, Tietojen kaavinta verkosta

Näytä lisää: concurrent web scraping, efficient web scraping, web scraping blog, advanced web scraping python, python web scraping challenges, python async requests, web scraping tips, python async download, asynchronous web scraping python, python web scraping certificate, python asynchronous programming, how to make web scraping faster, data mining web scraping, scraping data web page excel, php web scraping store data, scraping data web match word, scraping data web pagejob, web scraping data collection website, web scraping data facebook, web scraping data extraction

Tietoa työnantajasta:
( 0 arvostelua ) Edmonton, Canada

Projektin tunnus: #25853299

18 freelanceria on tarjonnut keskimäärin 669$ tähän työhön

vladzolotukhin

Hello. As a web scraping and data mining expert by python and node.js, selenium web driver, I am glad to place the bid on your project. I have experienced LinkedIn profile scraping, amazon products scraping and ticket Lisää

$1000 CAD 5 päivässä
(32 arvostelua)
5.8
kirilnberezebko

Hi Kushal. I am a senior web and data scraping script developer with 6+ years of experience. I have worked on several similar projects scraping email address and phone number and specific datas from websites with pytho Lisää

$750 CAD 5 päivässä
(12 arvostelua)
5.4
arturkandalyan

Hi sir, I am a Web & Data Scraping Expert who have career for 7 years over. I am very happy to bid on your job. I have rich experience of scraping for collect data & contact & business information such as email, phone Lisää

$700 CAD 5 päivässä
(6 arvostelua)
4.8
mkhrsunil

Hello there, I have gone through your requirement and understood. I have few things to ask but let me answer first what you asked in description.. 1.) Do you have any questions about the job description? Yes.. Are yo Lisää

$850 CAD 10 päivässä
(7 arvostelua)
4.5
suyashdhoot

Hi I am a very experienced statistician, data scientist and developer. I have worked with data from several companies and have done projects involving high level quantitative analysis and data interpretation skills to Lisää

$1500 CAD 7 päivässä
(20 arvostelua)
5.3
biopku

Hi there I have 10 years experience in Python and web development. I did lots of projects in web world, develop in most web technologies. 8 years developing Django sites and services. I did lots of projects in scrapy a Lisää

$500 CAD 7 päivässä
(9 arvostelua)
4.3
matiasjumadiaz

1) Not now 2) Yeah, I am gonna use multi-threading and processing. 3) Please check this previous work example. - Realestate Scraping [login to view URL] 4) Lisää

$500 CAD 7 päivässä
(6 arvostelua)
4.0
skdcoding

You can also check out my recent review where I was commended for coding standards in particular. My Repositories: [login to view URL] You can take a look at my repos to check code quality ab Lisää

$650 CAD 3 päivässä
(9 arvostelua)
3.5
TTomaszW

Hello, sir. How are you? I have just read your project description and I can help you as my rich experiences. I have rich experiences in Web Scraping I am a new freelancer, but I can give you the perfect result you ar Lisää

$500 CAD 7 päivässä
(2 arvostelua)
2.9
CloudAdvisory

Hi, Your posting is very interesting. I have 8 years of experience in web development field. I have worked on several similar projects scraping email address and phone number and specific data from websites with pytho Lisää

$500 CAD 7 päivässä
(3 arvostelua)
2.0
ankleshsingh

Hi, Hope you are doing good. I have gone through your requirement and I do have skill set you are looking for. I am certified advanced RPA and QA Automation Professional. I have 2+ year of experience with Automation Lisää

$677 CAD 7 päivässä
(1 arvostelu)
2.1
SetDad

hello how are you? I am very interested in your project i have much experiences in web scraping. I am sure I can do your project I am ready to start work immediately thx

$500 CAD 7 päivässä
(0 arvostelua)
0.0
faddons

hi i am a fullstack web and application developer. i have about 9 years of experience and most of my activity in developing extensiosn and templates has been for cms.

$300 CAD 12 päivässä
(0 arvostelua)
0.0
Hansie19

Hi, I am medical laboratory [login to view URL] experience almost 18 years plus I had 8 years of international gulf [login to view URL] to covid [login to view URL] are in worst stage of our cariers. Relevant Skills and Experience I am fr Lisää

$556 CAD 10 päivässä
(0 arvostelua)
0.0
sabadashsvatosla

Hi I can answer the question. 1) I have some question. - Do you want to GUI scraping program? - Do you want to scrap all of that websites mentioned on attached file at the same time? - What is the meaning of m Lisää

$750 CAD 8 päivässä
(0 arvostelua)
0.0
astrasoft8

Dear Prospect Hiring Manager. Thank you for giving me a chance to bid on your project. I am a serious bidder here and I have already worked on a similar project before and can deliver as u have mentioned "I can do th Lisää

$500 CAD 7 päivässä
(0 arvostelua)
0.0
miro8818

Dear Sir I am very motivated in your project. I have many experiences of web scraping and have been working for 5+ years. I already have done many shoppingmall, sports betting bots such as amazon, alliexpress, bet365, Lisää

$555 CAD 7 päivässä
(0 arvostelua)
0.0
DigitalWorld2

Why Choose Me ! - I have been working with the Canadian based Company (Elevate-inc) which is positive review generating site. Basically we Scrap Reviews and integrate it with our portal. - I have Multiple Sample Scra Lisää

$750 CAD 30 päivässä
(0 arvostelua)
0.0