Find Jobs
Hire Freelancers

Selenium and Scrapy web scraper doesn't work (score = 0.1)

$30-250 USD

Käynnissä
Julkaistu lähes 5 vuotta sitten

$30-250 USD

Maksettu toimituksen yhteydessä
Hello, I own a web scraper in Python 3, which uses Selenium. It connects to good quality private proxy and scrapes the website using Chrome browser, exporting content to HTML files. The problem is the scraper is getting 0.1 score (1.0 is maximum), which is too low and searching on site is being blocked due to unusual traffic. When searching manually without using the scraper, my score is 0.9 (when logged to Gmail account). I assume that the problem is Google's verification system v3 (no images, just score) is detecting the bot. Scraper remembers the login and password to Gmail account, as it should increase the score but Google blocked the try of logging in, due to critical error (unauthorized use of password). Scraper uses chrome driver. 0.5 score would be enough to scrape. I own a source code. If you are interested in fixing this problem, please post an offer. Important: Please describe what is your idea on fixing the problem. If you advice to create totally new scraper, please also describe how it would work. I respect your know-how, but please be precise as possible. Thank you.
Projektin tunnus (ID): 19978336

Tietoa projektista

12 ehdotukset
Etäprojekti
Aktiivinen 5 vuotta sitten

Haluatko ansaita rahaa?

Freelancerin tarjouskilpailun edut

Aseta budjettisi ja aikataulu
Saa maksu työstäsi
Kuvaile ehdotustasi
Rekisteröinti ja töihin tarjoaminen on ilmaista
12 freelancerit tarjoavat keskimäärin $151 USD tätä projektia
Käyttäjän avatar
hi, I have read you description in detail. I have experience with SCRAPY AS WELL AS SELENIUM scrapers in PYTHON. I can look at the code and resolve the issue. ALSO, i believe scrapers made using WEBREQUESTS are faster and more reliable as compare to SELENIUM based scrapers. But lets discuss the details in private chat. Looking forward to your reply.
$155 USD 3 päivässä
5,0 (349 arvostelua)
7,5
7,5
Käyttäjän avatar
Dear Sir. Glad to meet you. I'm Web developer specializing in web scraping crawling and indexing web pages. Skills: python, scrapy, selenium, requests, beautifulsoup, mechanize, lxml, urllib2, automation, bots, spiders I worked with 50+ differenct individuals/startups/companies and helped them archive their goals. Here's what you will get by working with me: * Working only on projects that I can deliver 100% * Ridiculous quality of work * Smart suggestions on increasing results/revenue and decreasing costs * Responsibility for tasks/project I work on * Reliability * Short response time * Good sense of humor ;) Hope I could awake you interest in me. Thank you for your attention. Shuaixing.(call me Shuaixing L.)
$200 USD 7 päivässä
5,0 (34 arvostelua)
6,8
6,8
Käyttäjän avatar
Dear Hiring Manager.⭐ I am a python programmer and I have faced many python problems.. Hope you give this project to me.. Relevant Skills and Experience I have worked with python + web development with Django framework and worked with several libraries in python including Selenium, Beautiful Soup, Flask, Django, Scrapy and many more. I have that much experience to complete your project. If you give me an opportunity I assure you that I will give you my best. Thanks . :)
$100 USD 1 päivässä
5,0 (84 arvostelua)
6,4
6,4
Käyttäjän avatar
want to see the source code. can try with proxy rotation and user agent. These are my skills related to web scraping and crawling Have done scraping in CasperJS Phantomjs, python. Have done testing and automation with selenium also. Know to deal with database like mongo, mysql, Elasticsearch. Also know to handle proxy and captcha while scrapping.
$166 USD 2 päivässä
4,9 (64 arvostelua)
6,3
6,3
Käyttäjän avatar
Hi there, I have scraped amazon, aliexpress, yellow pages ,yelp ,zomato etc. I have 500GB internet with 20mbps speed, i have 6 systems I can do it. I have done many related projects like this. If you are provide this work, it will help my career also. Give me a chance to do this. Waiting for your precious reply. Thank you. Please Check my mastery work at:- https://www.freelancer.in/u/Stephenrajs *Why you are choose me 1. 24/7 hours support 2. Quick response 3. Deliver on time 4. Smooth communication. I assure that I can satisfy you completely and want to have a long term relationship with you. Best Regards Stephenrajs
$90 USD 3 päivässä
4,9 (130 arvostelua)
6,0
6,0
Käyttäjän avatar
Hi, Python Ninja here! I have understood your requirement with the Selenium and Scrapy web scraper work, and I am very confident with that. I can make it work for you. I am Usman. I have 6 years of experience in the Web & Mobile App Development Department and Designing. I have reviewed the description and understand it very well. I want to start the project right now, just need to discuss it little to more clear the image of the project. Please come to inbox so that we can lock the scope of the project. I am an expert in my domain areas like PHP, Laravel, CodeIgniter, My SQL, Angular, React, Node, Mongo DB, Ionic, React Native, Android, Java, IOS, Swift & Golang. I have extensive knowledge of Html5, Css3, Javascript, Ajax, jQuery, Bootstrap, and Payment Integration modules as well. I assure you that you will experience the WOW factor in my work. I am looking forward to your response. Many Thanks Usman Adil
$250 USD 5 päivässä
4,5 (24 arvostelua)
5,2
5,2
Käyttäjän avatar
Hey there! Selenium scraper here, I can do any sort of data mining or web scraping that you need to be done in a reasonable amount of time. I have years of Python, HTML, CSS and JavaScript experience under my belt and am ready to apply it to whatever your project requires! Here are some examples of things I've done for clients before: Writing custom website scrapers to get you the data you need! Scraping census data on a large scale and finding correlations in the data Mining hundreds of thousands of tweets and performing sentiment analysis Transforming data into Python data structures like pandas data frames. Creating PDF and document scrapers to help you sift through thousands of documents in minutes and get exactly the information you're after. Creating & managing a Django, Flask, or other pythons web-framework website. E-commerce data harvesting/Scraping bots/scripts development. Web Scraping of any publicly available data. Web Scraping of the business listing. Data collection from any directory/listing. Any type of Web Automation. Kindly reply to me, I am waiting for your kind response Thanks
$120 USD 3 päivässä
5,0 (10 arvostelua)
4,6
4,6
Käyttäjän avatar
The blocking issue can be because of Google ReCaptcha. i faced this problem in when i was try to scrap google search results on Mass amount. i didnt totally bypass it but i was able to avoid it for much longer time using different country domain of google while searching google. it helped me to raise my score from 15 to 70 approximate (max is 100)
$50 USD 5 päivässä
5,0 (1 arvostelu)
0,5
0,5
Käyttäjän avatar
Hello there, Hope you are doing good...!!! As per your requirement, I am very much suitable for the job kindly invite me for a chat for further discussion regarding the project. I have more than 6 years of experience in web & mobile app development you will get good quality work from me. Thank you
$155 USD 3 päivässä
0,0 (0 arvostelua)
0,0
0,0

Tietoja asiakkaasta

Maan JAPAN lippu
Tokyo, Japan
5,0
1
Maksutapa vahvistettu
Liittynyt huhtik. 18, 2019

Asiakkaan vahvistus

Kiitos! Olemme lähettäneet sinulle sähköpostitse linkin, jolla voit lunastaa ilmaisen krediittisi.
Jotain meni pieleen lähetettäessä sähköpostiasi. Yritä uudelleen.
Rekisteröitynyttä käyttäjää Ilmoitettua työtä yhteensä
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Ladataan esikatselua
Lupa myönnetty Geolocation.
Kirjautumisistuntosi on vanhentunut ja sinut on kirjattu ulos. Kirjaudu uudelleen sisään.