Resolve Scrapy Crawler Errors: Python / Scrapy / Scrapyd / MySQL / Splash
$250-750 USD
Maksettu toimituksen yhteydessä
Developer needed with Python / Scrapy / Scrapyd / MySQL / Splash.
The code repository is in bitbucket.
1. Task One: Video information not recording to page_video table. Our crawler is working but not saving this data to this table.
2. Task Two: Resolve Crawler Errors
We have added the ability to crawl javascript with Scrapy by using Splash. We have a few errors that we need resolving.
a. Prevent a request to [login to view URL] of splash server from happening.
b. Do not forward head requests to splash
c. There's a lot of errors like this
2018-01-29 07:56:07 [[login to view URL]] ERROR: Error downloading <POST [login to view URL]>
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/twisted/internet/[login to view URL]", line 1299, in _inlineCallbacks
result = [login to view URL](result)
File "/usr/local/lib/python2.7/site-packages/scrapy/core/downloader/[login to view URL]", line 53, in process_response
spider=spider)
File "/usr/src/app/InnoCrawler/downloadermiddleware/[login to view URL]", line 177, in process_response
request, response, spider
File "/usr/src/app/InnoCrawler/downloadermiddleware/[login to view URL]", line 156, in _convert_from_splash
for row in data['history']:
KeyError: 'history'
Projektin tunnus: #16535481
Tietoa projektista
16 freelanceria on tarjonnut keskimäärin $515 tähän työhön
Hello Dear! I am very interested in your project. I have high skills in Python. Specially, I have rich experience in Scrapy/Scrapyd. If you contact me, we can discuss on your project. Thansk
Hello? How are you? My name is GuangJun I have read your brief so i can discuss the project with you. Hope to work with you. you can see my reviews. Thanks.
Hi there..... Warm Greetings We came along with your request for Resolve Scrapy Crawler Errors: Python / Scrapy / Scrapyd / MySQL / Splash and we reviewed your project description. We'd like to help you with conf Lisää
hi, sir. i am a python expert. so if you give me this project, i can finish it in time. the price may be negotiable. thanks.
I believe that my 9+ years of programming experience makes me highly qualified for this project. I believe that I would be an asset to this project. Linux, PHP, Python, Software Architecture, Web Scraping,
Hello I'm experienced Python developer and I'm very interested with the task. Let's discuss it in detail over the chat. Regards, Alex.
Hello dear, I hope this proposal find you well, I saw your job posting online, I would like to request you please have a look into my previous works. I am a full-time freelancer with over 5 years of vast experience i Lisää
Hi Sir, I can do the project with ease, i just need the bitbucket repository access, and I am ready to fix those errors. I am proficient with splash, scrapy, scrapyd, and Mysql. I can easily fix Task1 but when it com Lisää
