Developer needed with Python / Scrapy / Scrapyd / MySQL / Splash.
The code repository is in bitbucket.
1. Task One: Video information not recording to page_video table. Our crawler is working but not saving this data to this table.
2. Task Two: Resolve Crawler Errors
a. Prevent a request to [url removed, login to view] of splash server from happening.
b. Do not forward head requests to splash
c. There's a lot of errors like this
2018-01-29 07:56:07 [[url removed, login to view]] ERROR: Error downloading <POST [url removed, login to view]>
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/twisted/internet/[url removed, login to view]", line 1299, in _inlineCallbacks
result = [url removed, login to view](result)
File "/usr/local/lib/python2.7/site-packages/scrapy/core/downloader/[url removed, login to view]", line 53, in process_response
File "/usr/src/app/InnoCrawler/downloadermiddleware/[url removed, login to view]", line 177, in process_response
request, response, spider
File "/usr/src/app/InnoCrawler/downloadermiddleware/[url removed, login to view]", line 156, in _convert_from_splash
for row in data['history']:
20 freelanceria on tarjonnut keskimäärin %project_bid_stats_avg_sub_26% %project_currencyDetails_sign_sub_27% tähän työhön
Hello Dear! I am very interested in your project. I have high skills in Python. Specially, I have rich experience in Scrapy/Scrapyd. If you contact me, we can discuss on your project. Thansk
Hello? How are you? My name is GuangJun I have read your brief so i can discuss the project with you. Hope to work with you. you can see my reviews. Thanks.
I believe that my 9+ years of programming experience makes me highly qualified for this project. I believe that I would be an asset to this project. Linux, PHP, Python, Software Architecture, Web Scraping,
I am full-stack developer and built so many webite using python,laravel,mysql. I can fix your bugs asap and complete your project successfully. I am faster programmer and keep high-quality,fresh code and deadline.