Developer needed with Python / Scrapy / Scrapyd / MySQL / Splash.
The code repository is in bitbucket.
1. Task One: Video information not recording to page_video table. Our crawler is working but not saving this data to this table.
2. Task Two: Resolve Crawler Errors
a. Prevent a request to [login to view URL] of splash server from happening.
b. Do not forward head requests to splash
c. There's a lot of errors like this
2018-01-29 07:56:07 [[login to view URL]] ERROR: Error downloading <POST [login to view URL]>
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/twisted/internet/[login to view URL]", line 1299, in _inlineCallbacks
result = [login to view URL](result)
File "/usr/local/lib/python2.7/site-packages/scrapy/core/downloader/[login to view URL]", line 53, in process_response
File "/usr/src/app/InnoCrawler/downloadermiddleware/[login to view URL]", line 177, in process_response
request, response, spider
File "/usr/src/app/InnoCrawler/downloadermiddleware/[login to view URL]", line 156, in _convert_from_splash
for row in data['history']:
18 freelanceria on tarjonnut keskimäärin %project_bid_stats_avg_sub_26% %project_currencyDetails_sign_sub_27% tähän työhön
Hello Dear! I am very interested in your project. I have high skills in Python. Specially, I have rich experience in Scrapy/Scrapyd. If you contact me, we can discuss on your project. Thansk
Hello? How are you? My name is GuangJun I have read your brief so i can discuss the project with you. Hope to work with you. you can see my reviews. Thanks.
I believe that my 9+ years of programming experience makes me highly qualified for this project. I believe that I would be an asset to this project. Linux, PHP, Python, Software Architecture, Web Scraping,