Build a module that scrapes html element info/dimensions after JS has been extracted.

We are building a scraping tool, and we need some assistance with executing Javascript and getting some data back.

You can use any common technology stack you like.

We need a simple HTTP API that will help us with scraping Javascript websites.

The requirements are:

A] There will be three endpoints:

Endpoint 1: api/page-content?url={webpageURL}

Take a web page URL, execute the Javascript then return the HTML that results.

Endpoint 2: api/image-dimensions?url={webpageURL}&imgUrl={imageURL}

Take a webpage URL and an image URL, and execute the Javascript, then return a JSON object with the height and width of the specified image as displayed on the page.

Note: this is not the actual height and width of the image file, but the display dimension of the image object on the page. Naturally the exact value depends on the screen resolution. You can set the screen resolution to something like 1200x1600

Often the actual image is much larger than the displayed image, we only want the displayed dimensions (in pixels).

Endpoint 3: api/largest-image?url={webpageURL}

Take a web page URL, execute the JS, then returns the imageURL, height and with of the largest image

The largest image is the one where the displayed size= height * width is the greatest.

Again height and width are based on the displayed dimensions, not the actual height/width.

C] There should be no need for adding anything to the server. We just deploy the app in a docker container and then it should run. If you need to for example add a .exe file to the server, this is not a suitable solution.

D] It should be able to use a proxy server for IP rotation. we will provide the host and login details of the proxy server for testing.

E] There should be simple authentication to provide unwanted access. But the authentication params should be hardcoded/in a config file, and not require database access.

We will need you to write and deploy the application - most likely we can host it with our PAAS company (depends on your chosen stack), otherwise it will be deployed on Digital Ocean (we will provide the account).

For us, this is very urgent so speed is important.

In your bid please specify the tech stack (or stacks) you are able to use, ask any questions you may have, and specify approximately how long this will take - or at least when you can start.

Look forward to working with you.

Edit: letter numbering is missing a B - this is an error. The components should be A, B, C, D.

Taidot: HTML, Javascript, Python, Tietojen kaavinta verkosta

Näytä lisää: scrape javascript generated content python, python requests execute javascript, how to scrape javascript content python, scraping javascript forms with python, web scraping javascript tutorial, scraping javascript rendered web pages python, javascript web scraping example, dryscrape vs selenium, joomla custom module php html, javascript move html element location, onmouse image html show info, tournament module websites html, calendar module websites html, html element flash, dotnetnuke module excel html, build reservation form html, html layout info submitted, build sql filter html page, fake html referer info, excel data html element

Tietoa työnantajasta:
( 1 arvostelu ) Nedlands, Australia

Projektin tunnus: #16280869

13 freelanceria on tarjonnut keskimäärin %project_bid_stats_avg_sub_26% %project_currencyDetails_sign_sub_27% tähän työhön


Hello I have done a lot of scrapping and can scrap any site(even protected one) on challenge. My distinction, payment after your complete satisfaction with the resulted task. Relevant Skills and Experience http:/ Lisää

$1441 AUD 20 päivässä
(204 arvostelua)

Hello Sir I can design & develop high quality website to fulfill all of your requirements. Our developed cross-browser website would be highly secure from HACKING and spamming. I can create web scrapper for you wi Lisää

$288 AUD 10 päivässä
(86 arvostelua)

Hello ! If we talk about the scraper then happy to inform you that, Recently i had finished one of the large project in python with DJango. Hope you will be surprised when you will look into the project, it's perfor Lisää

$1176 AUD 10 päivässä
(64 arvostelua)

Dear Employer, Hope you are great today..!! I have one through your job posting and founds our skills set perfectly matching. Our team has already successfully developed LinkedIn bot and bot for telegram is in pr Lisää

$2352 AUD 20 päivässä
(64 arvostelua)

Hi there, lets get in touch to discuss more details about the project. I believe this one is similar to our previous project. Thanks, Mirko

$1500 AUD 5 päivässä
(77 arvostelua)

We have expert team in javascript and api programming. I am sure i can deliver the final product on time. you will be very satisfy with our efforts. please contact me for any queries. thanks

$941 AUD 10 päivässä
(110 arvostelua)

Hello, After going through your project descriptions,I found your project doable by our team. Our Aim is to help you achieve your project’s goal. If you have few minutes lets chat. Thanks A lot.

$941 AUD 20 päivässä
(40 arvostelua)

Dear Hiring Manager, Thank you for reviewing my proposal.I have carefully gone through your job posting on " Build a module that scrapes html element info/dimensions after JS has been extracted " and I believe I am th Lisää

$2000 AUD 25 päivässä
(39 arvostelua)

I fully read your project description and understood your requirements I can make 3 endpoints which you want I can start now Thanks

$1500 AUD 10 päivässä
(17 arvostelua)

Hello there, I have 6+ years of experience in web development. I got your requirement, in short you need to create API which will be called from jQuery ajax. Those events will be on button click. If you think I can hel Lisää

$1176 AUD 15 päivässä
(29 arvostelua)

We have done many web scrapping tasks that you can check in our profile also . We found ourself perfect for this job . We are having very good impact in market since last few months with our very accurate timely deli Lisää

$1323 AUD 20 päivässä
(31 arvostelua)

hi there i have read your all requirements and can i give you desktop application which will all these things ?

$1323 AUD 20 päivässä
(33 arvostelua)

Dear Employer, I'm ready to work on "scrape html after JS has been extracted" project. Relevant Skills API, Excel, VBA Milestones 1156 AUD - code writing 289 AUD - userfriendly interface I would love to work on yo Lisää

$1446 AUD 21 päivässä
(12 arvostelua)