Käynnissä

c# Data gathering from web applications

Hi,

I have a program that gathers data from websites (adult). Each website is represented by a class with several methods (get articles from index, get data from article, etc.)

I need to create new classes for data collecting from a particular website (one for each website), no login is required. The data will then be structured as an object and returned.

I can provide you with some examples from other websites, usually it is very similar.

I have about 100 websites in a queue, waiting to be created. Some websites have a very similar structure, so the differences between those are minimal, usually some html tag has a different name/id.

I would prefer to split this project into more milestones - each milestone would represent 20 pages (or less/more depends what you prefer) and when the milestone is done, then you would start to work on a next batch of websites

First milestone will be a little bit harder/more complex (depends on your skills), but I need those sites to be done first.

It’s 6 very similar websites with almost identical html/JavaScript structure, but to access those websites there is a need to bypass double google captcha (at least from the server where I need the program to be running). Check no robot box, wait and check second no robot box, then the page loads, it’s google recaptcha).

In this case I would prefer a solution like this (if it is even possible):

- when captcha happens, sends the captcha link by email and wait till the captcha is filled on another device

- usually on mobile phone I will open the email and fill the captcha

- when it's done, process continues

Or if not possible, I would imagine something like this:

- when captcha happens, opens chrome with google captcha and waits till the captcha is done manually

- when captcha is filled, close/hide the chrome window and continue in the process

Other websites don't require the captcha bypass, so they won't be that complex.

When possible, I always prefer to use htmlagility pack, otherwise I'm using selenium chrome.

Except the captcha, it is not a complex scripting. Maybe a little time consuming, but I can provide you with a couple of done website where you can check how it was done and re-use the code for creating new classes.

In proposal, please let me know if you have done something similar in the past. And what would be the price.

Thank you and kind regards,

Matej

Taidot: .NET, C# -ohjelmointi, HTML, Javascript

Näytä lisää: php display data tables web page, clone web applications, data net web application excel macro, research data gathering, layout examples crud web applications, create single page web applications php, software database driven web applications, programming data entry web page, web applications data entry layouts, data gathering web site design, pda data gathering applications, web data gathering extjs, web page data gathering, importing data excel sheet j2ee struts web applications, gathering web data, web data gathering free, cool web applications data processing, web applications solution developer, SQL, ASP.NET,MVC ,C#, .NET ,Web applications , Windows Applications

Tietoa työnantajasta:
( 16 arvostelua ) Hradec Kralove, Czech Republic

Projektin tunnus: #23962153

Myönnetty käyttäjälle:

ZhangDa

Nice to meet you on freelancer. My name is Da Zhang. I am an experienced C# programmer with more than 3 years scraping experience. Look at my upwork profile also: [login to view URL]~016a0f5cbf2aeb73ea? Lisää

$250 USD 5 päivässä
(7 Arvostelua)
3.9

25 freelanceria on tarjonnut keskimäärin 508$ tähän työhön

geekydeveloper

Hello there, I have read out the details of your project. And I am one of the suitable candidates for your project. We have more than 9 years exp in the development. We have focused to deliver perfect valuable deliver Lisää

$700 USD 30 päivässä
(163 arvostelua)
7.6
Peymanmi

Dear, Thank you for your job posting I have read your requirement carefully and I understood what you want. I am very happy to see the job which is matching my skills and experiences. I have 19+ years of experiences in Lisää

$700 USD 7 päivässä
(3 arvostelua)
5.2
$444 USD 10 päivässä
(42 arvostelua)
5.4
wasleyabraao

Hi Matej You described proposal very carefully. Thanks again for your detailed explanation. I can help you with this project. I will use selenium mainly for this project and can do this project in 5 days.

$600 USD 10 päivässä
(9 arvostelua)
5.1
fastwebsoft

Hello Sir, I have read your requirement carefully. you need a web application We can do this job easily.. I have already developed many WEB APPLICATION, ERP,CRM,School management system, Hospital management system, Lisää

$500 USD 12 päivässä
(18 arvostelua)
4.9
shiva1888

Hi I am a C#/html/JavaScript expert and I ONLY apply to jobs when I know I can do it. I am confident that I will deliver you the best solution possible and will exceed your expectations. I never left a client unsatisf Lisää

$750 USD 7 päivässä
(24 arvostelua)
4.8
kozachenkodr

Dear client. Hope your are doing well. What about using python for your jobs? I am a python web scraping expert, I did many scraping job with python. I know how to avoid the captcha and how to make the classes for spec Lisää

$500 USD 7 päivässä
(9 arvostelua)
4.5
songhku925

Hi. Excuse me, you only want C#? how about using python for your goal? I have over 5 years of experience with python. I have already completed several scraping projects on this site, so I can deliver 100% result to ti Lisää

$500 USD 3 päivässä
(9 arvostelua)
4.2
patrickcamacho

Hi there I have just read your post and interested in it. I am a web expert having great experience in .NET, C# Programming, HTML, Javascript. I will provide you the website as per your requirements where you can eas Lisää

$555 USD 7 päivässä
(5 arvostelua)
3.8
AMITALLIED

Hi I have done this type of project in past. I can do this project. I have few questions about the requirements. Please message me so that I can discuss the project in details with you. Warm Regards Amit Bandana

$611 USD 8 päivässä
(9 arvostelua)
3.3
maxthor

Greetings, this kind of job is my every day job, I am an Electronics Engineer/Software Developer, I have worked reversing both hardware & software (which I enjoy most, by the way).Here you can find a video sample of a Lisää

$600 USD 7 päivässä
(1 arvostelu)
3.0
webtheorems

Hi, I am an Expert Software achitect (12+ Years) and 100% confident to presentthe Possible Cheapest Solution for this work post. -I am ready for Technical Discussion/Interview -Available as per your time zone - Lisää

$744 USD 9 päivässä
(6 arvostelua)
3.1
wangcaocao

Hello, how are you? I had read your description very carefully. I can make the application for gathering data from multiple websites. I already had many experiences in bypassing google captcha with c#. Also I used for Lisää

$300 USD 7 päivässä
(2 arvostelua)
2.4
iSnopy

Dear Client I have read your description very carefully and I am sure that can complete your project perfectly in short time. The Honest and Hard Work is diffterent than other freelancers. I am a new Freelancer but a A Lisää

$250 USD 7 päivässä
(3 arvostelua)
2.0
milosstankovic33

Dear Client, I have read your requirement in detail. I have several projects of scraping with C#, Python, etc. We can start work immediately, and I can lead this project to perfect result. Thanks, Regards.

$500 USD 7 päivässä
(1 arvostelu)
1.6
vcechella

Hello, Thanks for your job posting. I read your requirements carefully and feel your job just fits my skills. I am good at web scraping using Python, c#, and PHP. And I have rich experience in this area. So I confident Lisää

$500 USD 5 päivässä
(2 arvostelua)
0.6
BigFTeam

Hi, We have over 11 years experience in developing web based software. As part of this, we have internally built scrapers to pull data from other websites and also for our own site for SEO audits. We have recently de Lisää

$500 USD 7 päivässä
(0 arvostelua)
0.0
mariakashifse

I can works wonders for you and enthusiast to do new and complex jobs . I am a Gold medalist in Information Technology I also have a team under me with different expertise (like developing , designing , content wri Lisää

$300 USD 5 päivässä
(0 arvostelua)
0.0
mmohsi

Dear Prospect Hiring Manager, After a full understanding of your project description, I can say I could help you with strong interests and desire. I will use a mixture of selenium and htmlagility pack for your proj Lisää

$750 USD 60 päivässä
(0 arvostelua)
0.0
NiravDalsaniya

Hello, I read your requirement and I am able to design a Project for you in .NET C# I'm a new freelancer, but Honesty and hard work is the difference between others. I am a C# and ASP.Net, MVC Expert. I can offer you Lisää

$500 USD 7 päivässä
(0 arvostelua)
0.0