Suljettu

Web Scraping on a Large Scale

I am currently in the process of gauging the feasability and scale of a web scraping project. The gist of this project is to build a database of all degrees offered by all universities in NSW, Australia (collecting key fields such as University, degree name, faculty, skills, related industries, description). This data needs to be scraped from University Handbooks, with the typical degree page looking like this: ([login to view URL]).

This is a large scale project. There are 43 universities in NSW, with each of these offering anywhere up to a few hundred different degrees (considering undergraduate and postgraduate).

I plan to manage this project myself, and will be available throughout to assist you in your work.

The method for me to receive the data would be me setting up a database on a cloud platform like Azure, then giving your IP temporary access (as well as giving you the required credentials) to read from and write to the database, in order to push the data. As the project begins, we can discuss database structure or other possible delivery methods.

I am putting down a price estimate only because this is a requirement for posting a job on this platform - please ignore it when deciding on your price estimation.

What I am looking for now is a justification for your belief in your ability to complete this project, and a rough time and cost estimate. If I go ahead with this, I would be looking to begin this project in about a month.

Taidot: Tietojen kaavinta verkosta

Näytä lisää: web scraping visual basic, web design salary per month, web scraping flash sites, web scraping jobs, google base web scraping, upload video web php large, data mining web scraping, small medium large scale projects, web scraping projects, web scraping practice projects, web scraping experience, web scraping password protected, large scale web game, large scale front end j2ee web development projects, large scale web development, large scale web development cost, php net large scale web application, large scale web application, design large scale web application, information architecture for the world wide web designing large scale web sites 3rd edition

Tietoa työnantajasta:
( 3 arvostelua ) Yowie Bay, Australia

Projektin tunnus: #18073070

59 freelanceria on tarjonnut keskimäärin %project_bid_stats_avg_sub_26% %project_currencyDetails_sign_sub_27% tähän työhön

jeweljitu

Dear Employer, I am a manual web searching and web scraping expert.i will collect all university [login to view URL] you provide me with the template, please?. I am a highly trained Data Entry, Resear Lisää

$2722 AUD 30 päivässä
(637 arvostelua)
8.3
seaanddream

Hi, my name is Selim. I read your `Web Scraping on a Large Scale` project descriptions carefully before bidding. I checked your requirements as well... I got what you need and ready to go ahead as soon as we can clar Lisää

$3000 AUD 30 päivässä
(249 arvostelua)
7.6
mirniyazuddin92

Hi there, I can build a database of all degrees offered by all universities in NSW, Australia. If you have questions or doubts about anything, please feel free to ask me. Sincerely, Mir

$2000 AUD 30 päivässä
(155 arvostelua)
7.1
jubair7

I'm still interested, if you consider. Let's discuss! Hello there, hope you're doing well. I have gone through your details and am happy to mention that I am interested in your project. And it would be best if you c Lisää

$1500 AUD 15 päivässä
(144 arvostelua)
7.1
bouslimi1979

Hi, I need your assistance to show me what to gather exactly, so I can estimate the price and the delay. thanks ***************************************************

$2222 AUD 30 päivässä
(125 arvostelua)
7.0
Kalpanasekhar

Hi, This project seems familiar to the project we recently worked on for UK Colleges. We would like to discuss directly regarding the scope of this project. We understood your requirement and would definitely do Lisää

$2222 AUD 30 päivässä
(39 arvostelua)
6.8
zekovicm

Hi there,I am Miljan,Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this job ! I can start immediately and finish it within Lisää

$2500 AUD 15 päivässä
(64 arvostelua)
6.7
mmadi

Dear EthanDev I have gone through your project Web Scraping on a Large Scale Can we do quick chat right now so that we can discuss the project briefly? I do have similar experience doing something similar to yours Lisää

$2700 AUD 36 päivässä
(7 arvostelua)
6.0
zolayossa

I understand your requirements and what you wanted scrapped, and this is a large scale project that needs full time and dedication. I have a couple of questions. Please get back to me via chat and let's discuss detail Lisää

$2498 AUD 30 päivässä
(102 arvostelua)
6.2
diamond247

Hello, I have gone through your project details and understand the requirement properly as it needs data for the universities and details. We are a team here and complete more than 200 similar projects. And compl Lisää

$1500 AUD 30 päivässä
(112 arvostelua)
6.4
adeelpirzada

Hi, there i have done scrapping almost on Half of Worldwide web including eCommerce giants(Amazon,eBay,craigslist) News Feed, Social media websites, API's. I develop my own tools based on client requirements with Mu Lisää

$2222 AUD 30 päivässä
(15 arvostelua)
5.5
masterlancer999

Hello, I'm an expert of web scraping. I've worked with so many startups/companies in big data processing and web scraping. I have lots of experience and skills; I've already extracted so many different kinds of d Lisää

$2500 AUD 30 päivässä
(15 arvostelua)
5.3
DavidLindon

would be written in C# and a custom scraper written for each of the 43 universitys

$1666 AUD 18 päivässä
(24 arvostelua)
5.5
dkarataev

Hi EthanDev, I find myself eligible for this job since I am having 7 years of extensive experience I am Good at Web Scraping EthanDev, Please Send a message so that we can discuss more about this project

$2500 AUD 10 päivässä
(5 arvostelua)
5.0
stevobujica91

Hello. I'm a Python developer with a lot of experience in web scraping. Although I think this project probably need manual scraping but I can use automated scraping tool if possible and needed. My price and time is Lisää

$2000 AUD 20 päivässä
(20 arvostelua)
5.0
mingxiao2008

Hello Sir! I have checked your requirements carefully and think I can do your jobs. I have good experience in web scraping areas. Please check out my skills and past projects. If you discuss details with me to assi Lisää

$1500 AUD 30 päivässä
(5 arvostelua)
4.8
plabon979207

Hello There I have read the full job description carefully and understand. It will be great that you will assist in the work. so there will be no mistake. I am interested to work with you in this project. Thanks Lisää

$1500 AUD 30 päivässä
(84 arvostelua)
5.2
prakash2813

Hi there, I have strong expertise in web crawling, web scraping, website monitoring bots and web automation. I have made lots of web scraping and bot tools to automate and scrape data in large scale and export them i Lisää

$2500 AUD 10 päivässä
(5 arvostelua)
4.7
istiyaqueahmed

Hi, I can handle this task. --------------------------------------------------------- Budget 1500 AUD in 30 days Service will be provided until you get satisfied.

$1500 AUD 30 päivässä
(23 arvostelua)
4.5
Reflexlogic

Hi, Can we build database on any other cloud platform instead of Azure? I have reviewed requirements in detail and would like to mention that we can scrap and build a database of all degrees offered by all universit Lisää

$3000 AUD 30 päivässä
(2 arvostelua)
4.4