Seeking bids for creating a web scraper to collect well permitting data from the Texas Railroad Commission. The program needs to be written in C# so it can be integrated into the rest of our data system.
The URL that needs to be queried is;
[login to view URL]
The workflow is as follows. In the upper left portion of the form is a drop-down list labelled “District:”. The URL needs to be queried multiple times, once for each of the thirteen districts. The list of districts is stable and will not change. At the bottom of the form, are fields to put in From and To Date values for both Approved and Submitted permits. A date range will be passed into the program. The same date range should be entered into both the Approved and Submitted fields.
When the District is selected and the date fields filled with the input dates, the “Submit” button on the bottom left of the form is selected. Another page is then displayed with the query results in tabular form. Depending on how many search results are returned, the results may be spread across several pages. In the bottom left corner of the form is the “Download” button. This downloads a .csv file with the returned data for all results, regardless of how many pages the data is spread over.
The program needs to download the .csv file and place it into memory in a linked list of lines, or some other memory collection mechanism, so it can be walked through and parsed, line by line. If no results fit the search criteria, the query form is redisplayed with the text “(Ewa_117) No results found.” displayed at the top of the page.
The work required for this bid stops at putting the .csv file into memory. I will do the rest of the processing of parsing the data and placing it into our database.
38 freelanceria on tarjonnut keskimäärin %project_bid_stats_avg_sub_26% %project_currencyDetails_sign_sub_27% tähän työhön
I can provide you C# code that will get information you need using HttpWebRequest. It will process each district, download CSV and parse it into array of dictionaries so you can process it later.
Hi. I am a very experienced web developer and have completed many scraping projects since becoming a freelancer. I have my own tools I use for web scraping and deliver top quality results for my client.
Hi there, I have checked the details I have rich experienced with .NET, C# Programming, Software Architecture, Web Scraping. Please initiate chat so we can discuss this job.