I am seeking to have someone help me create a web scraping application for apartment market research.
I want to scrape all of the information about an apartment building from apartments.com. Here is an example page:
[url removed, login to view];srank=1&mfcisapicommand=quicksearch&qsearchtype=1&propname=copley&state=na&page=summary&prvpg=7&qsearchtype=1&srt1=0.55&srt2=0.55&srt3=0.55
I want to suck down all of the info about a building into two tables: one with the table of rents for the building, and one with all of the rest of the info about the building. Also I want to save all of the pictures for each building to a folder. (I can send a table with the fields I want, along with a markup of the datapage showing exactly how I want info captured).
I want the application to submit a list of zip codes to search at [url removed, login to view]; follow through to the links to the relevant results, scrape the info on each building, and go to the next zip code in the list.
There are several off the shelf web scraping programs which automate this whole process, including submitting the searches and crawling the results. One of the most popular is called Web Scraper Plus and is made by Velocity Scape. I am partial to it; there are several others like it out there. I want an off the shelf program to be used so it can be tweaked easily in the future. I can buy a license to the program for the developer if necessary.
After the application for [url removed, login to view] is set up, I want to do the same thing for 2 similar sites, but for now lets just talk about [url removed, login to view] to keep it simple.
See the attached file for a sketch of how I want the data parsed.
In your bid, please tell me if you would use a program or technology other than Web Scraper Plus.