The short project description is:
The Crawler acceps from user administration pannel the following data:
a) website URL in input field for URL
b) The Crawler waits for all webpage to load and all scripts to finnish loading and executing on webpage.
c) The Crawler has input fields where user can add HTML tags, id's and classes for identifing relevant data in order from parent element to child element or from child element to parent element .
d) The Crawler stores relevant data ( images, static and/or dynamic links, of images and/or other resources and/or element attributes and/or css styles) and stores them in Crawler database.
e) The Crawler has input fields to define links to other webpage (added by url with parent-to-child or child-to-parent elements relations similar to how it grabs products) and insert collected data as defined by user. (for example: newswebsite is the Crawled website it has title , text text text and a class>id>id>ul>li>a >img with relevant data user is able to click + icon on input field and add the number of elements necessary to define the target relevant data. The scaper creates tables or collections with the relevant data and the user defined containing elements and url where the information was found. On next step the user can define another webpage with input field data by class or id or html tag or database using database user and password provided by the user where the crawler inserts collected data either in the webpage with input fields or in the database).
20 freelanceria on tarjonnut keskimäärin %project_bid_stats_avg_sub_26% %project_currencyDetails_sign_sub_27% tähän työhön
HI I am interested in your project. I am experienced enough in scrapy projwct such as web crawlwer, spider. I already worked bet365, betfait, amazon, ...scrapy projects. Please share your work with me. Thank you
Dear I am an expert in web scraping. In before, I developed many spiders using scrapy. For example, I did scrape sites of products for CEOs(ebay, welivv and so on). Thanks.