Website Crawler

I need a website crawler. I have some base code ( it scans for meta tags one site at a time, and you have to enter each site manually. ) It would need:

1. Scan websites, and log them in the database.

2. Pickup links from sites and index those.

3. Never stop unless it's out of links to index.

4. If the site has no meta tags, take content from the page.

5. copy some content from each page ( like 150 words ) and place that in the database...

6. Will allow me to change a setting about weather or not a site can be scanned twice. ( A Check Box )

I'm guessing roughly 60 lines of code. I'd love to have a pause button, but it's not necessary.

Taidot: Perl, PHP

Näytä lisää: check website crawler, website crawler, scan copy, database crawler, d scans, crawler, button tags, code scan, scan code, website base, 150 words website, enter website database, code weather, perl log, crawler website, crawler code, enter page website, weather code, stop page, code database scan, links copy website, page love, scan page, php scan code database, php database crawler

Tietoa työnantajasta:
( 1 arvostelu ) Fountain, United States

Projektin tunnus: #391845

3 freelanceria on tarjonnut keskimäärin %project_bid_stats_avg_sub_26% %project_currencyDetails_sign_sub_27% tähän työhön


Hello,Please refer your [url removed, login to view] you.

$200 USD 5 päivässä
(49 arvostelua)

Check your pmb ;)

$60 USD 2 päivässä
(22 arvostelua)

check your pm.

$250 USD 7 päivässä
(6 arvostelua)