Simple Program To Remove Duplicate Domains From Text File

I need someone to write a simple program which can do the following -

Scan a .txt file which contains 1 URL per line. Check to see whether the domain name of any line is duplicate, and if it is, remove all instances of the URL.

For example - the program would check each line for [url removed, login to view] (tld being the domain extension such as .com, .org, .net, .info etc) - if two lines containing [url removed, login to view] exist, then the ENTIRE LINE will be deleted (not just the [url removed, login to view]).

Each URL will consist of more than just the domain and tld, for example URL's may consist of [url removed, login to view] however it's important that if [url removed, login to view] is found to exist on 2 or more lines, then the ENTIRE line is removed, not just the [url removed, login to view] portion.

Quite simply I want a program that will check a text file with a list of URL's, find any duplicate domains, and then remove the ENTIRE URL of any lines containing that domain and tld.

It should be something simple like: "if [url removed, login to view] exist in 2 or more lines then remove all lines containing word1.word2".

Further Examples:

Let's say the .txt file looks like the following -

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]

This program should remove line 1 & line 2, since the domain and tld are the same. The text file should be left looking like -

[url removed, login to view]

[url removed, login to view]

Notice how [url removed, login to view] and [url removed, login to view] were NOT removed since there was only ONE instance of each domain and tld. If a domain and TLD only exist ONCE, then they should NOT be removed.

If you need further instructions please message me.


Taidot: C-ohjelmointi, PHP, Visual Basic

Näytä lisää: php remove text url, php remove line duplicate, net remove duplicate domains, program find, in line program, how to write on line, how to check a php program, find someone to write, find a program, php remove duplicate domains, remove duplicate domains, duplicate domain remove text file, url remove duplicate domain, remove duplicate domains php, scan duplicate url, vbscript list duplicate lines text file, remove duplicate text line, php program check dupliacte lines text file, file remove line, scan text files, php duplicate url delete, check duplicate lines file vbscript, find duplicates txt file, find duplicate text file, duplicate text remover

Tietoa työnantajasta:
( 2 arvostelua ) Sydney, Australia

Projektin tunnus: #443014

Myönnetty käyttäjälle:


Hi.. Am expert in developing Visual basic Application... Please check PM...

30 $ USD 1 päivässä
(12 arvostelua)

15 freelanceria on tarjonnut keskimäärin 33 $ tähän työhön


I will create siimple PHP script to parse text file. And output result according to your rules. Can be done today.

30 $ USD 1 päivässä
(1258 arvostelua)

Hello,Please refer your [url removed, login to view] you.

35 $ USD 1 päivässä
(86 arvostelua)

Please check PMB.. jexplorer

30 $ USD 0 päivässä
(153 arvostelua)

Please read my message.

30 $ USD 1 päivässä
(68 arvostelua)

Hello sir, I can do this for you and give you the script in few hours. Please select me and I will make it. Thanks

44 $ USD 0 päivässä
(39 arvostelua)

Hello. I will do it in one hour. Sergey S.

30 $ USD 1 päivässä
(11 arvostelua)

Dear Sir, Please check your PMB. Thank you, Best Regards, GWT

30 $ USD 0 päivässä
(2 arvostelua)

Hi, I can get this done for you today in VB.

30 $ USD 0 päivässä
(2 arvostelua)

Hello, I have lot of experience in programming. I can create the application exactly in accordance to your description using Java. You will be 100% satisfied with the work. Thanks.

50 $ USD 3 päivässä
(3 arvostelua)

I've got realy good skills in C/C++, also have deep intret in C/C++. Done alot of projects in C.

30 $ USD 1 päivässä
(0 arvostelua)

I am expert in regular expression. Please allow me help you.

30 $ USD 3 päivässä
(0 arvostelua)

I can do it in C++, VB or perl whatever language you prefer in one day. Ready to start as soon as you make your choice.

40 $ USD 1 päivässä
(0 arvostelua)

I'm a PHP professional, but it's not needed to be a professional to do this project. If I would be awarded, I'll give you a PHP script in 30 minutes or less afrer I realize that I've awarded for this job. Actually, it Lisää

30 $ USD 0 päivässä
(0 arvostelua)

very easy job. It can be done within a couple of hours or less. Regards.

30 $ USD 0 päivässä
(2 arvostelua)