duplicate entry finder

[badgolferman]

I have a text file with many e-mail addresses in it. I would like to
use something that would scour the file and remove duplicate entries.
What do you recommend?

 

[Al Klein]

I have a text file with many e-mail addresses in it. I would like to
use something that would scour the file and remove duplicate entries.
What do you recommend?

Alphebetize the list, then it'll be easy to spot dupes.

 

[Eric Huebner]

Am 04 Feb 2006 00:28:33 GMT schrieb badgolferman:

I have a text file with many e-mail addresses in it. I would like to
use something that would scour the file and remove duplicate entries.
What do you recommend?

If the entries are in separate lines, this can be done with some text
editors like PSPad. Just sort the lines and check "remove duplicates".
http://www.pspad.com

 

[Franklin]

I have a text file with many e-mail addresses in it. I would
like to use something that would scour the file and remove
duplicate entries. What do you recommend?

ExanDiff should do it easily.

http://www.prestosoft.com/ps.asp?page=edp_examdiff

 

[Dewey Edwards]

I have a text file with many e-mail addresses in it. I would like to
use something that would scour the file and remove duplicate entries.
What do you recommend?

Assuming your addys are one to a line, three years ago I wrote a very
small program for JohnF that will do this. 38kb.

The program will output your addys in the same order they are now in,
sans duplicates. The algorithm and data structure used eliminates the
need to sort the data. Computation time is negligable, total time
will be slightly longer than the time taken to just copy the file.
20meg file should be under 5 seconds.

Niow the warnings. The program currently requires a DOS window, no
GUI. There is currently an upper limit on file size - if you have
more than one hundred thousand addys to consider, you may be pushing
its limits. (Limits are based on text file size vs. memory).

My initial ANN and JohnF's comments are here

http://www.box-one.org/new-1701705-4530.html

Program is no-install with readme/help flle included n the zip and is
here

http://home.nycap.rr.com/dewed/rm_dup_lines_v0.1.zip