Remove duplicates in files
Posted: Wed Jul 02, 2008 8:18 pm
I am wondering if wildedt will do this. If so, I am buying it for sure.
Let's say I have 100 files. I don't want to lose the integrity of any of the files.
Lets say I have the following files
File1.txt
File2.txt
File3.txt
File4.txt
Here are the same 4 files and their contents:
file1.txt
anything
something
whatever
file2.txt
friendly
almost always
Tuesdays
something
file3.txt
every Friday
next Wednesday
anything
file4.txt
blues
not yet
already
anything
What I want is to only leave 1 occurrence of each line in 1 of the files. It does not matter which file it is left in. Files 1,3 and 4 have the line anything. The deduping action I am looking for is to leave the word anything in only 1 of those files.
The results files would look like this:
file1.txt
anything
something
whatever
file2.txt
friendly
almost always
Tuesdays
file3.txt
every Friday
next Wednesday
file4.txt
blues
not yet
already
I need the deduping per line not per instance. For instance it would
remove more than one line that said "the big red dog" but would not
remove lines that only said "the big red"
Thanks and I hope this was clear.
Darin
Let's say I have 100 files. I don't want to lose the integrity of any of the files.
Lets say I have the following files
File1.txt
File2.txt
File3.txt
File4.txt
Here are the same 4 files and their contents:
file1.txt
anything
something
whatever
file2.txt
friendly
almost always
Tuesdays
something
file3.txt
every Friday
next Wednesday
anything
file4.txt
blues
not yet
already
anything
What I want is to only leave 1 occurrence of each line in 1 of the files. It does not matter which file it is left in. Files 1,3 and 4 have the line anything. The deduping action I am looking for is to leave the word anything in only 1 of those files.
The results files would look like this:
file1.txt
anything
something
whatever
file2.txt
friendly
almost always
Tuesdays
file3.txt
every Friday
next Wednesday
file4.txt
blues
not yet
already
I need the deduping per line not per instance. For instance it would
remove more than one line that said "the big red dog" but would not
remove lines that only said "the big red"
Thanks and I hope this was clear.
Darin