Page 1 of 1

Finding duplicate lines

Posted: Wed Sep 19, 2007 11:22 am
by acessn
Hi.

I need to search a file for duplicate entries.
I don't need to replace them, just find them, as they need to be dealt with manually.

The duplicate lines will probably be next to each other as in the example below.

Example file:
-----------------------------------------------------------------------
# Hyperfinds
All_Home_And_Scheduled_In=Alle ansatte og planlagte overføringer
All_Home_And_Scheduled_In=Alle ansatte og planlagte overføringer
All_Home_And_Scheduled_In=Alle ansatte og planlagte overføringer
All_Home_And_Scheduled_In=Alle ansatte og planlagte overføringer
All_Home_And_Transferred_In=Alle ansatte og overførte
All_Home_And_Transferred_In=Alle ansatte og overførte
All_Home_And_Transferred_In=Alle ansatte og overførte
All_Home_And_Transferred_In=Alle ansatte og overførte
All_Home_Locations=Alle steder
All_Home_Locations=Alle steder
V4_4_All_Home_And_Transferred_In=Alle hjemme og overførte (org.kart)
V4_4_All_Home_And_Transferred_In=Alle hjemme og overførte (org.kart)

# Organization Maps
JOB=Jobb
ORGANIZATION=Organisasjon
ORGANIZATION=Organisasjon
ROOT=Rot

-------------------------------------------------------------------

Any ideas on how to achieve this?

Thanks,
Bjorn

Posted: Wed Sep 19, 2007 12:23 pm
by ben_josephs
We'd like to be able use ^(.*)\n\1$ to search for adjacent duplicate lines. Unfortunately, TextPad's regular expression recogniser doesn't allow back-references (such as \1) to refer back over a newline, so we can't.

More powerful regex recognisers do permit this. For example, in WildEdit (http://www.textpad.com/products/wildedit/):
Find what: ^(.*)\r?\n\1$

[X] Regular expression

Options
[X] '.' does not match a newline character
But WildEdit, although it's designed "to make the same changes to a set of text files in a folder hierarchy", isn't an editor in the normal sense.

Posted: Thu Sep 20, 2007 6:27 am
by acessn
Thanks, I used that expression in another editor and it works fine.

Rgds,

Bjorn