Finding duplicate lines

General questions about using TextPad

Moderators: AmigoJack, bbadmin, helios, Bob Hansen, MudGuard

Post Reply
acessn
Posts: 10
Joined: Thu Jul 06, 2006 12:20 pm

Finding duplicate lines

Post by acessn »

Hi.

I need to search a file for duplicate entries.
I don't need to replace them, just find them, as they need to be dealt with manually.

The duplicate lines will probably be next to each other as in the example below.

Example file:
-----------------------------------------------------------------------
# Hyperfinds
All_Home_And_Scheduled_In=Alle ansatte og planlagte overføringer
All_Home_And_Scheduled_In=Alle ansatte og planlagte overføringer
All_Home_And_Scheduled_In=Alle ansatte og planlagte overføringer
All_Home_And_Scheduled_In=Alle ansatte og planlagte overføringer
All_Home_And_Transferred_In=Alle ansatte og overførte
All_Home_And_Transferred_In=Alle ansatte og overførte
All_Home_And_Transferred_In=Alle ansatte og overførte
All_Home_And_Transferred_In=Alle ansatte og overførte
All_Home_Locations=Alle steder
All_Home_Locations=Alle steder
V4_4_All_Home_And_Transferred_In=Alle hjemme og overførte (org.kart)
V4_4_All_Home_And_Transferred_In=Alle hjemme og overførte (org.kart)

# Organization Maps
JOB=Jobb
ORGANIZATION=Organisasjon
ORGANIZATION=Organisasjon
ROOT=Rot

-------------------------------------------------------------------

Any ideas on how to achieve this?

Thanks,
Bjorn
ben_josephs
Posts: 2461
Joined: Sun Mar 02, 2003 9:22 pm

Post by ben_josephs »

We'd like to be able use ^(.*)\n\1$ to search for adjacent duplicate lines. Unfortunately, TextPad's regular expression recogniser doesn't allow back-references (such as \1) to refer back over a newline, so we can't.

More powerful regex recognisers do permit this. For example, in WildEdit (http://www.textpad.com/products/wildedit/):
Find what: ^(.*)\r?\n\1$

[X] Regular expression

Options
[X] '.' does not match a newline character
But WildEdit, although it's designed "to make the same changes to a set of text files in a folder hierarchy", isn't an editor in the normal sense.
acessn
Posts: 10
Joined: Thu Jul 06, 2006 12:20 pm

Post by acessn »

Thanks, I used that expression in another editor and it works fine.

Rgds,

Bjorn
Post Reply