Regex: bare CR; bare LF

bob_obob · Post by **bob_obob** » Mon Sep 22, 2003 9:34 pm

I have some damaged files to repair.
(Eudora v3 mbx files)

I have tracked down the problems to the occasional failure of Eudora to insert a proper CR/LF sequence before writing a new message to the "folder" file. Since Eudora uses its own "toc" file it doesn't have to care about the "From ???@???" lines always exactly following a CR/LF pair, but other flaws in Eudora occasionally destroy the toc files, leaving users with corrupt mailboxes, most often having multiple messages munged together.

Anyway, it's easy to deal with when the CR/LF is completely missing, I use a regex like this \([^^]\)\(From \?\?\?@\?\?\?\)
but if there's a bare CR, or a bare LF (I'm not sure; might be either) it doesn't fit that regex as so sometimes my corrections don't get all the corrupted places.

I wish I could just use a 'strip' type program to change them all into CR/LF pairs, but I'm afraid that causes other problems on (extremely rare) occasions when messages are actually supposed to contain those individual characters.

Is there something I can use in regex to search for CR alone, and is there something I can use to search for LF alone?

thanks

s_reynisson · Post by **s_reynisson** » Mon Sep 22, 2003 9:47 pm

What about going for the hex values?
From TP's help file:

Regular Expressions Examples
<snip>
Hex Characters
Any character can be represented by its hex value. This is specified with
the pattern \xdd, where dd is any 2-digit hexadecimal number, excluding
zero.

bob_obob · Post by **bob_obob** » Mon Sep 22, 2003 10:18 pm

nice try, but as far as I can tell that notation doesn't work for CR
here's the hex representation of one section of an unmodified file where I found the problem:

62 20 0d 46 72 6f 6d 20 3f 3f 3f 40 3f 3f 3f 20

that's
b
From ???@???

where the 'b' is followed by a space and a bare CR.

I would expect the search regex \x13F or \(\x13\)F
to find it, but it finds nothing at all in the entire file (which contains at least three other such flaws)

talleyrand · Post by **talleyrand** » Mon Sep 22, 2003 10:18 pm

And FWIW,
CR = \x0D
LF = \x0A

talleyrand · Post by **talleyrand** » Mon Sep 22, 2003 10:21 pm

Minor point of order, the \x means it's looking for hex values so your search for \13F is really searching for 319 which doesn't exist in ASCII. Don't mix and match your search types.

[edit]
And because I feel like being a bigger geek.

62 20 0d 46 72 6f 6d 20 3f 3f 3f 40 3f 3f 3f 20
Should be
b CR space F r o m space ? ? ? @ ? ? ? space
[/edit]

bob_obob · Post by **bob_obob** » Mon Sep 22, 2003 10:24 pm

yes, you're right, while I was typing here in the forum I made a silly transcription error.

\x0dF
\x0d\x46

either way it finds no matches in the entire file, yet in hex mode I can clearly see

62 20 0d 46 72 6f 6d 20 3f 3f 3f 40 3f 3f 3f 20

heck, just searching for \x0d finds nothing, in this multi-megabyte file of over a million lines...

talleyrand · Post by **talleyrand** » Mon Sep 22, 2003 10:30 pm

This is fun! Is the file opened in binary mode? I just edited up your file in a hex editor (xvi rocks!) and opened it in textpad and nothing was found when I searched on 0D. However, if I opened it in binary mode, then the search found it. Maybe that's what you need to do?

bob_obob · Post by **bob_obob** » Mon Sep 22, 2003 10:47 pm

I need to insert characters. I've never been aware of any way to do such a thing in Textpad's binary mode. In any event, it's not something I'd consider a solution.

I do appreciate the effort though.

I'll probably have to hork up a utility in VB. Would have been nice if TextPad could do it, but fixing file damage isn't really a text-editor task.

s_reynisson · Post by **s_reynisson** » Mon Sep 22, 2003 10:56 pm

This might be a bug, having to revert to binary mode to use a
basic regular expression, ie. \xdd. Perhaps the Textpad team
will explain this to us.

bob_obob · Post by **bob_obob** » Mon Sep 22, 2003 11:00 pm

my first guess would be that the \xnn notation works if what you're looking for is on a line, but CR and LF define the boundaries.

when the boundaries between different lines in the same file are not always the same boundary characters, a pretty good case can be made for the assertion that it doesn't really qualify as a text file

talleyrand · Post by **talleyrand** » Mon Sep 22, 2003 11:03 pm

Oh yeah, I forgot about the binary readonly piece. *whistles innocently* Well then, I certainly wish you luck. I was going to suggest a handy dandy hex editor like XVI but it won't handle the regexp.

Post by **MudGuard** » Tue Sep 23, 2003 6:25 am

Just another wild idea:

When saving the file, select Unix mode.
Close the file.
Reopen the file.
Save the file, selecting PC mode.

(if necessary, repeat with Mac instead of Unix mode)

I know that this is not what you really want, but it might work (I didn't test)

bob_obob · Post by **bob_obob** » Sat Oct 04, 2003 10:31 pm

I appreciate the additional suggestion, but (as stated in the initial message) there is a need to keep messages intact, even if a bare CR or LF occurs somewhere other than at the end of a message.

Community

Regex: bare CR; bare LF

Regex: bare CR; bare LF

doesn't seem to work, thanks

of course they are, but the error is only in my forum post

edit in hex mode?

Re: edit in hex mode?

pc -> unix -> mac save-mode, etc.