Selecting from ... to ... across many lines

wiskers67 · Post by **wiskers67** » Mon Apr 21, 2008 3:49 am

Hi,
I want to select and then copy into a new document a specific section of a log file (highlighted in BOLD font below). This section is always prefaced with the words SLOW or ERROR SLOW and ends with the words End Request after several new lines. The number of new lines is not constant however the Start/End search string is constant.

The component RX I have tried and which partially succeeds is :-
SL.* = selects from occurrences of SL to end of that line
Request$ = selects "End Request" at the end of a line.

My problem is I cant seem to combine these 2 successfully to select the opening and closing strings and all in between. I can select the start or the end but not both and all in between. I cannot program Perl which I have been told could easily do this, so I am hopeing the TextPad user Community may assist?

After selecting just the ERROR SLOW & SLOW reports from my log file I will then be selecting a few smaller pieces of code to finish with a final result as below which shows the slowest data layer from a web mapping application (in example layer = TR_ROAD and web mapping app = bnr_external_v1). The later can easily be done using simple copy/paste, I am really stuck on selecting just the ERROR SLOW & SLOW errors from my original html log files.

SLOW GET_IMAGE: bnr_external_v1
FEATURE LAYER: Minor Roads (id= TR_ROAD )
DATA RETRIEVAL TIME: 25.609000s
Total Request Time: 33.985000s

I have removed some security code below to ensure web apps are not compromised.

3: <a href="javascript:fnToggleObjDisplay('o10.4964');"> SLOW </a> --> 17/04/2008 8:45:46 AM --> GET_IMAGE: bnr_external_v1
<span class="showMe" id="o10.4964">
[17/04/2008 8:45:46 AM][4964 6252 INFO1] Begin Request
[17/04/2008 8:45:46 AM][4964 6252 INFO3] REQUEST: <?xml version="1.1" encoding="UTF-8"?>
<ARCXML version="1.1">
<REQUEST>
<GET_IMAGE>
<PROPERTIES>
<ENVELOPE minx="2500147.57476554" miny="2368031.16924575" maxx="2530983.72734026" maxy="2394313.16625419" />
<FEATURECOORDSYS id="102171" />
<FILTERCOORDSYS id="102171" />
<BACKGROUND color="255,255,255" transcolor="255,255,255" />
<IMAGESIZE width="887" height="756" dpi="72"/>
<LEGEND antialiasing="true" autoextend="true" cansplit="false" cellspacing="3" columns="1" display="false" font="Arial" height="300" layerfontcolor="0,0,0" layerfontsize="10" layerfontstyle="bold" reverseorder="false" splittext="(cont)" swatchheight="12" swatchwidth="18" title=" " titlefontcolor="0,0,0" titlefontsize="12" titlefontstyle="bold" valuefontcolor="0,0,0" valuefontsize="9" valuefontstyle="bold" width="250">
<LAYERS>
<LAYER id="AL:38284"/>
</LAYERS>
</LEGEND>
<LAYERLIST order="true" nodefault="true" dynamicfirst="false" >
</LAYERLIST>
<DRAW map="true" />
</PROPERTIES>
<LAYER type="acetate" name="Standard Markup" id="AL:38284">
<OBJECT units="pixel">
<TEXTMARKERSYMBOL fontcolor="0,0,0" glowing="255,255,255" antialiasing="true" font="Arial" fontsize="10" overlap="false" halignment="right" valignment="top" />
</TEXT>
</OBJECT>
<OBJECT units="pixel">
<TEXT coords="884 3" label="Map center: 2515566, 2381172">
<TEXTMARKERSYMBOL fontcolor="0,0,0" glowing="255,255,255" antialiasing="true" font="Arial" fontsize="10" overlap="false" halignment="left" valignment="top" />
</TEXT>
</OBJECT>
</LAYER>
</GET_IMAGE>
</REQUEST>
</ARCXML>
[17/04/2008 8:45:46 AM][4964 6252 INFO1] GET_IMAGE: bnr_external_v1
[17/04/2008 8:46:20 AM][4964 6252 STAT2] DATA RETRIEVAL TIME: 0.219000s
[17/04/2008 8:46:20 AM][4964 6252 STAT2] LABEL ENGINE TIME: 0.031000s
[17/04/2008 8:46:20 AM][4964 6252 STAT2] TOTAL PROCESSING TIME: 33.719000s
[17/04/2008 8:46:20 AM][4964 6252 STAT2] OUTPUT TIME: 0.250000s
[17/04/2008 8:46:20 AM][4964 6252 INFO3] RESPONSE: <?xml version="1.0" encoding="UTF-8"?>
<ARCXML version="1.1">
<RESPONSE>
<IMAGE>
<ENVELOPE minx="2500147.57476554" miny="2368031.16924575" maxx="2530983.72734026" maxy="2394313.16625419" />
<OUTPUT file="d:\arcims\output\bnr_external_v1_p0004549646252369.png" url="http://nremap.nre.vic.gov.au/output/bnr ... 252369.png" />
</IMAGE>
</RESPONSE>
</ARCXML>
[17/04/2008 8:46:20 AM][4964 6252 STAT1] Total Request Time: 33.985000s
[17/04/2008 8:46:20 AM][4964 6252 INFO1] End Request
</span>

ben_josephs · Post by **ben_josephs** » Mon Apr 21, 2008 11:44 am

Unfortunately, TextPad's aged regular expression engine is incapable of matching text containing an arbitrary number of newlines.

If your input is not too long, you could change the newlines to something else, do your stuff, then change them back.

Or you could record a macro that extends the selection from the beginning of the wanted text to its end, copies it, and pastes it elsewhere.

Alternatively, you could try WildEdit (http://www.textpad.com/products/wildedit/), which uses a far more powerful regular expression engine.

HerNameWasTextPad · Post by **HerNameWasTextPad** » Mon Apr 21, 2008 8:45 pm

TextPad can do it, in a macro, using regular expressions, with or without the help of bookmarks.

First do it manually, so that you can get an idea of what the macro will need to do. The use of bookmarks is recommended for manual execution, because they offer some visual assistance and comfort.

Since I'm familiar with TextPad's default regular-expression operators, instead of POSIX, I'll use that syntax.

Configure -> Preferences... -> Editor
Uncheck "Use POSIX regular expression syntax"

(1) Make sure that the file shows no bookmarks

(2) Using the Find dialog box, with the "Match case," "Regular expression," and "Wrap searches" boxes checked, "Mark All" $SLOW\|ERROR SLOW$.*$

(3) Using the Find dialog box, with the "Match case," "Regular expression," and "Wrap searches" boxes checked, "Mark All" End Request$

(4) Use F2 and Shift+F2 to check whether you've successfully bookmarked your lines

(5) Go to the SLOW line and position the cursor at the S in SLOW (or the E in ERROR)

(6) Search -> GoTo... -> Next Bookmark, Extend selection, and [Go To] button (or Ctrl+G, Release, Shift+Tab, Release, N, Tab, E, Enter)

(7) Shift+DownArrow (to include the last bookmarked line)

(8) Cut the large section of text to the clipboard

(9) Smile when you're done, and say, "I love this program."

Here's where it gets tricky, if you want to automate the process by a macro:

The macro will need to access a file in which your regular expressions are held as follows:

$SLOW\|ERROR SLOW$.*$

End Request$

Before the macro run, you would open an instance of TextPad containing two files: the one you want to process and the one that holds the regular expressions, held in file positions similar to those above, so that you can use simple cursor-moving keyboard shortcuts to access each regular expression when needed, and then Ctrl+F6 to the other file in the instance, in order to paste the regular expression into the appropriate dialog box. In this case, you need the Find dialog box, instead of the Replace dialog box.

When you eventually need to position the cursor at the S in SLOW, or the E in ERROR, in order to select the text to cut, you will need to "Find Next" (instead of Mark All) $SLOW\|ERROR SLOW$.*$ and then close the Find dialog box and press the LeftArrow key to position the cursor at the S (or the E). Further, you could extend the selection from within the Find dialog box itself, without using bookmarks.

On a side note, when the day comes that you need to use the Replace dialog box in a macro, your regular-expression-algorithm file (as I like to call it) should look like this:

"Find what" expression
"Replace with" expression

"Find what" expression
"Replace with" expression

etc.

This will allow you to position the cursor at the start of each regular expression, so that you can Shift+End, Ctrl+C, and Ctrl+F6 your way to the appropriate dialog box of the file you need to process.

Hope all of this turns your wheels of thought.

muggins · Post by **muggins** » Sat May 24, 2008 11:47 pm

The bookmarks are a good idea. I wish I would have seen your message a couple of weeks ago. I've been pulling hair out and scratching fingernails off trying to figure out a way to select across multiple lines.

I've been working on extracting geneology files from HTML websites and putting it into GEDCOM format. My father created the family geneology over 35 years in retirement and posted it to the web using a popular Geneology program FTW. I don't know where his orifginal gedcoms are at.

For some reason there are multiple variations on line breaks within the HTML. It was just a nightmare trying to put them together into some useable format.

The following should have been an easy regex:

Code: Select all

 <B><A NAME=I423>Michaelis, Henry Robert</A>- male

Code: Select all

  This is the format I need
                        1 @i423@ INDI
                        1 NAME Henry Robert /Michaelis/
                        1 SEX  M

The regex:

Code: Select all

 <B><A NAME=\(I[^>]*\)>\([^,] \([^<]*\)</A>- \(.\)*.*\n

REPLACE WITH:

Code: Select all

1 @\1@ INDI\n1 NAME \3 /\2/\n1 SEX \4\n

By removing the linebreaks I can use that same simple piece of code for over 10 thousand names. It took me longer to write this message then the time I spend macro-ing through the files now. Hope this can help somebody...