Automation - Suggestions Please

[Kelly]

I have several metadata exports from which I wish to extract only a particular set of data. I have doodled out some regex (man! I'm loving regex, my new best friend which are manually run (find/replace), one at a time, and sequenced.

What is the best practice in automating this task which has numerous directories and data files within them?



The regexes in particular:

Code: Select all

Sunday, October 20, 2013, 2:55 PM, VKB - Regex stuff

find expression 01
FILENAME=.*\\.*\\.*\\.*\.*\nDESCRIPTION=

replace expression 01
TILE NAME: 


find expression 02
.las

replace expression 02
(nothing)


find expression 03
AREA COUNT=.*\nLINE COUNT=.*\nPOINT COUNT=.*\n

replace expression 03
(nothing)


find expression 04
POINT CLOUD MEMORY=.*\..*\n*

replace expression 04
(nothing)


find expression 05
LIDAR OFFSET=.*\nLIDAR SCALE=.*\..*\..*\..*\nUPPER LEFT X=.*\..*\nUPPER LEFT Y=.*\..*\nLOWER RIGHT X=.*\..*\nLOWER RIGHT Y=.*\..*\nWEST LONGITUDE=.*\..*\nNORTH LATITUDE=.*\..*\nEAST LONGITUDE=.*\..*\nSOUTH LATITUDE=.*\..*\nPROJ_DESC=.*\nPROJ_DATUM=.*\nPROJ_UNITS=.*\nEPSG_CODE=.*\nCOVERED AREA=.*\..*\nLOAD_TIME=.*\..*\n

replace expression 05
(nothing)


find expression 06
LAS_VERSION=.*\..*\nGLOBAL_ENCODING=.*\nSYSTEM_ID=.*\nGEN_SOFTWARE=.*\nFLIGHT_TIME=.*\n

replace expression 06
(nothing)


find expression 07
LIDAR POINT COUNT=

replace expression 07
(nothing)


find expression 08
LIDAR POINT DENSITY=

replace expression 08
(nothing)


find expression 09
 samples / m\^2\nMIN ELEVATION=

replace expression 09
\n


find expression 10
MAX ELEVATION=

replace expression 10
(nothing)


find expression 11
 METERS

replace expression 11
(nothing)


find expression 12
,

replace expression 12
(nothing)


find expression 13
LAS_VERSION=.*\..*\nGLOBAL_ENCODING=.*\nGEN_SOFTWARE=.*\nFLIGHT_TIME=.*\n

replace expression 13
(nothing)

There may be other regexes that will be necessary, but this generally presents the idea of what I'm trying to accomplish next in line of my baby steps

Sample data:

Code: Select all

FILENAME=c:\mcctests\data\19_05604899.las
DESCRIPTION=19_05604899.las
AREA COUNT=0
LINE COUNT=0
POINT COUNT=0
LIDAR POINT COUNT=1,132,092
POINT CLOUD MEMORY=6.2 MB
LIDAR POINT DENSITY=0.66 samples / m^2
LIDAR OFFSET=( 0, 0, -0 )
LIDAR SCALE=( 0.01, 0.01, 0.01 )
UPPER LEFT X=559500.010
UPPER LEFT Y=4900499.990
LOWER RIGHT X=561000.000
LOWER RIGHT Y=4899000.000
WEST LONGITUDE=68° 15' 17.3974" W
NORTH LATITUDE=44° 15' 19.1066" N
EAST LONGITUDE=68° 14' 9.1458" W
SOUTH LATITUDE=44° 14' 30.0476" N
PROJ_DESC=UTM Zone 19 / NAD83 / meters
PROJ_DATUM=NAD83
PROJ_UNITS=meters
EPSG_CODE=26919
COVERED AREA=555.98 acres
LOAD_TIME=0.34 s
MIN ELEVATION=-1.43 METERS
MAX ELEVATION=33.08 METERS
LAS_VERSION=1.2
GLOBAL_ENCODING=1
SYSTEM_ID=NIIRS10
GEN_SOFTWARE=GeoCue GeoCoder
FLIGHT_TIME=03/03/11

[ben_josephs]

If the order of the lines is fixed and if LIDAR POINT COUNT < 1,000,000,000:

Find what: FILENAME=.*\nDESCRIPTION=(.*)\.las\n(?:.*\n)*?LIDAR POINT COUNT=(\d+),?(\d*),?(\d*)\n(?:.*\n)*?LIDAR POINT DENSITY=([\d.]+).*\n(?:.*\n)*?MIN ELEVATION=([-\d.]+).*\nMAX ELEVATION=([-\d.]+).*\n(?:.*\n)*?FLIGHT_TIME=.*\n

Replace with: TILE NAME:$1\n$2$3$4\n$5$6\n$7\n

or something similar.

[Kelly]

That's amazing - thank you Ben, very much!

Friedl's book will get here tomorrow

Really awesome - thank you so much.

Kelly