Issue with Reg Ex

[Drxenos]

There seems to be a bug in the regular expressions engine. I'm trying to do a "replace all" and half the time the replacement characters are garbage. I'll undo, and apply the regex again and it will work fine.

Find What: \<HANDBRAKE|FFMPEG|MKVMERGE|REMUX|COPY|MOVE|TEST\>
Replace with: <\U$&>

sample file:

Code: Select all

TITLE "0015 - S02E26 - The Space Museum, Part 1.mkv"
HANDBRAKE --min-duration 5 --previews 40:0 --input "D:\DVD\Doctor Who 0015 The Space Museum (1965)\883929689422D1" --title 4 --format av_mkv --output "N:\media-root\Media\TV\D\Doctor Who (1963)\0009-0017 Season 02 (1964)\0015 - S02E26 - The Space Museum, Part 1.mkv" --encoder x264 --encoder-preset slower --encoder-profile high --encoder-level 4.1 --quality "20.0" --vfr --auto-anamorphic --modulus 2 --detelecine --comb-detect --no-deinterlace --decomb --no-deblock --no-hqdn3d --no-nlmeans --no-unsharp --no-lapsharp --no-grayscale --markers --audio-fallback ac3 --audio-copy-mask "aac,ac3,eac3,truehd,dts,dtshd,mp3,flac" --audio "1,1,2" --aencoder "fdk_aac,copy:ac3,copy:ac3" --ab "192,192,192" --mixdown "dpl2,none,none" --normalize-mix "0,0,0" --arate "auto,auto,auto" --aname "","","" --subtitle "1,2" --subtitle-burn=none --subtitle-default=none
TITLE "0015 - S02E27 - The Space Museum, Part 2.mkv"
HANDBRAKE --min-duration 5 --previews 40:0 --input "D:\DVD\Doctor Who 0015 The Space Museum (1965)\883929689422D1" --title 5 --format av_mkv --output "N:\media-root\Media\TV\D\Doctor Who (1963)\0009-0017 Season 02 (1964)\0015 - S02E27 - The Space Museum, Part 2.mkv" --encoder x264 --encoder-preset slower --encoder-profile high --encoder-level 4.1 --quality "20.0" --vfr --auto-anamorphic --modulus 2 --detelecine --comb-detect --no-deinterlace --decomb --no-deblock --no-hqdn3d --no-nlmeans --no-unsharp --no-lapsharp --no-grayscale --markers --audio-fallback ac3 --audio-copy-mask "aac,ac3,eac3,truehd,dts,dtshd,mp3,flac" --audio "1,1,2" --aencoder "fdk_aac,copy:ac3,copy:ac3" --ab "192,192,192" --mixdown "dpl2,none,none" --normalize-mix "0,0,0" --arate "auto,auto,auto" --aname "","","" --subtitle "1,2" --subtitle-burn=none --subtitle-default=none
TITLE "0015 - S02E28 - The Space Museum, Part 3.mkv"
HANDBRAKE --min-duration 5 --previews 40:0 --input "D:\DVD\Doctor Who 0015 The Space Museum (1965)\883929689422D1" --title 6 --format av_mkv --output "N:\media-root\Media\TV\D\Doctor Who (1963)\0009-0017 Season 02 (1964)\0015 - S02E28 - The Space Museum, Part 3.mkv" --encoder x264 --encoder-preset slower --encoder-profile high --encoder-level 4.1 --quality "20.0" --vfr --auto-anamorphic --modulus 2 --detelecine --comb-detect --no-deinterlace --decomb --no-deblock --no-hqdn3d --no-nlmeans --no-unsharp --no-lapsharp --no-grayscale --markers --audio-fallback ac3 --audio-copy-mask "aac,ac3,eac3,truehd,dts,dtshd,mp3,flac" --audio "1,1,2" --aencoder "fdk_aac,copy:ac3,copy:ac3" --ab "192,192,192" --mixdown "dpl2,none,none" --normalize-mix "0,0,0" --arate "auto,auto,auto" --aname "","","" --subtitle "1,2" --subtitle-burn=none --subtitle-default=none
TITLE "0015 - S02E29 - The Space Museum, Part 4.mkv"
HANDBRAKE --min-duration 5 --previews 40:0 --input "D:\DVD\Doctor Who 0015 The Space Museum (1965)\883929689422D1" --title 7 --format av_mkv --output "N:\media-root\Media\TV\D\Doctor Who (1963)\0009-0017 Season 02 (1964)\0015 - S02E29 - The Space Museum, Part 4.mkv" --encoder x264 --encoder-preset slower --encoder-profile high --encoder-level 4.1 --quality "20.0" --vfr --auto-anamorphic --modulus 2 --detelecine --comb-detect --no-deinterlace --decomb --no-deblock --no-hqdn3d --no-nlmeans --no-unsharp --no-lapsharp --no-grayscale --markers --audio-fallback ac3 --audio-copy-mask "aac,ac3,eac3,truehd,dts,dtshd,mp3,flac" --audio "1,1,2" --aencoder "fdk_aac,copy:ac3,copy:ac3" --ab "192,192,192" --mixdown "dpl2,none,none" --normalize-mix "0,0,0" --arate "auto,auto,auto" --aname "","","" --subtitle "1,2" --subtitle-burn=none --subtitle-default=none
TITLE "0015-D1-01 - Defending the Museum.mkv"
HANDBRAKE --min-duration 5 --previews 40:0 --input "D:\DVD\Doctor Who 0015 The Space Museum (1965)\883929689422D1" --title 11 --format av_mkv --output "N:\media-root\Media\TV\D\Doctor Who (1963)\Special Features\0009-0017 Season 02\0015-D1-01 - Defending the Museum.mkv" --encoder x264 --encoder-preset slower --encoder-profile high --encoder-level 4.1 --quality "20.0" --vfr --crop "0:0:0:0" --auto-anamorphic --modulus 2 --detelecine --comb-detect --no-deinterlace --decomb --no-deblock --no-hqdn3d --no-nlmeans --no-unsharp --no-lapsharp --no-grayscale --markers --audio-fallback ac3 --audio-copy-mask "aac,ac3,eac3,truehd,dts,dtshd,mp3,flac" --audio "1,1" --aencoder "fdk_aac,copy:ac3" --ab "192,192" --mixdown "dpl2,none" --normalize-mix "0,0" --arate "auto,auto" --aname "","" --subtitle "1" --subtitle-burn=none --subtitle-default=none
TITLE "0015-D1-02 - My Grandfather, the Doctor.mkv"
HANDBRAKE --min-duration 5 --previews 40:0 --input "D:\DVD\Doctor Who 0015 The Space Museum (1965)\883929689422D1" --title 12 --format av_mkv --output "N:\media-root\Media\TV\D\Doctor Who (1963)\Special Features\0009-0017 Season 02\0015-D1-02 - My Grandfather, the Doctor.mkv" --encoder x264 --encoder-preset slower --encoder-profile high --encoder-level 4.1 --quality "20.0" --vfr --crop "0:0:0:0" --auto-anamorphic --modulus 2 --detelecine --comb-detect --no-deinterlace --decomb --no-deblock --no-hqdn3d --no-nlmeans --no-unsharp --no-lapsharp --no-grayscale --markers --audio-fallback ac3 --audio-copy-mask "aac,ac3,eac3,truehd,dts,dtshd,mp3,flac" --audio "1,1" --aencoder "fdk_aac,copy:ac3" --ab "192,192" --mixdown "dpl2,none" --normalize-mix "0,0" --arate "auto,auto" --aname "","" --subtitle "1" --subtitle-burn=none --subtitle-default=none
TITLE "0015-D1-03 - A Holiday for the Doctor.mkv"
HANDBRAKE --min-duration 5 --previews 40:0 --input "D:\DVD\Doctor Who 0015 The Space Museum (1965)\883929689422D1" --title 8 --format av_mkv --output "N:\media-root\Media\TV\D\Doctor Who (1963)\Special Features\0009-0017 Season 02\0015-D1-03 - A Holiday for the Doctor.mkv" --encoder x264 --encoder-preset slower --encoder-profile high --encoder-level 4.1 --quality "20.0" --vfr --crop "0:0:0:0" --auto-anamorphic --modulus 2 --detelecine --comb-detect --no-deinterlace --decomb --no-deblock --no-hqdn3d --no-nlmeans --no-unsharp --no-lapsharp --no-grayscale --markers --audio-fallback ac3 --audio-copy-mask "aac,ac3,eac3,truehd,dts,dtshd,mp3,flac" --audio "1,1" --aencoder "fdk_aac,copy:ac3" --ab "192,192" --mixdown "dpl2,none" --normalize-mix "0,0" --arate "auto,auto" --aname "","" --subtitle "1" --subtitle-burn=none --subtitle-default=none
TITLE "0015-D1-04 - Photo Gallery.mkv"
HANDBRAKE --min-duration 5 --previews 40:0 --input "D:\DVD\Doctor Who 0015 The Space Museum (1965)\883929689422D1" --title 9 --format av_mkv --output "N:\media-root\Media\TV\D\Doctor Who (1963)\Special Features\0009-0017 Season 02\0015-D1-04 - Photo Gallery.mkv" --encoder x264 --encoder-preset slower --encoder-profile high --encoder-level 4.1 --quality "20.0" --vfr --crop "0:0:0:0" --auto-anamorphic --modulus 2 --detelecine --comb-detect --no-deinterlace --decomb --no-deblock --no-hqdn3d --no-nlmeans --no-unsharp --no-lapsharp --no-grayscale --markers --audio-fallback ac3 --audio-copy-mask "aac,ac3,eac3,truehd,dts,dtshd,mp3,flac" --audio "1,1" --aencoder "fdk_aac,copy:ac3" --ab "192,192" --mixdown "dpl2,none" --normalize-mix "0,0" --arate "auto,auto" --aname "","" --subtitle "1" --subtitle-burn=none --subtitle-default=none
TITLE "0015-D1-05 - Promo.mkv"
HANDBRAKE --min-duration 5 --previews 40:0 --input "D:\DVD\Doctor Who 0015 The Space Museum (1965)\883929689422D1" --title 10 --format av_mkv --output "N:\media-root\Media\TV\D\Doctor Who (1963)\Special Features\0009-0017 Season 02\0015-D1-05 - Promo.mkv" --encoder x264 --encoder-preset slower --encoder-profile high --encoder-level 4.1 --quality "20.0" --vfr --crop "0:0:0:0" --auto-anamorphic --modulus 2 --detelecine --comb-detect --no-deinterlace --decomb --no-deblock --no-hqdn3d --no-nlmeans --no-unsharp --no-lapsharp --no-grayscale --markers --audio-fallback ac3 --audio-copy-mask "aac,ac3,eac3,truehd,dts,dtshd,mp3,flac" --audio "1,1" --aencoder "fdk_aac,copy:ac3" --ab "192,192" --mixdown "dpl2,none" --normalize-mix "0,0" --arate "auto,auto" --aname "","" --subtitle "1" --subtitle-burn=none --subtitle-default=none

[ben_josephs]

I can't reproduce this. What bad output are you seeing from that input?

Presumably you've selected Match case. Otherwise your command will replace

Code: Select all

--audio-copy-mask "aac,ac3,eac3,truehd,dts,dtshd,mp3,flac" --audio "1,1,2" --aencoder "fdk_aac,copy:ac3,copy:ac3"

with

Code: Select all

--audio-<COPY>-mask "aac,ac3,eac3,truehd,dts,dtshd,mp3,flac" --audio "1,1,2" --aencoder "fdk_aac,<COPY>:ac3,<COPY>:ac3"

Should you be anchoring the search to the beginnings of lines?

Glad to know you're watching the First Doctor!

[Drxenos]

Hi Ben. Thanks for responding. The replacement characters appear random (a lot of Asian characters). I tried to copy/paste the result, but TextPad complains about null characters. So, I make screenshots.

Thank you!
Yes, I LOVE Doctor Who. I have them all (except, of course, the missing ones) on DVD/Blu-ray.

A.jpg (7.59 KiB) Viewed 6431 times

B.jpg (64.88 KiB) Viewed 6431 times

[Drxenos]

Also: it only happens the first time for a given file. After that, it works correctly.

[bbadmin]

How big is the file (lines and characters)?

[Drxenos]

How big is the file (lines and characters)?

Hi. It happens with files of various sizes. This particular one is 18 lines and is just under 8K. The longest line is 856 characters.

Thanks.

[Drxenos]

FYI: this happens in both 9.2.0 and 9.16.1.

[bbadmin]

Please check your PMs.

[AmigoJack]

sample file:

That's only sample text, not the exact file that could have a weird encoding. Please attach one such sample file to a post - this way the bytes of that file remain unchanged and we have the slight chance to reconstruct the issue.

[bbadmin]

Drxenos sent me a zipped copy of the file, the contents of which are identical to the sample text, so invisible characters or weird encoding do not seem to be the cause.

The fact that it's intermittent makes me wonder if it's a hardware issue, but why would it only show up in that context? For example, is it repeatable when run in a second instance of TextPad?

Memory problems can be checked as described here:

https://www.windowscentral.com/how-chec ... windows-10

I suppose disk problems might cause TextPad's code to be corrupted as it is read into memory. Worth doing a surface check of the drive as described here:

https://www.howtogeek.com/1033/how-to-u ... n-windows/

Any other ideas?

[Drxenos]

Drxenos sent me a zipped copy of the file, the contents of which are identical to the sample text, so invisible characters or weird encoding do not seem to be the cause.

The fact that it's intermittent makes me wonder if it's a hardware issue, but why would it only show up in that context? For example, is it repeatable when run in a second instance of TextPad?

Memory problems can be checked as described here:

https://www.windowscentral.com/how-chec ... windows-10

I suppose disk problems might cause TextPad's code to be corrupted as it is read into memory. Worth doing a surface check of the drive as described here:

https://www.howtogeek.com/1033/how-to-u ... n-windows/

Any other ideas?

Thank you for your support and courtesy. I'm sorry you can't reproduce it. I doubt it's a issue with my machine as I can reproduce it consistently on two different machines. It always happens the very first time I run the replace. The second and succeeding times are always successful.

Thanks.

Edit: I should note that I'm a software engineer and rely heavy on my equipment. Thus, I scan then regularly for issues (i.e., memory, disc, etc.).

[Drxenos]

I thought this had gone away because it stopped occurring. I figured out it ONLY happens when "Match case" is NOT checked.

Thanks.

[bbadmin]

Could it be a display artefact? If you minimize and restore TextPad, are the characters then shown correctly?

[Drxenos]

Could it be a display artefact? If you minimize and restore TextPad, are the characters then shown correctly?

That's a good thought, but no dice. Sorry.

I was hoping to look at the actual character codes to see if there was a pattern. I can't copy/paste because it complains of null characters (even just selecting the "messed up" part). I can't save because it says it'll replace invalid characters. There no way to switch to "hex mode" on the fly.

Any ideas?

[bbadmin]

Try saving as UTF-8.