Community

It would be nice if there was a feature to set a custom "end of line" character. I'm working with data files where the data is plain-text but separated by custom characters and it's all on a single "line" within the file. I'd like to be able to view it and have it displayed such that the data element separator causes each element to be shown on a new line. Ideally, if I hit <enter> while editing the file, this custom EOL char would be inserted rather than the normal EOL char ('\n').

I've searched around for a text editor that does this, and the only one I found is 8 years old and is terribly annoying to work with. I'm a big fan of textpad, and would love for this feature to be added.

In the meantime, if anyone knows of a good text editor that can do this, please let me know. Ideally it would be able to handle extremely large files without crashing (250MB+) but this would be a nice-to-have.

I would also like to point out that these data files use a NULL character as the data separator. It looks like currently, you cannot even cut/copy/paste NULL characters. It would be great if this could be fixed as well. Otherwise, I need a new text editor =(

So, not only do I need the ability to set a custom EOL character, I need to be able to set it to a NULL character ('\0')

I'm not sure if i understand what you mean with custom EOL and why \n doesn't work for you. After reading what you said i would think that a search and replace where you search for the "data element separator" (whatever that is) and replace with "\n" would do what you want. Also remember to have regular expression checked during the search and replace. If you don't have regulard expression checked you will get the text \n instead of the control character for newline. \n is for newline so you should then get a file with multiple lines instead of everything on a single line like you describe. You could replace with anything else you prefer if \n (new line) is not what you want. If you want to replace some kind of separator with a newline then "\n" and regular expression should work. If this still doesn't work the way you want it to be it would be nice with an example to explain what you mean since i don't understand what you want in that case.

Regarding large files textpad should be able to handle files larger then 250mb which isn't really large, but it depend on how much ram you got and the swapfile size. There are text editors that can handle very large files like 250 gigabyte+ if you need that, but 250mb isn't really large. Those text editors have no requirements to ram/swapfile either to open large files and one of them support multithreading so you don't have to wait for the whole file to be loaded and can start working with the file right away while it's continually loading the rest of the file in background. I wish textpad could implement such a feature. I'm not sure if the textpad support forum is the place to advertise for other text editors, but there are others that is very good at the stuff that textpad cannot do.

Regarding the NULL character that isn't actually valid for textfiles so if you have some kind of program that create textfiles with null characters i would say that the problem isn't textpad, but the program that created the textfiles. You could do a search and replace to replace the null characther with anything you like (\n for newline, some other control characters or any other valid character). In this case there are other text editors as well that detect null characters when opening the file and ask you if you want them to be converted to something else. Some is able to do this automatically as well where you specify what should be the default to replace the null characters. Using textpad i think you have to do a search and replace for the null characters before you can copy/paste like you describe.

There is some more info regarding large files here: http://forums.textpad.com/viewtopic.php?t=5195

Thanks for the reply gan.

To briefly address the large file issue, it's not a problem with TextPad, I only mentioned it in case someone had a recommendation for an editor that did handle EOL chars as I would like (to ideally also handle large files well).

As far as the program that generates the files I'm dealing with, well, it's my company's applications, and it's pretty widely used, so changing that is not really an option.

To clarify a bit, the files do not contain any EOL characters ('\n'). The data elements are separated by NULL characters instead. However, the data itself is all plaintext between the NULL characters. I would like to be able to view and edit the data such that the NULL characters are interpreted and displayed as new lines, but saved as NULL characters in the file, so I can easily read and edit the data rather than have it all mashed together which is difficult to read. Basically, be able to set a character that is interpreted as the EOL character instead of '\n'.

Part of the problem is that TextPad doesn't properly handle a find-replace of NULL characters. A NULL character is represented as '\0'. A find-replace for this does not work. Without reg expressions turned on, it fails to find any (as it is searching for a string "\0"). With reg expressions on, it finds and replaces instances of the character '0'. (Note that I tried entering it as \0 with no quotes, with single quotes, and with double quotes just to be sure.) Furthermore, TextPad does not allow copy/paste of a NULL character, so I can't copy it from the data and paste it into the find/replace dialog.

I found one program that will allow me to find/replace NULL characters. However, to edit the data for testing, I have to find/replace all NULL characters with '\n', find and edit the data I want, and then replace all '\n' with NULL characters before saving. This is do-able, but a hassle, and in general I don't like it Pro as much as TextPad.

bobthe wrote:Thanks for the reply gan.
To briefly address the large file issue, it's not a problem with TextPad, I only mentioned it in case someone had a recommendation for an editor that did handle EOL chars as I would like (to ideally also handle large files well).

Well there is a couple of other text editor i know that handle both. One of them is pretty good and able to handle large files excellent and convert NULL's to space. The other i don't like at all, but got a lot of feature and able to customize what to do with NULL's.

bobthe wrote:As far as the program that generates the files I'm dealing with, well, it's my company's applications, and it's pretty widely used, so changing that is not really an option.

I understand that could be hard to change and not an option.

bobthe wrote:To clarify a bit, the files do not contain any EOL characters ('\n'). The data elements are separated by NULL characters instead. However, the data itself is all plaintext between the NULL characters. I would like to be able to view and edit the data such that the NULL characters are interpreted and displayed as new lines, but saved as NULL characters in the file, so I can easily read and edit the data rather than have it all mashed together which is difficult to read. Basically, be able to set a character that is interpreted as the EOL character instead of '\n'.

I don't think that is possible to do with textpad. You could easily replace the NULL's with a search and replace where you search for "\x00" and replace with anything you like, but then you cannot save with the NULL's again. You could use "word wrap", but that works if you only want to read the files and doesn't really help much if you need to edit the files since you cannot copy NULL's and stuff like that.

bobthe wrote:Part of the problem is that TextPad doesn't properly handle a find-replace of NULL characters. A NULL character is represented as '\0'. A find-replace for this does not work. Without reg expressions turned on, it fails to find any (as it is searching for a string "\0"). With reg expressions on, it finds and replaces instances of the character '0'. (Note that I tried entering it as \0 with no quotes, with single quotes, and with double quotes just to be sure.) Furthermore, TextPad does not allow copy/paste of a NULL character, so I can't copy it from the data and paste it into the find/replace dialog.

That is not correct usage of the regular expression. Search for "\x00" (without the quotes) to find NULL's then it will work with textpad. \0 does not represent the hex for NULL, but 0x00 does and using regular expression you would search using \x00. "\0" represent the decimal and not the hex for NULL and you need to specify as hex since you cannot search using the decimal like that. Using textpad "\x" where the x is a number is used for another purpose with textpad.

bobthe wrote:I found one program that will allow me to find/replace NULL characters. However, to edit the data for testing, I have to find/replace all NULL characters with '\n', find and edit the data I want, and then replace all '\n' with NULL characters before saving. This is do-able, but a hassle, and in general I don't like it Pro as much as TextPad.

I never tried using textpad to replace something with NULL's so not sure if that works, but replacing NULL's with anything else work fine. Using a hex editor this would be easy of course. You could use a hex editor that accept command line parameters and then add it as an external tool in textpad to do this conversion with a shortcut key or a toolbar button. Like one button that convert from 0x00 to something and another button that convert back. If you are familiar with something like C++ creating such a conversion tool that can be started from textpad with command line parameters is just 10-20 lines of code. Another option would be to use a hex editor or some other text editor that handle the NULL's the way you want.

gan wrote:That is not correct usage of the regular expression. Search for "\x00" (without the quotes) to find NULL's then it will work with textpad. \0 does not represent the hex for NULL, but 0x00 does and using regular expression you would search using \x00. "\0" represent the decimal and not the hex for NULL and you need to specify as hex since you cannot search using the decimal like that. Using textpad "\x" where the x is a number is used for another purpose with textpad.

Ah, I see. I don't work much in hex so I wasn't aware of this solution. I'll give it a shot and see if that works for me.

gan wrote:I never tried using textpad to replace something with NULL's so not sure if that works, but replacing NULL's with anything else work fine. Using a hex editor this would be easy of course. You could use a hex editor that accept command line parameters and then add it as an external tool in textpad to do this conversion with a shortcut key or a toolbar button. Like one button that convert from 0x00 to something and another button that convert back. If you are familiar with something like C++ creating such a conversion tool that can be started from textpad with command line parameters is just 10-20 lines of code. Another option would be to use a hex editor or some other text editor that handle the NULL's the way you want.

A hex editor wouldn't help me any more unless I could have it interpret the NULL characters as line breaks anyhow. I think my best bet is probably you're suggestion of writing a macro to do the conversion for me. If I can't just write a macro to do a find/replace using \x00 then I can easily write a C++ script to do this for me. Short of an editor that will allow me to choose a custom character to be interpreted as a line break, this is my best bet.

Thanks for all the help!

As a sidenote, there was a text editor that I once saw that was capable of displaying NULL characters as line breaks. Unfortunately, I haven't been able to find it since. So, it's possible, I'm just not sure if it's feasible for TextPad to include a feature like this.

bobthe wrote:A hex editor wouldn't help me any more unless I could have it interpret the NULL characters as line breaks anyhow. I think my best bet is probably you're suggestion of writing a macro to do the conversion for me. If I can't just write a macro to do a find/replace using \x00 then I can easily write a C++ script to do this for me. Short of an editor that will allow me to choose a custom character to be interpreted as a line break, this is my best bet.

Well i was thinking a hex editor that convert NULL to line break so you can edit using textpad and then convert back again after you are done with textpad. If the hex editor support command line parameters you might be able to add this as an external tool in textpad and then do the conversion with a single click from withint textpad. Creating a small application to convert could basically do the same thing.

bobthe wrote:As a sidenote, there was a text editor that I once saw that was capable of displaying NULL characters as line breaks. Unfortunately, I haven't been able to find it since. So, it's possible, I'm just not sure if it's feasible for TextPad to include a feature like this.

I'm not sure what text editor that would be since i never seen that feature, but even if it could be possible to include this feature in textpad i wouldn't hold my breath waiting. There been a lot of feature requests for textpad for some years now and people are still waiting for most of them. So i'm pretty sure you need a LOT of patience if you are waiting for this to be included in textpad since the development is almost standing still it seems. Neither do i think this will be the first feature to be included since NULL characters doesn't really belong in a text file unless textpad include more hex editor features in the future. You could post a feature request under "Enhancement Suggestions" and hope that it will be included in the future though.

Do you have perl?

To convert nulls to newlines:
perl -n0e "chop ; print \"$_\n\"" file.null > file.crlf

To convert newlines to nulls:
perl -ne "chomp ; print \"$_\0\"" file.crlf > file.null

Well, TextPad correctly replaced the NULL characters using '\x00' and converted them to line breaks '\n'. However, when I tried the reverse, it replaced the line breaks '\n' with the literal string "\x00".

bobthe wrote:Well, TextPad correctly replaced the NULL characters using '\x00' and converted them to line breaks '\n'. However, when I tried the reverse, it replaced the line breaks '\n' with the literal string "\x00".

Like i said there is most likely no way to get it back to include the NULL's using textpad. You could create a small program using the language you prefer like c++ or visual basic, use a hex editor or perl. ben_josephs gave you an example and creating a batchfile and add as external tool in textpad you should be able to do this with a single click or shortcut key from within textpad.

gan wrote:Like i said there is most likely no way to get it back to include the NULL's using textpad. You could create a small program using the language you prefer like c++ or visual basic, use a hex editor or perl. ben_josephs gave you an example and creating a batchfile and add as external tool in textpad you should be able to do this with a single click or shortcut key from within textpad.

Yep, got it. I'll likely go the C++ route as I'm very familiar with C++ (and have never used perl, even though the code does look very simple). I'll play around with it when I have some time.

Thanks to both of you for all the help and quick replies.