Copying huge blocks
Moderators: AmigoJack, bbadmin, helios, Bob Hansen, MudGuard
-
Russell May
- Posts: 6
- Joined: Mon Apr 26, 2004 2:42 pm
Copying huge blocks
How large a block should be copyable? While editing a large file (about 950K lines) using columnar block mode in Textpad and insert mode in the computer, I could not copy blocks (about the left-hand half of the lines) larger than about 350K to 400K lines. I worked around it by dividing the file into three parts. Total memory used was over 1GB, exceeding real memory but not virtual memory in Win 2000.
-
s_reynisson
- Posts: 940
- Joined: Tue May 06, 2003 1:59 pm
-
Russell May
- Posts: 6
- Joined: Mon Apr 26, 2004 2:42 pm
Undo might have had something to do with it but I am not sure because I did not get any notices about running out of memory.
As an experiment I doubled my pagefile size and re-ran the editing session. It was to rearrange a 27MB 310K line file so that side-by-side pages about 60 lines each were made sequential in a column about half as wide. I had learned a little about what was needed to do this, so this time it grew to only 57MB 657K lines instead of 78Mb 950K lines. Then I accidentally ran an unending replacement. I got a notice that minimum total memory (now 2GB) had been exceeded, but I let the process continue until memory usage was about 2.4GB. Max is now 3.5GB. I undid the replacement, saved the file, and cancelled Undo. Memory usage drifted slowly down to about 950MB. Then I re-enabled Undo and attempted more editing. Textpad (?) almost immediately notified me that I was out of memory. I don't recall what I was attempting when that happened. I restarted the computer, restarted Textpad and continued editing the saved file. Everything went okay, including cutting and pasting to move the left half of the file relatively up about 60 lines, and then deleting a huge number spaces which I had generated in preparation for the move. After the move they were leading spaces for alternate pages, so deleting them made all of the pages into one fairly uniform column. This time I had no problem copying and pasting all of the huge columnar block as one section. Total memory usage peaked at about 645MB. The finished file had about 31MB 587K lines.
I don't know whether anything in Textpad need be fixed because this was definitely an unusual situation, but Textpad did apparently foul up somehow in the first editing session. (Could it be me that fouled up? Maybe, but of course I don't think so.)
As an experiment I doubled my pagefile size and re-ran the editing session. It was to rearrange a 27MB 310K line file so that side-by-side pages about 60 lines each were made sequential in a column about half as wide. I had learned a little about what was needed to do this, so this time it grew to only 57MB 657K lines instead of 78Mb 950K lines. Then I accidentally ran an unending replacement. I got a notice that minimum total memory (now 2GB) had been exceeded, but I let the process continue until memory usage was about 2.4GB. Max is now 3.5GB. I undid the replacement, saved the file, and cancelled Undo. Memory usage drifted slowly down to about 950MB. Then I re-enabled Undo and attempted more editing. Textpad (?) almost immediately notified me that I was out of memory. I don't recall what I was attempting when that happened. I restarted the computer, restarted Textpad and continued editing the saved file. Everything went okay, including cutting and pasting to move the left half of the file relatively up about 60 lines, and then deleting a huge number spaces which I had generated in preparation for the move. After the move they were leading spaces for alternate pages, so deleting them made all of the pages into one fairly uniform column. This time I had no problem copying and pasting all of the huge columnar block as one section. Total memory usage peaked at about 645MB. The finished file had about 31MB 587K lines.
I don't know whether anything in Textpad need be fixed because this was definitely an unusual situation, but Textpad did apparently foul up somehow in the first editing session. (Could it be me that fouled up? Maybe, but of course I don't think so.)
-
s_reynisson
- Posts: 940
- Joined: Tue May 06, 2003 1:59 pm
Found a 26Mb text file on my HD so here goes. Rebooted my XPproSP2 system with TP473, using 1Gb memory and 1,5Gb pagefile, Commit Charge(K) Limit reads 2520284. Commit Charge(K) Peak starts at 115908. Start TP with a 26Mb/687559 line file, undo is enabled. Commit Charge(K) Peak reads 152400.
Trying to follow your most splendid post I set block mode on, copy a column 40 characters wide starting from line one to end of file. Paste that block in a new file. In the original block I do a find/replace on all characters with a space using the regex find "." and replace with " ". Insert 60 blank lines at top of document. Back to the temp file and copy all, paste the original block back into original document starting from line one. Commit Charge(K) Peak now reads 1.8Gb.
All I'm really doing is that I'm "lifting" the left half of the document up 60 lines, the first 40 characters.
I save the file, takes a while, and exit TP. This is where it gets interesting. TP took around 20 minutes to return the memory, there was no TP listed under Windows Task Manager Applications, but it was there under Processes. When it had returned about 800Mb it reported an error and that it would have to be shut down. I pressed ok and the rest of the memory was returned instantly. Perhaps this is worth taking a better look at for Helios!
Oh, pls note that I'm using a dirt cheap HD, ATA100 with 2Mb buffer.
Trying to follow your most splendid post I set block mode on, copy a column 40 characters wide starting from line one to end of file. Paste that block in a new file. In the original block I do a find/replace on all characters with a space using the regex find "." and replace with " ". Insert 60 blank lines at top of document. Back to the temp file and copy all, paste the original block back into original document starting from line one. Commit Charge(K) Peak now reads 1.8Gb.
All I'm really doing is that I'm "lifting" the left half of the document up 60 lines, the first 40 characters.
I save the file, takes a while, and exit TP. This is where it gets interesting. TP took around 20 minutes to return the memory, there was no TP listed under Windows Task Manager Applications, but it was there under Processes. When it had returned about 800Mb it reported an error and that it would have to be shut down. I pressed ok and the rest of the memory was returned instantly. Perhaps this is worth taking a better look at for Helios!
Oh, pls note that I'm using a dirt cheap HD, ATA100 with 2Mb buffer.