Partial File Loading

Ideas for new features

Moderators: AmigoJack, bbadmin, helios, Bob Hansen, MudGuard

Post Reply
bluntelk
Posts: 2
Joined: Fri Nov 21, 2003 4:52 am

Partial File Loading

Post by bluntelk »

if this has been suggested, sorry for not seeing the existing thread.

The Problem:
Opening large files makes textpad go non responsive as it tries to load the file into memory.

Possible sollution:
Only load the first x megabytes of the file (when opening) the file

eg - 100 meg log file
user opens it
textpad load the first 2 meg.
as the user scrolls down (or across) the log textpad keeps a track of where they are in the file - loading the relavant portions of the files as they go

so in effect - if you loaded up a 100 meg file, it would load 2 meg at a time as you scrolled down.

as for updated files, the trick will still work.

as for "the size" of the files to do this, let it be user definable.

The End Result:
you will be able to load up 1gb files, make changes in your favourite editor and save them without hassle.

what do you think? a good or bad idea?
User avatar
ramonsky
Posts: 88
Joined: Fri Nov 14, 2003 10:54 am

Post by ramonsky »

Doesn't this happen already, thanks to the miracle of Virtual RAM (aka Swap-File) that is built into Windows?

The problem you describe is a problem which affects all sorts of programsL image editors; music editors; 3D-model editors, etc. They all eventually have to face the possibility that the document might not fit in RAM. Almost universally, the application simply relies on the operating system to manage the swap file. It's a good system. It works. Sure - things slow down when you hit the out-of-real-RAM point, but that would still be true even if TextPad handled the situation internally.

But text editors are the LEAST affected by such considerations, because text files are relatively small compared with most other kinds of document. In general, text files are tiny compared to (for example) video clips. If you're running out of RAM when editing a text file, then either you have humungously huge text files or a miniscule RAM,

Just buy more RAM. Seriously.

Jill
User avatar
maniac
Posts: 33
Joined: Thu Sep 25, 2003 4:16 am
Contact:

Post by maniac »

First of all, I don't think he was actually talking about loading it from memory, rather, streaming it from the file, which wouldn't deal with a swapfile. However, this would be really hard to implement, at least in my opinion.
_maniac_
User avatar
kaimiddleton
Posts: 13
Joined: Wed Nov 12, 2003 12:48 am
Location: San Francisco
Contact:

vedit

Post by kaimiddleton »

vedit, which has been around for a while, purports to handle large files well. It ain't cheap tho'.

http://www.vedit.com/
Quickly edit huge, tricky files:
  • Database, log, mainframe files
    Binary, control & graphics chars
    Fast, flexible, multi-key sorting
    Huge 100+ Megabyte files
    Lines with 100,000+ columns
Post Reply