How to select random lines from text?

[saaron]

Hello, I've searched for the answer to this, but I can't seem to find just what I'm looking for. I have a text file of 39,000 lines - they're text data records. I want to select and extract 1,000 random lines from this file into another text file. Is there some simple way I can do this? Thanks for any advice that anyone can offer!

Sabra

[s_reynisson]

Here's one way, a bit long and the random factor is questionable to
say the least

1. Using POSIX regular expression and the find dialog play with the value
to look for and the column number:

Code: Select all

^.{80}X (Looks for the value X in column 80)

In the find dialog box choose mark all, TP will tell you how many line were marked.
If you don't like the numer of marked lines goto Search->Clear all bookmarks
(Ctrl-Shift-F2) and start again.
To get more random selection you could look for a range of values in a
specific column like this: ^.{80}[a-z] or a number range ^.{80}[0-9]
or both like ^.{80}[a-z0-9].
2. When you got around 1000 marks, goto Search->Invert all bookmarks
and then Edit->Delete->Bookmarked lines
3. Save as <new file name>

To use POSIX goto Configure->Preferences->Editor and tick the
"Use POSIX regular expression syntax" box. In the find or replace
dialog tick the "Regular expression" box.
HTH

[talleyrand]

If you want something a little more random.
Install Python and you're good to go.

Copy this code and paste into Textpad.
Save it to something like C:\sarron.py
Update the fileName and the number of lines you want

C:\>python sarron.py

You can also run it straight out of Texpad. Set the command to wherever you installed Python (probably C:\python23\python.exe)

Code: Select all

import sys
import random

def makeRandomList(origList, randomSize):
   """
   randomly select entries from our list until we have reached our desired count
   """
   l = []

   #Chose one or the other
   allowDuplicates = True
   allowDuplicates = False
   while (len(l) < randomSize):
      #Get a random entry from the list
      choice = random.choice(origList)

      #The possibility exists that the chosen line will already exist in the
      #  list.  True randomness means the choice made now is not influenced
      #  by previous choices.
      if (allowDuplicates):
         l.append(choice)
      else:
         #Go here if you want to get lines that you haven't already gotten
         # This has the potential of causing an infinite loop if the number of
         # unique entries in the origList is less than randomSize
         if (choice not in l):
            l.append(choice)
   return l


def loadList(fileName):
   """
   open up a file and read all the lines into a list
   """
   try:
      f = open(fileName, 'r')

   except:
      print 'Failed to open file ' + fileName
      sys.exit(0)
   #EDIT 12Dec03 21:15 CST 
   # added the \n to split on newlines instead of whitespace
   return f.read().split('\n')


def main():
   #update this
   numberOfLines = 3
   #and this
   fileName = r'C:\winnt\system.ini'
   print makeRandomList(loadList(fileName), numberOfLines)

if __name__ == '__main__':
   main()