Find in text file the first word starting with ...

Solved
Emzo9 Posted messages 6 Status Member -  
mamiemando Posted messages 33174 Registration date   Status Moderator Last intervention   -

Hello,

How can I find the first word starting with a given string in a large text file (about 3000 words, one word per line, in alphabetical order)? I do it with a FOR loop, but is it possible to do it faster, with a direct search instruction, somewhat like seek()?

4 answers

  1. PierrotLeFou
     

    seek() is used to reposition within a file. Where are you going to reposition yourself?
    How do you do it now?
    You say "starting" with a string. Do you know slicing? (if you know the length of the string?)
    Start with this before thinking about more sophisticated methods.
    I do it easily with files of over 300,000 words.
    It shouldn't take as long with about 3,000 words.

    0
  2. PierrotLeFou
     

    repositioning?

    No way to delete?

    0
  3. mamiemando Posted messages 33174 Registration date   Status Moderator Last intervention   7 944
     

    Hello,

    A 3000-word file is small and therefore your function does not specifically require optimization.

    Moreover, there is not really a way to speed up the code (what will be "slow" is reading the file and loading the 3000 words into memory). So for my part, I would just write this.

    toto.py

    #!/usr/bin/env python3 from pathlib import Path filename = Path("data.txt") assert filename.exists() with open(filename) as f: lines = [ line.strip() for line in f.readlines() if line.startswith("str") ] print(lines)

    data.txt

    aaa bbb str1 ssstr2 str2aa str3 xxstr

    Execution:

     ['str1', 'str2aa', 'str3']

    If you are aiming for (really much) larger dictionaries, you might consider serializing them (see for example the pickle or dill modules).

    Then, if the question is how to efficiently search for the index of an element in a sorted list (and in memory), you might consider a binary search. See here for more details.

    Good luck

    0
  4. Emzo9 Posted messages 6 Status Member
     

    Thank you both.

    Here is what I had done:

    # Find all the words in Dico.txt, # starting with deb, variable defined beforehand. with open(r"Dico.txt", "r") as f: for word in f: if word[0:len(deb)] == deb: # process_word --> call of the function # dealing with each relevant word

    I see that you both think it doesn't need optimization.

    And the method with "startswith" proposed by mamiemando is quite compact and elegant. I will do that.

    In fact, what I had in mind was something like the FIND instruction, followed by a string, which used to give, in dBase IV, the index of the first word starting with that string in the file directly. Without a loop. But well, I will stick to "startswith". Problem solved.

    A BIG THANK YOU to both of you.

    0
    1. mamiemando Posted messages 33174 Registration date   Status Moderator Last intervention   7 944
       

      In fact, it's mainly that you can't do much better without using an intermediate structure like a Trie. Generating such a structure only becomes worthwhile if the corpus is very large (which is not your case) and/or if we are making a very large number of searches.

      0