Find in text file the first word starting with ...
SolvedHello,
How can I find the first word starting with a given string in a large text file (about 3000 words, one word per line, in alphabetical order)? I do it with a FOR loop, but is it possible to do it faster, with a direct search instruction, somewhat like seek()?
4 answers
-
seek() is used to reposition within a file. Where are you going to reposition yourself?
How do you do it now?
You say "starting" with a string. Do you know slicing? (if you know the length of the string?)
Start with this before thinking about more sophisticated methods.
I do it easily with files of over 300,000 words.
It shouldn't take as long with about 3,000 words. -
-
Hello,
A 3000-word file is small and therefore your function does not specifically require optimization.
Moreover, there is not really a way to speed up the code (what will be "slow" is reading the file and loading the 3000 words into memory). So for my part, I would just write this.
toto.py
#!/usr/bin/env python3 from pathlib import Path filename = Path("data.txt") assert filename.exists() with open(filename) as f: lines = [ line.strip() for line in f.readlines() if line.startswith("str") ] print(lines)data.txt
aaa bbb str1 ssstr2 str2aa str3 xxstr
Execution:
['str1', 'str2aa', 'str3']If you are aiming for (really much) larger dictionaries, you might consider serializing them (see for example the pickle or dill modules).
Then, if the question is how to efficiently search for the index of an element in a sorted list (and in memory), you might consider a binary search. See here for more details.
Good luck
-
Thank you both.
Here is what I had done:
# Find all the words in Dico.txt, # starting with deb, variable defined beforehand. with open(r"Dico.txt", "r") as f: for word in f: if word[0:len(deb)] == deb: # process_word --> call of the function # dealing with each relevant word
I see that you both think it doesn't need optimization.
And the method with "startswith" proposed by mamiemando is quite compact and elegant. I will do that.
In fact, what I had in mind was something like the FIND instruction, followed by a string, which used to give, in dBase IV, the index of the first word starting with that string in the file directly. Without a loop. But well, I will stick to "startswith". Problem solved.
A BIG THANK YOU to both of you.
-
In fact, it's mainly that you can't do much better without using an intermediate structure like a Trie. Generating such a structure only becomes worthwhile if the corpus is very large (which is not your case) and/or if we are making a very large number of searches.
-