[plug] importing a large text database for fast search

Michael Holland michael.holland at gmail.com
Fri Sep 2 14:30:14 WST 2011


Suppose you had a large database - 1.7GB, with about 250k records in a CSV file.
Each record has 8 fields - 7 headers plus a body.
You might use a PERL script to split to files, sort into folders by
embassy name, convert the ALLCAPS to more legible case, and remove the
quote escaping from the body.
Maybe add links to a glossary for the more obscure military/diplomatic
terms and acronyms.
But greping all this data is still slow. What is a good way to store
it in Linux, with a full text index?



More information about the plug mailing list