[plug] speed: find vs ls

[Brad Campbell]

On 28/7/22 20:22, Onno Benschop wrote:
> Interesting observation.
> 
> The glob with ls will only have to check 170 files (or as many as match the glob if there are more directories), where as you say find does not. It would be interesting to see if using *find . -type f | fgrep bkb.rhash.crc32 *gives a quicker response.

To quote Dr Teeth, "Easier done than said!".

I've picked the worst case directory to traverse. It has 399 entities at level 1. 234 files and 165 directories.
I'll start each test with an echo 2 > /proc/sys/vm/drop_caches and I've snipped the output from each test (a list of 7 names)

root at rpi31:/mnt/backup/work# ls -d ??????-???? | wc -l
439

root at rpi31:/mnt/backup/work# time ls ??????-????/bkb.rhash.crc32

real	0m13.161s
user	0m0.004s
sys	0m0.162s

root at rpi31:/mnt/backup/work# time find . -maxdepth 2 -type f | fgrep bkb.rhash.crc32

real	6m1.423s
user	0m1.685s
sys	0m12.119s

root at rpi31:/mnt/backup/work# time find . -maxdepth 2 -type f -name bkb.rhash.crc32

real	6m1.035s
user	0m1.981s
sys	0m12.146s

> That said, I've had the exact opposite experience with a massive directory with log files where find . -type f -print0 gave me a file list MUCH faster than ls, to the point where bash ran out of resources.
> 
> The point being that finding a solution is often specific to each unique problem.

Absolutely.

> Thank you for sharing.
> 
> We should do more of that here.

We used to.

It's gradually trailed off since I first signed up some 24 years ago, but if I do the sums that means I've been using Linux longer than I haven't.

I can't figure out whether that's due to the rise of the "Google library" or people just aren't getting their hands dirty anymore but I do miss the technical discussions.

Brad
-- 
An expert is a person who has found out by his own painful
experience all the mistakes that one can make in a very
narrow field. - Niels Bohr