[plug] ext4 fragmentation

byron ester byronester at gmail.com
Fri Mar 17 13:54:33 AWST 2017

Thanks for sharing Brad!

I've often wondered about fragmentation in Linux filesystems and why it's
not talked about (that I've seen).


Cheers for the info and real world data.


On Fri, Mar 17, 2017 at 1:06 PM, Brad Campbell <brad at fnarfbargle.com> wrote:

> G'day all,
> I'm approaching my 6 yearly HD replacement on my bulk storage array.
> Current array is 14 x 2TB drives in a RAID6 with an ext4 filesystem.
> This array was re-built with a fresh format in 2013, so it has about 4
> years heavy use on the filesystem.
> I can stream direct from the md block device at about 1.2GB/s, but I was
> finding when copying files across the network I was getting about ~150MB/s
> (tar and netcat). I tried streaming some larger files (~1TB) to /dev/null
> and saw the same transfer rates. So 1.2GB/s from the array and 150MB/s from
> the filesystem. What gives?!?
> Fragmentation. You know, that thing _everyone_ says just isn't a problem
> on Linux? Twaddle.
> I ran e4defrag over a directory of video files (~1.5TB). This took 2 days,
> but afterwards the average transfer rate (tar to /dev/null) went from about
> 150MB/s to 500MB/s. The 1TB file mentioned above went from ~150MB/s to
> ~600MB/s.
> Having said that, the 1TB file copied across to the replacement array was
> giving me ~900MB/s until I used e4defrag on it. Now it's down to 500MB/s.
> So things are not perfect in defrag land. None the less, fragmentation does
> become an issue on heavily used, mostly full filesystems over a long period
> of time, and e4defrag does have tangible benefit in helping clean this up.
> Now I wish someone had a good offline defragger that planned ahead and
> really reorganized the drives like they do for FAT and NTFS. Even as a once
> every couple of year thing. On large arrays you just can't copy the files
> somewhere else, then copy them back to defragment them.
> Just more rambling as I come across stuff that I think is worth mentioning.
> _______________________________________________
> PLUG discussion list: plug at plug.org.au
> http://lists.plug.org.au/mailman/listinfo/plug
> Committee e-mail: committee at plug.org.au
> PLUG Membership: http://www.plug.org.au/membership
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.plug.org.au/pipermail/plug/attachments/20170317/d5de7154/attachment.html>

More information about the plug mailing list