[plug] ext4 fragmentation
byron ester
byronester at gmail.com
Fri Mar 17 13:54:33 AWST 2017
Thanks for sharing Brad!
I've often wondered about fragmentation in Linux filesystems and why it's
not talked about (that I've seen).
http://askubuntu.com/questions/221079/how-to-defrag-an-ext4-filesystem
Cheers for the info and real world data.
Byron
On Fri, Mar 17, 2017 at 1:06 PM, Brad Campbell <brad at fnarfbargle.com> wrote:
> G'day all,
>
> I'm approaching my 6 yearly HD replacement on my bulk storage array.
> Current array is 14 x 2TB drives in a RAID6 with an ext4 filesystem.
> This array was re-built with a fresh format in 2013, so it has about 4
> years heavy use on the filesystem.
>
> I can stream direct from the md block device at about 1.2GB/s, but I was
> finding when copying files across the network I was getting about ~150MB/s
> (tar and netcat). I tried streaming some larger files (~1TB) to /dev/null
> and saw the same transfer rates. So 1.2GB/s from the array and 150MB/s from
> the filesystem. What gives?!?
>
> Fragmentation. You know, that thing _everyone_ says just isn't a problem
> on Linux? Twaddle.
>
> I ran e4defrag over a directory of video files (~1.5TB). This took 2 days,
> but afterwards the average transfer rate (tar to /dev/null) went from about
> 150MB/s to 500MB/s. The 1TB file mentioned above went from ~150MB/s to
> ~600MB/s.
>
> Having said that, the 1TB file copied across to the replacement array was
> giving me ~900MB/s until I used e4defrag on it. Now it's down to 500MB/s.
> So things are not perfect in defrag land. None the less, fragmentation does
> become an issue on heavily used, mostly full filesystems over a long period
> of time, and e4defrag does have tangible benefit in helping clean this up.
>
> Now I wish someone had a good offline defragger that planned ahead and
> really reorganized the drives like they do for FAT and NTFS. Even as a once
> every couple of year thing. On large arrays you just can't copy the files
> somewhere else, then copy them back to defragment them.
>
> Just more rambling as I come across stuff that I think is worth mentioning.
>
> _______________________________________________
> PLUG discussion list: plug at plug.org.au
> http://lists.plug.org.au/mailman/listinfo/plug
> Committee e-mail: committee at plug.org.au
> PLUG Membership: http://www.plug.org.au/membership
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.plug.org.au/pipermail/plug/attachments/20170317/d5de7154/attachment.html>
More information about the plug
mailing list