[plug] RPi USB drive backups

[Brad Campbell]

On 20/3/25 08:03, Chris McCormick wrote:
> Hey all,
> 
> Unfortunately I couldn't make it to the recent RPi meetup, otherwise that would have been a good place to ask about this.
> 
> Anyway, I have an RPi with a powered USB hub and two USB hard drives plugged in doing backups. The second drive is a mirror of the first which I have traditionally done with rsync. I recently replaced one of the drives and I now need to mirror the full drive to the new drive. I've been using rsync to do this but it takes ages just to do the initial diff and craps out after a while with IO errors and unmounts the destination drive. The total size of data is 2.3Tb.
> 
> What is a good way to do a fast copy of all of the data on one USB drive to the other? I'd prefer not to have to stuff around with partitions if possible as they have slightly different partitions on each drive and different labels etc.
> 
> LLMs were no help at all with this so I'm relying on the PLUG brains trust! 😅 Thanks for any suggestions.

I agree entirely with Chris in that you likely have issues. I have a pair of 4TB drives by USB on a pi and can shovel data around with no errors.

Having said that, using rsync for an initial backup can be slow, particularly when there are a lot of hard-links.

I use tar. Off the top of my head, something like tar -cpC /source . | tar -xC /target

I don't exactly recall the specifics because I always fiddle, tweak and dry run to get the best speeds.

If doing it across a network, I use the same tar commands but usually with compression if the source machine is fast enough (pipe through pigz), and I pipe them over netcat.
It can be up to an order of magnitude faster than rsync when just replicating a filesystem.

Regards,
Brad