[plug] Deduplication - LessFS, opendedup or zfs

[Nick Bannon]

On Sat, Oct 01, 2011 at 07:53:10PM +0800, Scott Middleton wrote:
[deduplication server]
> I have been reading a bit about it but one of my concerns is that I will be
> copying over a limited bandwidth WAN. I understand that the FS will use less
> space but what about less bandwidth in the transfer? Traditionally I use
> rsync but since I also want to back up VirtualBox VMs, I do not want to
> upload all of them and want the Deduplication to process on the client side
> so that only changed blocks are transferred.

As part of a regular backup scheme, you mean? This could do the job:
http://braawi.org/obnam/

dedup on the client side is necessary for the encrypted backups, the
backup storage server end doesn't get the keys.

Those filesystem approaches (LessFS, opendedup or ZFS) aren't directly
going to help you cut down duplicate network transfers, though if you
already have one of those running it would make it a bit less painful to
make an occasional offsite copy of some related VM image files. (Copy the
"template" files, which will be fast, then use zsync, rsync, "rsync --inplace"
as before)

Nick.

-- 
   Nick Bannon   | "I made this letter longer than usual because
nick-sig at rcpt.to | I lack the time to make it shorter." - Pascal