[plug] Deduplication

[Adon Metcalfe]

Yeah I'd say BTRFS itself is very stable on kernel 3.12+, and bedup had a
bug with sqlite (https://github.com/g2p/bedup/pull/45) fixed in the latest
version which was causing some issues, since those 2 changes we have found
it pretty nice.

We did note with BTRFS if you have a high snapshot load (like a few
snapshots a minute on big volumes) and/or are trying to send multi terabyte
volumes it can still trip up on occasion, but using it as a standard
filesystem with snapshotting for backups we haven't managed to trigger any
of those issues (all the corruption/space issues I believe were resolved
around 3.10, and while you can still 'fragment' yourself out of space it's
relatively easy to recover from with the btrfs balance tool, and you can
avoid it completely long term by making 'balances' part of your maintenance
schedule)


On 12 May 2014 12:05, Will Dowling <will at autodeist.com> wrote:

> Bedup seems at first glance to heavily integrate with brtfs, so I'm
>> unsure how this would help in the stability stakes.
>>
>
> Bedup is a tool to do deduplication on btrfs specifically, and whilst it
> works reasonably well it crashes for me trying to use it on sufficiently
> large volumes (>10TB).
>
> Are you looking for a copy-on-write filesystem or just a tool to detect
> duplicate files on any fs? If it's the latter a tool like duff would work
> well and should be available by APT.
>
>
> Will Dowling
>
> F: +61 (08) 6364 4880
> F: +61 (08) 6364 4881
> E: will at autodeist.com
>
> _______________________________________________
> PLUG discussion list: plug at plug.org.au
> http://lists.plug.org.au/mailman/listinfo/plug
> Committee e-mail: committee at plug.org.au
> PLUG Membership: http://www.plug.org.au/membership
>



-- 
Regards,

Adon Metcalfe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.plug.org.au/pipermail/plug/attachments/20140512/4b1f933c/attachment.html>