[plug] pigz --independent, gzrt/gzip file recovery, was Re: Tapes

[Brad Campbell]

On 8/3/23 23:12, Nick Bannon wrote:
> On Wed, Mar 08, 2023 at 09:43:39PM +0800, Andrew Furey wrote:
>> Also pigz is a must for getting data to the tape fast enough to avoid
>> shoeshining. (At least in our case with LTO3/4 it made a huge difference…
>> LTO8 I couldn’t seem to get it there fast enough regardless.)
>>
>> (tar -cf /dev/st0 -I pigz source, rather than tar -czf.)
> 
> `pigz` has another superpower for seekability/partial recovery, in
> combination with https://www.urbanophile.com/arenn/hacking/gzrt/gzrt.html /
> DebianPkg:gzrt , namely `pigz --independent` .
> 
> What if you had gigabytes of vital backup with missing, corrupt blocks
> partway through? can you recover some of the rest? I think with hardware
> tape compression that should usually be possible, but it's sadly missing
> from most software-compression-format options.

This was going to be my next line of enquiry and precisely for this reason.
The advantage of tar being the ability to skip broken bits and recover the
rest, but traditionally when you pipe that through compression and/or
encryption you lose the ability to recover an archive that has had a hole
punched in it. That, combined with block based symmetric encryption should
give the ability to recover more of what is recoverable.

Nice head start. Thanks!

Regards,
Brad
-- 
An expert is a person who has found out by his own painful
experience all the mistakes that one can make in a very
narrow field. - Niels Bohr