[plug] backups

[W.Kenworthy]

Yes, its a great program, but if it wont work then thats that.  There's
some tricks that can extend how many files it can handle but eventually
you run out of memory -  It is possible to scp the files across the
first time, then it can work as it is only caching the metadata of files
that have changed, but it wont do 100000 files from scratch (without
manual interventions like running multiple partials).  Doesnt help me in
this case!  Took some 15 hrs but the jobs done via scp/openvpn over wap
using bzip2/tar.  The limiting factor was that it was cpu bound, not
channel bandwidth.  Next time I'll try gzip and see if its better than
bzip2 for this.

BillK


On Sat, 2006-04-22 at 09:23 +0800, Stuart Midgley wrote:
> I completely disagree.
> 
> I use rsync to backup over a TB of data on a daily basis (with 10's  
> of GB of changed files per day).  It has many great features that  
> when used correctly make it one of the best backup solutions around.
> 
> Some of the features I use:
> 	incremental backups
> 	exclude certain types of files
> 	compress over the wire
> 	ssh for security
> 	can have multiple streams in action at once
> 	transfere whole or partial files
> 	restartable
> 	
> and the list goes on.  Once you start using it correctly, it is  
> extremely flexible, fast and works really really well.
> 
> If you need assistance, or want to see our backup scripts, I'm more  
> than happy to help or post info.
> 
> Stu.
> 
> 
> 
> On 21/04/2006, at 12:20, W.Kenworthy wrote:
> 
> > rsync is great for partial backups, but fails on large whole disks  
> > with
> > may thousands of files.  This is what I use for the partial backups I
> > do.  Unison is also good (uses the rsync protocol), but also doesnt do
> > large disks well.  Also, they dont have a compressed target.
> >
> > BillK
> 
> --
> Dr Stuart Midgley
> Industry Uptake Program Leader
> iVEC, 'The hub of advanced computing in Western Australia'
> 26 Dick Perry Avenue, Technology Park
> Kensington WA 6151
> Australia
> 
> Phone: +61 8 6436 8545
> Fax: +61 8 6436 8555
> Email: industry at ivec.org
> WWW:  http://www.ivec.org
> 
> 
> 
> _______________________________________________
> PLUG discussion list: plug at plug.org.au
> http://www.plug.org.au/mailman/listinfo/plug
> Committee e-mail: committee at plug.linux.org.au