[plug] Constant rsync'ing...

[Bill Kenworthy]

Try unison as has been reccomended a number of times now.  It caches the
info so the first time it runs its slower than rsync, and it is also
slower to transfer the data: *BUT* the next time it is quite fast if you
only have a few changes.  i.e., 6.3g takes only a few seconds across a
100M network, only a little longer across a modem - any changes are on
top of the checking of course!  3 way syncs are no problem: I do by
using one machine as the master, the other two sync to that.  Master is
purely a convenience, as its easier to work that way, than try and sync
fully meshed.

BillK

On Mon, 2003-03-17 at 14:26, Trevor Phillips wrote:
> On Thursday 13 March 2003 17:56, James Devenish wrote:
> >
> > If you ask a question like "triggered by the filesystem itself" without
> > describing what operating system, hardware platform, and filesystem you
> > are using then you are surely indicating to us that you want answers
> > about the Linux/i386/ext2 combination. I don't know about that.
> 
> Sorry. Linux/i386/ext3 - although the FS format is negotiable. I'd prefer a 
> solution that works per-directory, rather than per-partition, but I'll still 
> consider a per-partition solution if it's worth it.
> 
> > In the case of the latter, I will *again* mention that Unison fits this
> > bill. In fact, I imagine it can do the former if you disable 'fastcheck'
> > (but obviously that means it has to compare file contents on both
> > machines and that would be slow). Fastcheck could potentially overlook
> > changes (unlikely unless you actually set out to thwarte it), though it
> > always does a 'safe check' before overwriting any files with supposed
> > "new" versions.
> 
> I considered Unison, and keep meaning to try it out in this role. Would it 
> really be any faster than rsync's "building file list" phase, though?
> 
> One of the complications is this isn't just 2 machines - it's a cluster of 3, 
> and so the changes need to be replicated from one primary server to 2 other 
> machines. This is part of the problem with rsync - it does the "building file 
> list" twice on the current "server", to replicate to both the other machines 
> acting as clients.
> 
> A "find /top/directory" takes about 30 seconds to run (and finds, for one of 
> the directories, over 154 thousand files).
> 
> I should set some time aside to grab a coupla PCs and try mirroring of the 
> same quantity of files between them using different techniques...
> 
> On Monday 17 March 2003 13:00, Tony Breeds wrote:
> >
> > For 2.4 series kernels look at intermezzo.  It's not 100% of what you're
> > after but I think it'll comes close enough.
> >
> > http://www.inter-mezzo.org/  for mor info.
> 
> Intermezzo looks cool and scary. ^_^
> 
> It looks more like what I had in mind, although it does operate per-partition, 
> which would require a considerable change in structure.
> 
> Somewhat related to Intermezzo - does anyone know if it's possible to rescale 
> a loop-back filesystem semi-dynamically? eg; You create a 100Mb file, ext3 
> it, Intermezzo it, then 3 months later decide it'd be better to have it 
> 200Mb. Any way to "parted" the file (short of creating a new one of the new 
> size, duplicating content, etc...)? ^_^
-- 
Bill Kenworthy <billk at iinet.net.au>