[plug] Debmirror or what?

[Chris Caston]

On Wed, 2004-05-12 at 23:59, Harry wrote:
> On Wed, 12 May 2004 23:07:50 +0800 Cameron Patrick
> <cameron at patrick.wattle.id.au> wrote:
> 
> > | What is the best way to achieve my goal?
> > 
> > debmirror is probably the easiest way.  The other alternative is
> > The Rsync Command Line From Hell:
> > 
> > rsync   --recursive --times --links --hard-links \
> > 	--verbose --compress --timeout=120 \
> > 	--delete --delete-after --delete-excluded \
> > 	--exclude 'Contents-*.gz' \
> > 	--exclude binary-arm/         --exclude "*_arm.deb" \
> [...]
> > You can't narrow it down to just one of stable/testing/unstable this
> > way, though.
> 
> Because you are going to rsync all files from the pool without context
> of which distribution they are part of, is this right ? I can see the
> advantage of deb-mirror rather than using rsync as a blunt instrument. 
> 
> >  This command will also get you the source for
> > everything, though you can get rid of it by adding more --exclude
> > bits.
> 
> The exclude switches can be put into an exclude file so a simple rsync
> script looks like:
> 
> #!/bin/sh
> if  ( pidof -x rsync > /dev/null ) ; then
>   echo "Already running"
> else
> 
> /usr/bin/rsync -rltvz --stats --delete --delete-excluded --partial
> --exclude-from=/root/bin/rsync-exclude.txt "ftp.it.net.au::debian/*"
> /mirror/debian/ > /tmp/rsync-au.log
> 
> /usr/bin/rsync -rltvz --stats --delete --delete-excluded --partial
> --exclude-from=/root/bin/rsync-exclude.txt "ftp.it.net.au::debian-non-US/*"
> /mirror/debian-non-US/ >/tmp/rsync-au-us.log
> 
> fi
> 
> You can tail -f /tmp/rsync-au.log and watch progress.
> 
> I can send you my exclude file originally given to me by James
> Bromberger and since tweaked as the architectures and naming changed.
> I don't think James would mind. The last rsync I ran at poledra (many
> thanks to Informed Technology) was a few weeks ago and builds an 18.3G
> mirror which is woody/sarge/sid of binary i386 only.  
> 
> If you consistently install identical packages, how about looking at
> apt-proxy ? I haven't used it but I remember Scott from LinuxIT
> gave it good reports for their office debian installations. It
> will maintain a much more compact mirror (proxy) but I guess it may need to
> have access to an external mirror so it can check it's versions are fresh.
>  

That's a good idea. Perhaps it's worth trying out all the options. Each
one is probably best for a different situation. 

Sid gets a lot of updates so it may not be a good idea to sync every
single package all the time.

> All the best
> Harry
> 
> It might be fun to setup a few servers at the installfest so people can
> park a machine and rsync a mirror. Even 18G might be served to a few
> machines if they have all day to do it. Once you have a base mirror,
> keeping it up to date isn't as difficult.

That's a good idea. Perhaps if the mirror has a gigabit nic and some
SATA ports it will speed things up a bit. I now use external SATA drives
and a PCI SATA controller for backing up peoples systems in place of USB
2.0 ext drives. 

thanks,

Chris Caston
-- 
Linux is ready for the desktop like a Boeing F-22 is ready for the
run-way.