Warning "Contains linux content" Was: Re: [OT] Re: [plug] File Conversions

[Brad Campbell]

Daniel Foote wrote:
>> OBLC: I had a fun weekend converting an entire Debian etch system to 
>> boot and run from initramfs
>> rather than an nfs root. It's pretty minimal but the initramfs is 
>> about 60MB and loads over PXE. The
>> whole thing runs from tmpfs and does the job very nicely. Live and 
>> learn, I wondered if it could be
>> done.
> 
> If I may, how did you make that work? Just the other day I set up two
> machines to netboot and use a remote NFS root (the same NFS root - the
> debian package nfs-booted has a number of tweaks to make this work
> properly - creating RAM disks for things like /var/log).

nfs-booted you say.. I must check that out.

I have 2 storage boxes here, one has 15 250GB drives in a RAID-6 and one has 4 200GB in a RAID-5. I 
used to have 2 complete copies of the nfs-root and mount them separately. Now I use a single 
initramfs and a very early script in rcS.d that sets whatever parameters are different between the 2 
boxes based on eth0 mac address (mainly hostname actually).

The kernel is non-modular and I have the drivers for both boxes built in, so PXE loads the one 
kernel then loads the initramfs and away they go.

> I was also a few weeks ago trying to netboot an ancient 386 just for
> kicks, and was thinking of using a very small initrd for it's root. So
> far, I've made the machine boot a kernel, but that's about it. I have
> two things working against me: the machine has 8MB of RAM, and Debian
> is compiled for 486 or better machines these days... resulting in code
> that won't run on that machine... I had to manually compile a kernel
> for it, but 2.6.21 boots quite happily on it.

Hrm.. you will be pretty tight, even my Belkin ADSL router has 16mb of ram. An nfs root will be ok, 
but you *will* need swap, and swap over nfs particularly on small memory machines has a tendency to 
deadlock. I had better luck with swap over nbd but there is still a theoretical deadlock there (not 
that I ever hit it).

My biggest issue converting a debian install to initramfs was making udev work properly, and that 
was a matter of adding a single line in /etc/udev/udev.conf

no_static_dev="1"

Once I did that it "just worked".

I've had a play with a few extra bits and bobs to make the root visible (which it's not by default 
if you just use a single initramfs). Without a visible root, apt-get would not run as it could not 
tell how much space it has to play with.

To get around that I had to generate an /init and I recompiled busybox-static to include 
switch_root, then I just did this..


brad at storage2:/etc/udev$ cat /init
#!/bin/bash

echo Creating Root
mkdir /newroot
echo Mounting Root
mount -t tmpfs none /newroot
echo Copying Root
cp -ax / /newroot/
mkdir -p /newroot/lib/modules/`uname -r`/
touch /newroot/lib/modules/`uname -r`/modules.dep
echo Going to switch root. Hang on to your jubblies!
exec /newroot/bin/busybox switch_root /newroot /sbin/init

Now I can use the machine as a fully functioning debian system all running from ram. No changes are 
ever persistent (which is what I wanted) so I can apt-get install packages and play with them 
knowing full well that next time I reboot they'll be gone.

If I want to make a permanent change, I chroot into the master environment on the server, make my 
changes, updates, additions or deletions then exit out of that and run the ./build script which 
generates an updated initramfs. Reboot the two boxes and they are running the latest built environment.

It's ugly, but it works. Here is the build script.

brad at storage2:/etc/udev$ cat /build
#!/bin/bash

if [ "`id -u`" != 0 ] ; then
         echo You are not root!
         exit 1
fi;
REV=`cat etc/revision.number`
REV=$((REV+1))
echo > etc/motd.tail
echo Brads Storage System Revision : $REV >> etc/motd.tail
echo Last built : `date` >> etc/motd.tail
echo >> etc/motd.tail
echo Building new initrd
rm var/lib/dpkg/*-old
rm var/cache/apt/*.bin
rm var/lib/apt/*
find . | grep -v "usr/share/doc/" | cpio -o -H newc | gzip | pipebench >/tftpboot/storage-initrd.gz.1
mv /tftpboot/storage-initrd.gz.1 /tftpboot/storage-initrd.gz
echo Done building revision $REV
echo $REV > etc/revision.number


Honestly I love PXE booting. The reason I did it this way was I've enabled tickless on these 
machines (as they are idle a lot) and with nfs-root they were waking up 100-200 times per second 
doing stat() on some file(s) on the root which was hammering the network and keeping the power 
consumption up. Now with them running completely from RAM I'm down to about 15 wakeups/sec. I know 
why it's that high but I've not bothered fixing it just yet. Probably saved about 10-20W all up but 
the challenge was fun.

Oh, one machine has 1GB and the other has 1.5GB so running from RAM is no hassle at all, they are 
only nfs servers after all.

Brad
-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams