Populating Initramfs with BusyBox and uClibc http://busybox.net/~landley/ols2006 Who the heck am I? Rob Landley BusyBox maintainer (officially since February, unofficially after 1.0) Wrote Documentation/filesystems/ramfs-rootfs-initramfs.txt in kernel. slide 1: Introduction differences between initramfs and initrd: ramfs vs block - less memory - dynamic resizing, no limits - rootfs always mounted for same reason pid1 unkillable. archive vs image - no filesystem driver - ramfs tiny, always loaded in 2.6. Wrapper around vfs cache. bundled vs unbundled - both available to ramfs init vs linuxrc - linuxrc sucky. - Not PID 1. - unpredictable PID, killable, zombies, etc. - strange flow control, unnecessarily complicated. - setup then return to kernel so it can mount root? (nfs?) - you're right there already, just do it yourself. slide 2: Introduction advantages of initramfs - fundamental sequencing problem: mount "/" before running init. - Device enumeration complicated. (USB, anyone? SCSI longstanding) - NFS mounts require dhcp and login. - encrypt, compress, - modular filesystem driver, build into kernel. Magic format. Special. - root ext3. Initrd ext2 _not_ ext3, journal on ramdisk! Build in ext2 driver then can't get rid of it. - Binary only firmware for some devices (network cards). - Too cheap to spring for ram or flash. - Require a cross-compiler for mips to build your kernel? - initramfs has no chicken and egg problems ala nfs mounts - very simple implementation. (Small, cross-platform.) - One file. Bootloader simple to set up. - actually init - no assumptions about what you're going to do. Stay running. - root mount not "magic". Built-in root, ignorable. - moves more control to userspace - laptops, embedded systems, clusters, mainframes slide 3: Build a "hello world" program to put in initramfs. #include #include int main(int argc, char *argv[]) { printf("Hello world!\n"); sleep(9999999); return 0; } gcc hello.c -o hello ./hello Sleep instead of returning avoids "kernel panic, tried to kill init" later. slide 4: Get the "hello world" program to run in a chroot environment. Shared libraries: The executable doesn't work by itself: chroot . /hello Use the ldd command to find shared libraries. (I made a script.) mkdir walrus ./mkchroot.sh walrus a.out chroot walrus /a.out It's big: du walrus: 1.3 megabytes. Plus your kernel. For hello world. gets gzipped down to 600k, but still. strip - library already stripped with --strip-unneeded -Os, we're optimizing the idle task here. Statically link: gcc hello.c -o hello -static -s A 400k hello world. That's glibc for you. Can we do better? slide 5: Get the size down by building against uClibc. - gcc sucks, but no time to go into depth and breadth of gcc's suckage. - lots of ways to get a uClibc toolchain buildroot.uClibc.org, gentoo embedded, I did one (landley.net/code/firmware) Here are some prebuilt ones, via my day job: http://crossdev.timesys.com/download/latest/i686/toolchain/i686-linux-toolchain-fat.tar.gz Extract into root directory, as root. (Yeah, I know.) They don't supply an ldd that works with my script. (use readelf -d) PATH=/opt/timesys/toolchains/i686-linux/bin/:$PATH i686-uclibc-gcc \ hello.c -o hello -Os -s hello + lib/ld-uClibc.so.0 + lib/libc.so.0 252k: smaller than statically linked hello world from glibc. Build statically against uClibc (add -static): 7k slide 6: Build a User Mode Linux kernel to try out initramfs. Get current kernel source, configure (I'm using miniconfig), build UML: wget http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.17.tar.bz2 tar xjvf linux-2.6.17.tar.bz2 patch -p1 -i ../miniconfig.patch make miniconfig MINICONFIG=../linux-miniconfig-uml ARCH=um optional: make ARCH=um menuconfig (general setup->small systems->load all) make ARCH=um note: CONFIG_HZ error means it's distclean time. You have leftover ARCH=i386 symlinks in your build directory. Try again. Boot User Mode Linux with hostfs, demonstrate basic User Mode Linux. ./linux rootfstype=hostfs rw init=/bin/sh whoami mount -t procfs /proc /proc cat /proc/cpuinfo ps cat/proc/mounts exit - panics the kernel when PID 1 exits, but who cares? slide 7: Build our "hello world" initramfs into UML as directory. - Grab above statically linked hello. As a directory: echo 'CONFIG_INITRAMFS_SOURCE="../walrus"' >> ../linux-miniconfig-uml make ARCH=um miniconfig MINICONFIG=../linux-miniconfig-uml make ARCH=um - Make it actually run: ./linux - doesn't find it, because it's not called /init, so: ./linux rdinit=/hello - no console! Downside of the directory method, must make /dev/console as root. mknod dev/console c 5 1 The default initramfs has a /dev/console in it. Note for my fellow ubuntu users, to fix a bug in 2.6.17 kernel: sed -i 's/gawk/awk/' scripts/gen_initramfs_list.sh - make ARCH=um; ./linux rdinit=/hello kill it from another window, UML puts console into raw mode, but PID 1 not killable. slide 8: Build our "hello world" initramfs into UML as script. - scripts/gen_initramfs_list.sh creates a file list. Using file list to create archive doesn't require root. Don't ask me why it isn't chmod +x, I dunno. - point CONFIG_INITRAMFS_SOURCE at script, it'll autodetect type. - Grab the statically generated script from last time Slide 9: Build cpio archive yourself. Create: cd "$1"; find . | cpio -o -H newc | gzip - yeah, you have to do it from the current directory. Extract: cpio -i -d -H newc -F initramfs_data.cpio --no-absolute-filenames slide 10: Build a static busybox for initramfs. - Build a statically linked busybox, run it as standalone shell. wget http://busybox.net/downloads/busybox-1.2.0.tar.bz2 tar xvjf busybox-1.2.0.tar.bz2 cd busybox-1.2.0 make defconfig make menuconfig -> switch on static in first menu make Now run it as standalone shell under UML with rdinit= - Transfer to new RFS (hostfs) with switch_root, by hand. Slide 11: Now boot a real kernel under QEMU. - Build a real kernel, run under QEMU. Package external initramfs with cpio, show initrd=. Show builtin too. Build second initrd with data and show initrd overlay on builtin base. echo -e "lash\ndev/console" | cpio -o -H newc | gzip > blah.cpio.gz qemu /dev/zero -kernel arch/i386/boot/bzImage (-nographic append="console=ttyS0") (-rdinit blah.cpio) Slide: Do real things with it: - shared libraries, directory layout, /dev, /proc, /sys - Setup quick init script. mdev -s, reproduce root= - The firmware Firmware Linux initramfs. - encfs Slide: Things there won't be time to do: Extracting initramfs image from existing kernel. - Cross-compile for PPC (kernel, busybox, uClibc) - Talk about klibc. - uClibc full fledged replacement for glibc. Build anything against it. klibc isn't. Doesn't even build BusyBox. - Talk about the many ways that, gcc sucks. The uClibc guys recommend building gcc from source. (I run sed against the source code.) There's a wrapper too, and various ways it can break. Newer gccs break less, but can still break. (libgcc_s) Klibc also uses the wrapper.