Saturday, June 9, 2012

How to recompile the linux kernel for specific hardware in Debian 6

I have Debian 6 installed on my mini-ITX system.

I want to try recompiling the kernel to speed up boot and reduce the footprint on the slow, old system. Eventually, I want to use the lessons learned to recompile Ubuntu 12.04 to work on the system. Currently, 12.04 will not boot, since the old VIA 32-bit processor does not fully support x686 nor pae.

If the kernel and modules are small enough, and customized for the equipment, we can even get rid of the initrd at boot. That will also speed up the boot a lot!

Step 1: Collect CPU and module information 

We can compile the CPU for the specific processor and modules we actually use. On such a resource-constrained system, getting rid of a lot of unused bloat is welcome.

The following command shows that we have a VIA Samuel 2 processor, 533MHz, 32-bit system.

$ cat /proc/cpuinfo 
processor : 0
vendor_id : CentaurHauls
cpu family : 6
model  : 7
model name : VIA Samuel 2
stepping : 3
cpu MHz  : 533.370
cache size : 64 KB
fdiv_bug : no
hlt_bug  : no
f00f_bug : no
coma_bug : no
fpu  : yes
fpu_exception : yes
cpuid level : 1
wp  : yes
flags  : fpu de tsc msr cx8 mtrr pge mmx 3dnow
bogomips : 1066.74
clflush size : 32
cache_alignment : 32
address sizes : 32 bits physical, 32 bits virtual
power management:

The following command lists most of the modules used (modules used during boot then unloaded do not show up here).

$ lsmod
Module                  Size  Used by
fuse                   43213  1 
snd_via82xx            15020  0 
gameport                6105  1 snd_via82xx
snd_ac97_codec         79248  1 snd_via82xx
ac97_bus                 710  1 snd_ac97_codec
snd_mpu401_uart         4155  1 snd_via82xx
snd_rawmidi            12737  1 snd_mpu401_uart
snd_pcm                46098  2 snd_via82xx,snd_ac97_codec
snd_seq_device          3661  1 snd_rawmidi
snd_timer              12304  1 snd_pcm
via_ircc               13188  0 
snd                    34142  7 snd_via82xx,snd_ac97_codec,snd_mpu401_uart,snd_rawmidi,snd_pcm,snd_seq_device,snd_timer
vt8231                  8678  0 
irda                   71690  1 via_ircc
snd_page_alloc          5021  2 snd_via82xx,snd_pcm
i2c_viapro              4387  0 
shpchp                 21168  0 
parport_pc             15747  0 
evdev                   5520  4 
parport                22170  1 parport_pc
button                  3578  0 
processor              23003  1 
pcspkr                  1235  0 
serio_raw               2912  0 
i2c_core               12739  1 i2c_viapro
soundcore               3294  1 snd
crc_ccitt               1039  1 irda
pci_hotplug            18493  1 shpchp
ext3                   92928  3 
jbd                    28166  1 ext3
mbcache                 3482  1 ext3
sd_mod                 25961  5 
crc_t10dif              1012  1 sd_mod
ata_generic             2239  0 
fan                     2590  0 
pata_via                5721  4 
uhci_hcd               16021  0 
libata                117427  2 ata_generic,pata_via
via_rhine              14427  0 
thermal                 9210  0 
ehci_hcd               28859  0 
thermal_sys             9346  3 processor,fan,thermal
scsi_mod              105921  2 sd_mod,libata
mii                     2654  1 via_rhine
usbcore                99061  3 uhci_hcd,ehci_hcd
nls_base                4445  1 usbcore

The following command tells us that the current version of the kernel in use is 2.6.32-5-486:

$ uname -a
Linux mini-deb6 2.6.32-5-486 #1 Sun May 6 03:29:22 UTC 2012 i686 GNU/Linux

Step 2: Install the kernel source

The kernel source code is needed. "Recompiling" a kernel actually just means compiling a new version of the kernel from source. The process is covered very well in this forum post. We will install all the needed packages for the recompile, install the linux source code too /usr/src, and copy the existing old kernel config so we have a nice baseline if we mess it up.

# apt-get install ncurses-dev kernel-package cpio initramfs-tools linux-base module-init-tools udev linux-source-2.6.32-5-486
# cd /usr/src
# tar -xjf linux-source-*.tar.bz2
# ln -s linux-source-* linux
# cd linux
# cp .config .config-original-2.6.32

One tiny extra step - we'll be manually rebooting at the end of the process, so I reset my grub timeout to 15 seconds so I have time to select the kernel I want without rushing. Grub will grab this new value when it automatically updates in Step 4.

# nano /etc/default/grub
(change the line GRUB TIMEOUT= )

Step 3: Edit the config file, then compile

Now we're ready to modify the .config file.

# make menuconfig 

Menuconfig takes about an hour, perhaps more. To build a really minimal kernel, go through and switch all of the options off. Next, go back and look at the list of modules. Search for each module (search is the forward-slash key '/'), and turn each back on. Make sure each is compiled into the kernel, not added as a module. Some options require you to search for prerequisites and turn those on first. Like I said, it took me over an hour to work through all the modules on the list.

When finished, save the config. Then we're ready to compile:

# make-kpkg clean
# make-kpkg --append-to-version=-custom1 kernel_image kernel_headers

Iterate the version number (-custom1 to -custom2) or you'll lose track of progress very quickly! If you want an initrd, add --initrd to the line.

Compiling on my slow machine takes about 2.5 hours. I run make-kpkg inside a screen session. I can ssh in, start compiling, detach the screen, and return in a few hours to see the result.

Step 4: Install the new kernel

The make-kpkg command creates a lovely new package for us in /usr/src. Install it.

# ls -l /usr/src
total 97892
lrwxrwxrwx  1 root root       19 Jun  2 19:20 linux -> linux-source-2.6.32
-rw-r--r--  1 root root  6060196 Jun  2 13:56 linux-headers-2.6.32-custom1_2.6.32-custom1-10.00.Custom_i386.deb
-rw-r--r--  1 root root  4170548 Jun  2 13:47 linux-image-2.6.32-custom1_2.6.32-custom1-10.00.Custom_i386.deb
drwxr-xr-x 25 root root     4096 Jun  9 14:02 linux-source-2.6.32
-rw-r--r--  1 root root 65220157 May  5 18:19 linux-source-2.6.32.tar.bz2
drwx------  2 root root    16384 May 27 17:36 lost+found

# dpkg -i  /usr/src/linux-image-2.6.32-custom1_2.6.32-custom1-10.00.Custom_i386.deb

The install will also create an updated initrd (if you asked for one) and add the appropriate grub entries. If you changed the Grub Timeout value in Step 2, the update will reflect that, too.

Change one setting in the new /etc/grub/grub.cfg. Normally, grub tells the kernel to load the boot drive read-only, then initrd unmounts it and remounts it read-write. Since we don't have an initrd, we must change that grub behavior. For the first few iterations of kernel testing, this doesn't matter - it wont affect anything until you reach busybox prompts because the root filesystem cannot be loaded:

# nano /etc/grub/grub.cfg

Look for the root=/dev/sda1 ro stanza in the last line of the appropriate menuentry, and change ro (read only) to rw (read/write). Here's an example of an unchanged menuentry:

menuentry 'Debian GNU/Linux, with Linux 2.6.32-custom8' --class debian --class gnu-linux --class gnu --class os {
        insmod part_gpt
        insmod ext2
        set root='(hd0,gpt1)'
        search --no-floppy --fs-uuid --set 89b0fb32-b7e3-46e8-878b-5da4cd69fbdd
        echo    'Loading Linux 2.6.32-custom8 ...'
        linux   /boot/vmlinuz-2.6.32-custom8 root=/dev/sda1 ro console=tty1 console=ttyS0,115200n8

Reboot into the new kernel.

# shutdown -r now

Step 5: Lather, Rinse, Repeat

Keep the hardware handy - you need to see the console boot messages, and you need access to the power switch.

The first few times you try this, the kernel is likely to fail. Either you forgot to add a module, or you set something wrong. Record the error off the screen, and jump into a search engine to see what it means.

Kernel failures fall into several broad categories. Each is usually due to the same cause - a misconfigured kernel .config file. Happily, these are sequential. As you move down the list, you're getting closer to final success.
- The easiest to diagnose is the simple refusal to start. The kernel fails a basic start test, and tells you in plain language that it won't start because it doesn't support this processor or the processor doesn't support this service, etc.
- Once the kernel passes those start tests, the next set of failures are kernel panics. This means that the kernel has loaded, but some piece of hardware or initrd instruuction or filesystem has mystified it, so it barfs a bunch of diagnostic information and freezes.
- Once the kernel is over its panic attacks, the next set of failures are dropping to the busybox prompt when the root filesystem fails to be recognized or loaded. This usually means either a misconfigured root= or UUID= in grub, or a missing or incorrect filesystem module.
- Finally, the system boots and works...mostly. Some application or piece of hardware doesn't work, though.

Reboot into the original working kernel (hard reboot, using the power switch).

Remove the non-working kernel. This will also remove the appropriate initrd (if any), and appropriate grub entries.

# apt-get remove linux-image-2.6.32-custom1

Return to Step 3, change the config file to fix the problem, and compile again.

Step 6: The new kernel works!

Once the new kernel works, clean up. Backup the working config file, remove the source code package, and delete the 'linux' symlink so a future recompile doesn't accidentally overwrite this working version.

# cp .config .config-2.6.32-custom5   # or whatever custom iteration number you want
# apt-get remove linux-source-2.6.32
# cd /usr/src
# rm linux

Now you can brag to your friends about how easy it was...

Let's take a quick look at the old and new kernels side-by-side:

$ ls -l /boot
total 17052
-rw-r--r-- 1 root root  110922 May  6 07:49 config-2.6.32-5-486
-rw-r--r-- 1 root root   55311 Jun 10 08:54 config-2.6.32-custom8
drwxr-xr-x 3 root root    4096 Jun 10 11:48 grub
-rw-r--r-- 1 root root 8722934 May 27 15:54 initrd.img-2.6.32-5-486
-rw-r--r-- 1 root root 1249814 May  6 07:49
-rw-r--r-- 1 root root 1333763 Jun 10 11:00
-rw-r--r-- 1 root root 2192352 May  6 07:48 vmlinuz-2.6.32-5-486
-rw-r--r-- 1 root root 3734224 Jun 10 11:00 vmlinuz-2.6.32-custom8

The actual kernel, vmlinuz, is 1.5MB (70%) larger, but no longer has an initrd (saves 8.7 MB) or /lib/modules directory (saves 67 kB). Boot time savings seems negligible so far

Here is the final working config file.

No comments: