Installing Debian 6 onto an SD card, installing into an old VIA EPIA-5000L motherboard

In this post I discussed my new-to-me fanless motherboard, and how I got it to turn on/off, and test the boot.

Next step is to install a hard drive and install Debian 6.

Hardware

This hardware will eventually become a router or light-duty server, so the OS doesn't need huge storage. I used:

• SD-to-IDE adapter (ebay, $8.88)
• The adapter reqires a FDD (floppy disk) power connector. Since my power supply doesn't have that connector, I picked up a molex (IDE HDD power) to FDD adapter (amazon, $3.92)
• 8GB SD card (left over from a now-dead camera). An SD card can also be plugged in to my Ubuntu 11.10 laptop. This turned out to be critically useful for me

Install Failures

Using the same method as this post to make bootable USB drives, I tried the latest Debian 6 businesscard and netinstall disks. Both failed at various points. Sometimes the iso was corrupted during download. Sometimes there was a failure writing the new partition table. Sometimes the table was successfully written, but the new partition couldn't be mounted for install. Sometimes the install was successful, but grub failed to install.

One by one, I winnowed through the issues. MD5 for the iso downloads. Pull the card and reformat it in my laptop to fix the partition table. Finally, I got consistent failures: The businesscard iso failed during the install debootstrap, and the netinstall iso failed to create an initramfs and grub.

The businesscard iso could successfully drop to a working shell, so I used these instructions to manually install debootstrap and see if I could bypass the installer failures. This worked well, though slowly (writing to the SD card is much slower than to a real hard drive), and my many mistakes after this point were a learning experience in filesystem corruption, chroot, initramfs, and grub.

fsck, chroot, initramfs, and grub

Booting to a live environment, dropping to a shell, debootstrapping to an installed system, and then rebooting to test it...only to discover it's unbootable...take a long time for each cycle, mostly due to the SD card slow write speed, and the SD card's seeming love of getting it's filesystem corrupted somewhere in the process each time.

I sped up the process and injected filesystem checks by moving most of it onto a Debian VM on my laptop. The SD card mounts as a USB drive, so the VM must be USB-capable. Lesson Learned: Before mounting/unmounting the card, fsck it! Every time! This is how I got rid of the filesystem corruption, which seemed to be happening from something unclean during shutdown (almost true - I'll tell you the real story later). On some occasions the filesystem was so messed up that I simply wiped it and copied a new system over from the VM.

Now, I could reliably build a system that would boot...but grub would complain or initramfs would drop to a shell. On two occasions it successfully booted, then froze (and corrupted the filesystem). Testing with variations in /boot/grub/grub.cfg fixed grub to boot reliably...only to later learn that grub 'error' wasn't an error at all, but a different configuration problem I had caused somewhere else.

That left a frustrating initramfs refusal to mount the card. I could mount it manually, but that resulted in kernel panic. After a lot of digging through initramfs scripts, I discovered a packaging bug - initramfs relies on klibc-tools...but the dependency is not listed in Debian 6! (It is correct in Ubuntu 10.10). One apt-get later and the system booted reliably every time...but the filesystem was frequently corrupting.

Apt activities that triggered man-db began returning "fopen: Permission denied" errors. Fixed with chown -R man:root /var/cache/man


And the result...

I finally determined that the frequent filesystem corruptions were due to a defective SD-to-IDE adapter. I tested the card reader on my laptop - no problems, so it wasn't the laptop, the card, or my handling. The proof was when the SD card was freshly installed, fsck'd before removal from the laptop, fsck'd again upon insertion to the Epia system, and run for an hour. Missing and corrupted file errors began to proliferate. Still need to check the RAM.


Lessons Learned
1) fsck the SD card before every mount and after every unmount
2) Manually check for the existence of /boot/grub/grub.cfg, /boot/grub/normal.mod, and /sbin/init
3) Use blkid to get the UUID of the root partition. Check that against /boot/grub/grub.cfg

Steps to succesfully creating the SD card
Without wandering down my many blind paths, here is the final process to install a bootable Debian 6 onto an SD card.

1) Use a VM (I used Virtualbox) with USB access on a fast machine (my laptop).
2) Within the VM, create a chroot environment and use debootstrap to create the new linux system that will be copied onto the card.
3) Prepare the SD card, copy the environment, and install grub.
4) Ensure a good system after the first boot from the card.

Creating a VM is beyond the scope of this discussion.

Creating the new linux system happens entirely on the VM. Create the base debian install

# mkdir /var/card-env
# debootstrap --arch=i386 squeeze /var/card-env

Depending on your network connection, debootstrap may take a long time.


Mount the system-required stuff

# mkdir /var/card-env/proc
# mount --bind /proc /var/card-env/proc
# mkdir /var/card-env/dev
# mount --bind /dev /var/card-env/dev
# mount --bind /dev/pts /var/card-env/dev-pts
# mkdir /var/card-env/sys
# mount --bind /sys /var/card-env/sys

Copy useful stuff before chrooting


These files must be edited later in the process

# cp /etc/fstab /var/card-env/etc/
# cp /etc/network/interfaces /var/card-env/etc/network/
# cp /etc/hosts /var/card-env/etc/

Chroot into the card-env

# chroot /var/card-env

Set up the package manager. Do these in the chroot environment.

# echo "deb-src http://ftp.us.debian.org/debian squeeze main" >> /etc/apt/sources.list
# echo "deb http://security.debian.org/ squeeze/updates main" >> /etc/apt/sources.list
# echo "deb-src http://security.debian.org/ squeeze/updates main" >> /etc/apt/sources.list
# apt-get update

If any apt-get errors pop up, take time and fix them now. Do these in the chroot environment.

If any command-not-found errors occur:

Use 'which commandname' to find the full path
for example, 'which date' returns '/bin/date'
Use dpkg -S full-path to find the package name
for example, 'dpkg -S /bin/date' returns 'coreutils'
Reinstall that package
for example, 'apt-get remove coreutils && apt-get install coreutils'

Edit hostname, hosts, and interfaces. Do these in the chroot environment

# echo MyNewHostName > /etc/hostname
# nano /etc/hosts                # Change the hostname
# nano /etc/network/interfaces   # Reduce interfaces to just to loopback device
                               # udev will add the rest anyway upon boot

Add your root and user accounts. Do these in the chroot environment

# passwd root
# adduser MyName

Add essential elements to the new filesystem. Do these in the chroot environment

# apt-get install locales
# dpkg-reconfigure locales     # Choose a UTF-8 in your preferred language
# apt-cache search linux-image # Choose a kernel that matches your architecture
# apt-get install klibc-tools initramfs-tools linux-image-VERSION-i486 grub-pc

Take a few minutes and troubleshoot any kernel-install or initramfs issues. Do this in the chroot environment. Grub will have a lot of incorrect information, and we'll fix that after we plug in the SD card.

Add desired optional packages. Do this in the chroot environment

# apt-get install openssh-server     # For example

Exit the chroot environment, then unmount the system-essentials

# exit
# umount /var/card-env/proc
# umount /var/card-env/dev-pts
# umount /var/card-env/dev
# umount /var/card-env/sys

You now have a complete Debian environment (except for grub and fstab), ready to copy from the VM onto the SD card.

Plug in the SD card to the VM machine, and add the SD card (as a USB) to the VM. Leave it unmounted. udev assigned my card as /dev/sdb.

Partition the card as desired. I created /dev/sdb1 (bootable ext3), and /dev/sdb5 (swap). Do these steps as root. Card must be plugged in but unmounted for these steps:

# mke2fs -j /dev/sdb1   # Assuming you haven't formatted it yet
# mkswap /sev/sdb5   # Assuming you haven't formatted it yet

Mount the card

# fsck /dev/sdb1
# mount /dev/sdb1 /mnt

Change grub's device map to the SD Card

# ls /dev/disk/by_id    # Note the SD Card's ID string
# nano /var/card-env/boot/grub/device.map  # Change HD0 to the SD card

Change grub's config file and fstab to the SD Card partition

# blkid        # Note the linux partition's UUID string
# nano /var/card-env/boot/grub/grub.cfg    # Change all the UUID strings to the SD Card Partition
# nano /var/card-env/etc/fstab             # Change the UUID strings on the root and swap lines

Copy the environment over to the card

# cp /var/card-env/* /mnt/
# grub-install --root-directory /mnt /dev/sdb

Final checks

# blkid   # Note the UUID of the card boot partition
# nano /mnt/boot/grub/grub.cfg   # Check for the same UUID
# ls -l /mnt/sbin/init    # Check for existence
# ls /mnt/boot            # Check for an initrd.img file and a vmlinuz file

Unmount and fsck the SD card

# umount /mnt
# fsck /dev/sdb1

You now have a complete bootable Debian 6 installed on the SD card, ready to try booting from.

During the first boot from the SD card, do the following:

# update-initramfs -u  # Get rid of "Can't find disk" errors
# tune2fs -c 1 /dev/sda # Set the flag to run fsck every boot until further notice
# ifconfig -a   # Note the MAC address and interface of the ethernet port

Keep the current system interfaces, delete leftovers from the laptop environment

# nano /etc/udev/rules.d/70-persistent-net-rules

Bring up the network and prepare apt-get

# ifconfig up eth1  # Your eth may differ
# dhclient eth1
# apt-get update
# apt-get upgrade

Add essential system components

# apt-get install rsyslog