Sunday, August 16, 2020

Installing Home Assistant Core in an LXD Container (Part 2)

Last time, we built a basic LXD container, and then build HomeAssistant inside.

This time, we're going to add a few more elements.

  • We're going to do all the steps on the Host instead of diving inside the container. So we're going to use lxc exec and lxc push. The goal is to make spinning up a new container scriptable
  • We're going to start/stop the HomeAssistant application using a systemd service
  • We're going to keep the data and config outside the container and use an lxd disk device to mount the data. Even if we destroy the container, the data and config survive to be mounted another day.

Preparing LXD

We're going to skip LXD initialization in this example. There's one addition from last time: We're going to add shiftfs, which permits us to chown mounted data. The macvlan profile and shiftfs enablement are persistent -- if you already have them, you don't need to redo them. All of these commands occur on the Host (we have not created the container yet!)

   # Create a macvlan profile, so the container will get it's IP Address from
   # the router instead of the host. This works on ethernet, but often not on wifi 
   ip route show default 0.0.0.0/0
   lxc profile copy default lanprofile
   lxc profile device set lanprofile eth0 nictype macvlan
   lxc profile device set lanprofile eth0 parent enp3s5

   # Test that macvlan networking is set up
   lxc profile show lanprofile
     config: {}
     description: Default LXD profile  // Copied. Not really the default
     devices:
       eth0:                           // Name, not real device
         nictype: macvlan              // Correct network type
         parent: enp3s5                // Correct real device
         type: nic

   # Enable shiftfs in LXD so data mounts work properly
   sudo snap set lxd shiftfs.enable=true
   sudo systemctl reload snap.lxd.daemon

   # Test that shiftfs is enabled:
   Host$ lxc info | grep shiftfs
    shiftfs: "true"

Create the Container and Initial Configuration

If LXD is already set up, then start here. We will mount the external data location, set the timezone and do all that apt setup. But this time, we will do all the commands on the Host instead of inside the container. We will also create the sources.list file on the host and push it into the container.

   # Create the container named "ha"
   lxc launch -p lanprofile ubuntu:focal ha

   # Mount the existing HomeAssistant data directory
   # Skip on the first run, since there won't be anything to mount
   # Shiftfs is needed, else the mounted data is owned by nobody:nogroup
   # Chown is needed because shiftfs changes the owner to 'ubuntu'
   lxc config device add ha data_mount disk source=/somewhere/else/.homeassistant path=/root/ha_data
   lxc config device set ha data_mount shift=true
   lxc exec ha -- chown -R root:root /root

   # Set the timezone non-interactively
   lxc exec ha -- ln -fs /usr/share/zoneinfo/US/Central /etc/localtime
   lxc exec ha -- dpkg-reconfigure -f noninteractive tzdata

   # Reduce apt sources to Main and Universe only
   # Create the new sources.list file on the host in /tmp
   # Paste all of these lines at once into the Host terminal
   cat <<EOF > /tmp/container-sources.list
   deb http://us.archive.ubuntu.com/ubuntu/ focal main universe
   deb http://us.archive.ubuntu.com/ubuntu/ focal-updates main universe
   deb http://security.ubuntu.com/ubuntu focal-security main universe
   EOF

   # Push the file into the container
   lxc file push /tmp/container-sources.list ha/etc/apt/sources.list

   # Apt removals and additions
   lxc exec ha -- apt autoremove openssh-server
   lxc exec ha -- apt update
   lxc exec ha -- apt upgrade
   lxc exec ha -- apt install python3-pip python3-venv

Create the Venv, Build HomeAssistant, and Test

This method is simpler than all that mucking around activating and venv and paying attention to your prompt. All these command are issued on the Host. You don't need a container shell prompt.

   # Setup the homeassistant venv in a dir called 'ha_system'
   #We will use the root account since it's an unprivileged container.
   lxc exec ha -- python3 -m venv --system-site-packages /root/ha_system

   # Build and install HomeAssistant
   lxc exec ha -- /root/ha_system/bin/pip3 install homeassistant

   # Learn the container's IP address. Need this for the web browser. 
   lxc list | grep ha

   # Run HomeAssistant
   lxc exec ha -- /root/ha_system/bin/hass -c "/root/ha_data"

   # Use your browser to open the the IP address:8123
   # HA takes a couple minutes to start up. Be patient.
   # Stop the server from within the Web UI or ^C to exit when done.

Start HomeAssistant at Boot (Container Startup)

The right way to do autostart is a systemd service file on the container. Like with the sources.list file, we will create it on the host, then push it into the container, then enable it. There's one optional ExecPreStart line - it will slow each startup slightly while it checks for and installs updates.

   cat <<EOF > /tmp/container-homeassistant.service
   [Unit]
   Description=Home Assistant
   After=network-online.target

   [Service]
   Type=simple
   User=root
   PermissionsStartOnly=true
   ExecPreStart=/root/ha_system/bin/pip3 install --upgrade homeassistant
   ExecStart=/root/ha_system/bin/hass -c "/root/ha_data"

   [Install]
   WantedBy=multi-user.target
   EOF

   # Push the .service file into the container, and enable it
   lxc file push /tmp/container-homeassistant.service ha/etc/systemd/system/homeassistant.service
   lxc exec ha -- systemctl --system daemon-reload
   lxc exec ha -- systemctl enable homeassistant.service
   lxc exec ha -- systemctl start homeassistant.service

Now we can test it. The last command should start HA. The same command with 'stop' should gracefully stop HA. Restarting the container should gracefully stop HA, and then restart it automatically. Your web browser UI should pick up each stop and start. You did it!


Final Notes

Remember how you start without any HomeAssitant data to mount? Now that you have a running HA Core, you can save a set of data:

   lxc file pull ha/root/.homeassistant /somewhere/else/.homeassistant --recursive

And remember to clean up your mess when youare done:

   lxc stop ha
   lxc delete ha

Saturday, August 15, 2020

Installing Home Assistant Core in an LXD Container (Part 1)

I've been running HomeAssistant Core reliably in an LXD container for almost two years now, so it's probably time to start detailing how to do it.

This is a step-by-step example of how to do it for folks who aren't very familiar with LXD containers and their features.

Installing LXD (documentation)

If you haven't used LXD before, you need to install it (it's a Snap) and initialize it (tell it where the storage is located). The initialization defaults are sane, so you should not have problems.

   sudo snap install lxd
   sudo lxd init

Container Profile: Macvlan Networking (optional)

A macvlan profile is one easy way for the container to get it's IP address from the router instead of the host. This means you can use a MAC Address filter to issue a permanent IP address. This works on ethernet, but often not on wifi. You only need to set up this profile ONCE, and it's easiest to do BEFORE creating the container. Since the container doesn't exist yet, all of these commands are done on the Host.

   # Get the real ethernet device (enp3s5 or some such)
   ip route show default 0.0.0.0/0

   # Make mistakes on a copy
   lxc profile copy default lanprofile

   # Change nictype field to macvlan
   #  'eth0' is a virtual device, not a real eth device
   lxc profile device set lanprofile eth0 nictype macvlan

   # Change parent field to real eth interface
   lxc profile device set lanprofile eth0 parent enp3s5

Create the Container

Create a new container named 'ha'. This command is done on the Host.

   # Create the container named "ha"
   lxc launch -p lanprofile ubuntu:focal ha

   # Learn the container's IP address. Need this for the web browser. 
   lxc list | grep ha

   # Get a root shell prompt inside the container
   lxc shell ha

Initial Setup in the Container

Let's get a shell set up timezone and apt. These commands are done on the Container root prompt.


   // This is one way to set the timezone
   dpkg-reconfigure tzdata

   // Reduce apt sources to Main and Universe only
   cat <<EOF > /etc/apt/sources.list
   deb http://us.archive.ubuntu.com/ubuntu/ focal main universe
   deb http://us.archive.ubuntu.com/ubuntu/ focal-updates main universe
   deb http://security.ubuntu.com/ubuntu focal-security main universe
   EOF

   // Tweak: Remove openssh-server
   apt autoremove openssh-server

   // Populate the apt package database and bring the container packages up-to-date
   apt update
   apt upgrade
   
   // Install the python packages needed for HomeAssistant
   apt install python3-pip python3-venv

   # Setup the homeassistant venv in the root home dir (/root)
   # --system-site-packages allows the venv to use the many deb packages that are already
   #    installed as dependencies instead of donwnloading pip duplicates 
   python3 -m venv --system-site-packages /root

Install and Run HomeAssistant

Now we move into a virtual environment inside the container, build HomeAssistant, and give it a first run. If you try to build or run HomeAssistant outside the venv, it will fail with cryptic errors.

   // Activate the installed venv. Notice how the prompt changes.
   root@ha:~# source bin/activate
   (root) root@ha:~#
   
   // Initial build of HomeAssistant. This takes a few minutes.
   (root) root@ha:~# python3 -m pip install homeassistant

   // Instead of first build, this is where you would upgrade
   (root) root@ha:~# python3 -m pip install --upgrade homeassistant

   // Initial run to set up and test.
   (root) root@ha:~# hass

   // After a minute or two, open the IP Address (port 8123). Example: http://192.168.1.18:8123
   // Use the Web UI to shut down the application. Or use CTRL+C.

   // Exit the venv
   (root) root@ha:~# deactivate

   // Exit the container and return to the Host shell.
   root@ha:~# exit
   Host:~$


There's a lot more to talk about in future posts:

  • The systemd service that starts HomeAssistant at container startup.
  • Creating an LXD disk device to keep the HomeAssistant data in. If I rebuild the container for some reason, I can simply connect it to the data.
  • Adding a USBIP client. The Z-Wave controller is elsewhere in the building, and USBIP lets me control it like it's attached to the host. That also means adding a USB device to the container.
  • Collecting Host hearbeat statistics for the HomeAssistant dashboard, and pushing those into the container regularly.
  • Backing up and restoring HomeAssistant data and configurations.

Friday, August 14, 2020

LXD Containers on a Home Server

LXD Containers are very handy, and I use them for quite a few services on my home hobby & fun server. Here's how I set up my containers after a year of experimenting. Your mileage will vary, of course. You may have very different preferences than I do.

1. Networking:

I use macvlan networking. It's a simple, reliable, low-overhead way to pull an IP address from the network DHCP server (router). I set the IP address of many machines on my network at the router.

The container and server cannot communicate using TCP/UDP with each other. I don't mind that.

You only need to set up this profile once for all containers. Simply specify the profile when creating a new container.

   // 'Host:$' means the shell user prompt on the LXD host system. It's not a shell command

   // Learn the eth interface: enp3s5 in this example
   Host:$ ip route show default 0.0.0.0/0

   // Make mistakes on a copy
   Host:$ lxc profile copy default lanprofile

   // Change nictype field. 'eth0' is a virtual device, not a real eth device
   Host:$ lxc profile device set lanprofile eth0 nictype macvlan

   // Change parent field to real eth interface
   Host:$ lxc profile device set lanprofile eth0 parent enp3s5

   // Let's test the changes
   Host:$ lxc profile show lanprofile
     config: {}
     description: Default LXD profile  // This field is copied. Not really the default
     devices:
       eth0:                           // Virtual device
         nictype: macvlan              // Correct network type
         parent: enp3s5                // Correct real device
         type: nic
       root:
         path: /
         pool: containers-disk         // Your pool will be different, of course
         type: disk
     name: lanprofile


2. Creating a Container

Create a new container called 'newcon':

   Host:$ lxc launch -p lanprofile ubuntu:focal newcon
      // 'Host:$'        - user (non-root) shell prompt on the LXD host
      // '-p lanprofile' - use the macvlan networking profile
      // 'focal'         - Ubuntu 20.04. Substitute any release you like


3. Set the Time Zone

The default time zone is UTC. Let's fix that. Here are two easy ways to set the timezone: (source)

   // Get a root prompt within the container for configuration
   // Then use the classic Debian interactive tool:
   Host:$ lxc shell newcon
   newcon:# dpkg-reconfigure tzdata

   // Alternately, here's a non-interactive way to do it entirely on the host
   Host:$ lxc exec newcon -- ln -fs /usr/share/zoneinfo/US/Central /etc/localtime
   Host:$ lxc exec newcon -- dpkg-reconfigure -f noninteractive tzdata


4. Remove SSH Server

We can access the container from the server at anytime. So most containers don't need an SSH server. Here are two ways to remove it

   // Inside the container
   newcon:# apt autoremove openssh-server 
   
   // Or from the Host
   Host:$ lxc exec newcon -- apt autoremove openssh-server


5. Limit Apt sources to what the container will actually use

Unlike setting the timezone properly, this is *important*. If you do this right, the container will update itself automatically for as long as the release of Ubuntu is supported (mark your calendar!) If you don't get this right, you will leave yourself an ongoing maintenance headache.

   // Limit the apt sources to (in this example) main from within the container
   newcon:# nano /etc/apt/sources.list
         // The final product should look similar to:
         deb http://archive.ubuntu.com/ubuntu focal main           
         deb http://archive.ubuntu.com/ubuntu focal-updates main           
         deb http://security.ubuntu.com/ubuntu focal-security main 

   // Alternately, *push* a new sources.list file from the host.
   # Create the new sources.list file on the host in /tmp
   cat <<EOF > /tmp/container-sources.list
   deb http://us.archive.ubuntu.com/ubuntu/ focal main
   deb http://us.archive.ubuntu.com/ubuntu/ focal-updates main
   deb http://security.ubuntu.com/ubuntu focal-security main
   EOF
   
   // *Push* the file from host to container
   Host:$ lxc file push /tmp/container-sources.list newcon/etc/apt/sources.list


6. Install the Application

How you do this depends upon the application and how it's packaged.



7. Update Unattended Upgrades

This is the secret sauce that keeps your container up-to-date. First, let's look at a cleaned-up version of the first 20-or-so lines of /etc/apt/apt.conf.d/50unattended-upgrades inside the container:

                    What it says                             What it means
           ------------------------------------------      -----------------------
   Unattended-Upgrade::Allowed-Origins {
           "${distro_id}:${distro_codename}";              Ubuntu:focal
           "${distro_id}:${distro_codename}-security";     Ubuntu:focal-security
   //      "${distro_id}:${distro_codename}-updates";      Ubuntu:focal-updates
   //      "${distro_id}:${distro_codename}-proposed";     Ubuntu:focal-proposed
   //      "${distro_id}:${distro_codename}-backports";    Ubuntu:focal-backports
   };

...why, those are just the normal repositories! -security is enabled (good), but -updates is disabled (bad). Let's fix that. Inside the container, that's just using an editor to remove the commenting ("//"). From the host, it's a substitution job for sed:

   Host:$ lxc exec newcon -- sed "s\/\ \g" /etc/apt/apt.conf.d/50unattended-upgrades

Third-party sources need to be updated, too. This is usually easiest from within the container. See this post for how and where to update Unattended Upgrades with the third-party source information.



8. Mounting External Media

Some containers need disk access. A classic example is a media server that needs access to that hard drive full of disorganized music.

If the disk is available across the network instead of locally, then use plain old sshfs or samba to mount the network share in /etc/fstab.

If the disk is local, then first mount it on the Host. After it's mounted, use an lxd disk device inside the container. A disk device is an all-in-one service: It creates the mount point inside the container and does the mounting. It's persistent across reboots...as long as the disk is mounted on the host.

   // Mount disk on the host and test
   Host:$ sudo mount /dev/sda1 /media
   Host:$ ls /media
      books         movies       music

   // Create disk device called "media_mount" and test
   Host:$ lxc config device add newcon media_mount disk source=/media path=/Shared_Media
   Host:$ lxc exec newcon -- ls /Shared_Media
      books         movies       music

If the ownership of files on the disk is confused, and you get "permisson denied" errors, then use shiftfs to do the equivalent of remounting without suid

   Host:$ lxc exec newcon -- ls /Shared_Media/books
      permission denied

   // Enable shiftfs in LXD, reload the lxd daemon, and test
   Host$ sudo snap set lxd shiftfs.enable=true
   Host$ sudo systemctl reload snap.lxd.daemon
   Host$ lxc info | grep shiftfs
    shiftfs: "true"

   // Add shiftsfs to the disk device
   Host$ lxc config device set newcon media_mount shift=true

   Host:$ lxc exec newcon -- ls /Shared_Media/books
      boring_books       exciting_books        comic_books        cookbooks