Wednesday, February 19, 2020

Pushing a file from Host into an to LXD Container

One of the little (and deliberate) papercuts of using unprivileged LXD containers is that unless data flows in from a network connection, it likely has the wrong owner and permissions.

Here are two examples in the HomeAssistant container.

1. The HA container needs to talk to a USB dongle elsewhere in the building. It does so using USBIP, and I discussed how to make it work in this previous post.

2. I want the HA container to display some performance data about the host (uptime, RAM used, similar excitements). Of course, it's a container, so it simply cannot do that natively without using lots of jiggerypokery to escape the container. Instead, a script collects the information and pushes the information into the container every few minutes.

     $ sudo lxc file push /path/to/host/file.json container-name/path/to/container/

Easy enough, right.

Well, not quite. Home Assistant, when installed, creates a non-root user, and puts all of it's files in a subdirectory. Add another directory to keep things simple, and you get:

     /home/homeassistant/.homeassistant/external_files/

And, unfortunately, all those subdirectories are owned by a non-root user. So lxc cannot 'push' all the way into them (result: permission error).

    -rw-r--r-- 1   root root  154 Feb 19 15:34 file.json

The pushed file can only be pushed to in the wrong location, and gets there with the wrong ownership.



Systemd to the rescue: Let's create a systemd job on the container that listens for a push, then fixes the location and the ownership.

The feature is called a systemd.path.

Like a systemd timer it consists of two parts, a trigger (.path) and a service that gets triggered.

The .path file is very simple. Here's what I used for the trigger:

[Unit]
# /etc/systemd/system/server_status.path
Description=Listener for a new server status file

[Path]
PathModified=/home/homeassistant/.homeassistant/file.json

[Install]
WantedBy=multi-user.target

The service file is almost as simple. Here's what I used:

[Unit]
# /etc/systemd/system/server_status.service
Description=Move and CHOWN the server status file

[Service]
Type=oneshot
User=root
ExecStartPre=mv /home/homeassistant/.homeassistant/file.json /home/homeassistant/.homeassistant/external_files/
ExecStart=chown homeassistant:homeassistant /home/homeassistant/.homeassistant/external_files/file.json

[Install]
WantedBy=multi-user.target

Finally, enable and start the path (not the service)

sudo systemctl daemon-reload
sudo systemctl enable server_status.path
sudo systemctl start server_status.path


Sunday, February 2, 2020

Advanced Unattended Upgrade (Ubuntu): Chrome and Plex examples

Updated Aug 26, 2020

This is a question that pops up occasionally in various support forums:

Why doesn't (Ubuntu) Unattended Upgrades work for all applications? How can I get it to work for my application?

Good question.

Here is what happens under the hood: The default settings for Unattended Upgrades are for only packages in the "-security" pocket of the Ubuntu repositories.

Not "-updates", not "-backports", not "-universe", not any third-party repositories, not any PPAs. Just "-security".

This is a deliberately conservative choice -- while the Ubuntu Security Team keeps it's delta as small as possible, it's a historical fact that even small security patches have (unintentionally) introduced new bugs.



Here's how you can override that choice. 

Let's take a look at the top section of file /etc/apt/apt.conf.d/50unattended-upgrades, and focus on the "Allowed-Origins section." It's edited for clarity here:

Unattended-Upgrade::Allowed-Origins {
     "${distro_id}:${distro_codename}";
     "${distro_id}:${distro_codename}-security";
//   "${distro_id}:${distro_codename}-updates";
//   "${distro_id}:${distro_codename}-proposed";
//   "${distro_id}:${distro_codename}-backports";
};

There, you can see the various Ubuntu repo pockets.

You can also see that most of the options are commented out (the "//"). If you know how to use a basic text editor and sudo, you can safely change those settings. Warning: You can break your system quite horribly by enabling the wrong source. Enabling "-proposed" and other testing sources is a very bad idea.



How to add the -updates pocket of the Ubuntu Repos?

I've done this for years, BUT (this is important) I don't add lots of extra sources. Simply uncomment the line.

   "${distro_id}:${distro_codename}-updates";

That's all. When Unattended Upgrades runs next, it will load the new settings.

Bonus: Here's one way to do it using sed:

   sed -i 's~//\(."${distro_id}:${distro_codename}-updates";\)~\1~' /etc/apt/apt.conf.d/50unattended-upgrades


How to add the -universe pocket of the Ubuntu Repos?

You can create a '-universe' line like the others, but it won't do anything. It's already handled by the "-updates" line.



How to add a generic new repository that's not in the Ubuntu Repos?

Add a line in the format to the end of the section:

    //    "${distro_id}:${distro_codename}-backports";
    "origin:section"       <-------- Add this format
    };

The trick is finding out what the "origin" and "section" strings should be.

Step 1: Find the URL of the source that you want to add. It's located somewhere in /etc/apt/sources.list or /etc/apt/sources.list.* . It looks something like this...

    deb http://security.ubuntu.com/ubuntu eoan-security main restricted universe multiverse
      ...or...
    deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main
      ...or...
    deb https://downloads.plex.tv/repo/deb/ public main

Step 2: Find the corresponding Release file in your system for the URL.

    http://security.ubuntu.com/ubuntu eoan-security
      ...becomes...
    /var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_eoan-security_InRelease


    http://dl.google.com/linux/chrome/deb/ stable
      ...becomes...
    /var/lib/apt/lists/dl.google.com_linux_chrome_deb_dists_stable_InRelease


    https://downloads.plex.tv/repo/deb/ public
      ...becomes...
    /var/lib/apt/lists/downloads.plex.tv_repo_deb_dists_public_Release

Step 3: Use grep to find the "Origin" string.

    $ grep Origin /var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_eoan-security_InRelease
    Origin: Ubuntu

    $ grep Origin /var/lib/apt/lists/dl.google.com_linux_chrome_deb_dists_stable_InRelease
    Origin: Google LLC

    $ grep Origin /var/lib/apt/lists/downloads.plex.tv_repo_deb_dists_public_Release
    Origin: Artifactory

Step 4: With the Origin string and Section (after the space in the URL), we have all the information we need:

    "Ubuntu:eoan-security"
       ...or...
    "Google LLC:stable"
       ...or...
    "Artifactory:public"

You're ready to add the appropriate string to the config file.

Bonus: Here's one way to isolate most of these using shell script

    package="google-chrome-stable"
    url=$(apt-cache policy $package | grep "500 http://")
    var_path=$(echo $url | sed 's~/~_~g' | \
           sed 's~500 http:__\([a-z0-9._]*\) \([a-z0-9]*\)_.*~/var/lib/apt/lists/\1_dists_\2_InRelease~')
    origin=$(grep "Origin:" $var_path | cut -d" " -f2)
    section=$(echo $url | sed 's~500 http://\([a-z0-9._/]*\) \([a-z0-9]*\)/.*~\2~')
    echo "$origin":"$section"

Step 5: Run Unattended Upgrades once, then check the log to make sure Unattended Upgrades accepted the change.

    $ sudo unattended-upgrade
    $ less /var/log/unattended-upgrades/unattended-upgrades.log   (sometimes sudo may be needed)

You are looking for a recent line like:

    2020-02-02 13:36:23,165 INFO Allowed origins are: o=Ubuntu,a=eoan, o=Ubuntu,a=eoan-security, o=UbuntuESM,a=eoan, o=UbuntuESM,a=eoan-security, o=UbuntuESM,a=eoan-security

Your new source and section should be listed.



Summary for folks who just want to know how to update Chrome (stable)

  1. Edit (using sudo and a text editor) the file /etc/apt/apt.conf.d/50unattended-upgrades
  2. In the section "Unattended-Upgrade::Allowed-Origins {", add the following line BEFORE the final "};"
    "Google LLC:stable"


Summary for folks who just want to know how to update Plex

  1. Edit (using sudo and a text editor) the file /etc/apt/apt.conf.d/50unattended-upgrades 
  2. In the section "Unattended-Upgrade::Allowed-Origins {", add the following line BEFORE the final "};"
    "Artifactory:public"