Upstart Jobs at login

Login is not the same is startup. Let's just get that out of the way first.

• Startup is the time between boot and the login screen. It's the habitat of system jobs.
• Login is the time after you enter your password. It's the habitat of user jobs.

The easy way to run a task at login is to run a script from your .bashrc.
And the (deceptively not-) easy way to run a task at logout is to run a script from your .bash_logout

But today we're not doing it the easy way. Today we're going to use dbus and Upstart.

Emitting Upstart Signals from your .bashrc

It's terribly easy.

1) Emit a user-level Upstart signal by adding a line to .bashrc:

# Upstart signal that .bashrc is running
initctl emit "I_AM_LOGGING_IN"

2) Add a user-level Upstart job to ~.config/upstart/ for one user, or to /usr/share/upstart/sessions/ for all users:

# /home/$USER/.config/upstart/login_test.conf
description "login test"
start on I_AM_LOGGING_IN            # Start criteria
exec /bin/date > /tmp/login_test    # Do something

3) Open a new terminal window (to load the new .bashrc). When you open the window, the Upstart job creates the tempfile at /tmp/login_test.

Clean up: Restore your bashrc, and delete the sample Upstart job.

Can I emit system-level Upstart signals from .bashrc?

Not directly. The script runs as a user, not as root.

You can use a secondary method of triggering system-level Upstart signals, like sending a Dbus signal, or manipulating a file, or connecting to a socket.

Can I emit Upstart signals from .bash_logout?

No.

Using initctl emit in .bash_logout will merely result in an error. The user-level Upstart daemon seems to be terminated before .bash_logout is run. The command will return a cryptic "Rejected send message"error from PID 1 (system Upstart). Since .bash_logout is not running as root, it cannot emit system-level signals.

Also, GUI terminal programs do not not run .bash_logout, unless you specify compatibility (with a flag) when you start.

That easy way of doing login actions is still too hard

Boy, are you difficult to please.

Okay, there is an even easier way, but it's more complicated to explain: Instead of .bashrc emitting an Upstart event, let Upstart listen for a dbus signal.

Here is an example of the dbus message that occurs when I login via SSH to a new session. This signal is emitted by systend-logind every time a new TTY, ssh, or X-based GUI login occurs.

The signal is not emitted when you are in a GUI environment and simply open a terminal window - that's not a login, that's a spawn of your already-existing GUI environment:

signal sender=:1.3 -> dest=(null destination) serial=497 
  path=/org/freedesktop/login1; interface=org.freedesktop.login1.Manager;
  member=SessionNew
    string "4"
    object path "/org/freedesktop/login1/session/_34"

The important elements are the source, the "SessionNew" signal, and the path of the new session.

Aside, let's query systemd-logind to find if the login is to a TTY, X session, or SSH. logind has lots of useful information about each session:

$ dbus-send --system                              \ 
            --dest=org.freedesktop.login1         \
            --print-reply                         \
            --type=method_call                    \
            /org/freedesktop/login1/session/_34   \ # Path from the signal
            org.freedesktop.DBus.Properties.Get   \
            string:org.freedesktop.login1.Session \
            string:Service
method return sender=:1.3 -> dest=:1.211 reply_serial=2
   variant       string "sshd"

It's right. I did connect using ssh.

Now let's construct an Upstart job that runs when I login via a TTY, X Session, or SSH. We will use Upstart's built-in dbus listener.

# /home/$USER/.config/upstart/login_test.conf
description "login test"
start on dbus SIGNAL=SessionNew     # Listen for the dbus Signal
exec /bin/date > /tmp/login_test    # Do something

• Now, whenever you login to a TTY, X session, or SSH session, the job will run.
• If your job needs to tell the difference between those sessions, you know how to find out using dbus.
• If *everybody* needs the job, place it in /usr/share/upstart/sessions/ instead of each user's .config/upstart/

What about super-easy logout jobs?

Logout jobs are harder, and generally not recommended. Not super-easy. They are hard because you can't guarantee they will run. Maybe the user will hold down the power button. Or use the "shutdown -h now" command. Or the power supply sent a message that the battery only has 60 seconds of life left. Or the user absolutely cannot miss that bus....

Here's the dbus signal that systemd-logind emits when a TTY, X, or SSH user session ends:

signal sender=:1.21 -> dest=(null destination) serial=286 
  path=/org/freedesktop/Accounts/User1000; 
  interface=org.freedesktop.Accounts.User; member=Changed

All this tells me is that User1000 now has a different number of sessions running. Maybe it's a login (yes, it emits the same signal upon login). Maybe it's a logout.

Sure, we can do a login-and-logout Upstart job...

# /home/$USER/.config/upstart/login_test.conf
description "login and logout test"
start on dbus SIGNAL=Changed INTERFACE=org.freedesktop.Accounts.User
exec /bin/date > /tmp/login_test

...but then you need logic to figure out who logged in or logged out, and whether it's an event you care about. Certainly doable, but probably not worthwhile for most users.

In other words, if you want to backup-at-logout, you need to structure it as a backup-then-logout sequence. Logout is not an appropriate trigger to start the sequence...from the system's point of view.

But I really want to do a job a logout!

Okay, here's how to do a job when you log out of the GUI environment. Logging out is the trigger. This won't work for SSH or TTY sessions.

The Upstart jobs /etc/init/lightdm.conf and/etc/init/gdm.conf emit a system-level "desktop-shutdown" signal when the X server is stopped. You can use that job as your start criteria.

# /etc/init/logoff_test.conf
description "logout test"
setuid some_username        # Your script probably doesn't need to run as root
start on stopping lightdm   # Run *before* it is stopped
exec /bin/date > /tmp/logoff_test

The amazing simplicity of sshfs

So I'm out for a few minutes, waiting in a lounge during a children's choir rehearsal. And I want something off my home server.

sudo apt-get install sshfs
ssh_port=12345                  # Use your own port number, of course
server_name=myserver.dyndns.org # Use your own server, of course
user_name=myname                # Your username on the server
local_mount_point=/path/to/local/mountpoint
mkdir ${local_mount_point}
fwknop -A 'tcp/${ssh_port}' -s -D ${server_name}
sshfs ${user_name}@${server_name}:/path/on/server ${local_mount_point} -p ${ssh_port}

And SHAZAM! Remote server path shows up in File Manager.

No mucking with Samba or PPTP or other strangeness. Just

sshfs me@myserver.dyndns.org:/mnt/Common/ /tmp/server/ -p 12345

To unmount:

fusermount -u ${local_mount_point}

Firewall improvement with fwknop

fwknop is an improvement upon the older technique of port-knocking. Quite simply, it looks like an ordinary failed attempt to connect blocked by a closed firewall...but something about the packet identifies it as trustable, and causes the host system to do an action (like briefly open a firewall port).

There are safeguards to prevent the packet from being spoofed, and until the packet is received my SSH port stays safely blocked by the firewall - preventing the use of clever undiscovered exploits.

This is pretty cool because it means that my ssh port can be safely closed behind a firewall, yet still accessible to me (and only me) on the internet.

There are a whole bunch of moving pieces here (firewall, ports, services, client, keys), so I'll talk a lot of detail about the installation and configuration, the some options, then try an example or two. I haven't even touched the options for using my GPG-key for knocking!


Planning: Ahead of time, fill in this list:

• I want the knock to occur on UDP port #: (default 62201)
• I want be able to open ports for the following services: (example: sshd)
• Those services run on ports: (example: tcp/22)
• I want my knock password (key) to be: (8 characters min -or- GPG)
• I want to be notified of a successful knock by: (log/email/none)
• The interface I want to protect is: (only one, usually the same internet-facing interface described in your firewall)

Install and configure fwknop on the server:

1. Server Install:

   apt-get install fwknop-server   # Debian

2. Enable fwknopd to autostart upon boot. Edit /etc/default/fwknop-server

   START_DAEMON="no"  #From
   START_DAEMON="yes" #To

3. Customize the server setup by editing /etc/fwknop/fwknop.conf:
   

   EMAIL_ADDRESSES             root@localhost;  # From
   EMAIL_ADDRESSES             MyLoginAccountName@localhost;  #to
   
   PCAP_INTF                   eth0;  # From
   PCAP_INTF                   ppp0;  # To
   
   PCAP_FILTER                 udp port 62201;                         # From
   PCAP_FILTER                 udp port <some other port number>; # To

   PCAP_INTF should be the same internet-facing interface used in your firewall (for example, ppp0). You can only choose ONE interface. In this case, ppp0 protects the internet-facing interface only so it does not protect against, say, hackers on the LAN wifi.
   PCAP_FILTER is the udp port to listen for the knock on. Remember that PCAP_FILTER port number - you'll need that again on the client.
   
4. Set the server access controls and keys by editing /etc/fwknop/access.conf:
   

   # Change all three of the following, if needed.
   OPEN_PORTS: tcp/22,tcp/23,tcp/24,udp/85;
   KEY: <Your-8-digit-key>;
   FW_ACCESS_TIMEOUT: 30;

   OPEN_PORTS is the list of ports fwknopd is permitted to open. The client will request a specific port on the list. Remember that 8-digit key, you'll need it again on the client.
   
5. Edit the firewall to close all fwknop-protected ports. The fwknopd daemon will tell iptables to open the port as needed.

   iptables -A INPUT -i lan0 -p tcp -dport 22 -j ACCEPT
   iptables -A INPUT -i ppp0 -p tcp -dport 22 -j REJECT

   The first line opens the firewall for ssh from the LAN (not protected by fwknopd). The second line closes the firewall for ssh from the internet-facing ppp0. If the default policy is already DROP or REJECT, you don't need a separate entry at all and the second line can be deleted. fwknopd can open a port regardless of the default policy or any explicit closure rule.
   
6. Restart fwknopd (and optionally the firewall) to load the new config(s):
   

   fwknopd -Restart

Optional Server Notes:

1. E-mail spamming of successful knocks: I got tired of that pretty fast, so I moved the notifications to a log:
   • In /etc/fwknop/fwknop.conf, change ALERTING_METHODS from ALL; to noemail;
   • Create a new file /etc/rsyslog.d/fwknop.conf:

     # Log fwknopd actions to syslog, auth.log, and iptables.log
     :msg, contains, "fwkno" -/var/log/auth.log
     :msg, contains, "fwkno" -/var/log/iptables.log #my own custom iptables-related log

   • Restart rsyslogd to reload the new config: service rsyslog restart
2. Server Testing Tip: To prevent locking yourself out of the server (and feeling like a chump), here's a way to test fwknop without erroneously locking yourself out:
   
   • Edit /etc/ssh/sshd.conf to add a second ssh port (let's say port 2233)
   • Restart sshd to load the new config: service ssh restart
   • Use netstat -tulpn to ensure sshd is really listening on both ports
   • Edit your firewall so the new port (2233) is either explicitly closed or closed by the default policy (it doesn't matter which as long as it's closed). If necessary, reload your firewall to apply the changes.
   • Configure fwknopd to listen on your LAN interface, and only to control the ssh test port.
   • Do your testing and experimenting. Your original ssh port is unaffected.
   • When testing is complete, edit /etc/ssh/sshd.conf to remove the testing port, remove any testing-port related lines in your firewall, and edit /etc/fwknopd/access.conf to remove the test port.

Install and configure fwknop on the client:

1. Client Install:

   sudo apt-get install fwknop-client   # Ubuntu

2. Usage with manually-entered key:
   

   $fwknop --Server-port 12345 --Access 'tcp/22' -R --Destination myserver.dyndns.org
   Enter your key: <Your 8-digit key>
   $ssh -p 22 myserver.dyndns.org

   Server-port is the PCAP_FILTER, the UDP port to send the knock on.
   Access is the firewall port to open. It must be on the server's OPEN_PORTS list.
   The 8-digit key must be the same, of course, as on the server.
   The followup ssh command to connect on the briefly-open port is just an example.
3. Create a keyfile for automatic use (optional): Use the format servername: key

   echo "myserver.dyndns.org: <Your 8-digit-key> > .knock-keyfile

4. Usage with keyfile key (optional - handy for scripting):

   $fwknop --Server-port 12345 --Access 'tcp/22' -R --Destination myserver.dyndns.org --get-key .knock-keyfile
   $ssh -p 22 myserver.dyndns.org

Access your 24-hour IRC client/logbot from anywhere

I use irssi (IRC client) running on a 24/7 server to maintain my IRC presence. When I want to connect to IRC, I ssh into the server, and attach into the running IRC session.

ssh me@myserver.local   # Log into the server
screen -r            # Attach to the irssi session
#(Do stuff)
#CTRL+a, CTRL+d to detach irssi
exit                 # Logout of the server

It's a little different if I want to log in from elsewhere on the internet:

ssh -p PortNumber me@myserver.dyndns.org    # Log into the server
screen -r            # Attach to the irssi session
#(Do stuff)
#CTRL+a, CTRL+d to detach irssi
exit                 # Logout of the server

Here's one way to simplify it:

1. ssh takes in-session commands:ssh destination "command1; command2"
2. ssh has a -t flag, forcing a pseudo-terminal connection (screen needs this to work)
3. bash uses basic if/them logic a couple ways.
4. determine the network using the 'route' command

So you can set up an ssh command like ssh -t me@myserver 'screen -dr pts-0' to connect to the server and then the screen session. (I already knew that my screen session was at pts-0; it tells you when you detach). When you detach from the screen session (CTRL+a, CTRL+d), the ssh session closes automatically.

One more piece of the puzzle is my network logic - I connect differently if on the LAN or out in the wider world. I use route | grep -q myserver.local to determine if I'm on the home network or not (return is '0' if home network, '1' if internet). Of course, your server name will vary.

The easy, logical result:

LocalNetwork=$(route | grep -q myserver.local)
if $LocalNetwork; then
   ssh -t me@myserver.local 'screen -dr pts-0'
else:
   ssh -t -p PortNumber me@myserver.dyndns.org 'screen -dr pts-0'
fi

We can simplify this even further using bash logical expressions instead of if/then/else statements. It's a single line, but more cryptic:

route | grep -q myserver.local && ssh -t me@myserver.local 'screen -dr pts-0' || \
ssh -t -p PortNumber me@myserver.dyndns.org 'screen -dr pts-0'

SSH suddenly stops working

I tried to ssh to my XO laptop today, but got the following error:

me:~$ssh user@(IP addr of xo_laptop)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
blah:blah:blah:blah:blah:blah:blah:blah:blah:blah.
Please contact your system administrator.
Add correct host key in /home/USER/.ssh/known_hosts to get rid of this message.
Offending key in /home/USER/.ssh/known_hosts:NUMBER
RSA host key for (IP addr of xo_laptop) has changed and you have requested strict checking.
Host key verification failed.
me:~$

Why? Probably because I upgraded the OLPC laptop.

How to fix it?

1. Use ssh-keygen -R (IP addr) to purge the old key.
2. Connect again like normal ssh user@xo_laptop to install the new (current) key.