Friday, November 27, 2009

Quick and Easy CD-ripping

Ripping a CD to add to my music collection using the command-line application abcde.

To install abcde, use sudo apt-get abcde

To rip a CD, insert the CD and use the terminal. This will rip the CD to FLAC (lossless) format, and eject the CD when complete:

cd Music
abcde -x -o flac

Sunday, November 15, 2009

Linux converter for Microsoft .lit files

.lit is a proprietary format, and must be converted using the C-lit application, included in the 'epub-utils' package. It can then be read with the 'fbreader' e-book reader.

To install epub-utils and fbreader, use sudo apt-get epub-utils fbreader

To convert a book file, use lit2epub /path/to/book.lit

To read a converted book, open fbreader and point it to /path/to/book.epub

Monday, July 13, 2009

Video, photos, and music sharing with my phone

I have a snazzy new geek-phone, a Shuttle from Virgin Mobile. It has a 4GB MicroSD slot so files can be exchanged with my Xubuntu system.

  • Phone Video is in .3G2 format. The video plays in Totem, but the sound doesn't. YouTube properly uploads and converts the .3G2 videos (no need to share by e-mail). I plan to archive the .3G2 videos in their original format. This command successfully converts the .3G2 to a .avi for viewing by totem:
    $ mencoder media/disk/my_flix/inputfile.3g2 -ovc lavc -lavcopts vcodec=msmpeg4v2 -oac mp3lame -lameopts vbr=3 -o Videos/outputfile.avi
  • Computer Video Haven't figured out the right command to convert movies to .3G2 yet.
  • Phone Pictures are in .jpg format. Xubuntu's standard Image Viewer is very effective for weeding through them.
  • Computer Pictures Large pictures take a long time to show up - a 2.4 MB image froze the phone for 30 seconds. The following script uses Imagemagick to resize the photos to a smaller size:
    $ convert Pictures/filename -size 320x240 /media/disk/my_pix/filename.jpg
  • Computer Music My FLAC music needs to be converted to .mp3 to play on the phone. Soundconverter does it, but some results refuse to play...haven't figured out the pattern yet.
  • Important information-
    UUID=5D18-F276  # The 4GB MicroSD Card's UUID. Find using syslog after plugging it in
    /media/disk     # Default mount point
    /dev/sdb1       # Device location
    /dev/sdb1 /media/disk vfat rw,nosuid,nodev,uhelper=hal,utf8,shortname=winnt,uid=1000 0 0   # /etc/mtab entry for the disk
    /media/disk/my_flix   /media/disk/my_music   /media/disk/my_pix    # Available folders
    Camera Resolutions: 1280x960 1024x768 640x480 320x240 176x144 160x120
    Video Camera Resolution: 176x144
    Screen Resolution: 220x176, 262k colors

Thursday, July 9, 2009

Installing Xubuntu 9.04 on an emachines E625-5192

Received my new laptop today - I need it for the fall, and got it a little early due to a sale.

What went well: I created a set of Restore DVDs (in case I want Windows back), then removed windows and installed Xubuntu 9.04 full-disk. Copied over my old /home directory, and installed all my favorite apps. E-mail, web, games, .bashrc, most dektop settings, etc. transferred without a hiccup. Recreated my crontab. Wireless networking and video work great. Built-in card reader reads all cards from my cameras and phone. Machine is noticeably faster. FN-key brightness control works. FN-key multimedia controls work.

Solved Problems:

  • Sound was tinny and a bit faint. Headphone jack works, not tinny, but low volume. Solution: Adding the following lines to /etc/modprobe.d/alsa-base.conf
    somwhat improved speakers and fixed the headphone jack.
    # Added by me on <date> while troubleshooting audio
    options snd-hda-intel model=6stack

  • The Volume-FN keys didn't work. Solution: Mapped FN-audio up, down, and mute keys by mapping them to aumix using these instructions.

  • The suspend FN-key didn't work. Solution: Enabled it in the gnome-power-manager preferences. It was set up to work out-of-the-box, but disabled by default.

  • DVD: DVDs play only after changing CDROM permissions after each disc insertion. Workaround: Created an alias in .bashrc as the 'fixcd' command

  • TomTom GPS automatic update doesn't work on Linux - Win and OSX only. Department of Defense forms and other applications are Windows-specific. Workaround: Installed a Virtual Machine (VM) to host an occasional-use XP instance.

  • Phone-made .3g2 video files play with weird audio - need a way to convert them. Computer .wma songs don't play on phone - need a lossless storage format, and a way to convert them to .mp3. Workaround: Upload videos to YouTube instead of the local hard drive.

  • Need password to restore from suspend (Xubuntu issue, not hardware-related). Workaround: Remove the screensaver package.

  • Unsolved Problems: No built-in webcam. Keyboard is different, and will take time to get used to - many typos in the meantime.

    Monday, July 6, 2009

    Creating Custom color palletes using The GIMP

    My spouse wants to create art using carefully-arranged Rubik's Cubes. She needs a tool to do it: She needs to manipluate images to the right color and pixellation to look cube-ready (Solving them to the displayed configuration is another issue).

    So let's set up The GIMP to do it.

    Preparation: GIMP needs to know what colors to use, so let's create a custom color pallette. (This only needs to be done the first time)

    1. Find an image or two with cube colors (Google is handy for this). Copy the images to your clipboard.
    2. Import the image to GIMP: File -> Create -> From Clipboard
    3. Create the new color palette: Image -> Mode -> Indexed -> Use Custom Palette -> Open The Palette Selection Dialog -> New Palette
    4. Select the six colors: Use the color-picker tool to change the foreground color, then right-click in the palette-editor to add each color.
    5. Save the new palette with a name you will remember, like 'Cube Colors'

    Changing an image

    1. Import the image into GIMP
    2. Reduce the image to the six cube colors - Image -> Mode -> Indexed -> Use Custom Palette -> Cube Colors
    3. Reduce the image to manageable size with Image -> Scale Image Pick a small size that is a multiple of 3 (3 rows/colums on each cube)
    4. NEED a way to split the image into 3x3 (blown up to 9x9) squares for each cube.

    Scripting for batch-resizing: This looks possible using Imagemagick - see "using pre-defined color maps" for an easy way to get Imagemagick to reduce the colors.

    Saturday, July 4, 2009

    Batch converting photos using Imagemagick

    Our store has a 7-inch photo frame, with a 2GB stick in it. So we can just drag-and-drop lots of photos onto the stick.

    But how can we use Imagemagick, the command-line photo editor, to batch-resize a whole bunch of images...perhaps as part of a script?

    Here's one way to resize a single photo:
    $ mogrify -sample 480 foldername/filename.jpg

    Here's one way to resize a whole directory of files from 4000 x 3000 to 400 x 300.
    $ mogrify -sample 400 foldername/*.jpg

    This resizes the original(s). Making copies (before the resize) is a separate command:
    $ mkdir copies
    $ cp original/image.jpg copies/
    $ mogrify -sample 400 copies/image.jpg
    $ mv copies/image.jpg stick/

    Thursday, July 2, 2009

    Installing a Samsung SCX-4725FN printer under Xubuntu 9.04

    Instead of the installation disks, I used these instructions

    Then go into Settings -> Printer -> Add New Printer and let it autodetect the new printer on the network.

    Wednesday, July 1, 2009

    Xubuntu 9.04 user-level login scripts

    Here is how to get a custom script to run at XFCE startup:

    1. Create a generic startup script and save it as You can put anything in the script; in there now is only a logger so you know it's working.
      # This script is run automatically by xfce4-desktop during system startup.
      logger -i "Running the custom startup script"

    2. Create the following entry as /home/me/.config/autostart/MyStartup.
      [Desktop Entry]
      Exec=bash /home/me/Scripts/

    Sunday, June 28, 2009

    Xubuntu desktop wallpaper from a website (NOAA Radar)

    This post has been superseded by a newer post. Much of this information is still valid, but the newer post has more detail amd examples, and includes more services.

    Living in the midwest, I check the weather radar a lot to protect my laundry drying outside from lots of pesky thunderstorms. So I want to make the radar image my desktop picture in Xubuntu, and to have it automatically update.

    To get the image: I'm using It is from the United States National Weather Service and updates every 5-6 minutes.

    A shell script to refresh the radar image as the desktop picture:

    # This is the path of the final output files that get read by other processes.
    # Working files show up here, too. You may wish to create your own directory.
    Username=<your username>
    # This is the closest weather station for observations. Find your weather 
    # station: Format is four letters. 
    # All UPPER CASE. For example, 'KMKE' for Mitchell Field in Milwaukee, WI
    # This is the closest weather radar. Find your radar:
    # Check the link to your weather radar, for example:
    # The radar name is in the 'rid=' field. In this example, mkx is Milwaukee, WI
    # Format is UPPER CASE. For example, 'MKX' for Milwaukee.
    # (OPTIONAL) The height of your top menu bar, in pixels.
    # The radar image is padded by this amount on the top edge so the menu doesn't
    # block the timestamp in the image.
    # Download the radar image.
    echo "Weather Update: Downloading the most recent radar image available..."
    curl -o $Local_Name $Radar_Url
    # (OPTIONAL) Use imagemagick to pad the image top so the timestamp is not 
    # blocked by the menu bar.
    #convert $Local_Name -background none -splice 0x${Radar_Image_Top_Padding} $Local_Name
    # Refresh desktop background with 'xfdesktop -reload'. NOTE - some versions 
    # of XFCE flicker all the icons brighter when this occurs, providing visual 
    # feedback that the refresh occurred. An Alternate method to avoid the 
    # flicker is below.
    # The 'DISPLAY=:0.0' prefix is required so root processes like cron and 
    # Upstart can process it.
    # The 'sudo -u $Username' is required because a root process (like an Upstart 
    # trigger) may be trying to change a user's desktop. Sudo changes the command 
    # to run as user instead of root. Use your username, of course.
    DISPLAY=:0.0 sudo -u $Username xfconf-query -v -c xfce4-desktop -p /backdrop/screen0/monitor0/image-path -s $Local_Name
    DISPLAY=:0.0 sudo -u $Username xfdesktop -reload
    # Alternate method to avoid the flicker by changing desktop pictures for 
    # just a moment, then changing it back.
    #DISPLAY=:0.0 sudo -u $Username xfconf-query -v -c xfce4-desktop -p /backdrop/screen0/monitor0/image-path -s /usr/share/xfce4/backdrops/xfce4logo.png
    #DISPLAY=:0.0 sudo -u $Username xfconf-query -v -c xfce4-desktop -p /backdrop/screen0/monitor0/image-path -s $Local_Name
    I'll save this script as, and make it executable with sudo chmod +x

    1. The DISPLAY=:0.0 element is explained here.
    2. The xfconf-query command, and how to change the background using DBus, are discussed in the XFCE forums

    Updating the desktop picture manually:

    Since we have it in a shell script already, we can create a .bashrc alias to run the script manually.
    nano .bashrc opens the .bashrc for editing.
    Add the line alias radar='/home/me/' to the bottom of the file and save it.
    Open a new terminal window (terminals only read the .bashrc upon starting) and try the command radar.

    Updating the desktop picture automatically: Since we can update the desktop image using bash commands, let's make a crontab entry to update the desktop image automatically. Here's what it looks like - a crontab entry with just the shell command:

    # m h  dom mon dow   command
    */20 * * * * /home/me/
    Note that the desktop picture will refresh every 20 minutes.

    Explaining the cron instructions:

    # m h dom mon dow command - That's just part of the crontab

    */20 * * * * - tells the machine to run the script every 20 minutes. */5 * * * * will run the script every five minutes.

    > /dev/null at the end (optional) - tells the machine to not e-mail me every time the script runs.

    Make sure the command is all on one line (not wrapped), or you'll get crontab errors.

    Saturday, June 27, 2009

    Watching a DVD in Xubuntu 9.04

    After the previous reinstall two months ago, DVDs stopped working.

    Here's how I got it to work:

    1. Add the medibuntu repository, if you haven't already
      sudo cp /etc/apt/sources.list /etc/apt/sources.list~   # Backup the sources.list file
      sudo mousepad /etc/apt/sources.list &                  # Open the sources.list file in an editing window
        ## In sources.list, append the following two lines at the bottom, then save (don't close it)
        ## Medibuntu
        deb jaunty free non-free
        ## If you use debtorrent, use: deb debtorrent://localhost:9988/ jaunty free non-free
      sudo apt-get update
      sudo sudo apt-get install medibuntu-keyring
      sudo apt-get update
    2. Use these two terminal commands to add the correct software (source):
      sudo apt-get install totem-xine libxine1-ffmpeg libdvdread4
      sudo /usr/share/doc/libdvdread4/
      # Note - this installs the libdvdcss2 package, which is *not* in the repositories.
      # If you use this list of packages to rebuild your system, for example by using a 
      # Jablicator metapackage, it will fail due to this missing dependency.
    3. I installed ubuntu-restricted-extras for unrelated reasons. So I don't *think* it's necessary.
      sudo apt-get install ubuntu-restricted-extras
    4. Finally, I got an error message when I put in a dvd:
      "Could not open location; you might not have permission to open the file." This is indeed a permission issue. Fix it with:
      sudo chmod 755 /media/cdrom0
      And then try opening the DVD from within your player application (Totem).

    Reinstalling Xubuntu 9.04

    About a month ago, audio suddenly stopped working. Rather than troubleshoot, I decided to might be faster. Unlike last time, this time was a complete reinstall to get rid of certain dependency problems that had also cropped up.

    • Wireless networking worked immediately.
    • Oops. I didn't backup any of my *hidden* files - suddenly all of my e-mail and web caches are...gone. Lesson - keep older backups. Lesson - copy hidden files, too.
    • I tried using Jablicator to save my package list, but hit a failure - it couldn't resolve libdvdcss2 because it didn't come from the Ubuntu repositories.
    • Audio works in the Listen player - except wma files. Totem plays the wma, so it's not a codec or dependency issue. Streaming .pls works.
    • Video works. Added Flash, and YouTube and Hulu work. DVD read works after just a bit of tweaking

    Sunday, May 31, 2009

    Creating a metapackage with jablicator

    A convenient way to create a metapackage is the jablicator command-line application, included in the package of the same name.

    Jablicator creates a metapackage with all current packages as the dependencies. So you can edit the dependencies to limit metapackage, or take a snapshot of your packages for easy restoring after an update.

    Monday, April 27, 2009

    The Evolution plugin for Mail Notification doesn't work/exist (LP#355209)

    In mail-notification 5.4 in Ubuntu Jaunty (9.04), the following files are installed to /usr/lib/evolution/2.24/plugins/ instead of /usr/lib/evolution/2.26/plugins/:

      A simple workaround:
    1. Copy or link the files to the correct directory:
      sudo ln -s /usr/lib/evolution/2.24/plugins/org-jylefort-mail-notification.eplug /usr/lib/evolution/2.26/plugins/
      sudo ln -s /usr/lib/evolution/2.24/plugins/ /usr/lib/evolution/2.26/plugins/
    2. Restart evolution and go to Edit -> Plugins -> Jean-Yves Lefort's Mail Notification. Check the box.
    3. Right-click on the Mail-Notification icon -> Properties. Add your evolution e-mailbox.

    Saturday, April 25, 2009

    Upgrading from Xubuntu 8.04 to 9.04

    It's finally time to reinstall to dist-upgrade, which I haven't done in a year.
    Burning the 9.04 CD: Instead of installing Brasero or another burner, I used the command wodim dev=$PATH-TO-DEVICE driveropts=burnfree -v -data $PATH-TO-ISO, so in my case wodim dev=/dev/scd0 driveropts=burnfree -v -data /home/me/Ubuntu\ Images/xubuntu-9.04-desktop-i386.iso. Very easy and fast that way.

    Using the 9.04 LiveCD installer: Very simple. One hiccup when automatic partitioning failed. I chose to reuse my existing partition *without* formatting it first, and (COOL!) my /home directory was untouched. All my preferences and saved fies were still if they had been migrated. Networking and sound worked immediately from the default installation.

    Enabling Medibuntu and debtorrent: Medibuntu is for non-free packages like skype. Debtorrent is a method of using torrents instead of mirrors to download. Both require changes to the /etc/apt/sources.list file. debtorrent instructions

    sudo cp /etc/apt/sources.list /etc/apt/sources.list~   # Backup the sources.list file
    sudo mousepad /etc/apt/sources.list &                  # Open the sources.list file in an editing window
      ## In sources.list, append the following two lines at the bottom, then save (don't close it)
      ## Medibuntu
      deb debtorrent://localhost:9988/ jaunty free non-free
    sudo apt-get update
    sudo sudo apt-get install medibuntu-keyring debtorrent apt-transport debtorrent
      ## In sources.list, substitute each occurrence of the string 'deb http://' prefix with 'deb debtorrent://localhost:9988/'
      ## Save and close the sources.list file.
    sudo apt-get update

    Bringing back my favorite apps: Using this table, it's pretty easy to figure out what to install and remove. Downloading all this stuff took about 40 minutes.

    The droid fonts are nice, but not special.
    SubjectPackages I RemovedPackages I AddedInstall Notes
    Evolution needed a couple restarts to start working properly
    Download-at-first-need for audio also works well.
    Remote desktopvinagreopenssh-server
    xtightvncviewer required manual config: sudo update-alternatives --set vncviewer /usr/bin/xtightvncviewer
    blues-gnome gets rid of bluetooth-applet
    pmount mounts usb drives as user instead of root

    Several launcher icons were missing - the launchers were still in the right place and fully functional, but the application (like bluefish) was gone. After reinstallation, most images came back automatically. A couple needed to be reassociated with the image by right-clicking on 'properties'.

    Two important shortcuts were missing.
    • The Home folder icon on the desktop was gone - fixed by right clicking on the desktop, then Desktop Settings -> Icons.
    • The shortcut to my imadison script was missing from /bin. Easy to replace with sudo cp -l /bin/imadison

    The crontab was gone and had to be recreated.

    The mail-notification icon couldn't find evolution (Bug 355209). The bug report has the simple workaround.

    Wednesday, April 15, 2009

    Using rmadison and apt-cache

    rmadison is part of the devscripts package. It's a fantastic little tool that tells you which version of a package is in which release of Ubuntu or Debian. It's also the quickest way to see if a package is in one of them at all - very handy to check package requests on Launchpad and in Brainstorm.

    apt-cache is part of the default Ubuntu install. It's very handy to find the right package name, dependencies, and other clues when tracking down the correct package for Launchpad or Brainstorm.

    Tuesday, April 7, 2009

    Using debtorrent to contribute to the community

    Debtorrent is a way to download packages using a torrent instead of a mirror. It's also a way to contribute to the community by reducing the need for mirrors.
    Installing debtorrent: (instructions)
    1. Install from the command line ( sudo apt-get install debtorrent apt-transport-debtorrent ), or synaptic.
    2. Edit the lines in /etc/apt/sources.list to take advantage of debtorrent
      deb etch main contrib non-free                              #OLD
      deb debtorrent://localhost:9988/ etch main contrib non-free         #NEW
      # Do not modify deb-src lines
    3. Reload the package list with sudo apt-get update
    Using debtorrent: Debtorrent runs in the background automatically. You don't need to start it or stop it. To check on what it's doing, use the web interface at http://localhost:9988/

    Wednesday, April 1, 2009

    One IM application to rule them all...

    I use Pidgin, the default IM client on Xubuntu 8.04. But I also use the Department of Defense's AKOIM. And my laptop doesn't have a built-in webcam or microphone, so family uses Skype's IM. I really don't want three IM's open (one tying up a browser window and java!), so here's how I consolidated them into just Pidgin.

    AKO Instant Messenger uses a standard xmpp protocol, so Pidgin can talk to it. All your contacts migrate like magic! If you have AKO/DKO access, get the details here. If you don't have AKO/DKO access, then safely ignore this paragraph.

    Skype uses a proprietary protocol, so it's integration is limited. The skype4pidgin plugin (.deb package) shares Skype contacts and IM with Pidgin...but Skype must still be running alongside Pidgin to work...though many fewer windows must be open. After installation, BOTH Skype and Pidgin must be restarted. I had to edit the Skype options manually to turn off chat notifications - otherwise both apps whine when a new message arrives.

    Sunday, March 22, 2009

    Scanning for wireless networks

    Two methods to scan for wireless networks. One requires sudo/root, the other requires Network Manager.

    #! /usr/bin/env python
    """This python 2.5 script uses iwlist to scan for nearby wireless networks. It must be run as sudo/root to work."""
    import subprocess as SU
    command = ['iwlist', 'eth1', 'scan']
    output = SU.Popen(command, stdout=SU.PIPE).stdout.readlines()
    data = []
    for item in output:
        print item.strip()
        if item.strip().startswith('ESSID:'): 
            data.append(item.lstrip(' ESSID:"').rstrip('"\n'))
        if item.strip().startswith('Quality'): 
            data.append(int(item.split()[0].lstrip(' Quality=').rstrip('/100 ')))
        if item.strip().startswith('Encryption key:off'): data.append('OPEN')
        if item.strip().startswith('Encryption key:on'): data.append('encrypted')        
    print data
    #! /usr/bin/env python
    """This python 2.5 script uses dbus to query Network Manager, which scans regularly for wireless networks. It does NOT require root/sudo."""
    import dbus
    item = 'org.freedesktop.NetworkManager'
    path = '/org/freedesktop/NetworkManager/Devices/eth1'
    interface = item + '.Device'
    bus = dbus.SystemBus()
    data = []
    wireless = dbus.Interface(bus.get_object(item, path), interface)
    for network_path in wireless.getNetworks():
        network = dbus.Interface(bus.get_object(item, network_path), interface)
        data.append(network.getName())                        # also network.getProperties[1]
        data.append(network.getStrength())                    # also network.getProperties[3]
        if network.getEncrypted(): data.append('encrypted')
        else: data.append('OPEN')
    print data

    Wednesday, March 11, 2009

    Replacing os.popen() with subprocess.Popen() in Python

    In Python, you can execute shell commands using the os.popen method...but it's been deprecated in favor of a whole new command.

    # The old way, which worked great!
    import os
    shell_command = 'date'
    event = os.popen(shell_command)
    stdout = event.readlines()
    print stdout
    # The new way, which is more powerful, but also more cumbersome.
    from subprocess import Popen, PIPE, STDOUT
    shell_command = 'date'
    event = Popen(shell_command, shell=True, stdin=PIPE, stdout=PIPE, 
        stderr=STDOUT, close_fds=True)
    output =
    print output
    # The new way, all in one line (a bit uglier), works in Python3!
    import subprocess
    output = subprocess.Popen('date', stdout=subprocess.PIPE)
    print output

    Sunday, March 8, 2009

    Using DBus on Xubuntu 8.04

    This post is obsolete has been superseded by my 2012 dbus tutorial series.

    DBus is a system that permits different applications to exchange information. Tutorial Reference Other Reference.
    Sometimes, DBus crashes upon restart from a suspend or hibernation. These bash commands will help you figure out if it has crashed, and how to restart it.

    $ps -e | grep `cat /var/run/dbus/pid` # Confirm if DBus is running by checking for the PID number in the list of live processes.
                                          # If DBus is running, this will return the process number. 
                                          # If not, it will return nothing.
    $sudo rm /var/run/dbus/pid            # Remove the stale pid file so DBus can be restarted.
    $sudo dbus-daemon                     # Start DBus again.

    A python script uses DBus to see if the network connection is available by asking Network Manager:

    #! /usr/bin/env python
    import dbus
    bus = dbus.SystemBus()
    item = 'org.freedesktop.NetworkManager'
    eth0_path = '/org/freedesktop/NetworkManager/Devices/eth0'
    eth1_path = '/org/freedesktop/NetworkManager/Devices/eth1'
    interface = 'org.freedesktop.NetworkManager.Devices'
    # There are two possible network interfaces: eth0 (wired) and eth1 (wireless).
    eth0 = dbus.Interface(bus.get_object(item, eth0_path), interface)
    if eth0.getLinkActive(): print('The wired network is up') # getLinkActive() is a boolean, TRUE if the network link is active
    eth1 = dbus.Interface(bus.get_object(item, eth1_path), interface)
    if eth1.getLinkActive(): print('The wireless network is up')
    This shell script does exactly the same thing, using the same DBus call:
    # This shell script checks Network Manager if the network is up, using dbus as the communications medium.
    # There are two possible network interfaces: eth0 (wired) and eth1 (wireless). Of course, you may need to alter these to meet your own circumstances.
    # The basic format of dbus-send is: dbus-send --system --dest=org.freedesktop.NetworkManager /org/freedesktop/NetworkManager/Devices/eth0 --print-reply org.freedesktop.NetworkManager.Devices.eth0.getLinkActive
    result_eth0=`dbus-send --system --dest=$DEST $PPATH'/eth0' --print-reply $DEVICE'.eth0.getLinkActive'`
    shaved_eth0=`echo $result_eth0 | cut -d ' ' -f8`
    if [ $shaved_eth0 = 'true' ]; then echo 'The wired network is up'; fi
    result_eth1=`dbus-send --system --dest=$DEST $PPATH'/eth1' --print-reply $DEVICE'.eth1.getLinkActive'`
    shaved_eth1=`echo $result_eth1 | cut -d ' ' -f8`
    if [ $shaved_eth1 = 'true' ]; then echo 'The wireless network is up'; fi

    A Python script that queries Network Manager to get the list of wireless networks.

    import dbus
    item = 'org.freedesktop.NetworkManager'
    path = '/org/freedesktop/NetworkManager'
    interface = item + '.Device'
    network_list = []
    bus = dbus.SystemBus()
    # Create a Network Manager interface and get the list of network devices
    event = dbus.Interface(bus.get_object(item, path), interface)
    # Create an interface for each device
    # Query each interface to see if it's wireless
    # Query each wireless interface for the networks it sees
    for device in event.getDevices():
        device_interface = dbus.Interface(bus.get_object(item, device), interface)
        if device_interface.getType() == 2:  # 0 unknown, 1 wired, 2 wireless
    # Reformat the network names in the list to be more readable
    if network_list:
        for entry in network_list:
            #print entry    # String of path/to/network_name
            entry_list = entry.split('/')
            print entry_list[-1]  # String of network_name

    A Python listener that catches the changes in wireless signal strength using both available methods.

    import dbus, gobject
    from dbus.mainloop.glib import DBusGMainLoop
    def print_device_strength1(*args):  #Use the received signals
        signal_strength = args[1]
        print ('Signal Strength (Method 1): ' + str(signal_strength) + '%')
    def print_device_strength2(*args, **kwargs):  #Use the received signals
        signal_strength = args[1]
        print ('Signal Strength (Method 2): ' + str(signal_strength) + '%')
    DBusGMainLoop(set_as_default=True)    # Set up the event loop before connecting to the bus
    bus_object = dbus.SystemBus()
    # The variables you need. I used the shell command 'dbus-monitor --system' to find this information
    sender = 'org.freedesktop.NetworkManager'
    path = '/org/freedesktop/NetworkManager'
    interface = sender
    member = 'DeviceStrengthChanged'
    # Method 1 - bus_object.proxy_object.connect_to_signal(method, action, filter, message_parts)
    proxy_object = bus_object.get_object(sender, path)
    proxy_object.connect_to_signal(member, print_device_strength1, dbus_interface = interface)
    # Method 2 - bus_object.add_signal_receiver(action, [filters])
    bus_object.add_signal_receiver(print_device_strength2, dbus_interface = interface, member_keyword = member)
    # Start the loop
    loop = gobject.MainLoop()

    Thunar responds beautifully to D-Bus. Introspection is fully set up, so it's easy to use with the d-feet application. Useful for launching programs, opening folders and windows, and manipulating the trash. Launching a program by this method means that the the window manager launches the program, not the script or terminal, so the program can remain open after the script or terminal terminates.

    #!/usr/bin/env python
    import dbus
    item = ('org.xfce.Thunar')
    path = ('/org/xfce/FileManager')
    interface = ('org.xfce.FileManager')
    event = dbus.Interface(dbus.SessionBus().get_object(item, path), interface)
    # These three lines at the end of the script open the file's 'properties' window
    display = (':0')         # The current session screen
    uri = ('/home/me/')
    event.DisplayFileProperties(uri, display)
    # These three lines at the end of the script launch a new application
    display = (':0')         # The current session screen
    uri = ('/usr/bin/gftp-gtk')
    event.Launch(uri, display)
    # These four lines at the end of the script open a folder window and optionally select a file
    display = (':0')         # The current session screen
    uri = ('/home/me/.cron')
    filename = ('anacrontab.daily')
    event.DisplayFolderAndSelect(uri, filename, display)

    A sample hal script.

    """This python 2.5 script uses dbus to check if the lid switch is open.
    Based on an original python script at"""
    import dbus
    dest = 'org.freedesktop.Hal'
    hal_path = '/org/freedesktop/Hal/Manager'
    hal_interface = 'org.freedesktop.Hal.Manager'
    udi_interface = 'org.freedesktop.Hal.Device'
    # Get the list of possible input switches. The return is a list of paths.
    bus = dbus.SystemBus()
    hal = dbus.Interface(bus.get_object(dest, hal_path), hal_interface)
    list_of_udi_paths = hal.FindDeviceByCapability('input.switch')
    # Filter the list for the word 'lid'. Print the status for each one.
    for udi_path in list_of_udi_paths:
        udi = dbus.Interface(bus.get_object(dest, udi_path), udi_interface)
        if udi.GetProperty('button.type') == "lid":
            # The button.state.value is FALSE if the lid is open.
            if udi.GetProperty('button.state.value'): print ('Lid is closed')
            else: print ('Lid is open') 
        else: print ('Problem: I could not find the lid switch. Sorry.')


    • The D-feet application is very handy for exploring DBus, and figuring out how to communicate with it. It's available in the Ubuntu repositories.
    • More information on the destination/path/interface settings is available from each application's XML config files, found in the /etc/dbus-1/system.d and /session.d directories.
    • The system-tools-backends DBus interfaces look promising, with methods for network interfaces, time, users and groups, and more. But I couldn't get any of it to work. One hint suggested that the DBus message must be sent by root instead of user.
    • xfce4-terminal has a Launch method, seemingly for launching items in the terminal (source code). I can see how that would be handy, but I couldn't get it to work.

    Wednesday, March 4, 2009

    Bash and Python scripts to unzip and modify an OpenOffice .odt document

    .odt files are actually containers (you can see one using unzip -l document.odt). Within the container, content is in the content.xml file. Script info source

    Here's what I've figured out about opening, modifying, and saving the content of an .odt file:

    • To open the container for editing:
      # bash
      $unzip path/container.odt content.xml
      $unzip path/container.odt content.xml -d working/dir/path/
          # -d places content.xml in a different directory.
          # Creates a content.xml file where you want it.
      # python
      >>>import zipfile
      >>>odt_file = zipfile.ZipFile('path/to/file.odt','a')  
      >>>    # Options are 'r'ead only, 'w'rite only, and 'a'ppend to existing
      >>>raw_xml ='content.xml') 
      >>>    # Reads content.xml in as a string, doesn't place a file.

    • Modify the content.xml file by hand, or using a script, or using Python.
      >>> # Tip for using python: ElementTree is good at XML, and it can parse a file, but it cannot parse a string!
      >>> # So here's how to make a string look like a file using python's StringIO module.
      >>>import StringIO, xml.etree.ElementTree as ET
      >>>fakefile = StringIO.StringIO(raw_xml)   # Pretend raw_xml string is a file called fakefile
      >>>tree = ET.parse(fakefile).getroot()     # Parse the fakefile
      >>>fakefile.close()                        # close() is desired by StringIO to free the buffer
      >>> # Make changes to your tree here.

    • To restore the container with modified content:
      # bash
      zip -j path/container.odt working/dir/path/content.xml
          # The -j flag adds the file 'content.xml' instead of the useless 'path/content.xml'. You need this!
      rm working/dir/path/content.xml    # Clean up
      # python
      >>>new_xml = ET.tostring(tree) # If you're exporting from an ElementTree
      >>>odt_file.writestr('content.xml', new_xml)
    • Putting it all together in bash:
      cd path/to/working/directory
      cp path/to/template_filename.odt working_file.odt
      unzip working_file.odt content.xml
      # Change the xml...somehow
      zip -j working_file.odt content.xml
      rm content.xml
    • Putting it all together in python:
      def edit_the_odt_content(template_filename):
          """Exposes the content of an .odt file so you can modify it."""
          import os, shutil, StringIO, zipfile, xml.etree.ElementTree as ET
          shutil.copyfile(template_filename, 'working_file.odt') # Copy the template into a working file
          odt_file = zipfile.ZipFile('working_file.odt','a')
          xml_string ='content.xml')              # Read the zipped content.xml within the .odt as a string
          raw_xml = StringIO.StringIO(xml_string)                # Pretend the read string is a file so ElementTree will parse it
          tree = ET.parse(raw_xml).getroot()                     # Convert raw string to ElementTree
          office_namespace = '{urn:oasis:names:tc:opendocument:xmlns:office:1.0}'
          body = tree.find(office_namespace + 'body')            # Search the tree to find the elements you want to change
          text = body.find(office_namespace + 'text')
          new_text = your_function_to_modify_the_xml(text)       # You can now change the XML any way you wish
          body.remove(text)                                      # Replace the old XML with the new
          new_xml = ET.tostring(tree)                            # Convert the modified ElementTree back into an XML string
          odt_file.writestr('content.xml', new_xml)              # Write the string into the zipped content.xml
          odt_file.close()                                       # Close the zip archive (important!)
    • Bug: Don't use the zip -m flag! It looks handy, claiming to delete the content.xml file from your file system after adding it to the archive...but instead it will unpredictably delete without adding to the archive.

    • You can avoid the whole "containers" muddle by saving an OpenOffice document as a flat file (.fodt). There's no zipping or unzipping, just open the file in an editor - it's xml already. Open the modified .fodt with OpenOffice, and your document is right there. Er, be sure your version of OO supports .fodt before using it. My Mac doesn't, for example.

    • OpenOffice also has it's own script classes for Python and C, called UNO. However, I haven't taken time to dig around through it.

    Tuesday, March 3, 2009

    Python Script on Windows XP

    Trying to adapt the Linux-built rental invoice script onto the Windows XP bookkeeping computer

    cmd - run this command at the run prompt to get a windows command line.
    C:\Python26\python  - run python from the windows command line.
    C:\Python26\python "C:\Documents and Settings\All Users\Documents\" - run the script from the command line

    Tuesday, February 24, 2009

    You Are Here on a Google Map

    This post is obsolete and has been superseded

    The following python 2.x script shows your current (GPS-enabled) location on a Google Map. A useful learning experience:

    #!/usr/bin/env python
    """This is a python 2.5 script that plot's a GPS receiver's location on the
    Google Maps website.
    In order to work, you need a network connection, an attached GPS receiver, and
    the GPS daemon (gpsd).
    import os
    #import subprocess and SU
    import gps
    import dbus
    import sys
    def test( ):
        """ Step 1: Test for the existence of a running gpsd, test for the existence of an open network connection, and test for a firefox process.
        If any fail, give an error message, don't try to recover. FUTURE: Could also use DBus to test for firefox and gpsd."""
        process_list = os.popen('ps -e')    # os.popen is deprecated in favort of subprocess.Popen
        #process_list = SU.Popen(['ps','e'], stdout=SU.PIPE)  
        gpsd_existence_flag = 0
        firefox_existence_flag = 0
        for line in process_list.readlines():
            if line.count('gpsd') > 0: gpsd_existence_flag = 1
            if line.count('firefox') > 0: firefox_existence_flag = 1
        if not gpsd_existence_flag:
            print ("gpsd is not running. Use 'gpsd -b /dev/ttyUSB0' to start it, and then try again.")
        else: print ('Checking...found gpsd')
        if not firefox_existence_flag:
            print ("firefox is not running. Please start it and try again.")
        else: print ('Checking...found firefox')
        bus = dbus.SystemBus()
        nm_item = ('org.freedesktop.NetworkManager')  # This string gets used a lot
        nm_path = ('/org/freedesktop/NetworkManager')
        nm_device = ('org.freedesktop.NetworkManager.Device')
        list_of_interface_paths = dbus.Interface(bus.get_object(nm_item, nm_path), nm_device).getDevices()
        found_network_flag = 0
        for interface_path in list_of_interface_paths:
            one_interface = dbus.Interface(bus.get_object(nm_item, interface_path), nm_device)
            if one_interface.getLinkActive():   # True if there is an active network on this interface
                if one_interface.getType() == 2: # 0 unknown, 1 wired, 2 wireless
                    print('Checking...found the wireless network') 
                    found_network_flag = 1
                elif one_interface.getType() == 1: 
                    print('Checking...found the wired network')
                    found_network_flag = 1
        if found_network_flag: return
            print ("cannot find a network connection. Please connect and try again.")
    def get_position_fix( ):
        """Step 2: Get a position fix from gpsd."""
        session = gps.gps('localhost','2947')  # Open a connection to gpsd
        session.query('p')                     # Get the location fix 
        lat = session.fix.latitude
        lon = session.fix.longitude
        print ('Location is ' + str(lat) + ' latitude and ' + str(lon) + ' longitude.')
        return (lat, lon)
    def show_map(lat_lon_tuple):
        """Step 3: Submit the position fix to Google Maps. Note that the parentheses '()' in the URL must be escaped '\' to work.
        Sample URL format:,+-122.41175+(You+can+insert+your+text+here)&iwloc=A&hl=en"""
        url_string = ('' + str(lat_lon_tuple[0]) + ',+' + str(lat_lon_tuple[1]) + '+\(You+Are+Here\)&iwloc=A&hl=en')
        os.popen('firefox ' + url_string)
    # Run this script as a standalone program
    if __name__ == "__main__" :
        location = get_position_fix()

    Monday, February 23, 2009

    GPS and Xubuntu 8.04

    This post is obsolete and has been superseded

    I'm experimenting with USB GPS receiver (dongle). It's a Canmore GT-730F that I received in January 2009. Here's what I've learned so far.

    Manually getting data using the command line (source):

    1. Check dmesg, the kernel log, to find out where the device has been mounted. In my case, it mounts reliably to /dev/ttyUSB0. If it doesn't mount, try the command sudo modprobe pl2303 to load the correct USB driver.
    2. Set the data output rate to 4800 baud using the stty command: stty 4800 > /dev/ttyUSB0
    3. Read the data stream using the cat command: cat /dev/ttyUSB0
    4. You should see a set of data scroll down the screen. Use CTRL+C to end the test.

    The Linux GPS Daemon (gpsd) is the central clearinghouse for receiving GPS data from the receiver, buffering it, and forwarding it to the applications that want it. gpsd has features to broadcast to dbus (system bus), update ntpd, and respond to a multitude of specific queries from clients. References: Project home page, gpsd man page, and a great example program

    $sudo apt-get install gpsd gpsd-clients  # Installs the daemon (gpsd) and test set (gpsd-clients) packages
    $gpsd -b /dev/ttyUSB0                    # Start gpsd, telling it where to find the receiver
    $cgps                                    # Current satellite data - great way to test that the receiver and gpsd are working

    gpsfake is a gpsd simulator. It tricks gpsd into reading from a logfile instead of a real GPS device. Very handy for testing without actually using a GPS dongle. It is included with the gpsd package, and has a man page for additional reference. To make a logfile, and then to use gpsfake:

    $cat /dev/ttyUSB0 > path/to/testlog  # Create the log file. Use CTRL+C to end the capture.
    $gpsfake path/to/testlog             # Run gpsd simulator (not a daemon - it will occupy the terminal)

    Python interface to gpsd (python-gps) is a programming tool to build your own gps-aware application.

    >>>import gps                             # Load the module
    >>>session = gps.gps('localhost','2947')  # Open a connection to gpsd
    >>>session.query('o')                     # See man gpsd(8) for the list of commands
    >>>print session.fix.latitude             # Query responses are attributes of the session
    >>>dir(session)                           # To see the possible responses
    >>>del session                            # Close the connection to gpsd

    In this case, it seems that I need a periodic session.query('p'), which just gives lat/lon and timestamp.

    Time might be an issue, since the system and the GPS may think the time is different. To see if it's an issue, compare them using the python script below. In my tests, they vary from 0.08 to 1.3 seconds apart, not enough to worry about. GPS timestamps use GMT, not localtime.

    #!/usr/bin/env python
    import calendar, time, gps
    system_time = calendar.timegm(time.gmtime())  # System time (in seconds)
    session = gps.gps('localhost','2947')         # Open a connection to gpsd
    session.query('p')                            # See man gpsd(8) for the list of commands
    gps_time = session.timings.c_recv_time        # GPS time (in seconds)
    print ('The time difference is ' + str(system_time - gps_time) + ' seconds.')

    MGRS (military map coordinates) conversion to/from latitude and longitude is not currently available in Ubuntu...that I can find. The dongle documentation doesn't mention MGRS at all. An online converter is available. The proj package looks promising, but I haven't figured it out yet. Perhaps Lat/Lon -> UTM -> MGRS?

    DBus access appears to be undocumented...but there are tantalizing hints on Google that examples are out there. I can listen to dBus using cgps to make traffic, then dbus-monitor --system to see it.

    The best storage format for tracklogs, routes, and waypoints seems to be GPX format, since it's easy to understand and cgpxlogger, included with gpsd, will create an XML track in GPX 1.1 format. Google's KML is more powerful, but also much more complex. GPSbabel universal data translator is a command-line application that translates one file type to another, and does convert GPX <-> KML.

    $cgpxlogger -i 30 > path/to/logfile         # Save data every 30 seconds to an XML file
    $gpsbabel -i gpx -f path/to/gpx_file -x duplicate -o kml -F path/to/kml_file
    $#gpsbabel [options] -i INTYPE -f INFILE -x FILTER -o OUTTYPE -F OUTFILE

    GPSdrive navigation system looks cool, but I couldn't get maps to load, so it's utility was limited. However, it seems that online plotting of tracklogs, routes, and waypoints is possible on Google Maps (and Yahoo Maps, and others). One example is the cool GPS Visualizer.

    Gypsy is an alternative daemon, but not currently in Debian or Ubuntu, so I haven't tried it. Last release 0.6 in March 2008.

    GPSman device manager is an app I know nothing about. I couldn't get it to work, so I removed it. The dongle seems small enough and simple enough that it may not need to be 'managed' at all.

    Friday, February 20, 2009

    Using Python to reformat the xml within .odt files

    A quick python 2,x script to copy, unzip, and uniformly reformat the XML of an .odt file. It adds indentations and line breaks. Useful to debug my invoicing script, which muddles with the xml files, by making the files diff-able and easier to read and easier to search.

    #!/usr/bin/env python
    import os
    import xml.etree.ElementTree as ET
    odt_path_and_file = 'path/to/file.odt'
    # This function was copied from
    def indent(elem, level=0):
        i = "\n" + level*"  "
        if len(elem):
            if not elem.text or not elem.text.strip():
                elem.text = i + "  "
            if not elem.tail or not elem.tail.strip():
                elem.tail = i
            for elem in elem:
                indent(elem, level+1)
            if not elem.tail or not elem.tail.strip():
                elem.tail = i
            if level and (not elem.tail or not elem.tail.strip()):
                elem.tail = i
    odt_filename = odt_path_and_file.split('/')[-1]
    folder_name = ('Desktop/' + odt_path_and_file.split('/')[-1].rstrip('.odt'))
    os.popen('rm -r ' + folder_name) #Delete any old working files
    os.popen('mkdir ' + folder_name)
    os.popen('cp ' + odt_path_and_file + ' ' + folder_name)
    os.popen('unzip ' + folder_name + '/' + odt_filename + ' -d ' + folder_name)
    reply = os.popen('ls ' + folder_name)
    file_list = [filename.rstrip('\n') for filename in reply.readlines() if filename.count('.xml') > 0]
    for file in file_list:
        print ('Parsing ' + folder_name + '/' + file)
        tree = ET.parse(folder_name + '/' + file)
        tree.write(folder_name + '/' + file)
        print ('Completed ' + file)

    Using Python to compare .odt files

    A quick python script to copy, unzip, and reformat an .odt file. It adds indentations and line breaks. Useful to debug my invoicing script, which muddles with the xml files, by making the files diff-able and easier to read and easier to search.

    #!/usr/bin/env python
    import os
    import xml.etree.ElementTree as ET
    odt_path_and_file = 'path/to/file.odt'
    # This function was copied from
    def indent(elem, level=0):
        i = "\n" + level*"  "
        if len(elem):
            if not elem.text or not elem.text.strip():
                elem.text = i + "  "
            if not elem.tail or not elem.tail.strip():
                elem.tail = i
            for elem in elem:
                indent(elem, level+1)
            if not elem.tail or not elem.tail.strip():
                elem.tail = i
            if level and (not elem.tail or not elem.tail.strip()):
                elem.tail = i
    odt_filename = odt_path_and_file.split('/')[-1]
    folder_name = ('Desktop/' + odt_path_and_file.split('/')[-1].rstrip('.odt'))
    os.popen('rm -r ' + folder_name) #Delete any old working files
    os.popen('mkdir ' + folder_name)
    os.popen('cp ' + odt_path_and_file + ' ' + folder_name)
    os.popen('unzip ' + folder_name + '/' + odt_filename + ' -d ' + folder_name)
    reply = os.popen('ls ' + folder_name)
    file_list = [filename.rstrip('\n') for filename in reply.readlines() if filename.count('.xml') > 0]
    for file in file_list:
        print ('Parsing ' + folder_name + '/' + file)
        tree = ET.parse(folder_name + '/' + file)
        tree.write(folder_name + '/' + file)
        print ('Completed ' + file)

    Tuesday, February 10, 2009

    Bean on Mac OS 10.5

    Our old copy of Appleworks from a previous Mac is sputtering and becoming unreliable in 10.5, so I'm looking for a replacement. OpenOffice and AbiWord are cluttered and unpleasant (spouse dislikes them), but Bean seems pleasant enough to fit our needs. Happily, it reads and writes simple documents to .rtf, .doc, and .odt.

    Saturday, January 24, 2009

    Writing to syslog using shell scripts

    logger is a simple command to write a string to /var/log/syslog. For example:

    $ logger -i "This is a test string"
    $ logger -is -p -f /var/log/syslog This is another test string
        -i includes the PID
        -s sends a copy to stdout (the screen)
        -p is the priority. See the available priorities at 'man logger'
        -f is the log to append to
    And here's how to log from a crontab:
    * * * * * /usr/bin/logger -i crontab test

    Friday, January 23, 2009

    Moving Cron jobs to Anacron

    Cron is a great way to run recurring jobs. But some jobs need to run weekly...and sometimes the computer is turned off, so the cron job doesn't run. So I'm going to migrate some jobs to anacron. Cron runs once each minute, checking if the time matches anything in the crontab list. Anacron, however, runs once each hour (triggered by cron, restart, or resume) and checks the interval in days since a job was last run.


    • Anacron is a root/sudo-level command. Running it as a user will silently FAIL.
    • Anacron will silently FAIL to run scripts with periods '.' in the filename.
    • Anacron stores the timestamps of each job's last run in /var/spool/anacron/JOBNAME. This is handy to change while testing.

    There are two ways to run a command using anacron. You can place the command directly in the anacrontab (/etc/anacrontab), or you can put a script in one of the periodic folders (/etc/cron.daily, /etc/cron.weekly, or /etc/cron.monthly)Here are some examples:

    • For comparison, here's an example cron.daily entry that runs at 07:25 each morning:
      # min hr dom mon dow   command
      25 07 * * * date > /home/YOUR_USERNAME/.cron/test_file_1
    • This /etc/anacrontab entry runs each time anacron is called (days = 0), which is very useful for testing. Start anacron manually with sudo anacron -d.
      #days delay  jobname   command
      0 0 anacron-test2  date > /tmp/crontab-test-ouput-everytime
    • This /etc/anacrontab entry runs once daily (days = 1), no matter how many times anacron is called automatically or manually.
      #days delay  jobname   command
      1 0 anacron-test3  date >> /tmp/crontab-test-output-daily

    Friday, January 2, 2009

    Web Scraper for string prices

    I successfully tested a web scraper in python. It scrapes about 20 web pages for the prices of violin strings, then puts the prices in an OpenOffice document for handy printing. It is structured so I can add other output formats, and I could add an XML file to track prices over time or just print changes.

    I'm installing it on the store iMac, and setting it as a daily recurring job. The finished file just pops onto the desktop, marked with the date.

    A future version may compare prices from multiple sites.

    The script and template live in the standard location for user scripts, /Users/username/Library/Scripts/scriptname/