Thursday, January 22, 2015

Is your team using Find-a-Task?

Find-a-Task is the Ubuntu community's job board for volunteers.

 

Is your team listed?


It's the place for volunteers to find new, interesting, fulfilling ways to contribute to Ubuntu.
It's the place for them to discover your team or project.

Get listed today!


We have made it super easy to get your team onto Find-a-Task: No login, no editing. Just jump into #ubuntu-community-team with a volunteer role in mind:

Example:
  • Category: Programming
  • Role: QML De-frobber
  • Very short description: Get rid of QML Frob with the Ubuntu Frobbing Team
  • Landing page: http://wiki.ubuntu.com/FrobbingTeam/QMLDefrobber

That's right...you can list technical roles, too.


Nah, I don't want volunteers


If the old way is working for you, and your team has lots of spare capacity, then more power to you! Please share your secret sauce.

But if you could use a few more hands to grab a work item or two, a Find-a-Task listing is fast and simple.

You really should, you know.

Tuesday, January 13, 2015

Introducing Ubuntu Find-A-Task



The Ubuntu Community website has an awesome new service: Find-A-Task

It's a referral service - it helps volunteers discover teams and tasks that match their interests.

  • Link to it!
  • Refer new enthusiasts toward it!
  • Advertise your teams and projects on it!

Give it a try and see how it can work for your team or project.


How do I get my team listed?


So easy and so fast.
  1. What volunteer role do you want to advertise?
  2. What's a very short, exciting description of the role?
  3. Which Find-A-Task paths do you think this role is appropriate for? 
  4. Create a great landing page on the wiki. (example)
    • Drop by #ubuntu-community-team and let us know.
      • Role: Frobishers, 
      • Description: "Help Frobnicators add fabulous Frob!"
      • Path: One, in the Coding and Development submenu
      • Landing URL http://wiki.ubuntu.com/FrobTeam/Frobishers

    Your landing page:


    This is a volunteer's first impression of your team. Make it shine.

    When volunteers show up at your wiki page, they are already interested. They want to know how to set up, who to contact, and how to get started on their first easy work item. They want instructions and details.

    If you don't provide what they want, they may move on to their next choice. Find-A-Task makes it easy for them to move on.



    Credits


    Tremendous thanks to:

    Friday, December 19, 2014

    Wordpress and LAMP in an LXC Container

    Container

    I need to test some html running in Wordpress.
    But I don't use Wordpress...obviously.

    In Linux, that's easy to fix. Just install Wordpress.
    Oops, not so easy: Wordpress pulls in an entire LAMP stack with it.
    That's a lot to pollute my laptop with just to do some testing.

    Containers to the rescue!
    Let's spin up a container, install LAMP and Wordpress inside it, run the tests, then destroy the container.

    Cargo

    The Router


    I'm going to open a whole new port on my router's firewall for this, so others can help me test.

    On the router, I want to forward port 112233 to the similar port on the server.

    On the server, I want to forward port 112233 to the container's port 80. (That part is later)


    Creating the container


    Thanks the the amazing Stephane Graber for his detailed instructional series on how to create and use a container This is a slightly different setup than he did. Instead of installing directly on, say, a laptop, I'm installing the container onto a server that I access via ssh.

    Three moving pieces: Laptop (me), Server (headless), Container (added to server)

    So, from Laptop, I ssh into Server normally.

    On SERVER:
    sudo apt-get install lxc               # Install the container system
    sudo lxc-create -t ubuntu-cloud -n c1  # Download and install a 195MB cloud image of Ubuntu 14.10
    sudo lxc-start -n wp1 -d               # Boot the image in the background (name is 'wp1')
    sudo lxc-info -n wp1                   # Discover the IP of the image
        Name:           c1
        State:          RUNNING
        IP:             10.0.3.201         # <-- Ooh, there is the IP
    ping 10.0.3.201                        # Check network connectivity on the container
        PING 10.0.3.201 (10.0.3.201) 56(84) bytes of data.
        64 bytes from 10.0.3.201: icmp_seq=1 ttl=64 time=0.081 ms
        64 bytes from 10.0.3.201: icmp_seq=2 ttl=64 time=0.085 ms
    
    # Forward port 112233 to the container
    sudo iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 112233 -j DNAT --to-destination 10.0.3.201:80
    sudo iptables -A FORWARD -p tcp -d 10.0.3.201 --dport 80 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT


    Set up LAMP and Wordpress within the container


    The container is installed, started, and responds to ping.

    On SERVER:
    sudo lxc-console -n wp1    # Login to the console of wp1 (username: ubuntu, password: ubuntu)

    A couple words about the container.
    I'm going to skip the part where you would delete the 'ubuntu' user and add your own admin user and password. But you should do it.
    Also, two pieces of information you should have handy before going farther.
    1. Server's IP address on the LAN (mine is 192.168.0.101)
    2. The real, fully qualified domain name (freds_blog.fred.com) that readers on the internet will use
    The FQDN will be used to generate the MySQL account for the blog, and the create the wordpress config file. The LAN address will be used to link to the same accounts.

    Special thanks to the Ubuntu community for putting together this fantastic tutorial about how to install wordpress in Ubuntu.

    # On CONTAINER
    sudo apt-get install mysql-server wordpress   # 49 packages, 28MB download

    Let's jump over to Laptop for a moment and check that Server's port forwarding and Container's apache service are working: Open a browser window, and look for http://192.168.0.101:112233. You should get the apache default page. Success! Okay, now back to Container:

    # On CONTAINER
    sudo ln -s /usr/share/wordpress /var/www/html/wordpress   # Make Wordpress accessible from apache
    
    # Create mysql account, and link it to Wordpress
    sudo gzip -d /usr/share/doc/wordpress/examples/setup-mysql.gz
    sudo bash /usr/share/doc/wordpress/examples/setup-mysql -n freds_blog_fred_com freds_blog.fred.com  
    
    # Prevent a Wordpress error (can't locate config file) from within the LAN
    # by linking the LAN addr to existing FQDN config file
    sudo ln /etc/wordpress/config-freds_blog.fred.com.php /etc/wordpress/config-192.168.0.101.php
    
    exit
    <ctrl+a, q> to exit the console


    Use Wordpress


    The setup is complete. Since I'm on my LAN, I point my Laptop browser to http://192.168.1.101/wordpress/ and I get the Wordpress setup screen.

    Outside, on the big wide internet, I would point it to http://freds_blog.fred.com/wordpress/ (er, that's an example - you already know that's not my real blog).

    Wordpress is ready for my data.


              Bye, Container!

    Destroying the container


    This is one of the true joys of LXC

    # On SERVER
    sudo lxc-stop -n wp1
    sudo lxc-destroy -n wp1


    And all the work is wiped out forever....

    Remember to clean up:
    • Uninstall lxc from the server
    • Delete the iptables rules on the server
    • Close the port on the router firewall

    Saturday, December 13, 2014

    Say, what are you doing this afternoon?

    Bus Stop - Under The Rain by Leonid Afremov
    Here's a happy little afternoon project for new users trying to play with a new
    language or script, and getting their feet in the open-source ecosystem.

    Your phone's handy weather app depends upon the goodwill of a for-profit data provider, and their often-opaque API (14 degrees? Where was it observed? When?) That's a shame because most data collection is paid for by you, the taxpayer.

    Let's take the profit-from-data out of that system. Several projects have tried to do this before (including libgweather), but each tried to do too much and replicate the one-data-provider-to-rule-them-all model. And most ran aground on that complexity.


    Here's where you come in

    One afternoon, look up your weather service's online data sources. And knock together a script to publish them in a uniform format.

    Here's the one I did for the United States:
    Worldwide METAR observation sites
    US DOD and NOAA weather radars
    US Forecast/Alert zones

    • Looking for data on non-METAR (non-airport) observation stations, weather radar sites, and whatever forecast and alert areas your country uses.

    • Use the same format I did: Lat (deg.decimal), Lon (deg.decimal), Location Code, Long Name. Use the original source's data, even if it's wrong. Area and zones should use the lat/lon of the centroid.

    • The format is simple CSV, easy to parse and publish.

    • Publish on GitHub, for easy version control, permalinking, free storage, and uptime.

    • Here's the key: Your data must be automatically-updated. Regularly, your program must check the original source and update your published version. How I did it with a cron job. Publish both the data and your method on GitHub. 

    • When you have published, drop me an e-mail so I can link to your data and source.

    If you do it right, one afternoon to setup your country's self-updating database. Not a bad little project, you learn a little, and you help make the world a better place.


    My country doesn't have online weather data

    Sorry to hear that. You're missing some great stuff.

    If you live in a country with a reasonably free press and reasonably fair elections, make a stink about it. You are probably already paying for it through taxes, why can't you have it?

    If you live somewhere else, then next time you have a revolution or coup, add 'open data' to the long list of needed reforms.


    What will this accomplish?

    This will create a free, sustainably updated, uniform, crowdsourced set of accurate worldwide data that will be easy to compile into a single global database. If you drop out, your online code will ensure another volunteer can step in.

    This is one fundamental tool that other free-weather projects have lacked. And any weather project can use this.

    The global database of locations is really small by most database standards. Small enough to easily fit on a phone. Small enough to be bundled with apps that can request data directly from original sources...once they can look up the correct source to use.


    How will this change the world?

    It's about simple tools that make
    it easy to create free, cool software.
    And it's about ensuring free access to data you already paid for.

    Not bad for one afternoon's contribution.

    Friday, December 12, 2014

    How to set up automatic Github uploads

    Back to basics

    For years, I have been playing with US Weather Data.

    A couple years ago, I came up with a lookup server for US weather locations, and -of course- had all kinds of plans for how to expand it to more services and more countries.

    So, of course, it went nowhere.

    This week, I took a look at the issue again, and came up with a simple set of lookup tables (yes, a tiny database) to replace the server. It reduced the amount of code by about 70%, and it's finally in a maintainable, expandable form. It's simple, and ready for a few drive-by patches.

    Which leads to the next question...how to share it with the world?

    Well, in this decade, sharing software and data means putting it on GitHub.


    Creating the GitHub repositories

    We have a Python 3 script that creates the database, another Python 3 example script that consumes the database, and the three CSV files that make up the database tables. Most consumers will probably just want the data for their own purposes - they really don't need the scripts (and it would be quite rude for millions of users to hit NOAA servers to produce their own databases each week)

    So I created two repositories on GitHub, one for the data and another for the optional scripts.

    If you haven't used GitHub, you first go online and create your account and create the repositories (with initial README file) in a web browser. Then:

    cd /tmp
    git clone https://github.com/ian-weisser/data   # Create a clone of the online repository
    cd data
    cp /home/somewhere/else/data/* ./               # Add existing data files into the local clone
    git add ./*                                     # Tell git to track the new data files
    git commit -m "Add initial data"
    git push origin master                          # Push the changes back to GitHub
      Username for 'https://github.com':
      Password for 'https://me@github.com': 
    cd /tmp 
    rm -rf /tmp/data                                # Clean up

    Overall, it's pretty simple. For the next iteration:
    • We must automate the upload credentials so it doesn't prompt for a username and password each time.
    • We must add a cron job that creates the data for upload.
    • Let's move this project from /tmp on my laptop to the server where it will live permanently. This also means that the server should pull the latest Python script...which is also hosted on GitHub.

    Let's setup the server:


    sudo apt-get install git    # On this server, about 3 MB of download
    sudo apt-get install python3-httplib2  # Script dependency
    mkdir uploads
    cd uploads
    git clone https://github.com/ian-weisser/data          # Setup the local 'data' repository
    git clone https://github.com/ian-weisser/weather       # Setup the local 'software' repository
    weather/nws_database_creator.py                        # Use the script to update CSVs in ~/uploads/data
    cd data
    git config --global user.email "my_email@example.com"  # First time setup
    git config --global user.name "My Name"                # First time setup
    git config credential.helper store                     # First time using this repo setup
    git add -A
    git commit -m "Test update"
    git push origin master
      Username for 'https://github.com': my-github-login-name   # First time using this repo setup
      Password for 'https://my-github-login-name@github.com':   # First time using this repo setup
    


    Automating the process

    So now subsequently, without all the first time setup:

    #!/bin/sh
    
    dir=/home/ian/uploads
    hub="https://github.com/ian-weisser"
    now=$(/bin/date +%F)
    
    cd $dir/weather
    /usr/bin/git pull "$hub"/weather      # Update the database_creator script
    
    cd $dir/data
    /usr/bin/git pull "$hub"/data         # Need to pull to be able to push later
    
    $dir/weather/nws_database_creator.py  # Do the update
    /usr/bin/git add -A                   # Record any changes
    /usr/bin/git commit -m "Scheduled update $now"
    /usr/bin/git push origin master
    exit 0
    


    And now we can run the shell script weekly from a user cron job:

    5 6 * * 7 /home/ian/uploads/cron/weekly_weather > /home/ian/uploads/weather.log 2>&1


    A couple of tests, and it works.

    It's completely automatic.

    Once a week, my server downloads the latest make-a-database script, builds the database, and uploads the database to GitHub for the world to use.


    The data


    If you want to see the final product (the lat/lon lookup tables for US weather), you can get them at:
    https://raw.githubusercontent.com/ian-weisser/data/master/metar.csv
    https://raw.githubusercontent.com/ian-weisser/data/master/radar.csv
    https://raw.githubusercontent.com/ian-weisser/data/master/zone.csv

    They will, of course, be automatically updated weekly.

    Tuesday, July 22, 2014

    Etherape as root

    This has popped up as an occasional question in the forums.
    If you monitor network activity using Etherape, it's worthless as a user level application. It must be run as root. But running as root requires opening a terminal and entering a password.

    Here's how to change the settings in Ubuntu 14.04 to create an Etherape (as root) launcher in Unity that won't require a password. It has only been tested with 14.04. Unity development moves fast - this method may not work for a different version of Ubuntu.

    It requires a minor edit to the /etc/sudoers file, and minor edits to two .desktop files. It's easy to undo if you change your mind.


    1) I like to document my changes. I have three changes to keep track of, in case I change my mind. So I'm adding a new subdirectory to keep track of this change.

    mkdir my_customizations/etherape_launcher




    2) Edit the sudoers file. This is the dangerous part. Pay close attention.

    You need to know your login account name (fred, Jane, dog, etc.)
    You need to know your hostname as shown in /etc/hostname.

    $ sudo visudo


    At the bottom of the file, add one line. Spacing, spelling, and CAPS are important.

    account_name hostname = NOPASSWD: /usr/bin/etherape


    Use your account_name and hostname, of course.

    Do a CTRL+X to exit, and hit Y to confirm the exit.
    Now look carefully at the screen.

    If visudo is happy with your changes, it will print no output, but simply return you to a command prompt.

    If the screen shows an error, type 'edit' and go right back in to fix the error. If you don't know how to fix the error, delete your added line and save. You MUST NOT exit with a broken sudoers file - a broken sudoers file is 100% guaranteed to break your system! It's very important that you pay close attention to any error messages visudo prints when you try to exit.



    3) Test: Open a NEW terminal window, and try 'sudo etherape'.
    It should no longer ask for a password.

    If you get sudo errors instead, you have broken your system! Stop! Save all your data! Reboot into a recovery console (since you cannot use sudo) and edit /etc/sudoers to undo your changes. Go to an Ubuntu help channel and ask for help on how to do this!



    4) Now that we know the sudo change works, move the sudoers line to a separate file

    We used visudo to ensure the correct syntax and a working sudoers file. But it's not a great idea to leave undocumented system changes laying around in system-installed (and system-rewritable) files. So let's move the customization to a separate file so we can keep track of it.

    $ sudo nano /etc/sudoers.d/etherape

    File contents:

    # Allows 'sudo etherape' to be run, including from a launcher, wthout password
    
    my_account my_hostname = NOPASSWD: /usr/bin/etherape


    Remember to change my_account and my_hostname to the proper values!

    Test your change using 'sudo etherape'. It should work without a password, and without an error message.

    Go back into visudo and delete the etherape line at the bottom.

    Test your change once more using 'sudo etherape'. It should work without a password, and without an error message.


    Now /etc/sudoers is back to it's original state, and our change is safely in a separate file. If we want to delete the change, we simply delete /etc/sudoers.d/etherape.



    5) Let's link this new sudoers file to our documentation so we can find it again.

    $ sudo ln /etc/sudoers.d/etherape \
              my_customizations/etherape_launcher/




    6) Edit the etherape-root .desktop file

    $ sudo nano /usr/share/applications/etherape-root.desktop


    File contents:

    [Desktop Entry]
    Name=EtherApe (as root)
    Type=Application
    Categories=GNOME;System;Network;
    Comment=Graphical Network Monitor
    Comment[es]=Monitor Gráfico de Red
    #TryExec=su-to-root                            <-- Comment out
    #Exec=su-to-root -X -c /usr/bin/etherape        <-- Comment out
    Exec=/usr/bin/sudo /usr/bin/etherape -i wlan0     <-- Add line
    Icon=etherape
    Terminal=false




    7) Edit the etherape (user) .desktop file to hide it. We don't want to delete it; the package manager will expect it to be there.

    $ sudo nano /usr/share/applications/etherape.desktop


    #[Desktop Entry]           <-- Comment out
    Version=1.0
    Encoding=UTF-8
    Name=EtherApe
    GenericName=EtherApe
    Comment=Graphical Network Monitor
    Comment[es]=Monitor Gráfico de Red
    Exec=etherape
    Terminal=false
    Type=Application
    Icon=etherape
    Categories=GNOME;System;Network;




    8) Let's save links to the changed desktop files so I can find them again someday:

    $ sudo ln /usr/share/applications/etherape-root.desktop \
              my_customizations/etherape_launcher/
    $ sudo ln /usr/share/applications/etherape.desktop \
              my_customizations/etherape_launcher/




    9) Test the new .desktop file.

    Open Unity's Dash.
    Close any existing search.
    Open a new search for Etherape. There should be only one option now: Etherape (as root).
    Open it. It should show all interfaces without requiring a password or prompting an error.
    Lock it to the Launcher (right-click).
    Close etherape. The launcher icon should remain.
    Open it again from the launcher. It should work

    How to recover from a broken sudo

    You cannot fix a broken sudoers file while logged in. You would need access to sudo for that...and it's broken. You can't get to the root account because it's locked in normal use.

    So you need to reboot into a condition where root is unlocked and available.

    1) Save all your work and then reboot.

    2) Hold down the left SHIFT button to get to the GRUB prompt.
    If you get a pretty splash screen, reboot and hold down the button earlier.

    3) At the Grub prompt, select Advanced Options

    4) At the Advanced Options prompt, select any Recovery Console

    5) Watch the boot technobabble run across your screen.

    6) At the Recovery Console prompt, select Drop To A Root Shell

    7) You should see a warning about a read-only filesystem.
    The prompt should look like '#'

    8) Remount the root filesystem as read-write instead of read-only. We want to make changes that stick.

    # mount -o remount /


    9) Try a test write. It should not cause an error:

    # date > /tmp/test


    10) Okay, now go into visudo and fix your mistake.

    11) When complete, reboot into normal Ubuntu.

    # shutdown -r now