Friday, December 28, 2012

From raw idea to useful source code

A couple months ago I had an Idea.

I even blogged about it: A lookup service for US National Weather Service codes. Those codes are necessary to access their machine-readable products.

In this post, I will show how I developed the idea into some code, how I grew the code into a project, added structure and version control, and finally moved the project onto online hosting.

This is not the only way to create a project.
This is probably not the best way for many projects.
It's just the way I did it, so you can avoid the most common mistakes.

You can see my final product hosted online at Launchpad.



From Idea to Code:

I know barely enough C to be able to ask there the bathroom is, so it's easier for me to use Python.

Code starts out as a single Python script:
- geolocation.py

As we add more features, a single script gets big and unwieldly, and we break it into smaller pieces.

For example, this structure easily allows more interfaces to be added.
- geolocation_service.py
- command_line_interface.py

Let's add a dbus interface, too. Dbus will semd messages to the interface if it knows about it. Let dbus know about it using a service file.
- dbus_interface.py
- dbus_service_file.service

Let's add an http interface, so not everyone in the world needs to download 5-6MB of mostly-unused lookup databases:
- http_interface.py
- specialized_webserver.py

Let's go back and formalize how we create the databases:
- database_creator.py

We have a lot of hard-coded variables in these scripts. Let's break them out into a config file.
- configfile.conf

There are other possible files, that we're not using. For example:
- Upstart config file (if we want the service to run/stop at boot or upon some system signal, lives in /etc/init)
- Udev rule file (if we want the service to run/stop when a device is plugged in, lives in /etc/udev/rules.d)

But that's a lot of files and scripts! 8 files, plus the databases.
 


Adding Version Control:

It's time to get serious about these eight files. We have invested a lot of time creating them, and it's time to start organizing the project so others can contribute, so we can track new bugs and features, and to protect all our invested work.

First, we need to introduce version control. Ideally, we would have done that from the start. But we didn't. So let's fix that.

Version control offers a lot of advantages:
    We can undo mistakes.
    It helps us package the software later.
    It helps us track bugs.
    It helps us apply patches.
    It helps us document changes.

There are plenty of good version control systems available. For this example, I'll use bazaar. The developers have a very good tutorial.

Installing bazaar:

$ sudo apt-get install bzr
$ bzr whoami "My Name "

Since we didn't start with proper version control, we need to create a new directory using version control, move our files into it, and add our files to version control.

$ bzr init-repo My_NEW_project_directory
$ bzr init My_NEW_project_directory
$ mv My_OLD_project_directory/* My_NEW_project_directory/
$ cd My_NEW_project_directory
$ bzr add *

Finally, we need to clean up the old directory, and commit the changes.

$ rm ../My_OLD_project_directory
$ bzr commit -m "Initial setup"


Organizing the code

My project directory is starting to get disorganized, with eight scripts and files, plus six database files, plus version control, plus more to come. I'm going to restructure my project folder like this:

My_project_directory
  +-- data   (all the database files)
  +-- src    (all the python scripts and other non-data files)
  +-- .bzr   (bzr's version control tracking)


Once version control is active, we cannot just move things around. We need to use the version control tools so it can keep tracking the right files.

$ bzr mkdir data src
$ bzr mv *.gz data/
$ bzr mv *.py src/
$ bzr mv dbus_service_file.service src/
$ bzr mv configfile.conf src/

See how bazaar adds the directories and performs the moves?

Now My_project_directory should be empty of files. Once reorganization is complete, remember to commit the change:

$ bzr commit -m "Reorganize the files to a better project structure"




Integrating into the system:

We have a problem with our eight files. They run beautifully, but only if they are in our home directory.

That won't work in the long run. A server should not be run as a user with shell access - that's a security hole. Nor should it be run out of a user's /home. Nor should it be run as root. Also, other applications that are looking for our server won't find it - all the files are in the wrong places.

So we need to put our files into the right places. And often that means fixing the scripts to replace hard-coded temporary paths (like '~/server/foo') with the proper locations ('/usr/lib/foo-server/foo').

Where are the right places?

The Linux Filesystem Hierarchy Standard (FHS) is used by Debian to define the right places.

Two files are directly user-launched in regular use:
- specialized_webserver.py: /usr/bin
- command_line_interface.py: /usr/bin

The database files are read-only and available to any application:
- database files: /usr/shared

Three files are launched or imported by other applications or scripts:
- geolocation_service.py: /usr/lib
- dbus_interface.py: /usr/lib
- http_interface.py: /usr/lib

One file is very rarely user-launched under unusual circumstances:
- database_creator.py

The  dbus service file will be looked for by dbus in a specific location:
- geolocation_dbus.service: /usr/share/dbus-1/services

Config files belong in /etc
- geolocation.conf: /etc

Makefiles make organization easier:

Now that we know where the right places are, let's create a Makefile that will install and uninstall the files to the right place. Our originals stay where they are - the makefile copies them during the install, and deletes the copies during uninstall.

Makefiles are really config files for the make application (included in the build-essential metapackage). Makefiles tell make which files depend upon which, which files to compile (we won't be compiling), and where the installed application files should be located, and how to remove the application.

Here is a sample makefile for my project (wbs-server):
DATADIR = $(DESTDIR)/usr/share/weather-location
LIBDIR  = $(DESTDIR)/usr/lib/wbs-server
BINDIR  = $(DESTDIR)/usr/bin
DBUSDIR = $(DESTDIR)/usr/share/dbus-1/services
CONFDIR = $(DESTDIR)/etc
CACHDIR = $(DESTDIR)/var/cache/wbs-webserver

install: 
 # Indents use TABS, not SPACES! Space indents will cause make to fail
 mkdir -p $(DATADIR)
 cp data/*.gz $(DATADIR)/

 mkdir -p $(LIBDIR)
 cp src/geolocation.py $(LIBDIR)/
 cp src/wbs_dbus_api.py $(LIBDIR)/
 cp src/wbs_http_api.py $(LIBDIR)/
 cp src/wbs_database_creator.py $(LIBDIR)/

 cp src/wbs_cli_api.py $(BINDIR)/
 cp src/wbs_webserver.py $(BINDIR)/
 cp src/wbs-server.service $(DBUSDIR)/
 cp src/confile.conf $(CONFDIR)/wbs-server.conf
 mkdir -p $(CACHDIR)

uninstall:
 rm -rf $(DATADIR)
 rm -rf $(LIBDIR)

 rm -f $(BINDIR)/wbs_cli_api.py
 rm -f $(BINDIR)/wbs_webserver.py
 rm -f $(DBUSDIR)/wbs-server.service
 rm -f $(CONFDIR)/wbs-server.conf
 rm -rf $(CACHDIR)

Let's save the makefile as Makefile, and run it using sudo make install and sudo make uninstall.

We run a test:

$ sudo make install
$ /usr/bin/wbs_cli_api.py zipcode 43210
bash: /usr/lib/wbs-server/wbs_cli_api.py: Permission denied

Uh-oh. Let's investigate:

$ ls -l /usr/lib/wbs-server/wbs_cli_api.py 
-rw-r--r-- 1 root root 3287 Dec 23 20:46 /usr/lib/wbs-server/wbs_cli_api.py

Aha. Permissions are correct, but the executable flag is not set. Let's uninstall the application so we can fix the makefile.

$ sudo make uninstall

In the makefile, we can make a few changes if we wish. We can set the executable flag. We can also create links or symlinks, or rename the copy.

For example, wbs_cli_api.py is a rather obtuse name for a command-line executable. Instead of copying it to /usr/bin, let's copy it to /usr/lib with its fellow scripts, make it executable, and create a symlink to /usr/bin with a better name like 'weather-lookup'

install:
        ...
 cp src/wbs_cli_api.py $(LIBDIR)/
 chmod +x $(LIBDIR)/wbs_cli_api.py
 ln -s $(LIBDIR)/wbs_cli_api.py $(BINDIR)/weather-lookup
        ...

uninstall:
        ...
 rm -f $(BINDIR)/weather-lookup
        ...


Another example: It's a bad idea to run a webserver as root. So let's add a couple lines to the makefile to create (and delete) a separate system user to run the webserver.

USERNAM = wbserver
        ...
install:
        ...
 adduser --system --group --no-create-home --shell /bin/false $(USERNAM)
 cp chgrp $(USERNAM) $(LIBDIR)/*
 cp chgrp $(USERNAM) $(CACHDIR)
 # Launch the webserver using the command 'sudo -u wbserver wbs-server'
        ...

uninstall:
        ...
 deluser --system --quiet $(USERNAM)
        ...




Sharing the code

Now we have a complete application, ready to distribute.

Thanks to the Makefile, we include a way to install and uninstall.

It's not a package yet. It's not even a source package yet. It's just source code and an install/uninstall script.

We can add a README file, a brief description of how to install and use the application.

We can also add an INSTALL file, detailed instructions on how to unpack (if necessary) and install the application.

It would be very very wise to add a copyright and/or license file, so other people know how they can distribute the code.

After all that, remember to add those files to version control! And to finally commit the changes:

bzr commit -m "Initial code upload. Add README, INSTALL, copyright, and license files."

Finally, we need a place to host the code online. Since I already have a Launchpad account and use bzr, I can easily create a new project on bzr.

And then uploading the version-controlled files is as simple as:

bzr launchpad-login my-launchpad-name
bzr push lp:~my-launchpad-name/my-project/trunk 

You can see my online code hosted at Launchpad.


Next time, we'll get into how to package this source code.

Sunday, December 16, 2012

Very Simple Database in Python

Experimenting with big lookup tables for my weather code lookup server. Instead of using a big configparse file, I want to try a small database.

Python's dbm bindings are included in the default install of Ubuntu. It's light and easy to use.

#!/usr/bin/env python3
import dbm.gnu                # python3-gdbm package
zipcodes = '/tmp/testdb'

# Create a new database with one entry
# Schema: Key is Zipcode
# Value is Observation_Station_Code, Radar_Station_Code, Forecast_Zone
zipc = dbm.gnu.open(zipcodes, 'c')
zipc['53207'] = b'kmke,mkx,wiz066'

# Close and reopen the database
zipc.close()
zipd = dbm.gnu.open(zipcodes, 'r')

# List of database keys
keys = zipd.keys()

# Retrieve and print one entry
print(zipd[keys[0]].decode('utf-8').split(','))

zipd.close()

It works very well and is very fast. It's not easy to view or edit the database with other applications, since it is binary (not text).

Saturday, November 24, 2012

Dbus Tutorial - GObject Introspection instead of python-dbus

Introduction
Introspection
Network Manager
Create a Service
GObject Introspection


In previous posts, I have looked at using the python-dbus to communicate with other processes, essentially using it the same way we use the dbus-send command.

There is another way to create DBus messages. It's a bit more complicated than python-dbus, and it depends upon Gnome, but it's also more robust and perhaps better maintained.

Using Gobject Introspection replacement for python-dbus is described several places, but the best example is here. Python-dbus as a separate bindings project has also suffered with complaints of "lightly maintained," and an awkward method of exposing properties that has been unfixed for years.


These examples only work for clients.  Gnome Bug #656330 shows that services cannot yet use PyGI.




Here's an example notification using Pygi instead of Python-DBus. It's based on this blog post by Martin Pitt, but expanded a bit to show all the variables I can figure out....


1) Header and load gi

#!/usr/bin/env python3

import gi.repository
from gi.repository import Gio, GLib


2) Connect to the DBus Session Bus
Documentation: http://developer.gnome.org/gio/2.29/GDBusConnection.html

session_bus = Gio.BusType.SESSION
cancellable = None
connection = Gio.bus_get_sync(session_bus, cancellable)


3) Create (but don't send) the DBus message header
Documentation: http://developer.gnome.org/gio/2.29/GDBusProxy.html

proxy_property = 0
interface_properties_array = None
destination = 'org.freedesktop.Notifications'
path = '/org/freedesktop/Notifications'
interface = destination
notify = Gio.DBusProxy.new_sync(
     connection,
     proxy_property,
     interface_properties_array,
     destination,
     path,
     interface,
     cancellable)


4) Create (but don't send) the DBus message data
The order is determined by arg order of the Notification system
Documentation: http://developer.gnome.org/notification-spec/#protocol

application_name = 'test'
title = 'Hello World!'
body_text = 'Subtext'
id_num_to_replace = 0
actions_list = []
hints_dict = {}
display_milliseconds = 5000
icon = 'gtk-ok'  # Can use full path, too '/usr/share/icons/Humanity/actions/'
args = GLib.Variant('(susssasa{sv}i)', (
                    application_name, 
                    id_num_to_replace,
                    icon, 
                    title,
                    body_text,
                    actions_list,
                    hints_dict,
                    display_milliseconds))


5) Send the DBus message header and data to the notification service
Documentation: http://developer.gnome.org/gio/2.29/GDBusProxy.html

method = 'Notify'
timeout = -1
result = notify.call_sync(method, args, proxy_property, timeout, cancellable)


6) (Optional) Convert the result value from a Uint32 to a python integer

id = result.unpack()[0]
print(id)

Play with it a bit, and you will quickly see how the pieces work together.



Here is a different, original example DBus client using introspection and this askubuntu question. You can see this is a modified and simplified version of the above example:

#!/usr/bin/env python3
import gi.repository
from gi.repository import Gio, GLib

# Create the DBus message
destination = 'org.freedesktop.NetworkManager'
path        = '/org/freedesktop/NetworkManager'
interface   = 'org.freedesktop.DBus.Introspectable'
method      = 'Introspect'
args        = None
answer_fmt  = GLib.VariantType.new ('(v)')
proxy_prpty = Gio.DBusCallFlags.NONE
timeout     = -1
cancellable = None

# Connect to DBus, send the DBus message, and receive the reply
bus   = Gio.bus_get_sync(Gio.BusType.SYSTEM, None)
reply = bus.call_sync(destination, path, interface,
                      method, args, answer_fmt,
                      proxy_prpty, timeout, cancellable)

# Convert the result value to a formatted python element
print(reply.unpack()[0])




Here is a final DBus client example, getting the properties of the current Network Manager connection

#!/usr/bin/env python3
import gi.repository
from gi.repository import Gio, GLib

# Create the DBus message
destination = 'org.freedesktop.NetworkManager'
path        = '/org/freedesktop/NetworkManager/ActiveConnection/19'
interface   = 'org.freedesktop.DBus.Properties'
method      = 'GetAll'
args        = GLib.Variant('(ss)', 
              ('org.freedesktop.NetworkManager.Connection.Active', 'None'))
answer_fmt  = GLib.VariantType.new ('(v)')
proxy_prpty = Gio.DBusCallFlags.NONE
timeout     = -1
cancellable = None

# Connect to DBus, send the DBus message, and receive the reply
bus   = Gio.bus_get_sync(Gio.BusType.SYSTEM, None)
reply = bus.call_sync(destination, path, interface,
                      method, args, answer_fmt,
                      proxy_prpty, timeout, cancellable)

# Convert the result value to a useful python object and print
[print(item[0], item[1]) for item in result.unpack()[0].items()]

As you can see from this example, dbus communication is actually pretty easy using GLib: Assign the nine variables, turn the crank, and unpack the result.

GeoClue vs Geocode-Glib

This post has been superseded by a more recent post with updated information.  

GeoClue, used in Gnome and Unity, is a Dbus service that consolidates location input from multiple sources to estimate a best location. It's lightly maintained, and the most recent maintainer has essentially deprecated it in favor of his newer Geocode-Glib

That announcement is here, plus a bit of reading-between-the-lines and a few subsequent clarifications.

The big advantage if geocode-glib0, is that it uses GObject introspection instead of specialty DBus bindings. The big disadvantage is that it leaves KDE and other non-Gnome environments unsupported...and has less functionality that the older Geoclue.

For now, it looks like I need to stick with GeoClue, and perhaps even help maintain it for my current weather-backend project. geocode-glib simply doesn't do the job I need GeoClue to do.


Here is an example of using python and geocode-glib to get geolocation information. I have not seen any examples of geocode-glib anywhere else, so I may be first here to use the lib with python:

#!/usr/bin/env python3
import gi.repository
from gi.repository import GeocodeGlib

# Create an object and add location information to it 
location = GeocodeGlib.Object()
location.add('city','Milwaukee')

# Do the gelocation and print the response dict
result = location.resolve()
[print(item[0], item[1]) for item in result.items()]

Give it a try...
The relevant packages are libgeocode-glib0 and gir1.2-geocodeglib-1.0
The Geocode-Glib source code and API reference are hosted at Gnome.

Saturday, October 13, 2012

Dbus Tutorial - Create a service

Introduction
Introspection
Network Manager 
Create a Service
GObject Introspection



A dbus service is usable by other applications. It listens for input from another process, and responds with output.

When you create a service, you need to make a couple decisions about when you want to start your service, and when you want to terminate it...

Start: Startup, login, first-use, on-demand?
End: Each time? logout? Shutdown?
In other words, is this a single-use service, or a forever-running daemon?

Happily, the actual code differences are trivial. Dbus itself can launch a service that's not running yet. (Indeed, a lot of startup and login depends on that!)



Example Dbus Service in Python3

Here's an example of  a self-contained daemon written in Python 3 (source). It's introspectable and executable from dbus-send or d-feet. When called, it simply returns a "Hello, World!" string.

It can be started by either dbus or another process (like Upstart or a script). Since it runs in an endless loop awaiting input, it will run until logout. It can also be manually terminated by uncommenting the Gtk.main_quit() command.

#!/usr/bin/env/python3
# This file is /home/me/test-dbus.py
# Remember to make it executable if you want dbus to launch it
# It works with both Python2 and Python3

from gi.repository import Gtk
import dbus
import dbus.service
from dbus.mainloop.glib import DBusGMainLoop

class MyDBUSService(dbus.service.Object):
    def __init__(self):
        bus_name = dbus.service.BusName('org.me.test', bus=dbus.SessionBus())
        dbus.service.Object.__init__(self, bus_name, '/org/me/test')

    @dbus.service.method('org.me.test')
    def hello(self):
        #Gtk.main_quit()   # Terminate after running. Daemons don't use this.
        return "Hello,World!"

DBusGMainLoop(set_as_default=True)
myservice = MyDBUSService()
Gtk.main() 



Daemon that runs all the time

Just run the script at startup (or login). Or send a dbus-send message to the service, and dbus will start it. It will be terminated as part of shutdown (or logout). While it's running, it's introspectable and visible from d-feet.


Dbus-initiated start

Add a .service file. This file simply tells dbus how to start the service.

Here's an example service file:

# Service file: /usr/share/dbus-1/services/test.service
[D-BUS Service]
Name=org.me.test
Exec="/home/me/test-dbus.py" 

Dbus should automatically pick up the new service without need for any restart. Let's test if dbus discovered the service:

$ dbus-send --session --print-reply \
       --dest="org.freedesktop.DBus" \
       /org/freedesktop/DBus \
       org.freedesktop.DBus.ListActivatableNames \
       | grep test
string "org.me.test"

The new service does not show up in d-feet until after it is run the first time, since before there is nothing to probe or introspect. But it does exist, and is findable and usable by other dbus-aware applications.

Let's try the new service:

$ dbus-send --session --print-reply \
--dest="org.me.test" /org/me/test org.me.test.hello

method return sender=:1.239 -> dest=:1.236 reply_serial=2
   string "Hello,World!"

$ dbus-send --session --print-reply \
--dest="org.me.test" /org/me/test org.me.test.Frank

Error org.freedesktop.DBus.Error.UnknownMethod: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/dbus/service.py", line 654, in _message_cb
    (candidate_method, parent_method) = _method_lookup(self, method_name, interface_name)
  File "/usr/lib/python3/dist-packages/dbus/service.py", line 246, in _method_lookup
    raise UnknownMethodException('%s is not a valid method of interface %s' % (method_name, dbus_interface))
dbus.exceptions.UnknownMethodException: org.freedesktop.DBus.Error.UnknownMethod: Unknown method: Frank is not a valid method of interface org.me.test

It worked! Dbus launches the script, waits for the service to come up, then asks the service for the appropriate method.Upon execution of the method, the waiting loop terminates, and the script finishes and shuts down.

As a test, you can see that 'hello' is indeed a valid method and returns a valid response, while the invalid method 'Frank' causes a not-found error.


Dbus-initiated stop

Dbus doesn't stop scripts or processes. But a script can stop itself.

In order to wait for input, python-dbus uses a Gtk.main() loop. In this case, simply uncomment the line Gtk.main_quit(). When the method is called,the main() loop gets terminated, and the script continues to the next loop or end.

If you use on-demand starting and stopping, be aware that the service will exist, but will be visible in d-feet or introspectable only for the few seconds it's actually running.



Obsolete: Before python included introspection, you needed to include an interface definition. But you don't need this anymore - introspection seems to have replaced it. Avoid confusion - some old tutorials out there still include it.

<?xml version="1.0" encoding="UTF-8"?>
<!-- /usr/share/dbus-1/interfaces/org.me.test.xml -->
<node name="/org/me/test">
        <interface name="org.me.test">
                <annotation name="org.freedesktop.DBus.GLib.CSymbol" value="server"/>
                <method name="EchoString">
                        <arg type="s" name="original" direction="in" />
                        <arg type="s" name="echo" direction="out" />
                </method>
                <!-- Add more methods/signals if you want -->
        </interface>
</node>




Wednesday, October 3, 2012

Python 3: Using httplib2 to download just a page header

I want to use the National Weather Service geolocator for my private weather script.

When my laptop moves to a new location, the script automatically figures out the correct weather to show.

The NWS geolocator uses web page redirection. If I tell it that I want the web page for "Denver, CO," it redirects me to a web page for the appropriate Latitude/Longitude. I don't actually want the web page - I get the data from other sources...but I do sometimes want that redirect so I can parse the lat/lon pair.


>>> import httplib2
>>> url = "http://www.example.org"
>>> h = httplib2.Http()
>>> h.follow_redirects = False
>>> head = h.request(url, "HEAD")
>>> head
({'status': '302', 'connection': 'Keep-Alive', 'location': 'http://www.iana.org/domains/example/', 'content-length': '0', 'server': 'BigIP'}, b'')
>>> head[0]['location']
'http://www.iana.org/domains/example/'


Saturday, September 29, 2012

Python3 configparse

configparse is a Python Module, part of the standard library, that read and writes config files and treats them almost like python dictionaries.

Well, almost.

Example dictionary
>>> test = {}
>>> test['Cleese'] = {'a':'1', 'b':'2'}
>>> test['John'] = {'f':True, 'g':'Hello'}

Example config object
>>> import configparser
>>> test = configparser.ConfigParser()
>>> test['Cleese'] = {'a':'1', 'b':'2'}
>>> test['John'] = {'f':True, 'g':'Hello'}


First, let's get the top-level list of sections:

Get the list of sections for a dict
>>>test
{'John': {'g': 'Hello', 'f': True}, 'Cleese': {'a': '1', 'b': '2'}}
>>> test.keys()
dict_keys(['John', 'Cleese'])
>>> list(test.keys())
['John', 'Cleese']

Let's try the same for a config object:
>>> test              
<configparser .configparser=".configparser" 0xb755c34c="0xb755c34c" at="at" object="object">
>>> test.keys()
KeysView(<configparser .configparser=".configparser" 0xb755c34c="0xb755c34c" at="at" object="object">)
>>> test.sections()
['Cleese', 'John']


Second, let's get the list of keys in one section:

Get the list of keys for one section of a dict
>>> test['Cleese']
{'a': '1', 'b': '2'}
>>> list(test['Cleese'].keys())
['a', 'b']

Same task for a config object:
>>> test['Cleese']
<section: cleese="cleese">
>>> test['Cleese'].sections()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'SectionProxy' object has no attribute 'sections'
>>> [a for a in test['Cleese']]
['a', 'b']


Third, let's get the key-value pairs in the same section:

Get the key-value pair for one section of a dict
>>> test['Cleese']
{'a': '1', 'b': '2'}
>>> [(a, test['Cleese'][a]) for a in test['Cleese'].keys()]
[('a', '1'), ('b', '2')]
>>> [print(a, test['Cleese'][a]) for a in test['Cleese'].keys()]
a 1
b 2
[None, None]

Same task for a config object:
>>> [(a, test['Cleese'][a]) for a in test['Cleese']]
[('a', '1'), ('b', '2')]
>>> [print(a, test['Cleese'][a]) for a in test['Cleese']]
a 1
b 2
[None, None]


Finally, let's dump ALL the key-value pairs in the whole object:

Iterate and dump all key-value pairs in a dict
>>> [[print(b, test[a][b]) for b in test[a].keys()] for a in test.keys()]
g Hello
f True
a 1
b 2
[[None, None], [None, None]]

Dump all in a configparse object
>>> [[print(b, test[a][b]) for b in test[a]] for a in test]
a 1
b 2
g Hello
f True
[[], [None, None], [None, None]]
>>> [[print(b, test[a][b]) for b in test[a].keys()] for a in test.sections()]
a 1
b 2
g Hello
f True
[[None, None], [None, None]]