Simutrans packaging bug

After a bit of scratching my head, I finally figured out that Simutrans 110.0.1 (the version in Ubuntu 11.10) and the Debian-packaged version that is actually in Ubuntu have a weird incompatibility.

About a year ago, Simutrans added a network-multiplayer-server feature. But Debian/Ubuntu-packaged clients can only communicate with Debian/Ubuntu-packaged servers. Similarly, manually-installed clients can only talk to manually installed servers.

Well, that's certainly not intended behavior - all the servers should be able to talk to all the clients, regardless of whether they were packaged or not!

So I filed LP #931181, because the Debian Games Team needs to know.
And I opened a thread on the developer's forum.

Clue: One developer immediately noted that it seemed to be a checksum issue. (Thanks, Dwachs!)

Well, let's check that. Let's take one item in the pakset, building.PostOffice.pak, and see how it checksum:

Let's hash it manually first:

$ openssl sha1 ~/simutrans110/simutrans/pak/building.PostOffice.pak 
SHA1(/home/me/simutrans110/simutrans/pak/building.PostOffice.pak)= 8b1b5caa71013474039f357094cd1716bd3800bd

$ openssl sha1 /usr/share/games/simutrans/pak/building.PostOffice.pak 
SHA1(/usr/share/games/simutrans/pak/building.PostOffice.pak)= 8b1b5caa71013474039f357094cd1716bd3800bd

So the those pakset items (and, by extension, probably the entire pakset) are identical.

Next, let's check the logs of success and failure:

Succesful (Ubuntu 11.10) server: Message: pakset_info_t::debug: PostOffice -> sha1 = B80C3C262A2758C8FC54FF129272F564B7DA8AAF

Succesful (Ubuntu 11.10) client: Message: pakset_info_t::debug: PostOffice -> sha1 = B80C3C262A2758C8FC54FF129272F564B7DA8AAF

Failed (Manually-installed) server: Message: pakset_info_t::debug: PostOffice -> sha1 = 263C0CB8C858272A12FF54FC64F57292AF8ADAB7

Failed (Ubuntu 11.10) client: Message: pakset_info_t::debug: PostOffice -> sha1 = B80C3C262A2758C8FC54FF129272F564B7DA8AAF

It seems that Dwachs put me on the right track. Given identical files to checksum, the two binaries checksum differently, and both do it differently than Ubuntu's default method, even though each claim to be using the same sha1!


Next, let's look for the changed sha-1 patch at the Debian Games Team git repository.

Aha! February 16, 2011: Add replacement for non-free SHA-1 implementation
Aha! February 16, 2011: get-orig-source: Remove non-free SHA-1 implementation.

The patch, is clearly a replacement for the original Simutrans trunk/utils/sha1.cc
And, looking at the current tree, the patch is still applied.

Er, did I just walk into a year-old dispute between Debian and Simutrans devs?

Next, I need to prove that the patch is the culprit. I need to apply the patch to a stock simutrans source, compile it, run it as a server (in debug mode), successfully join the game from a debian-install client.


Happily, I already have a stock 110.0.1 source, setup, and pakset in my simutrans110 directory, from earlier troubleshooting. Full instructions on how to compile are in this post.

Let's build the unpatched (stock) version first. This should be compatible with a manual install, and incompatible with the Debian version.

$ cp simutrans110 r4359-stock
$ cd r4359-stock/svn/trunk
$ make clean
$ make
$ strip sim
$ cp sim ~/r4359-stock/simutrans/sim-unpatched

Now, let's build a patched version. This should be compatible with the Debian version, and incompatible with a manual install.

$ sudo apt-get install libssl-dev
$ cp simutrans110 r4359-patched
$ cp Desktop/debian_patches_sha1-replacement.diff r4359-patched/svn/trunk
$ cd r4359-patched/svn/trunk

  --- Apply the patch ---
$ rm utils/sha1.*
$ patch Makefile debian_patches_sha1-replacement.diff 
$ patch utils/checksum.h debian_patches_sha1-replacement.diff
$ nano utils/sha1.cc
$ nano utils/sha1.h

$ make clean
$ make
$ strip sim
$ cp sim ~/r4359-stock/simutrans/sim-patched

Finally, we test.

Ubuntu 11.10's package install of Simutrans 110.0.1 can join a game served by sim-patched, but not a game served by sim-unpatched.

A manually-installed version of 110.0.1 is, as expected, the reverse. It can join a game served by sim-unpatched, but not a game served by sim-patched.

Creating a patch to fix an Ubuntu bug

Today I'm preparing a patch to fix Launchpad Bug #117984, which is also Gnome Bug #451734.

References: Ubuntu Wiki, Ubuntu Packaging Guide

It turns out that to do it properly, you need to make two patches. The first one is a .diff file for Debian and upstream. The second creates a .debdiff patch for Ubuntu.

There are circumstances, of course, where one of the patches is not neccessary, and some of these steps can be skipped.

Note: Use steps 1-9 & 14-15 for just an upstream (.diff) patch. Use steps 1-4,5-7,11-13 & 15 for just an Ubuntu (.debdiff) patch.
1. Open a terminal window
2. Create a working directory with the command mkdir working
3. Move to the working directory with the command cd working
4. Download the latest source package using the command apt-get source packagename. This method automatically appends an '.orig' suffix, and unpacks the file, too. DON'T download the source from packages.ubuntu.com; instead ADD the repos to your Software Sources control panel using the instructions at the Ubuntu Wiki
5. Make a copy of the unpacked folder with cp -r package-folder package-folder-orig
6. Go into the unpacked folder (not the orig) with cd package-folder.
7. Edit the file using nano path/to/file/to/fix. Fix the file
8. Return to the working directory with cd ..
9. Use diff -Nurp package-folder-orig package-folder > upstream-bug#.diff to create the upstream patch.
10. Go back into the unpacked folder (not the orig) with cd package-folder.
11. Use the command dch -i to update the changelog. Show the change and list the bug# fixed.
12. Use debuild -S -us -uc to create the debdiff patch.
13. Attach the .debdiff patch to the bug in Launchpad.
14. Attach the upstream .diff patch to the Launchpad bug AND the upstream bug.
15. Delete the working directory, and all contents.

Update February 5, 2009: Accepted and fixed!