Ian's TechBlog: 2011

Thursday, December 22, 2011

Missing a file during compile

During a compile, I kept getting 'missing file' errors.

Solution: Use packages.debian.org to fing the package that includes the missing file, so I can install it.

Friday, December 2, 2011

This post is an update to my original in March 2011.

Every time I reconnect to the internet, my router is given a different IP address by the upstream internet service provider. That's called dynamic hosting. But if I'm out on the internet, I cannot connect to my server unless I know the current IP. A Dynamic DNS (ddns) host simply tracks my ever-changing IP for me in an internet-accessible way.

The ddns host isn't psychic - I need a small piece of software (a ddns client) on my server that figures out whenever the IP changes, and sends that update to the ddns host.

The 'ddclient' package is the server program that sends the new IP to the ddns host. Install ddclient using the command apt-get install ddclient. The installer will ask questions about the newly-registered dynamic dns account.

There are two ways to run ddclient - as a daemon or as an occasionally-triggered non-daemon.
- To run ddclient as a daemon (always-on), edit the file /etc/init.d/ddclient, and change the line:
```
run_daemon=false  #from
run_daemon=true   #to
```
  Restart ddclient (service ddclient restart) to reload the new config.
- To run ddclient as a non-daemon, take a look in the /usr/share/doc/ddclient/examples directory for a bunch of possible trigger mechanisms.
  I chose to run ddclient whenever my server picks up a new IP address:
```
cp /usr/share/doc/ddclient/examples/sample-etc_dhclient-exit-hooks /etc/dhcp/dhclient-exit-hooks.d/ddclient
```
  It's not perfect for my use, but we'll fix that later.

That should be it. ddclient should automatically update the ddns host regularly (or event-driven) now.

Usage: If you're out on the internet and need to know your server's IP, just look it up:

dig +short myaccount.dyndns.org                              # Elsewhere on the internet
nslookup myaccount.dyndns.org ns.dyndns.org                  # Another way from the internet
http://www.dnscog.com/dig/myaccount.dyndns.org/              # As a web page

Or have the service look it up for you by simply using the domain name: ssh myaccount.dyndns.org

Kingbaron: Sometimes my server is also an internet-facing router, sometimes it's just a LAN server. I don't want it erroneously feeding LAN addresses to the ddns. Happily, the dhclient-exit-hook script thought of that already. If the new IP address is in the normal LAN range, ddclient ignores it.

Tuesday, November 29, 2011

Firewall improvement with fwknop

fwknop is an improvement upon the older technique of port-knocking. Quite simply, it looks like an ordinary failed attempt to connect blocked by a closed firewall...but something about the packet identifies it as trustable, and causes the host system to do an action (like briefly open a firewall port).

There are safeguards to prevent the packet from being spoofed, and until the packet is received my SSH port stays safely blocked by the firewall - preventing the use of clever undiscovered exploits.

This is pretty cool because it means that my ssh port can be safely closed behind a firewall, yet still accessible to me (and only me) on the internet.

There are a whole bunch of moving pieces here (firewall, ports, services, client, keys), so I'll talk a lot of detail about the installation and configuration, the some options, then try an example or two. I haven't even touched the options for using my GPG-key for knocking!

Planning: Ahead of time, fill in this list:

I want the knock to occur on UDP port #: (default 62201)
I want be able to open ports for the following services: (example: sshd)
Those services run on ports: (example: tcp/22)
I want my knock password (key) to be: (8 characters min -or- GPG)
I want to be notified of a successful knock by: (log/email/none)
The interface I want to protect is: (only one, usually the same internet-facing interface described in your firewall)

Install and configure fwknop on the server:

Server Install:

apt-get install fwknop-server   # Debian

Enable fwknopd to autostart upon boot. Edit /etc/default/fwknop-server
```
START_DAEMON="no"  #From
START_DAEMON="yes" #To
```
Customize the server setup by editing /etc/fwknop/fwknop.conf:
```
EMAIL_ADDRESSES             root@localhost;  # From
EMAIL_ADDRESSES             MyLoginAccountName@localhost;  #to

PCAP_INTF                   eth0;  # From
PCAP_INTF                   ppp0;  # To

PCAP_FILTER                 udp port 62201;                         # From
PCAP_FILTER                 udp port <some other port number>; # To
```
PCAP_INTF should be the same internet-facing interface used in your firewall (for example, ppp0). You can only choose ONE interface. In this case, ppp0 protects the internet-facing interface only so it does not protect against, say, hackers on the LAN wifi.
PCAP_FILTER is the udp port to listen for the knock on. Remember that PCAP_FILTER port number - you'll need that again on the client.
Set the server access controls and keys by editing /etc/fwknop/access.conf:
```
# Change all three of the following, if needed.
OPEN_PORTS: tcp/22,tcp/23,tcp/24,udp/85;
KEY: <Your-8-digit-key>;
FW_ACCESS_TIMEOUT: 30;
```
OPEN_PORTS is the list of ports fwknopd is permitted to open. The client will request a specific port on the list. Remember that 8-digit key, you'll need it again on the client.
Edit the firewall to close all fwknop-protected ports. The fwknopd daemon will tell iptables to open the port as needed.
```
iptables -A INPUT -i lan0 -p tcp -dport 22 -j ACCEPT
iptables -A INPUT -i ppp0 -p tcp -dport 22 -j REJECT
```
The first line opens the firewall for ssh from the LAN (not protected by fwknopd). The second line closes the firewall for ssh from the internet-facing ppp0. If the default policy is already DROP or REJECT, you don't need a separate entry at all and the second line can be deleted. fwknopd can open a port regardless of the default policy or any explicit closure rule.
Restart fwknopd (and optionally the firewall) to load the new config(s):
```
fwknopd -Restart
```

Optional Server Notes:

E-mail spamming of successful knocks: I got tired of that pretty fast, so I moved the notifications to a log:
- In /etc/fwknop/fwknop.conf, change ALERTING_METHODS from ALL; to noemail;
- Create a new file /etc/rsyslog.d/fwknop.conf:
```
# Log fwknopd actions to syslog, auth.log, and iptables.log
:msg, contains, "fwkno" -/var/log/auth.log
:msg, contains, "fwkno" -/var/log/iptables.log #my own custom iptables-related log
```
- Restart rsyslogd to reload the new config: service rsyslog restart
Server Testing Tip: To prevent locking yourself out of the server (and feeling like a chump), here's a way to test fwknop without erroneously locking yourself out:
- Edit /etc/ssh/sshd.conf to add a second ssh port (let's say port 2233)
- Restart sshd to load the new config: service ssh restart
- Use netstat -tulpn to ensure sshd is really listening on both ports
- Edit your firewall so the new port (2233) is either explicitly closed or closed by the default policy (it doesn't matter which as long as it's closed). If necessary, reload your firewall to apply the changes.
- Configure fwknopd to listen on your LAN interface, and only to control the ssh test port.
- Do your testing and experimenting. Your original ssh port is unaffected.
- When testing is complete, edit /etc/ssh/sshd.conf to remove the testing port, remove any testing-port related lines in your firewall, and edit /etc/fwknopd/access.conf to remove the test port.

Install and configure fwknop on the client:

Client Install:

sudo apt-get install fwknop-client   # Ubuntu

Usage with manually-entered key:
```
$fwknop --Server-port 12345 --Access 'tcp/22' -R --Destination myserver.dyndns.org
Enter your key: <Your 8-digit key>
$ssh -p 22 myserver.dyndns.org
```
Server-port is the PCAP_FILTER, the UDP port to send the knock on.
Access is the firewall port to open. It must be on the server's OPEN_PORTS list.
The 8-digit key must be the same, of course, as on the server.
The followup ssh command to connect on the briefly-open port is just an example.
Create a keyfile for automatic use (optional): Use the format servername: key
```
echo "myserver.dyndns.org: <Your 8-digit-key> > .knock-keyfile
```

Usage with keyfile key (optional - handy for scripting):

$fwknop --Server-port 12345 --Access 'tcp/22' -R --Destination myserver.dyndns.org --get-key .knock-keyfile
$ssh -p 22 myserver.dyndns.org

Booting an old VIA EPIA-5000L motherboard

I purchased an old VIA fanless mini-ITX motherboard on eBay, including CPU and RAM and shipping, for a reasonable price ($43). I want to play with it and see what I can get it to do. Maybe build a cool case for it...

MB attached to ordinary ATX power supply, VGA monitor, and PS/2 Mouse.
Tiny black rectangle on the right is the on/off jumper.

Step 1: Hook it up

Happily, I have an old PS/2 keyboard, VGA monitor, and ATX power supply handy.
Hook them all up...and nothing.

Of course, the power switch.
Get the manual, read the manual, find the jumper, and use a spare jumper to connect them for a moment.

Hooray! Fans on the power supply start to spin! Beep! Splash appears on the monitor!

I made the following BIOS changes:

Always restart upon power loss (since I don't have a power switch!)
Enable fast startup (skip memtest, faster boot)
Skip splash (annoying)
Skip BIOS summary (annoying)
Change the boot device to USB-ZIP (not USB-FDD, essential for USB-boot later)

Step 2: Create a Debian Boot USB drive.

This is a pretty old motherboard, so the standard USB-HDD booting option is missing. That means Live-USB (hybrid USB/CD) .iso files won't work without some tweaking. The available USB options are USB-ZIP and USB-FDD. A quick web search turns up this page of instructions, but it doesn't work. Further digging reveals this page, a version of which does work.

Select a USB Drive (small). Everything on it will be erased, so backup if needed.
On a different (working) Debian or Ubuntu system, as root (use 'sudo su' to get a root prompt on Ubuntu). Not all commands will work with sudo alone.
Plugin the USB drive. If your system automatically mounts it, then unmount (not eject) it.
Use dmesg to find the USB drive's /dev/sd* node. For example, mine was sdz and sdz3.
Format the USB drive to a bootable configuration. This will limit the accessible size to 250MB (the old ZIP-format maximum), so CD-sized live .isos won't work. See this page for mkdiskimage commands for other-size drives.
```
mkdiskimage -4 /dev/sdz 0 64 32   # 1GB and below USB drives only
```

Install the MBR:

cat /usr/lib/syslinux/mbr.bin > /dev/sdz

Get and install the boot image:

wget -O /tmp/boot.img.gz http://ftp.debian.org/debian/dists/squeeze/main/installer-i386/current/images/hd-media/boot.img.gz
zcat /tmp/boot.img.gz > /dev/sdz1

Download the Live Session image. In this case, I used a Debian businesscard installer simply because it was a very small download:
```
wget -O /tmp/debian-6.0.3-i386-businesscard.iso http://cdimage.debian.org/debian-cd/6.0.3/i386/iso-cd/debian-6.0.3-i386-businesscard.iso
md5sums /tmp/debian-6.0.3-i386-businesscard.iso
```
Compare the md5sum result to http://cdimage.debian.org/debian-cd/6.0.3/i386/iso-cd/MD5SUMS

Mount the USB drive, and install the Live Session image:

mkdir /tmp/usbmountpoint
mount /dev/sdz3 /tmp/usbmountpoint
cp /tmp/debian-6.0.3-i386-businesscard.iso /tmp/usbmountpoint/
umount /tmp/usbmountpoint

Check the resulting filesystem, and correct any errors (there may be a couple!)
```
fsck.msdos -r /dev/sdz3
```
My resulting filesystem is about 214MB, below the 250MB limit.
Clean-up is optional, since all working files are in /tmp and will be deleted upon reboot anyway.
Unplug the USB drive (it should already be unmounted)

Step 3: Boot Debian

The BIOS should be set to a startup device of USB-ZIP (step 1), and you should have a Debian USB drive (step 2). Put them together - try to boot the new motherboard from it. In my case, success!

I don't have a hard drive installed yet, so I'm not ready to actually install. But this was a successful power-on and boot.

UPDATE #1: For Xmas, one of the kids got a toy butterly-in-a-jar. It has a "Try Me" feature for in the store, a button that detached after opening the package. The button's connector looked just right to fit the power jumper, and indeed now I have a real power button for the motherboard. This turns out to be handy, because a stable system needs better power control than flipping the switch on the power supply...and my new silent power supply has no switch!

UPDATE #2: A chainloader can be used to bypass the USB-ZIP 250MB boot limit. I did a successful USB-ZIP boot to a complete Debian 6 system.

Sunday, November 27, 2011

GPS dongle, gpsd, and Ubuntu 11.10

This post is an update to my original experiment #1 and experiment #2 with a USB GPS dongle two years ago.

Install the hardware: Plug it in, of course.

Check the kernel message log to discover the GPS dongle location in the filesystem:

$ dmesg | grep tty
[1769190.919411] usb 2-1: pl2303 converter now attached to ttyUSB0

Test for signal with the following command:

$ cat /dev/ttyUSB0 
5�y�220,N,08754.5061,W,000.0,316.5,271111,,,A*71
$GPVTG,316.5,T,,M,000.0,N,000.0,K,A*0C
$GPGGA,173905.000,4259.7220,N,08754.5060,W,1,08,1.0,190.0,M,-33.8,M,,0000*63
$GPGSA,A,3,12,04,10,23,02,05,17,25,,,,,1.7,1.0,1.4*31
$GPRMC,173905.000,A,4259.7220,N,08754.5060,W,000.0,316.5,271111,,,A*71

Use CTRL+C to end the signal check.
If you get a bunch of zero-data, then try moving closer to a window, since GPS signals are line-of-sight. Or try a USB hub or extension cable, to reduce interference from your computer.

Install the gps daemon so you can use gps data.

```
sudo apt-get install gpsd gpsd-clients
```
Normally, udev will start gpsd automatically when a USB GPS device is inserted. If it fails to start after 5-10 seconds, you can start it manually:
```
gpsd /dev/ttyUSB0  #You must tell gpsd where to look for the dongle
```
Test the GPS and gpsd output using the gpsmon and cgps terminal programs, or the xgps program (all included in the gpsd-clients package)

Capture data for recycling (optional - handy for development). gpsfake will simulate gpsd output

$cat /dev/ttyUSB0 > path/to/datalog  # Create data file. Use CTRL+C to end the capture.
$gpsfake path/to/datalog             # Run gpsd simulator (not a daemon - it will occupy the terminal)

Command-line tools to get GPS data

Use the gpspipe command to get gpsd data:

$ gpspipe -w -n 5
netlib_connectsock() returns socket on fd 3
{"class":"VERSION","release":"2.95","rev":"2011-07-27T11:20:24","proto_major":3,"proto_minor":3}
{"class":"DEVICES","devices":[{"class":"DEVICE","path":"/dev/ttyUSB0"}]}
{"class":"WATCH","enable":true,"json":true,"nmea":false,"raw":0,"scaled":false,"timing":false}
{"class":"TPV","tag":"RMC","device":"/dev/ttyUSB0","time":1322418390.001,"ept":0.005,"lat":42.995410000,"lon":-87.908430000,"alt":195.400,"epx":11.739,"epy":17.294,"epv":36.800,"track":342.2000,"speed":0.000,"climb":0.000,"eps":34.59,"mode":3}
{"class":"SKY","tag":"GSV","device":"/dev/ttyUSB0","xdop":0.66,"ydop":1.15,"vdop":1.46,"tdop":1.01,"hdop":1.32,"gdop":2.22,"pdop":1.97,"satellites":[{"PRN":2,"el":77,"az":360,"ss":26,"used":true},{"PRN":10,"el":58,"az":72,"ss":32,"used":true},{"PRN":12,"el":51,"az":247,"ss":34,"used":true},{"PRN":5,"el":45,"az":192,"ss":21,"used":true},{"PRN":4,"el":40,"az":65,"ss":20,"used":true},{"PRN":25,"el":39,"az":292,"ss":21,"used":true},{"PRN":13,"el":15,"az":60,"ss":18,"used":true},{"PRN":24,"el":14,"az":258,"ss":0,"used":false},{"PRN":29,"el":12,"az":304,"ss":23,"used":false},{"PRN":17,"el":8,"az":121,"ss":17,"used":true},{"PRN":23,"el":7,"az":35,"ss":10,"used":false},{"PRN":31,"el":0,"az":334,"ss":0,"used":false}]}

Let's refine the command a bit, limit to the first "TPV" (time, position, velocity) sentence:

$ gpspipe -w -n 5 | grep -m 1 TPV
netlib_connectsock() returns socket on fd 3
{"class":"TPV","tag":"RMC","device":"/dev/ttyUSB0","time":1322418481.001,"ept":0.005,"lat":42.995406667,"lon":-87.908428333,"alt":195.300,"epx":10.003,"epy":17.383,"epv":36.800,"track":342.2000,"speed":0.000,"climb":0.000,"eps":34.77,"mode":3}

We can use basic shell commands to extract single variables from this sentence:

tpv=$(gpspipe -w -n 5 | grep -m 1 TPV | cut -d, -f4,6-8,13)
seconds=$(echo $tpv | cut -d, -f1 | cut -d: -f2)
latitude=$(echo $tpv | cut -d, -f2 | cut -d: -f2)
longitude=$(echo $tpv | cut -d, -f3 | cut -d: -f2)
altitude=$(echo $tpv | cut -d, -f4 | cut -d: -f2)
speed=$(echo $tpv | cut -d, -f5 | cut -d: -f2)

For fun, let's compare GPS time with system time using shell commands:

date -u +%s   # System utc time in seconds
gpspipe -w -n 5 | grep -m 1 TPV | cut -d, -f4 | cut -d: -f2   # GPS utc time in seconds

# Let's compare them side-by side
$ echo System: $(date -u +%s) , GPS: $(gpspipe -w -n 5 | grep -m 1 TPV | cut -d, -f4 | cut -d: -f2) 
netlib_connectsock() returns socket on fd 3
System: 1322419689 , GPS: 1322419689.000
# Our sytem time is within one second of GPS time. That's good!

For fun, let's use shell script to post the GPS location to Google Maps:
The format is simple enough: http://maps.google.com/?ll=40.480381,-92.373047&z=16 will return a valid map. So that's three variables: Latitude, longitude, and zoom Level.

tpv=$(gpspipe -w -n 5 | grep -m 1 TPV | cut -d, -f4,6-8,13)
latitude=$(echo $tpv | cut -d, -f2 | cut -d: -f2)
longitude=$(echo $tpv | cut -d, -f3 | cut -d: -f2)
zoom=20
map_url="http://maps.google.com/?ll=${latitude},${longitude}&z=${zoom}"
firefox $map_url

Python tools to get GPS data

Python bindings to gpsd are provided by the python-gps package.
```
sudo apt-get install python-gps
```
The python tools have changed a *lot* in two years. The old methods don't work anymore. No Python 3 version is available yet. There's a decent example here, but I couldn't get it to work.

Conversion between Lat/Lon and MGRS/UTM and other systems: gpsd doesn't do this, though apparently the geographiclib-tools package does. And so does this website.

Saturday, November 26, 2011

Ubuntu Brainstorm - inside the sausage factory

Brainstorm gets about 2,000 submissions each year.
What do we do with all those ideas?
How can they help you, without bogging you down?

Well, we close about 85% of them immediately. That's good - it keeps the system from being spammed by all those bugs, complaints, and other not-really-ideas.

About 2% get reviewed by the Ubuntu Technical Board. We try to get the most out of that 2%!

Most new Brainstormers are new to the Ubuntu community - they find Brainstorm in their first excursion looking for a way to contribute. A Brainstorm idea is their first attempt to speak up, so we try to handle them carefully.

(If you were, er, untactfully treated by any moderator in the past, then please accept my most humble apologies. If you find a recent idea we handled tactlessly, *please* let me know!)

The Idea Sandbox

Of course, most of those first-ideas aren't very good. My first idea was awful, too.
- Make it more like my old OS
- Fix this bug
- This doesn't work very well
- Hey, do this. It's self-evident
- Substitute application X for application Y in the default install
- Create a whole new distro for a very small audience
- Develop a whole new application
- I don't like this
- Someone should be in charge of this
- Make this incredibly minor change to one application
- If only my perfect idea would reach SABDFL, *he* would appreciate it, though none of you other fools do.

In the Sandbox, we gently address every idea. Is it a bug in the wrong venue? Is it a dispute or complaint in the wrong venue? Does it clearly describe a real problem that has multiple possible solutions? Has the problem already been solved some other way? Is this a duplicate? Was it discussed by the community before (and the result)? Is the submitter still interested in the idea, or was it a fire-and-forget notion? Is the issue handled within an existing project or team, or does it cross boundaries?

Most submitters, of course, abandon their idea when they realize that nobody will implement it for them...or that they need to do some research or file a bug report or otherwise become a contributing community member.

We do a lot of basic instruction on how the Ubuntu software ecosystem works, and try to change many of them from a customer-mindset to a community-member mindset. That's a hard transition, and we're still trying to find better ways to do it.

This is an opportunity to recruit new members for YOUR project/team, and turn a lot of potential complainers into contributors and advocates. We're trying, and good ideas on improving our methods are welcome. So are more drop-ins from other teams - come in, answer a few project-specific questions, and leave with the contacts for a half-dozen new people interested in your subject.

30-day voting

15% of that 2000 pass the Sandbox - the problem statement is real and clear, and it won't waste the community's time. Some of that 15% is marginal - the problem is real enough, but needs community input to refine.

We leave those marginal ones open for only 30 days, to give the submitter the public feedback they want. These have the clear understanding that we're leaving it open for feedback to help refine the issue - it won't be seen by the Ubuntu Technical Board.

30-day wonders are also helpful to developers who want to use Brainstorm for project-related polling or questions. Just ping a moderator in #ubuntu-brainstorm so we can skip it past the Sandbox for you.

6-month voting

Around 10% of the original 2000 are left open for the full 6-month voting period (and, indeed, left open long after that. We have still-open ideas from 2008).

If you find a question that's relevant to your project or team, and have an answer, please ping a moderator in #ubuntu-brainstorm, so we can put your answer in the "Developer Comments" section of the idea, and close it for you.

Similarly, if the issue has been raised and answered somewhere else (Launchpad, AskUbuntu, Blog Post), just post the link to the answer, and flag the idea so we know to close it.

The Ubuntu Technical Board

Every three months, the UTB selects 10 of the highest-voted ideas to review. 40 a year out of 2000 initial submissions, or around 2%. The 'review' isn't a promise to implement - indeed most result in a "hey, we agree this is a great idea, somebody in the community should implement this..." Some get added to UDS discussions, some turn into milestones for future releases. Some simply become bug reports.

Every idea that the UTB reviews that simply becomes a "hey, great" or a bug report is a wasted opportunity. You didn't need to wait three months for that kind of feedback - by being involved with the right project or team, or filing the bug report yourself, you could have done all that yourself!

We incorporate past UTB feedback into our moderation - for example we now routinely close ideas to create a whole new project unless they include a realistic resource plan. We encourage idea submitters to engage the appropriate bug tracker / upstream project / Ubuntu Team directly whenever possible. We want the best 40 new ideas before the UTB, real head-scratchers that will require research and create discussion and show up at UDS...and then as taskings.

We have resources for your project

Brainstorm doesn't need to be a filter for only the UTB - we can be a structured-community-feedback filter for *lots* of projects and teams. We have a lot of very smart and knowledgeable community members that help refine the ideas and solutions. If your project/team reviews Brainstorm input on a regular basis, please let us know. If you're looking for certain types of ideas, or want us to moderate your relevant ideas in a different way, just pop in to #ubuntu-brainstorm and let us know.

Are you ready for the challenge?

If you're an AskUbuntu or Forums or IRC guru looking for the next challenge to your technical and diplomatic skills, looking to help shape the next generation of contributors...well, come on by.

NTP on a Debian 6 server

This post is an update to my original in March 2011.

NTP is useful to set the time of LAN devices. The correct time is essential for several services. I installed ntp on an almost-always-connected server, which will in turn broadcast the correct time to other devices on my LAN.

internet ntp servers  <--->  my net ntp client / LAN ntp server  <--->  my LAN ntp clients

The ntp program is both client and server (those used to be separate packages). The same ntp gets installed on all devices. You change the settings in the /etc/ntp.conf file. The version of ntp on those devices isn't important - different versions of ntp interact well together (unlike, say, different versions of the deluge torrent client/server)

These instructions are for debian 6, running as root. For Ubuntu, you need to prepend most commands with 'sudo'

Install NTP with the following command:
```
apt-get install ntp
```
Edit the server's /etc/ntp.conf file (if needed). If you want your device to get time from the default internet time servers (in most cases, this is exactly what you want), then skip this step and make no changes.

In my case, however, my server is both a client of those upstream internet time servers AND a time server to my LAN. Your LAN probably does not need a timeserver - I'm doing this for fun.
```
## Around Line 48
# If you want to provide time to your local subnet, change the next line.
# (Again, the address is an example only.)
#broadcast 192.168.123.255

# Since my LAN is 192.168.1.0, the corresponding broadcast address is at 192.168.1.255
broadcast 192.168.1.255
```
No changes to /etc/hosts.allow and /etc/hosts.deny are needed to limit access to ntpd. Access controls are set by the /etc/ntp.conf file, they default to nobody has any access, and we didn't change those rules.
xinted and dnsmasq do not interact with ntpd, and no special configuration is required.
Firewall rules to open udp port 123 for ntp sync messages. Normally, this won't be necessary. Since the ntp client sends the first packet to the upstream internet server, and outbound packets usually are not blocked, the firewall should automatically let the server's response through.

If an ntp client has a really locked-down firewall (blocking outgoing traffic), then open the client's UDP port 23 to outgoing traffic.
```
iptables -A OUTPUT -p udp --sport 123 -j ACCEPT
```
If you run a LAN-facing NTP server, then of course you must ensure that LAN-interface UDP port 23 is open to that same inbound traffic from your clients. (your -i interface name will be different)
```
iptables -A INPUT -i lan0 -p udp --dport 123 -j ACCEPT 
```
If you run an exposed internet-facing NTP server, then you should really be working with ntp.org for security, and to help support the ntp infrastructure!

In my case, no firewall changes were needed at all. My firewall blocks inbound packets from the internet, not LAN-only packets, and not outbound-packets (or responses).

Restart/update the firewall (how you do that is up to you), and then restart ntp to reload the changed config file. Let's test ntp to see if it's working:

$ ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
-w1-wdc.ipv4.got 10.0.77.54       4 u  396 1024  377   30.567    1.853   2.901
*ntp3.junkemailf 209.81.9.7       2 u  465 1024  377   64.788   -0.072   1.269
+vimo.dorui.net  208.90.144.52    3 u  633 1024  377   32.876   -1.081  30.206
+aeolus.seobeo.c 193.67.79.202    2 u  184 1024  377  110.840    0.577   1.790
 192.168.1.255   .BCST.          16 u    -   64    0    0.000    0.000   0.000

Looks correct. The first character of each row (*/+/-) is the status of each upstream server. See the explanation. The 'when' column is the number of seconds since the last check of that server. The'poll' column is the number of seconds between checks. The first four rows are upstream internet servers. The last row is the LAN broadcast, which is why it doesn't have an upstream status.

The other ntp clients on the LAN need to be modified to listen for the new ntp server. Left alone, they will pull time from the upstream internet time servers (good). But, of course, I now have a server broadcasting time, so let's set each device on my LAN to listen for it. Plus, I still want them to get time from upstream in case my ntp server goes down or starts sending bad time.

Log in to each device, and look at their /etc/ntp.conf (though it may be in a different location or names differently).

Here's an example /etc/ntp.conf change from an Ubuntu system on the LAN, starting at around Line #15. We added the LAN ntp server, and marked it as the preferred source. We added minpoll to the internet servers to increase the time between pollings.
```
# Specify one or more NTP servers.
server kiwkak.dyndns.org prefer

# Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board
# on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for
# more information.
server 0.ubuntu.pool.ntp.org minpoll 12
server 1.ubuntu.pool.ntp.org minpoll 12
server 2.ubuntu.pool.ntp.org minpoll 12
server 3.ubuntu.pool.ntp.org minpoll 12
```
Restart ntp (in debian 'service ntp restart'. For ubuntu 'sudo service ntp restart') to reload the new configuration file.

Let's also take a loot at their ntp status:
```
$ ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*adsl-76-229-175 184.105.182.7    3 u   28   64  377    0.274    0.805   0.209
 ntp1.Rescomp.Be 169.229.128.214  3 u  838 1024    3   63.979    3.262   0.424
 cheezum.mattnor 129.7.1.66       2 u  818 1024    3   34.490    0.145  25.810
 host2.kingrst.c 204.9.54.119     2 u  789 1024    3   22.359    0.721  17.671
 vf1.bbnx.net    128.4.1.1        2 u  824 1024    3   68.227   -0.820   1.466
```
This looks good. See how the local server is preferred, and gets checked more often than the backup upstream servers. And the backup upstreams are reachable, but not currently used.
The server itself may not always be the network router/firewall. I automagically switching between server between router and non-router roles using my kingbaron script. I use a script and two config files to change how ntp runs based on if the system is a router (Network King, runlevel 3) or just a LAN client (Faithful Baron, runlevel 4). The decision logic operates in runlevel 2.

Create two .conf files, one for each runlevel. Let's call them /etc/ntp-king.conf and /etc/ntp-baron.conf. Look at all the .conf stuff above for the contents of those files.

Use the following command once to automatically prevent ntp from starting in runlevel 2 (the decision logic level), and to stop ntp if you manually switch to runlevel 2 due to a loss of connectivity or for maintenance:
```
update-rc.d ntp stop 2
```
Add the following patch to /etc/init.d/ntp to restart ntp with the appropriate config files upon runlevel changes.
```
test -x $DAEMON || exit 5    # Insert after this line

current_runlevel=echo $(runlevel | cut -d' ' -f2)
if [ current_runlevel -eq "3" ]; then
     NTPD_OPTS="$NTPD_OPTS -c /etc/ntp-king.conf"
elif [ current_runlevel -eq "4" ]; then
     NTPD_OPTS="$NTPD_OPTS -c /etc/ntp-baron.conf"
```
This covers the common use cases (booting, manual force of decision logic), but doesn't cover everything - if I manually switch to runlevels 3/4/5, ntp won't automatically check the config.

Saturday, November 19, 2011

The Unexpected Success of Ubuntu Brainstorm

Once upon a time, there was a busy, vibrant community...with some communication issues. Yes, I'm talking about the Ubuntu community. Back in 2007.

So Ubuntu created a website to foster community discussion about ideas to improve Ubuntu. http://brainstorm.ubuntu.com

Conventional Wisdom from the old-timers who remember those heady days is that it didn't work out. The site was quickly overwhelmed with bugs (not ideas), complaints (not ideas), and unrealistic expectations. It didn't work.

And Conventional Wisdom had those facts right...but reached entirely the wrong conclusion! Brainstorm has been successful in ways that were unexpected at the time.

1) Brainstorm is indeed lousy at directly turning submissions into code.

Well, we knew this already, didn't we? There is no roomful of monkeys poised over their keyboards eagerly awaiting the next idea to begin cranking out Shakesperian code. There never was. There never will be. Implementation was never really expected to be a metric of success...community action was supposed to be the metric.

2) Brainstorm is a great early-warning system for controversy.

What? You can avoid getting blindsided? Brainstorm is awesome at reflecting the community's (lack of) consensus on a wide range of issues. For example, when 'close window' window button changed sides, the ideas on the topic were plentiful. Similarly, today most ideas are about Unity. Brainstorm is also a pretty good barometer of whether an issue is temporary or chronic, and our moderators are excellent at separating valid issues from mere whining (so you don't have to).

3) Brainstorm is a great place to recruit enthusiastic new members.

The website is a portal-of-entry for users who are just starting to get involved, but don't understand the community's structure yet, or how the various teams and projects interact.

4) Brainstorm keeps project community input manageable.

If you are involved with a project, you know how challenging it is to manage certain types of community feedback. Brainstorm has reviewers and moderators who do nothing but handle those tough cases. Better yet, it's set up to let the community itself moderate feedback for you. And it's set up so that you only answer an issue once...ever.

5) Brainstorm finds your blind spots.

Maybe it's documentation, maybe it's a workflow, maybe it's a use case, maybe it's a legacy, maybe it's something else. The trend of feedback across years helps you improve user experience.

So, in the spirit of Ubuntu Community Appreciation day,

- Thanks to nand for writing and maintaining Ideatorrent.

- Thanks to the best moderators and reviewers in the world: Vahan Harutyunyan, DarwinSurvivor, Komputes, DrG, alourie, andruk, and Thonixx

Your valiant efforts to raise the quality of discussions and ideas, and to uphold the Ubuntu Code of Conduct, challenge me to strive. I'm very proud to work with people of such high caliber.

Tuesday, November 1, 2011

Move IPTables log events to a separate logfile

Today some botnet tried to connect to my server over 26,000 times in five hours. They might still be trying.

I have strong firewall protection, and I log all those dropped packets from the firewall. but the records of more than 26,000 dropped packets is filling my syslog and making it unusable.

I used the instructions here to shift that reporting to a separate iptables log, plus enabled logrotate so it gets changed out daily.

Sunday, October 30, 2011

deluged on a server

I want to move my (quite limited) torrenting from my laptop onto the server. Here's how to run headless deluge on a server, and connect to it from the deluge client. (Instructions).

Since my server (Debian 6.0.3) is at version 1.2.3, but my Laptop (Ubuntu 11.10) is at version 1.3.3, using the GTK client won't work. I tired and tried, but ultimately the incompatibility defeated me...

...so instead of the GTK client, we'll use the web client. (There's also a console client)

Setup and start the deluged server

1) Run the following on the server as root:

apt-get install deluged deluge-console deluge-web
# To set this for automatic startup at boot, 
# see http://dev.deluge-torrent.org/wiki/UserGuide/InitScript

2) Run the following on the server as a user (not root)

deluged         # Run as USER to create the .config directory
pkill deluged   # Stop deluged

#Add an entry to the /home/USERNAME/.config/deluged/auth file
echo "USERNAME:my__deluge-only_password:5" >> /home/USERNAME/.config/deluged/auth

# Use deluge-console to change the config setting, allowing remote access.
# (For some reason, if you change .config/deluge/core.conf, the change is not persistent!)
deluge-console
config -s allow_remote True
exit

# Start deluged and deluge-web to launch the server and the web socket
deluged
deluge-web --fork

Connect to the server from the laptop:
Open a web browser to the server, port 8112: http: me.myserver.org:8112
Use the same password as the auth file (the "my_deluge-only_password")

And you should be in!

Try it with a small torrent (like a Debian Businesscard .iso)

Using runlevels to demote a network king to a mere baron

I have put enough services onto my server that it has become a single-point of failure. It's a router, a DSL modem, backup storage, and has network-shared drive, plus a few other cool things.

The key point of failure is the combination of Modem + Router. The PCI DSL card requires a custom driver, and sometimes after a system upgrade it needs to be reinstalled. The details of the wanrouter driver are here.

What I need is a failover mode: If the DSL fails to work, I want to use my old external modem and router. So I still want the system to run, but as a dchp-client server instead of a router.

Let's use some startup logic and runlevels to define two roles for the server: Network King routing over DSL, and faithful Baron merely connecting over wi-fi to the Linksys. And a bit of connective tissue so the machine automagically boots into the correct role, plus can be switched manually.

Internet <------+ Network King +-----> LAN

Internet <------+ Other router +------> LAN <------+ Faithful Baron

Runlevel 1 - startup (Don't touch this)
Runlevel 2 - network testing
Runlevel 3 - server (Network King)
Runlevel 4 - client (Faithful Baron)
Runlevel 5 - unchanged from stock install
Runslevel6 - reboot (Don't touch this)

Decision logic:
If the server can connect to the internet over the dsl interface, then it is a King
Otherwise, it is a Baron.

Issues:
1) I need to change the LAN IP address of the server so it doesn't conflict with the router anymore!
2) All changes to the server must be tracked and undoable

Changes to the external router
Port-forward from the internet to the Baron (easy, one DMZ setting or separate port-forward settings for each service)

Server setup changes
1) Create a new directory to hold the three small scripts we are going to make, so you can keep track!

mkdir /root/startup-scripts

2) It's possible to create one really ugly /etc/networking/interfaces file, but let's not do that. Instead, we'll create separate interface files for runlevels 3 and 4. For convenience, let's put a link to them next to the original interfaces file.

cp /etc/network/interfaces /root/startup-scripts/runlevel-3-interfaces
cp /etc/network/interfaces /root/startup-scripts/runlevel-4-interfaces
ln /root/startup-scripts/runlevel-3-interfaces /etc/network/interfaces
ln /root/startup-scripts/runlevel-4-interfaces /etc/network/interfaces

3) Edit the runlevel 3 file (/root/startup-scripts/runlevel-3-interfaces)

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# This file is ONLY for runlevel 3 (Network King [router] mode)

# The loopback network interface
auto lo
iface lo inet loopback

# The ethernet jack and wi-fi antenna in bridged server mode
iface eth0 inet manual
iface wlan0 inet manual
     up hostapd -B /etc/hostapd/hostapd.conf
     down ifconfig mon.wlan0 down
     down pkill hostapd
auto br0                       
iface br0 inet static
     # Adding and removing the slave eth0 and wlan0 interfaces
     # is handled by /etc/init.d/kingbaron
     address 192.168.1.1
     broadcast 192.168.1.255
     netmask 255.255.255.0
     network 192.168.1.0
     up hostapd -B /etc/hostapd/hostapd.conf
     up route add -net 239.0.0.0 netmask 255.0.0.0 br0
     down ifconfig mon.wlan0 down
     down pkill hostapd
     down route del -net 239.0.0.0 netmask 255.0.0.0 br0

iface dsl-provider inet ppp
     pre-up /sbin/ifconfig dsl0 up # line maintained by pppoeconf
     provider dsl-provider

auto dsl0
iface dsl0 inet manual

4) Edit the runlevel 4 file (/root/startup-scripts/runlevel-4-interfaces)

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# This file os ONLY for runlevel 4 (Network dhcp client mode)

# The loopback network interface
auto lo
iface lo inet loopback

# The ethernet jack in client mode
auto eth0
allow-hotplug eth0
iface eth0 inet dhcp

# The wi-fi antenna in client mode
auto wlan0
iface wlan0 inet dhcp
     pre-up ifconfig wlan0 down
     pre-up iwconfig wlan0 mode Managed
     pre-up iwconfig wlan0 essid MY_LAN_NAME

5) Let's edit the original /etc/network/interfaces file to reduce startup time by not automatically raising the eth0 and wlan0 interfaces. We don't need those in runlevel 2, since only the DSL line will needs to be brought up.

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The ethernet jack in client mode
# (Disabled during initial boot)
allow-hotplug eth0
iface eth0 inet manual

# The wi-fi antenna in client mode
# (Disabled during initial boot)
auto wlan0
iface wlan0 inet manual
     pre-up iwconfig wlan0 essid Klein-Weisser

# The following lines are auto-generated for the dsl connection

auto dsl-provider
iface dsl-provider inet ppp
     pre-up /sbin/ifconfig dsl0 up # line maintained by pppoeconf
     provider dsl-provider

auto dsl0
iface dsl0 inet manual

6) Create a new file for the startup testing and decision logic: /root/startup-scripts/kingbaron

#!/bin/bash

### BEGIN INIT INFO
# Provides:             runlevel_chooser
# Required-Start:       $network $remote_fs $syslog wanrouter
# Required-Stop:        $network
# Default-Start:        2
# Default-Stop:
# Short-Description:    Choose runlevels based on testing network connection to an interface
### END INIT INFO

# Functions

start_king_mode () {
   # If coming from runlevel N or 4, need to change from dhcp to static/Master
   [ runlevel=="4 2" ] && ifdown -a --interface=/etc/network/runlevel-4-interfaces
   [ runlevel=="N 2" ] && ifdown -a

   # If the br0 interface does not exist (coming from runlevel N), create it.
   [ $(brctl show | grep -c br0) -eq 0 ] && brctl addbr br0

   # Add the slave interfaces to br0
   [ $(brctl show | grep -c eth0) -eq 0 ] && brctl addif br0 eth0
   [ $(brctl show | grep -c wlan0) -eq 0 ] && brctl addif br0 wlan0

   # Bring up the King mode interfaces (except dsl0 and ppp0, which are already up)
   ifup -a -v --interfaces=/etc/network/runlevel-3-interfaces

   # Test that ifup worked
   [ $(ifconfig | grep -c mon.wlan0) -eq 0 ] && logger -i -s -t kingbaron "Failed to bring up wlan0...Sorry"
   [ $(ifconfig | grep -c br0) -eq 0 ] && logger -i -s -t kingbaron "Failed to bring up br0...Sorry"

   logger -i -s -t kingbaron "Network should be up now. If not, try 'ifconfig' for LAN interfaces and 'wanrouter' for the DSL interface"
   telinit 3
   exit 0
}

start_baron_mode () {
   # Shut down the dsl0 and ppp0 interfaces (not used in Runlevel 4).
   # Sometimes the signal needs to be sent twice
   [ $(wanrouter status | grep -c stopped) -gt 0 ] || wanrouter stop
   [ $(wanrouter status | grep -c stopped) -gt 0 ] || wanrouter stop

   # If coming from runlevel 3, need to change from static/Master to dhcp
   # If coming from runlevel N or 4, we can keep the same interfaces
   [ runlevel=="3 2" ] && ifdown -a --interface=/etc/network/runlevel-3-interfaces

   # If br0 is up, bring it down. If it still has slaves from runlevel 3, unslave them
   [ $(ifconfig | grep -c br0) -gt 0 ] && ifconfig br0 down
   [ $(brctl show | grep -c wlan0) -gt 0 ] && brctl delif br0 wlan0 && ifconfig wlan0 down
   [ $(brctl show | grep -c eth0) -gt 0 ] && brctl delif br0 eth0 && ifconfig eth0 down

   # If wlan0 is stuck in Master mode from runlevel 3, unstick it
   [ $(ifconfig | grep -c mon.wlan0) -gt 0 ] && pkill hostapd && ifconfig wlan0 down && iwconfig wlan0 Managed

   # Bring up the Baron Mode interfaces, ignoring anything already up
   ifup -a -v --interfaces=/etc/network/runlevel-4-interfaces

   # Sometimes the network fails to come up, especially if it didn't go down properly
   # Check for the most common errors (like WiFi not going up) and try to autofix
   if [ $(ifconfig | grep -A2 wlan0 | grep -c inet) -eq 0 ]; then
      logger -i -s -t kingbaron "Wireless failed to come up. Resetting and trying again..."
      ifconfig wlan0 down
      iwconfig wlan0 mode Managed
      iwconfig wlan0 essid MY_ESSID
      ifconfig wlan0 up
      dhclient -v wlan0
   fi
   logger -i -s -t kingbaron "Network should be up now. If not, try 'ifconfig' and 'iwconfig'"
   telinit 4
   exit 0
}

logger -i -s -t kingbaron "Testing for DSL connectivity"

# Check for the existence of a the DSL interface. If it exists, try to get a connection
# If the internet is reachable, goto runlevel 3 (King mode). Else goto runlevel 4 (Baron mode)

# If wanrouter is not already running, then start it
flag="wanrouter off"
[ $(wanrouter status | grep -c Connecting) -eq 0 ] && flag="wanrouter on"
[ $(wanrouter status | grep -c Connected) -eq 0 ] && flag="wanrouter on"
[ $flag=="wanrouter off" ] && wanrouter start

# If the test interface does not exist, then start client mode
if [ $(ifconfig | grep -c dsl0) -eq "0" ]; then
   logger -i -s -t kingbaron "The DSL interface (dsl0) does not exist. Entering dhcp client mode"
   start_baron_mode
fi

# If the test interface exists, then wait for wanrouter to start up
# Average start time is about 20 seconds
logger -i -s -t kingbaron "Found the DSL interface. Waiting up to 40 seconds for the DSL link (dsl0) to come up"
i="0"
while [ $i -lt 40 ]; do
   sleep 1
   i=$[$i+1]
   [ $(wanrouter status | grep -c Connecting) -gt 0 ] || i=100
done

# If time expires without the wanrouter starting, print the error and start client mode
if [ $i -lt 100 ]; then
   logger -i -s -t kingbaron "dsl0 interface failed to come up. Entering dhcp client mode"
   start_baron_mode
fi

# If the wanrouter comes up properly, then wait for a ppp connection
logger -i -s -t kingbaron "dsl0 up. Waiting up to 40 seconds for a PPPoE connection (ppp0)"
pon dsl-provider
i="0"
while [ $i -lt 40 ]; do
   sleep 1
   i=$[$i+1]
   [ $(route | grep -c default) -gt 0 ] && i=100
done

# If time expires without the ppp connection starting, print the error and start client mode
if [ $i -lt 100 ]; then
   logger -i -s -t kingbaron "ppp0 failed to come up. Entering dhcp client mode"
   start_baron_mode
fi

# If all has gone well, and the ppp connection comes up
logger -i -s -t kingbaron "ppp0 came up. This system has DSL connectivity. Starting Router services"
start_king_mode

7) Install the script:

chmod +x /etc/startup-scripts/kingbaron            # Make executable
ln /root/startup-scripts/kingbaron /etc/init.d/    # Hardlink to init.d
update-rc.d kingbaron defaults                     # Symlink to runlevel 2

8) Take a look at /etc/rc2.d/. See all those router and server services that shouldn't operate in runlevel 2? Or shouldn't operate in runlevels 2 and 4? Use the command "update-rc.d $Name disable 2" to disable the appropriate services in runlevel 2 (or 4).

...And do a whole lot of tweaking and testing, and voila! An automated detection and failover system. If the DSL line is active, router services start and the interfaces come up in static/master mode. If the DSL line isn't active, router services don't start and the interfaces come up in dhcp mode. Runlevel 2 is the decision mode, runlevel 3 is the King (router) mode, and runlevel 4 is the Baron (dhcp) mode.

All the server services (samba, cups, etc) still operate, regardless. And you can manually reset with the command 'telinit 2' to make the system reset the interfaces and router services.

TIP: Look for related posts using the kingbaron tag.

Friday, October 28, 2011

The better, easier way to install wanpipe

In this post from march 2011, I described my successful installation of Sangoma software for my Sangoma S518 PCI DSL Modem.

Well, for unknown reasons a couple weeks ago the software failed completely upon a reboot.

So it's time to reinstall. And this time, to do it the right way. (Instructions)

In this case, PPPoE, two interfaces are created, dsl0 and ppp0. Wanrouter creates the physical dsl0 interface, and the pon/poff creates the virtual ppp0 interface that actually shows up in the routing table and enables real network activity.

To install wanrouter, do the following as root:

#Create the build environment
apt-get install gcc g++ automake autoconf libtool make libncurses5-dev flex bison patch libtool linux-headers-$(uname -r)

# Get the latest driver tarball
cd /usr/src
wget ftp://ftp.sangoma.com/linux/current_wanpipe/wanpipe-3.5.23.tgz
tar xvfz wanpipe-<version>.tgz

# Create a deb package from the tarball
cd wanpipe-<version>
./Setup builddeb    # no make, no prepare. Setup does it all for you.
dpkg -i wanpipe_<version>

And that's all for the install.

It seems last time I made a few mistakes. Live and learn:
1) I missed a couple build dependencies like automake and autoconf
2) I used obsolete information to prepare for the installation
3) I completely missed the builddeb option!

Next, configure the dsl0 interface. The DSL line does not need to be connected:

wanrouter hwprobe  # Test if the card is detected
wancfg             # Ncurses tool to configure the wanpipe and interface
                   # Name the interface (foe example dsl0)
                   # Enable the startup and shutdown scripts
                   # Note the name of the configuration file (wanpipe1)

Let's test to make sure wanrouter works:

wanrouter start wanpipe1   # Test - should bring up interface
ifconfig                   # The interface should be on the list
ifconfig dsl0 down         # Test - should bring down interface
ifconfig                   # The interface should *not* be on the list
ifconfig dsl0 up           # Test - should bring up interface
ifconfig                   # The interface should be on the list
wanrouter stop wanpipe1    # Test - should being down interface
ifconfig                   # The interface should *not* be on the list

Next, create and configure the ppp0 connection. The DSL line must be connected for this step to work.

wanrouter start   # Activate wanrouter so pppoeconf can detect it
ppoeconf          # Configure ppp0
                  # The pppoeconf program will ask two important questions:
                  # 1) Do you want to start PPPoE at each startup? 
                  #  NO, because it will fail - dsl0 will not be ready yet.
                  # 2) Do you want to start PPPoE now? 
                  #  You can, but if there are any problems, the process will be 
                  #  orphaned. Kill it with the command 'poff -a'
                  # You can see the PPPoE configuration (link to the dsl0 interface) 
                  # in /etc/ppp/peers/dsl-provider. You can see your dsl username 
                  # and password in /etc/ppp/pap-secrets.

Configuration is complete. DSL connectivity should work now.

Let's test it:

To manually open/close the dsl connection:

wanrouter start     # To bring up the dsl0 interface. Wanrouter does this at boot already.
pon dsl-provider    # To bring up the ppp0 interface, which is the real PPPoE connection 
                    # (with an IP address).
plog                # A handy debugging tool. Take a quick look at the log
ifconfig            # The dsl0 interface does not have an IP. 
                    # The ppp0 interface does have an IP
route               # The upstream DSL server should be showing as a gateway,
                    # and a 'default' route should exist
ping <example>      # Test network connectivity
poff -a             # To close the PPPoE connection, and bring down the ppp0 interface
wanrouter stop      # To bring down the dsl0 interface. Wanrouter does this at shutdown already,

To automatically open/close the dsl connection: Insert a start script and a stop script as follows. Automatically starting takes about 45 seconds:

echo "pon dsl-provider" >> etc/wanpipe/scripts/wanpipe1-dsl0-start
echo "poff -a" >> etc/wanpipe/scripts/wanpipe1-dsl0-stop
# If these don't work, then you may need to use 'wancfg' to enable the start/stop scripts.

For my own reference, here's the Setup output:

me@mysystem:/usr/src/wanpipe-3.5.23# ./Setup builddeb
Enabling the FR Protocol
Enabling the CHDLC Protocol
Enabling the PPP Protocol
Enabling the X25 API Protocol
Enabling the ADSL Protocol
Enabling the Multi Protocol Driver
Enabling the AFT TE1 Support
Enabling the AFT TE3 Support

WARNING: Module installation dir mismatch!
Linux source name = 2.6.
Current image name = 2.6.32-5-686

If you are building wanpipe modules for the
currently running image, use the 2.6.32-5-686
directory (i.e. select 'y')

Install modules for current image: 2.6.32-5-686 ? (y/n) y

Installing modules for kernel: 2.6.32-5-686 !

----------------------------------------------------------
WANPIPE v3.5.23 Installation Script
Copyright (c) 1995-2011, Sangoma Technologies Inc.
----------------------------------------------------------

WANPIPE DEB BUILD

Wanpipe DEB will be build based on the
following Environment variables.

1. Linux Source Dir : /lib/modules/2.6.32-5-686/build

2. Linux Source Name : 2.6.32-5-686

3. Current Image Name: 2.6.32-5-686

4. CPU Type : i686

5. Wan Protocols : FR-CHDLC-PPP-MFR-MPROT-X25-AFT_TE1-AFT_TE3-ADSL-ATM

6. Build Directory : /usr/src/wanpipe-3.5.23/debbuild

Would you like to build WANPIPE DEB ? (y/n) y
Enabling the FR Protocol
Enabling the CHDLC Protocol
Enabling the PPP Protocol
Enabling the X25 API Protocol
Enabling the ADSL Protocol
Enabling the Multi Protocol Driver
Enabling the AFT TE1 Support
Enabling the AFT TE3 Support

WANPIPE INSTALLATION

You are about to install WANPIPE Multi-Protocol
TDM Voice & WAN Router into your system.

This script will examine your system, then install, create
and/or modify necessary files and directories.

You must have Linux Kernel Headers along with
full development tools (i.e. GNU C compiler and utilities)
installed in order to be able to install this product.

This script will automatically compile all WANPIPE kernel
drivers and install them in their appropriate directory.

If you are installing Wanpipe for ASTERISK/ZAPTEL this
script will will prompt you for zaptel source location.

If you have previoulsy installed WANPIPE, this release
will overrite/upgrade full release without the need to
uninstall first!

IMPORTANT:
It is always recommended to say YES to all options
prompted during the install!

Please visit: http://wiki.sangoma.com for more info.

Fixing file permissions...

Verifying files and fixing permissions ...Done
Checking for C developement tools ...(gcc) OK
Checking for C++ developement tools ...OK
Checking for Make utility ...OK
Checking for ncurses library ... OK
Checking for Perl developement tools ...OK
Checking for AWK ...OK
Checking for FLEX ...OK
Checking for Patch ...OK
Checking for libtermcap-devel...OK
Checking for bison...OK
Checking for libtool...OK

Installing WANPIPE Device Drivers: Linux KERNEL

To integrate WANPIPE Multi-Protocol Voice & WAN Router
modules into the Linux kernel, the kernel has to be
updated with latest wanpipe sources. Install will only
modify existing wanpipe source that is already in the
Kernel.

IMPORTANT:
It is always recommended to say YES to all options
prompted during the install!

Setting linux directory to /lib/modules/2.6.32-5-686/build

Upgrading WANPIPE kernel documentation ...Done.

Installing WANPIPE include headers ...Done.

WANPIPE device drivers upgraded successfully!

WANPIPE KERNEL DRIVER COMPILATION

The next step in WANPIPE installation involves compiling
WANPIPE kernel modules.

This script will compile and install WANPIPE modules
into the currently running linux kernel.

For greater customization you will be prompted to
select which Protocol/Drivers you would like to
build into the WANPIPE kernel modules.

Wanpipe for Asterisk/Dahdi/Zaptel
Default for Asterisk/Dahdi/Zaptel
Wanpipe for Wan Routing/API
Default for Wan/IP Routing and Data API
Wanpipe for Asterisk SMG/SS7
Default for Asterisk SS7
Wanpipe for TDM API
Default for FreeSwitch and Voice API

Custom Compilation:
------------------
Customise WANPIPE driver compilation to add only the
protocols that you need. This way one can reduce
the size of the WANPIPE kernel drivers.

Refer to http://wiki.sangoma.com for more info

Checking for SMP support ...Enabled.

Checking current processor type ...i686

Wan Update Flags: -DWANPIPE_MOD_266_FORCE_UPDATE

--------------------------------------------------
CFLAGS: gcc -Wp,-MD,.wanpipe.o.d -nostdinc -iwithprefix include -D__LINUX__ -Dlinux -D__KERNEL__ -I/usr/src/wanpipe-3.5.23/debbuild/usr/include/wanpipe -I/lib/modules/2.6.32-5-686/build/include -DMODULE -DWANPIPE_MOD_266_FORCE_UPDATE make: Entering directory `/usr/src/linux-headers-2.6.32-5-686'
make: Leaving directory `/usr/src/linux-headers-2.6.32-5-686'
--------------------------------------------------

Checking for REGPARM kernel option ...
Warning: Failed to determine regparm from Makefile defaulting to YES!

Enabled.

Compiling General WANPIPE Driver for 2.6.X Kernel .....Done.

WAN HWEC module enabled and compiled!

FR compiled for GCC Ver=3 Arch=i686 File:wanpipe_fr.gcc3.i686.regparm.o

PPP/CHDLC compiled for GCC Ver=3 Arch=i686 File:wanpipe_sppp.gcc3.i686.regparm.o

ATM compiled for GCC Ver=3 Arch=i686 File:wanpipe_lip_atm.gcc3.i686.regparm.o

ADSL binary compiled for GCC Ver=3 Arch=i686 File:wanpipe_adsl.gcc3.i686.regparm.o

ATM binary compiled for GCC Ver=3 Arch=i686 File:wanpipe_atm.gcc3.i686.regparm.o

Linking Wanpipe Driver and protocols ...Done.

Updating Kernel Modules ...Done.

Compilation Successful.

WANPIPE META CONFIGURATION

There are two configuration files associated with WANPIPE.

1) /usr/src/wanpipe-3.5.23/wanrouter.rc:
- defines locations of important files such as lock
and configuration files as well as start/stop
order of multiple WANPIPE devices.
2) /usr/src/wanpipe-3.5.23/wanpipe1.conf:
- main configuration file for each WANPIPE device.
- defines interfaces, hardware and protocol information.
- this file can be created using the 'wancfg' GUI
utility or manually based on sample files located
in /etc/wanpipe/samples.

Please read the WanpipeInstallation.(pdf/txt) manual for further
information.

WANPIPE UTILITIES SETUP

WANPIPE utilities are used to:
1) create configuration files: for Zaptel and Asterisk
/usr/sbin/wancfg_zaptel #Zaptel and Asterisk
/usr/sbin/wancfg_dahdi #Dahdi and Asterisk
/usr/sbin/wancfg_smg #BRI/SS7, Zaptel and Asterisk
/usr/sbin/wancfg_tdmapi #TDM API
2) create WANPIPE WAN/IP configuration files.
(/usr/sbin/wancfg)
3) start,stop,restart individual/all devices and interfaces.
(/usr/sbin/wanrouter)
4) debug line, protocol and driver problems.
(/usr/sbin/wanpipemon)
5) aid in WANPIPE API development
(/etc/wanpipe/api)

Refer to the WanpipeInstallation.(pdf/txt) for more information.

Compiling WANPIPE LibSangoma API library ...Done.

Compiling WANPIPE Utilities ...Done.

Compiling WANPIPE WanCfg Utility ...Done.

Compiling WANPIPE API Development Utilities ...Skipped, no makefile

Compiling WANPIPE HWEC Utilities ...Done.

WANPIPE Environment Setup Complete !!!

Installing WANPIPE Files ... !
Installing WANPIPE Utilities in /usr/src/wanpipe-3.5.23/debbuild/usr/sbin
Installing wanrouter.rc in /usr/src/wanpipe-3.5.23/debbuild/etc/wanpipe
Installing wanpipe libraries in /usr/src/wanpipe-3.5.23/debbuild/etc/wanpipe
Installing firmware in /usr/src/wanpipe-3.5.23/debbuild/etc/wanpipe/firmware
Installing documentation in /usr/src/wanpipe-3.5.23/debbuild/usr/share/doc/wanpipe
Installing sample api code in /usr/src/wanpipe-3.5.23/debbuild/etc/wanpipe/api
Installing AFT Firmware update utility in /usr/src/wanpipe-3.5.23/debbuild/etc/wanpipe/util
Installing driver headers in /usr/src/wanpipe-3.5.23/debbuild/etc/wanpipe/api/include/linux
Installing Hardware Echo Cancel Utilites

WANPIPE INSTALLATON: COMPLETE

WANPIPE installation is now complete. WANPIPE kernel drivers
and configuration/debug utilities have been compiled and installed.

1) Proceed to configure the WANPIPE drivers:
Asterisk/Zaptel : /usr/sbin/wancfg_zaptel
Asterisk/Dahdi : /usr/sbin/wancfg_dahdi
TDM API : /usr/sbin/wancfg_tdmapi
SMG SS7/BRI/PRI : /usr/sbin/wancfg_smg
WAN Routing/API : /usr/sbin/wancfg
2) Use the /usr/sbin/wanrouter startup script to start and stop
the router. (eg: wanrouter start)
3) To uninstall WANPIPE package run ./Setup remove

Please read http://wiki.sangoma.com for further instructions.

WANPIPE DEB BUILD SUCCESSFUL

The new rpm is located in /usr/src/wanpipe-3.5.23: wanpipe_3523-k266_i686.deb

To install run:
dpkg -i

Thursday, October 27, 2011

Update Problem - one bug, three symptoms

Upon upgrading in one afternoon from 10.10 to 11.04 and then to 11.10, I found a couple problems.

1) No boot! Nooooo!

I suspect I got bit by bug 858122. The workaround is in comment #22, and works.

But it made me wonder...

See, you can simply rename the symlinks for shutdown, but then the next time a package manager runs "update-rc.d", everything will get hosed again. Looks like each of 5 init.d scripts needs to be patched to show accurate #stop init information, and those patched pushed upstream...if upstream even has the problem.

That's why Comment #22 is a workaround instead of a fix.

2) Very slow boot

According to syslog,

Oct 19 13:54:06 cheesebot-supremo kernel: [ 3.362104] EXT3-fs (sda1): mounted filesystem with ordered data mode
Oct 19 13:54:06 cheesebot-supremo kernel: [ 33.253915] lp: driver loaded but no devices found

So let's try commenting out the lp driver in /etc/modules
I wonder...will CUPS still work after that?

Answer: Turns out to bot be an issue, but another arcane symptom of the same bug.

3) Boot Message "Waiting for Network Configuration"

Not really finding anything on it, seems polluted with #1.

Could there have been an initscripts install failure during an upgrade?

Answer: Turns out to not be an issue, but yet another arcane symptom of the same bug.

The takeaway: A reminder to be methodical attacking bugs. Fix one, test it, and move on.

Wednesday, October 19, 2011

Backup solution: rdiff-backup

It's time to finally get serious about my laptop backups.

I tried backup-manager and rsnapshot, but neither really met my needs for a simple, scriptable, push-to-remote-server-via-ssh backup solution.

Let's try rdiff-backup.

Here's what I think I want:

Triggered by laptop connecting to the network and/or cronjob
If the server's backup drive is not mounted, mount it
Do a diff-based backups
Unmount the backup drive

The heart of the backup is the rdiff-backup command. The rdiff-backup package must be installed on both client and server (they can be different versions).

# The basic rdiff-backup syntax is: rdiff-backup [options] [machine::]/path/to/original [machine::]/path/to/backup

# Here's a test command to backup my laptop /home directory to the server via ssh:
# option: --terminal-verbosity 5 (tell me a lot)
# option: --remote-schema (necessary for nonstandard ssh ports)
rdiff-backup --terminal-verbosity 5 --remote-schema 'ssh -p $PortNumber %s rdiff-backup --server' /home/me me@myserver.com::/mnt/Laptop/backups/me/

# Here's a test command to see how many backups are stored

rdiff-backup -l --terminal-verbosity 5 --remote-schema 'ssh -p $PortNumber %s rdiff-backup --server' me@myserver.com::/mnt/Laptop/backups/me

You can see how this can be scripted to make things easy:

#!/bin/sh

# This script backs up my laptop to the server's /mnt/Laptop-backup/backups directory

# Do pre-backup stuff

Schema="ssh -p $PortNumber %s rdiff-backup --server"

# Backup the home folder

rdiff-backup --remote-schema '$Schema' /home/me me@myserver.com::/mnt/Laptop/backups/me

# Backup /var, including the package list and cache and all the logs

rdiff-backup --remote-schema '$Schema' /var me@myserver.com::/mnt/Laptop/backups/var

# Backup /etc, including the firewall script and upstart/init scripts

rdiff-backup --remote-schema '$Schema' /etc me@myserver.com::/mnt/Laptop/backups/etc

# Backup /opt, mostly unpackaged non-debian stuff

rdiff-backup --remote-schema '$Schema' /opt me@myserver.com::/mnt/Laptop/backups/opt

# Do post-backup stuff

exit 0

What kind of pre- or post- backup actions? How about sanity checks, like that the server is available?

Since the server requires root to mount the backup drive, we need to create a socket for backups. The server will see which machine is asking to send or complete a backup, and mount/unmount the appropriate drive.
We need to write some logic to trigger the process.

Saturday, September 24, 2011

Access your 24-hour IRC client/logbot from anywhere

I use irssi (IRC client) running on a 24/7 server to maintain my IRC presence. When I want to connect to IRC, I ssh into the server, and attach into the running IRC session.

ssh me@myserver.local   # Log into the server
screen -r            # Attach to the irssi session
#(Do stuff)
#CTRL+a, CTRL+d to detach irssi
exit                 # Logout of the server

It's a little different if I want to log in from elsewhere on the internet:

ssh -p PortNumber me@myserver.dyndns.org    # Log into the server
screen -r            # Attach to the irssi session
#(Do stuff)
#CTRL+a, CTRL+d to detach irssi
exit                 # Logout of the server

Here's one way to simplify it:

ssh takes in-session commands:ssh destination "command1; command2"


ssh has a -t flag, forcing a pseudo-terminal connection (screen needs this to work)
bash uses basic if/them logic a couple ways.
determine the network using the 'route' command



So you can set up an ssh command like ssh -t me@myserver 'screen -dr pts-0' to connect to the server and then the screen session. (I already knew that my screen session was at pts-0; it tells you when you detach). When you detach from the screen session (CTRL+a, CTRL+d), the ssh session closes automatically.


One more piece of the puzzle is my network logic - I connect differently if on the LAN or out in the wider world. I use route | grep -q myserver.local to determine if I'm on the home network or not (return is '0' if home network, '1' if internet). Of course, your server name will vary.


The easy, logical result:

LocalNetwork=$(route | grep -q myserver.local)
if $LocalNetwork; then
   ssh -t me@myserver.local 'screen -dr pts-0'
else:
   ssh -t -p PortNumber me@myserver.dyndns.org 'screen -dr pts-0'
fi


We can simplify this even further using bash logical expressions instead of if/then/else statements. It's a single line, but more cryptic:



route | grep -q myserver.local && ssh -t me@myserver.local 'screen -dr pts-0' || \
ssh -t -p PortNumber me@myserver.dyndns.org 'screen -dr pts-0'








Ian




at

Saturday, September 24, 2011



No comments:
  




Share



Wednesday, September 7, 2011







More useful sshd on the gateway





I need access to my gateway machine (which runs irssi) while out in the world on the internet.


Currently, SSH access from the internet is blocked by firewall. There are a couple ways to enable it. I need to select one, add a secure connection method, and block brute-force methods to crack that connection.


The easiest way is to:

Open port 22 on the firewalliptables -A INPUT -p tcp --dport 22 -j ACCEPT
Add a method of blocking brute force attacks

# Allow incoming ssh attempts for the internet, and add them track them
#iptables -A INPUT -i ppp0 -p tcp -m tcp --dport 22 -m state --state NEW -m recent --set --name DEFAULT --rsource
# After three inbound ssh attempts in three minutes, drop any subsequent attempts until the three minutes expires
#iptables -A INPUT -i ppp0 -p tcp -m tcp --dport 22 -m state --state NEW -m recent --update --seconds 180 --hitcount 4 --name DEFAULT --rsource -j DROP
Check the settings of /etc/ssh/sshd_config

# What ports, IPs and protocols we listen for
Port 22
Check the settings of etc/hosts.allowsshd:              ALL
Restart the firewall so the new rules take effect. Restart sshd if any changes to sshd_config or hosts.allow were made.


Alternately, we can tell ssh to listen on a second (unadvertised) port. This may (or may not) be more secure than using firewall rules to block brute force attacks.

Open port 33 on the firewalliptables -A INPUT -p tcp --dport 33 -j ACCEPT
Change the settings of /etc/ssh/sshd_config

# What ports, IPs and protocols we listen for
Port 22
Port 33
Check the settings of etc/hosts.allowsshd:              ALL
Restart the firewall so the new rules take effect. Restart sshd.bash /etc/network/if-up.d/00-firewall
service restart ssh



Since the ddclient package is installed, I can now connect to my server from the internet using:

ssh [--port 33] accountname@myusername.dyndns.org








Ian




at

Wednesday, September 07, 2011



No comments:
  




Share








Thursday, September 1, 2011










Joined Google+ Today.

See what's going on with me








Ian




at

Thursday, September 01, 2011



No comments:
  




Share








Wednesday, August 24, 2011







Moving the TechBlog over to Blogger





August 24, 2011: After three years of self-publishing, I'm going to try the blog on a real blogging platform. I want those nifty feeds, plus tags to keep the posts organized.



Update Sept 21, 2011: Finally finished moving all the 2008 posts - 57 of them, apparently. I didn't move over some failed experiments like the dabbling with VOIP. So much has since changed that documenting prior experience will be useless. Moving on to 2009!








Ian




at

Wednesday, August 24, 2011



No comments:
  




Share








Wednesday, March 9, 2011







Add NTP to the router/gateway





This post is obsolete and has been superseded


NTP is useful to set the time of LAN devices. In addition, the 'adjtimex' package updates the hardware clock. (Slightly out-of-date instructions)

Install NTP with the following command:

apt-get install ntp adjtimex
Edit the /etc/ntp.conf file to broadcast time to the LAN:

## Around Line 48
# If you want to provide time to your local subnet, change the next line.
# (Again, the address is an example only.)
#broadcast 192.168.123.255
broadcast 192.168.1.0
Edit /etc/hosts.allow and /etc/hosts.deny to limit access to ntpd. Edit /etc/hosts.allow to look more like this:

sshd:              192.168.1.0/26
debtorrent-client: 192.168.1.0/26
ntpd:              192.168.1.0/26
ALL:               192.168.1.0/26
# /26 creates the range .1.0 - .1.62
And edit /etc/hosts.deny to look more like this:

sshd:              ALL
debtorrent-client: ALL
ntpd:              ALL
ALL:               ALL

xinted and dnsmasq do not interact with ntpd, and no changes are required.
Firewall rules to open those two listening ports. Edit the file /etc/network/if-up.d/00-firewall to add the following rules:

# ALLOW INCOMING OPEN PORTS TO THE SERVER FROM OUTSIDE HERE
#
# Allow incoming debtorrent requests on TCP ports 9899-9990
iptables -A INPUT -p tcp --dport 9899 -j ACCEPT
iptables -A INPUT -p tcp --dport 9900 -j ACCEPT
#
# Allow NTP syncs on UDP port 123
iptables -A INPUT -p udp --dport 123 -j ACCEPT 
iptables -A OUTPUT -p udp --sport 123 -j ACCEPT
#
#

Restart ntpd with these commands:

sh /etc/network/if-up.d/00-firewall  # Reload the firewall
service ntpd restart                 # Reload /etc/ntp.conf










Ian




at

Wednesday, March 09, 2011



No comments:
  




Share















Add a dynamic DNS client to the router/gateway





A ddns client will allow access to future SSH, VPN and other external services to access the server from the internet.



Register for a dynamic DNS service. Any good search engine can point you to a good service.

Install ddclient using the command apt-get install ddclient. The installer will ask questions about the newly-registered dynamic dns account. No further configuration seems needed.

Find the dynamic dns address, which is the router's IP address a couple ways.

route | awk '{ print $2 }' | sort | tail -n+4 | head -n+1    # If on the LAN
dig +short myaccount.dyndns.org                              # Elsewhere on the internet
nslookup myaccount.dyndns.org ns.dyndns.org                  # Another way from the internet
http://www.dnscog.com/dig/myaccount.dyndns.org/              # As a web page








Ian




at

Wednesday, March 09, 2011



No comments:
  




Share








Saturday, March 5, 2011







Installing a Sangoma S518 DSL Modem card in a Dell Optiplex GX60 running Debian 6





This post is obsolete and has been superseded

Ebay provided a new-to-me used DSL PCI modem card to replace my 10-year-old DSL modem. The old modem still works; this is purely for fun.
The card is recognized, but no kernel module is associated with it.
lspci -vv
01:07.0 Network controller: Globespan Semiconductor Inc. Pulsar [PCI ADSL Card] (rev 01)
 Subsystem: Globespan Semiconductor Inc. Device d018
 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping+ SERR+ FastB2B- DisINTx-
 Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
 Latency: 64 (750ns min, 50000ns max)
 Interrupt: pin A routed to IRQ 16
 Region 0: Memory at ff8f0000 (32-bit, non-prefetchable) [size=64K]
 Capabilities: [40] Power Management version 2
  Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
  Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Sangoma has a Linux driver, but it needs to be compiled, so it needs linux source code to refer to during compilation. The linux kernel itself does not need to be recompiled. It's annoying and time consuming, but it does work. Sangoma's information and downloads are at their wiki. Details of how to prepare the kernel source and headers is here.
Compatibility with networking and commands:

Do not make any changes to the /etc/network/interfaces file. The wanrouter program defines and brings up/down the dsl interface without using the file.
Ifup/ifdown does not work, because they rely on the interfaces file.
Ifconfig *does* work but only after a wanpipe is already active. Ifconfig up/down do work without restarting the wanpipe.
The nomenclature and order of events can be confusing:
wanrouter is the command that starts everything. It's just a bash script at usr/sbin/wanrouter. Don't be fooled by the name - it's not really a router. The wanrouter command turns on/off a wanpipe. A wanpipe is the low-level connection to the PCI card, and they create/destroy the dsl0 high-level network interface. Wanpipes are configured by the wancfg command.
pon/poff create a pppoe connection from the dsl0 interface to the upstream network provider. The pppoe connection, including dsl login/password, are configured by the pppoeconf command. pon/poff are actually just part of pppd, the ppp daemon, which creates another high-level interface, ppp0 to represent the actual live dsl link.
The upshot of all this is that wanrouter must create the dsl0 interface before pon can create the ppp0 connection (interface), and poff must terminate the ppp0 interface before wanrouter can destroy the dsl0 interface. Happily, wanpipes include a place to insert these commands so wanrouter appears to handle it all.


How to install the Sangoma wanpipe drivers, configure the card, configure the interface, and configure pppoe. The dsl line does not need to be plugged in until the final steps of configuring pppoe.
# Install the tools needed
apt-get install build-essential linux-source-2.6.32 linux-headers-2.6.32-5-686 libncurses5-dev bison libtool pppoe

# Get the Sangoma Wanpipe package. Unpack the linux-source and wanpipe packages 
cd /usr/src
wget ftp://ftp.sangoma.com/linux/current_wanpipe/wanpipe-3.5.18.tgz
tar zxvf wanpipe-3.5.18.tgz
tar xjf /usr/src/linux-source-2.6.32.tar.bz2

# Prepare the linux source for the wanpipe install script 
cd linux-source-2.6.32
cp /usr/src/linux-headers-2.6.32-5-686/Module.symvers ./
make oldconfig && make prepare && make modules_prepare

# Run the wanpipe install script
cd /usr/src/wanpipe-3.5.18
./Setup install
The script will ask for the linux source directory: /usr/src/linux-source-2.6.32. It will throw a lot of questions about using 2.6.32-5-686 instead, just answer yes and let the installer continue.

# When install is successfully completed
cd /home/USERNAME
wanrouter hwprobe  # Test if the card is detected
wancfg             # Ncurses tool to configure the wanpipe and interface
See the Sangoma Wiki for details, really all you need to choose is the interface name (for example, 'dsl0') 

wanrouter start wanpipe1   # Test - should bring up interface
ifconfig                   # The interface should be on the list
ifconfig dsl0 down         # Test - should bring down interface
ifconfig                   # The interface should *not* be on the list
ifconfig dsl0 up           # Test - should bring up interface
ifconfig                   # The interface should be on the list
wanrouter stop wanpipe1    # Test - should being down interface
ifconfig                   # The interface should *not* be on the list
Plug in the dsl connection in order to configure pppoe. Then run the PPPoE configuration program (pppoeconf). You need your dsl login and password at this point.

ppoeconf
The pppoeconf program will ask two important questions:

Do you want to start PPPoE at each startup? NO, because it will fail - dsl0 will not be ready yet
Do you want to start PPPoE now? You can, but if there are any problems, the process will be orphaned. Kill it with the command 'poff -a'
You can see the PPPoE configuration (linking it to the dsl0 interface) in /etc/ppp/peers/dsl-provider. You can see your dsl username and password in /etc/ppp/pap-secrets.
To manually open/close the dsl connection:

wanrouter start     # To bring up the dsl0 interface. Doing this at boot is part of the Wanrouter install
pon dsl-provider    # To bring up the ppp0 interface, which is the real PPPoE connection 
                    # (with an IP address). We'll automate this in the next section
plog                # A handy debugging tool. Take a quick look at the log
ifconfig            # The dsl0 interface does not have an IP, and the new ppp0 interface does have an IP
poff                # To close the PPPoE connection, and bring down the ppp0 interface
wanrouter stop      # To bring down the dsl0 interface. Doing this at shutdown is part of the Wanrouter install
To automatically open/close the dsl connection: Go back into wancfg. Edit the wanpipe1 file --> Interface Setup --> Interface Configuration --> Advanced options. Insert a start script and a stop script as follows:

pon dsl-provider    # Append this to the bottom of the START script

poff -a             # Append this to the bottom of the STOP script
Save the wanpipe1 config file, and let's test automatic dsl connection/disconnection:

wanrouter stop      # In case it was on.
ifconfig            # Neither dsl0 nor ppp0 interfaces should be live.
wanrouter start     # Bring up dsl0. The script should then bring up ppp0
ifconfig            # ppp0 should be up, and have an IP address. If not, try again - ppp0 is often missing the first time I try.
wanrouter stop      # Bring down the intefaces
ifconfig            # Should be back to the normal down state. ppp0 and dsl0 should not be showing.
Finally, test with a reboot and a shutdown to see in the interfaces change properly. Success! Time to clean up using the following commands:

apt-get remove build-essential linux-source-2.6.32 linux-headers-2.6.32-5-686 libncurses5-dev bison libtool
apt-get autoremove
rm -r /usr/src/wanpipe-3.5.18
rm -r /usr/src/linux-source-2.6.32


BUG: missing LSB tags and overrides. When I tried to install something else later, I got the following warnings:

insserv: warning: script 'K01wanrouter' missing LSB tags and overrides
insserv: warning: script 'wanrouter' missing LSB tags and overrides
A quick search on the warnings gave an answer. LSB tags are used by init, and the tags are easily added to the beginning of the /etc/init.d/wanrouter script. Here is a sample script that eliminated the warning:

#! /bin/sh        # Just to show where we are in the file

### BEGIN INIT INFO
# Provides:             wanpipe
# Required-Start:       $syslog
# Required-Stop:        $syslog
# Default-Start:        2 3 4 5
# Default-Stop:         
# Short-Description:    kernal support to DSL modem
### END INIT INFO




Final notes:

Three elements of the Sangoma package failed to compile: LibSangoma API library, LibStelephony API library, and API Development Utilities. I have seen no effect from those failures.
To uninstall WANPIPE package run ./Setup remove
There is additional documentation at /usr/share/doc/wanpipe
A firmware update utility is included in /etc/wanpipe/util
'wanpipemon' is an included diagnostic tool. The easiest way to use it is 'wanpipemon -g' for the ncurses gui.
Changing the default route to send packets across the dsl connection is beyond the scope of what I wanted to do. I just wanted to see if it worked.









Ian




at

Saturday, March 05, 2011



No comments:
  




Share








Wednesday, January 5, 2011







How to get Python to talk to QuickBooks Pro 2008 using QBFC





You can import/export and change most QuickBooks data 
(including most transactions) using the qbXML and QBFC methods. 
Quickbooks has a built-in COM listener, and it's possible for other 
applications to login through this interface...even if QB is not 
running.




Prerequisites



Python COM: The pywin32
 module enables python to talk COM. Look for the file matching your architecture AND python version. The downloaded .exe file should automatically install on your Windows system. You need pywin32 for both the QBFC method and the qbXML method. 


QBFC10.dll: The QBFC10.dll file, and the installer for it, is not included with Quickbooks, so you need to get it from the Quickbooks SDK available at http://developer.intuit.com, which means you'll need to register at that site.

This prerequisite only applies to QBFC. qbXML uses QBXMLRP2.dll, which is already included with Quickbooks.






Using qbXML




QBFC is just a thin object-oriented wrapper around qbXML, so the best way to understand QBFC is to look at some qbXML first.



#SAMPLE QBXML
<QBXML>                                   # XML container
   <QBXMLMsgsRq>                          # Container for multiple transaction types
      <InventoryAdjustmentQueryRs>        # List of transactions
         <InventoryAdjustmentRet>         # Container for each transaction
            <TxnID>12345</TxnID>          # Transaction detail
            <Memo>Something to say</Memo> # Transaction detail
         </InventoryAdjustmentRet>
      </InventoryAdjustmentQueryRs>
   </QBXMLMsgsRq> 
</QBXML>






As you see, getting transaction information in qbXML is like peeling an onion - layer after layer of containers. Here's how to do qbXML using Python 2.6:



#!usr/bin/python
import win32com.client
import xml.etree.ElementTree

# Connect to Quickbooks
sessionManager = win32com.client.Dispatch("QBXMLRP2.RequestProcessor")    
sessionManager.OpenConnection('', 'Test qbXML Request')
ticket = sessionManager.BeginSession("", 0)

# Send query and receive response
qbxml_query = """
<?qbxml version="6.0"?>
<QBXML>
   <QBXMLMsgsRq onError="stopOnError"> 
      <InventoryAdjustmentQueryRq metaData="MetaDataAndResponseData">
      </InventoryAdjustmentQueryRq>
   </QBXMLMsgsRq> 
</QBXML>
"""
response_string = sessionManager.ProcessRequest(ticket, qbxml_query)

# Disconnect from Quickbooks
sessionManager.EndSession(ticket)     # Close the company file
sessionManager.CloseConnection()      # Close the connection

# Parse the response into an Element Tree and peel away the layers of response
QBXML = xml.etree.ElementTree.fromstring(response_string)
QBXMLMsgsRs = QBXML.find('QBXMLMsgsRs')
InventoryAdjustmentQueryRs = QBXMLMsgsRs.getiterator("InventoryAdjustmentRet")
for InvAdjRet in InventoryAdjustmentQueryRs:
    txnid = InvAdjRet.find('TxnID').text
    memo = InvAdjRet.find('memo').text






See how each container needs to be opened to get to the bottommost data?



It's about the same in both qbXML and QBFC.






Using QBFC




Here's the same query in QBFC using Python 2.6:



#!usr/bin/python
import win32com.client
#No ElementTree needed, since no raw XML

# Open a QB Session
sessionManager = win32com.client.Dispatch("QBFC10.QBSessionManager")    
sessionManager.OpenConnection('', 'Test QBFC Request')
# No ticket needed in QBFC
sessionManager.BeginSession("", 0)

# Send query and receive response
requestMsgSet = sessionManager.CreateMsgSetRequest("US", 6, 0)
requestMsgSet.AppendInventoryAdjustmentQueryRq()
responseMsgSet = sessionManager.DoRequests(requestMsgSet)

# Disconnect from Quickbooks
sessionManager.EndSession()           # Close the company file (no ticket needed)
sessionManager.CloseConnection()      # Close the connection

# Peel away the layers of response
QBXML = responseMsgSet
QBXMLMsgsRq = QBXML.ResponseList
InventoryAdjustmentQueryRs = QBXMLMsgsRq.GetAt(0)
for x in range(0, len(InventoryAdjustmentQueryRs.Detail)):
    InventoryAdjustmentRet = QueryRs.Detail.GetAt(x)
    txnid = InventoryAdjustmentRet.TxnID.GetValue()
    memo = InventoryAdjustmentRet.Memo.GetValue()



The two small advantages of QBFC are:



Each transaction is a single variable with all the data appended at 
attributes. That seems slightly easier to deal with than using 
ElementTree to convert each transaction into a dict. But not much.


QBFC transactions seem to be about 20% faster than equivalent qbXML









Ian




at

Wednesday, January 05, 2011



2 comments:
  




Share




‹


›


Home


View web version





















Powered by Blogger.