It's called a feed because it (usually) includes an RSS update for changes.
There are lists of feeds on the Google wiki, and on the separate GTFS data website.
Each organization's GTFS file includes all their services, so some agency files can get pretty big, and get updated often. Any schedule change or route adjustment means a new release of the entire GTFS file. The file itself is merely a big zipfile, containing several csv files that are strangely required to be mislabelled as .txt.
Here's the contents of Milwaukee County Transit System's GTFS file:
$ unzip -l mcts.zip
Archive: mcts.zip
Length Date Time Name
--------- ---------- ----- ----
169 2014-01-10 05:01 agency.txt
40136 2014-01-10 05:00 calendar_dates.txt
5746 2014-01-10 05:01 routes.txt
307300 2014-01-10 05:00 stops.txt
35198135 2014-01-10 05:00 stop_times.txt
650622 2014-01-10 05:01 trips.txt
8369736 2014-01-10 05:01 shapes.txt
3490 2014-01-10 05:01 terms_of_use.txt
--------- -------
44575334 8 files
Yeah, 44MB unzipped.
But only 5MB zipped. Still not something you want to download every day to your phone.
Let's find a stop at Mitchell International Airport:
$ cat stops.txt | grep AIRPORT 7168,7168,AIRPORT,, 42.9460473, -87.9037345,,,1 7162,7162,AIRPORT & ARRIVALS TERMINAL,, 42.9469597, -87.9030569,,,0
It's right, there are two stops at the airport. Each stop has a latitude and longitude, a unique ID number, and a descriptive name. The final field designates a timepoint (1=Timepoint, 0=Not).
Let's try an intersection where two routes cross:
$ cat stops.txt | grep "HOWELL & OKLAHOMA" 709,709,HOWELL & OKLAHOMA,, 42.9882051, -87.9043319,,,1 658,658,HOWELL & OKLAHOMA,, 42.9885464, -87.9045333,,,1 $ cat stops.txt | grep "OKLAHOMA & HOWELL" 5152,5152,OKLAHOMA & HOWELL,, 42.9881561, -87.9046550,,,1 5068,5068,OKLAHOMA & HOWELL,, 42.9883466, -87.9041176,,,1
Here's a problem that will require some logic to solve. I consider the intersection to be one place (not a GTFS term). Many trips and routes can use the same stop. Multiple stops (GTFS terms) can exist at the same place. In this case, northbound, southbound, eastbound, and westbound buses each have a different stop at the same place.
This might make your job easier...or harder.
GTFS cares about trips and stops. It doesn't care that Stops #709 and #5152 are twenty meters apart, and serve different routes - that it's a transfer point. Nothing in GTFS explicitly links the two stops. Generally, you must figure out the logic to do that - you have the lat/lon and the name to work with.
GTFS does have an optional transfers.txt file, that fills in the preferred transfer locations for you. But that's for a more advanced exercise.
Let's see what stops at #709:
$ grep -m 5 ,709, stop_times.txt 4819177_1560,06:21:00,06:21:00,709, 14,,0,0 4819179_1562,06:49:00,06:49:00,709, 14,,0,0 4819180_1563,07:02:00,07:02:00,709, 14,,0,0 4819181_1564,07:15:00,07:15:00,709, 14,,0,0 4819182_1565,07:28:00,07:28:00,709, 14,,0,0
These fields are trip_id, arrival_time, departure_time, and stop-sequence (14th).
Let's see the entire run of trip 4819177_1560:
$ grep 4819177_1560 stop_times.txt 4819177_1560,06:09:00,06:09:00,7162, 2,,0,0 # Hey, look - stops out of sequence in the file 4819177_1560,06:09:00,06:09:00,7168, 1,,0,0 # Begin Trip 4819177_1560,06:11:00,06:11:00,7178, 3,,0,0 [...] 4819177_1560,06:20:00,06:20:00,8517, 13,,0,0 4819177_1560,06:21:00,06:21:00,709, 14,,0,0 # Howell & Oklahoma 4819177_1560,06:22:00,06:22:00,711, 15,,0,0 [...] 4819177_1560,07:17:00,07:17:00,1371, 66,,0,0 4819177_1560,07:19:00,07:19:00,6173, 67,,0,0 4819177_1560,07:20:00,07:20:00,7754, 68,,0,0 # End of trip
We can also look up more information about trip 4819177_1560:
$ grep 4819177_1560 trips.txt GRE,13-DEC_WK,4819177_1560,N BAYSHORE - VIA OAKLAND-HOWELL METROEXPRESS,0,515111,13-DEC_GRE_0_12
This needs a little more explanation
- route_id: Green Line (bus)
- service_id (weekday/days-of-service): 13-DEC_WK
- headsign: N BAYSHORE - VIA OAKLAND-HOWELL METROEXPRESS
- direction_id (binary, 0 or 1): 0
- block_id (useful only if the same bus changes routes): 515111
- shape_id (useful for route maps): 13-DEC_GRE_0_12
Let's look up the route_id:
$ grep GRE routes.txt GRE,MCTS, GRE,MetroEXpress GreenLine,,3,http://www.ridemcts.com/Routes-Schedules/Routes/GRE/,,
The full route name is MetroEXpress GreenLine, it's a bus (type-3 = bus) route, and we have the operator website for it.
Let's look up the service_id:
$ grep -m 10 13-DEC_WK calendar_dates.txt 13-DEC_WK,20140113,1 13-DEC_WK,20140114,1 13-DEC_WK,20140115,1 13-DEC_WK,20140116,1 13-DEC_WK,20140117,1 13-DEC_WK,20140120,1 13-DEC_WK,20140121,1 13-DEC_WK,20140122,1 13-DEC_WK,20140123,1 13-DEC_WK,20140124,1
Ah, this specific trip is a weekday (Monday-Friday) only trip.
Let's look up the route map shapefile for the trip:
$ grep 13-DEC_GRE_0_12 shapes.txt 13-DEC_GRE_0_12, 42.946054, -87.903810,10001 13-DEC_GRE_0_12, 42.946828, -87.903659,10002 13-DEC_GRE_0_12, 42.946824, -87.903588,10003 13-DEC_GRE_0_12, 42.946830, -87.903472,10004 [...] 13-DEC_GRE_0_12, 43.123137, -87.915431,670004 13-DEC_GRE_0_12, 43.123359, -87.915228,670005 13-DEC_GRE_0_12, 43.124016, -87.914535,670006 13-DEC_GRE_0_12, 43.124117, -87.914440,670007
The line for this trip has 520 points. That's pretty detailed.
So what do we know?
We know that Stop #709 is served by the GreenLine route, it's the 14th stop in direction 0, it's a bus line, we have all the times the stop is served, and we have the route website. We know the route map and all the other stops of any trip serving that stop.
How can we find the next scheduled bus at stop #709?
One way is to start with all trips that stop at #709 from stop_times.txt.
Since we probably know what time it is, we can filter out all the past times, and most of the future times. This leaves us with a nice, small list of, say, 10 possibles that include trips that don't run today at all (we must delve deeper to determine).
We can look up each of those trips in trips.txt, and get the route.
Each trip also includes a service_id code. The calendar_dates.txt file tells us which dates each service_id code is valid.
Right, we need to do three lookups.
The shell code gets a bit complex with three lookups, so I shifted to Python and wrote a basic next-vehicle-at-stop-lookup in about 160 lines. Python lists are handy, since it can handle all the stops at a location just as easily as a single stop. Python's zip module is also handy, so I can read data directly from the zipfile. But at 13 seconds, Python is probably too slow for this kind of application:
$ time ./next_bus.py Next departures from Howell & Okahoma 16:16 GRE N AIRPORT - VIA OAKLAND-HOWELL METROEXPRESS 16:22 GRE N BAYSHORE - VIA OAKLAND-HOWELL METROEXPRESS 16:26 51 OKLAHOMA - TO LAKE DRIVE 16:28 51 TO 124TH ST. - VIA OKLAHOMA 16:30 GRE N AIRPORT - VIA OAKLAND-HOWELL METROEXPRESS 16:35 GRE N BAYSHORE - VIA OAKLAND-HOWELL METROEXPRESS 16:43 51 TO 124TH ST. - VIA OKLAHOMA 16:44 GRE N AIRPORT - VIA OAKLAND-HOWELL METROEXPRESS 16:45 51 TO NEW YORK 16:45 GRE N BAYSHORE - VIA OAKLAND-HOWELL METROEXPRESS 16:56 GRE N BAYSHORE - VIA OAKLAND-HOWELL METROEXPRESS real 0m13.171s # Ugh. If I had started 13 seconds sooner, I wouldn't be bored now. user 0m10.740s sys 0m0.260s
All that time crunching the GTFS file has not gone unnoticed.
Trip planners (like Google) pre-process the data, mapping out and caching link-node and transfer relationships, limiting the trip data to the next hour or two (as appropriate), and using rather fancy algorithms to prune the link-node map to a likely set of possibilities before looking at trips along those links.
That's one reason Google Transit is much faster than 13 seconds.
But that's all advanced stuff.
Also advanced is how to integrate real-time data, which uses one of several different formats. Next time...
