Skånetrafiken’s open API at is what we’re using to collect data for kollektivt.se. Initially we thought that visualizing busses on a map would be a breeze, … This API offers basically only the core features of the travel planner Skånetrafiken has on their website. There is no concept of bus lines, e.g. no option to retrieve the route and stops for bus line 171. Instead we had to use Mechanize to do find specific lines at www.skanetrafiken.se and then BeautifulSoup to scrape the results. This way we could populate our database to make up for the lack of information in their API:s. To be exact, this is what we did to get all the information we needed for a bus line:
- In the Skånetrafiken travel planner, let Mechanize search for all busses passing stations “Malmö C” and “Gustav Adolfs Torg”. This includes all bus lines we are interested in at the moment
- Let Mechanize visit result pages for bus line 1-8 and put all stops in the database
- For each station, call the API and get the coordinates it
- Connect the stations by asking the API for exact routes and duration between two stations for a specific bus line (this step is not present in API:s from other providers, thus currently making Skånetrafiken the only option)
- Repeat step 4 in the other direction since the route might differ
- For both directions, find out to where the bus is heading for that direction. Kind of problematic since this is not always the same as the end station. A direction looks like “Västra Hamnen via Centralen” and you have to know where this is on the map to know which direction it is. We need this when we look at busses that arrive at each stop to determine in with direction the bus moves. This was solved by looking for departing busses at the end-stations where they only drive in one direction. Obviously
Now our database is populated with static information corresponding to thousands of requests for just a few bus lines. This step is necessary to avoid hammering the Skånetrafiken servers too much.
After this it’s time to gather the real-time data. Skånetrafiken offers time deviations rounded to minute precision. This is not that good for buses driving in crowded city streets but it’s better than nothing. After a chat with Skånetrafiken we learned that not all buses send their position due to different reasons so as a result some buses on kollektivt.se just move according to the time table. Even though they’re deviating.
We have an procedure that makes one API-call for every bus stop once every two minutes. This results in approximately 150 calls per minute which is quite a lot but it’s necessary evil since their API:s are not designed for supplying lines and coordinates. Our site put some extra pressure on Skånetrafiken’s servers, as can be seen below.
The collected data is used to interpolate and identify bus lines five times per second. These coordinates are exposed an own API which in turn is used by our own client which interpolates the coordinates further, based on the request rate.