Towards the perfect map pt 0.5

So it's been a long while that I talked about my quest towards the perfect map for in-car navigation and general orientation. This post is to fix that.

I started with marble, monav and OSM tiles. The tiles proved inefficient for in-car navigation because of the non-contrasting palette, which makes differentiating ways from blocks very difficult in the sunlight. Then I switched to creating maps with CloudMade, which at the beginning seemed enough, but then I noticed that the solution was not complete, as it lacked some features (in OSM terms) that couldn't be mapped. When I started to look for alternatives, I found a site that showed TileMill, which can use OSM data from a database, and specially its ability to create relief maps, with altitude coloring, slope and hillshading. I was doomed :)

I started exploring the relief part. relief data is available from several sources, the most prominent being the ones derived from the Shuttle Radar Topography Mission, like CGIAR-CSI or Jonathan de Ferranti. Discussing with a friend, he told me to stick to the first ones, as de Ferranti's data has been smoothed by hand and is not always the best. Also, his several sources have incompatible licenses, so it should be unusable. With this data I also generated contour lines, which you will recognize from topographic maps.

The next step is how to use OSM data. I started using GeoFabrik's extracts, first importing the data into a database, but this takes a lot of time, memory and disk space. I switched to the shapefiles, but this introduced several problems. First, the sum of the data contained in the several shapefiles for a region (landuse, natural, places, points, railways, roads and waterways) are not the entire set of data; for instance, all data related to ski lifts is not there. I wanted those. So I reluctantly reverted to importing the data in a database.

Then there's the problem of rendering a usable map. Both TileMill and OSM render the tiles using Mapnik, which implements the so called Painter's algorithm: you define layers and they're 'painted' from bottom up, one on top of the other. So, what do you put in a layer? You define a data source, which is a select in the database, and you link it to a style. A style defines several rules, which can provide extra filters (which technically you could do in the database, but simplifies the data source definitions), min and/or max scales (related to zoom levels) and how to paint them: fill a polygon, draw a line, put a symbol, write some text, etc.

Clearly, just like that, the complexity is big. Defining if a feature appears in a zoom level or not and how, specially if it changes between zoom levels, is quite complex. Then you have more factors: putting borders to a line is actually implemented as drawing two lines, one thicker, defining the borders, and then a thinner one on top of it, defining the inner part, done in two different layers. This is called casing. Also, think that you have to change the casing and the fill color when there is a tunnel or add an extra casing (layer) for a bridge, and even when there are several bridges on top of each other (think of complex highway junctions; implementet with more layers)! TileMill proved to be good for testing, but this level of nitpicking was too much for it.

For a while I flirted with the idea of generating the Mapnik input file with a structural description of what I wanted, but I aborted it before it became a monster. I learned something from Frankenstein!

So, why not come full circle and take a look at how the OSM tiles are generated? This was definitely the right move. Instead of trying to mimic the complex set of rules and layers that make OSM tiles perfect, just edit the xml files to take out uninteresting data, modify some zoom levels here and there to make some things appear sooner (I'm very interested in gas stations, hospitals, parkings, archaeological sites and viewpoints), and edit the colors. I can even export what I have done in TileMill to a Mapnik project, extract the part where a define the relief layers and add them as the background.

So the final setup is as following:

• Use elevation information from CGIAR-CSI, process them as TileMill says.
• Use Geofabrik's pbf extracts; import them in a PostGis Database as per this page in OSM's wiki.
• Use TileMill for initial designing, specially about color changes.
• Modify osm.xml by hand, picking out what's not wanted, changing zoom levels for some stuff.
• Use a script to extract colors and line widths from osm.xml; use the colors designed with TileMill.
• Use a modified version of generate_tiles.py to render big areas or special cities.

I almost have it finished with the scripts that allows me to iterate over the last two steps without much hand intervention (except tweaking the values, of course). When I'm done I'll publish it, then clean it up, then republish :)