.:: Marcos Dione/StyXman's glob ::. (Posts about hillshading)

Automating blender based hillshading with Python

Marcos Dione — Sun, 05 Nov 2023 16:19:45 GMT

Remember my Blend based hillshading? I promised to try to automate it, right? Well, it seems I have the interest and stamina now, so that's what I'm doing. But boys and girls and anything in between and beyond, the stamina is waning and the culprit is Blender's internals being exposed into a non-Pythonic API³. I swear if I worked in anything remotely close to this, I would be writing a wrapper for all this. But in the meantime, it's all a discovery path to something that does not resemble a hack. Just read some of Blender's Python Quickstart:

When you are familiar with other Python APIs you may be surprised that new data-blocks in the bpy API cannot be created by calling the class:

bpy.types.Mesh()
Traceback (most recent call last):
File "<blender_console>", line 1, in <module>
TypeError: bpy_struct.__new__(type): expected a single argument

This is an intentional part of the API design. The Blender Python API can’t create Blender data that exists outside the main Blender database (accessed through bpy.data), because this data is managed by Blender (save, load, undo, append, etc).

Data is added and removed via methods on the collections in bpy.data, e.g:

mesh = bpy.data.meshes.new(name="MyMesh")

That is, instead of making the constructor call this internal API, they make it fail miserably and force you to use the internal API! Today I was mentioning that Asterisk's programming language was definitely designed by a Telecommunications Engineer, so I guess this one was designed by a 3D artist? But I digress...

One of the first thing about Blender's internals is that one way to work is based on Contexts. This makes sense when developing plugins, where you mostly need to apply things to the selected object, but for someone really building everything from scratch like I need to, it feels weird.

One of the advantages is that you can open a Python console and let Blender show you the calls it makes for every step you make on the UI, but it's so context based that the results is useless as a script. Or for instance, linking the output of a thing into he the input of another is registered as a drag-and-drop call that includes the distance the mouse moved during the drag, so it's relative of the output dot where you started and what it links to also depends on the physical and not logical position of the things you're linking,

bpy.ops.node.link(detach=False, drag_start=(583.898, 257.74))

It takes quite a lot of digging around in a not very friendly REPL¹ with limited scrollback and not much documentation to find more reproducible, less context dependent alternatives. This is what's eating up my stamina, it's not so fun anymore. Paraphrasing someone on Mastodon: What use is a nice piece of Open Software if it's documentation is not enough to be useful²?

Another very important thing is that all objects have two views: one that has generic properties like position and rotation, which can be reacheched by bpy.data.objects; and one that has specific properties like a light's power or a camera's lens angle, which can be reached by f.i. bpy.data.cameras. This was utterly confusing, specially since all bpy.data's documentation is 4 lines long. Later I found out you can get specific data from the generic one in the .data attribute, so the take out is: always get your objects from bpy.data.objects.

Once we get over that issue, things are quite straightforward, but not necessarily easy. The script as it is can already be used with blender --background --python <script_file>, but have in account that when you do that, you start with the default generic 3D setup, with a light, a camera and a cube. You have to delete the cube, but you can get a reference to the other two to reuse them.

Then comes the administrative stuff around just rendering the scene. To industrialize it and be able to quickly test stuff, you can try to get command line options. You can use Python's argparser module for this, but have in account that those --background --python blender.py options are going to be passed to the script, so you either ignore unknown options or you declare those too:

mdione@ioniq:~/src/projects/elevation$ blender --background --python blender.py
Blender 3.6.2
Read prefs: "/home/mdione/.config/blender/3.6/config/userpref.blend"
usage: blender [-h] [--render-samples RENDER_SAMPLES] [--render-scale RENDER_SCALE] [--height-scale HEIGHT_SCALE] FILE
blender: error: unrecognized arguments: --background --python

Also, those options are going to be passed to Blender! So at the end of your run, Blender is going to complain that it doesn't understand your options:

unknown argument, loading as file: --render-samples
Error: Cannot read file "/home/mdione/src/projects/elevation/--render-samples": No such file or directory
Blender quit

The other step you should do is to copy the Geo part of GeoTIFF to the output file. I used rasterio, mostly because at first I tried gdal (I was already using gdal_edit.py to do this in my previous manual procedure), but it's API was quite confusing and rasterio's is more plain. But, rasterio can't actually open a file just to write the metadata like gdal does, so I had to open the output file, read all data, open it again for writing (this truncates the file) and write metadata and data.

Now, some caveats. First, as I advanced in my last post, the method as it is right now has issues at the seams. Blender can't read GDAL VRT files, so either I build 9 planes instead of 1 (all the neighbors are needed to properly calculate the shadows because Blender is also taking in account light reflected back from other features, meaning mountains) or for each 1x1 tile I generate another with some buffer. I will try the first one and see if it fixes this issue without much runtime impact.

Second, the script is not 100% parametrized. Sun size and power are fixed based on my tests. Maybe in the future. Third, I will try to add a scattering sky, so we get a bluish tint to the shadows, and set the Sun's color to something yellowish. These should probably be options too.

Fourth, and probably most important. I discovered that this hillshading method is really sensible to missing or bad data, because they look like dark, deep holes. This is probably a deal breaker for many, so you either fix your data, or you search for better data, or you live with it. I'm not sure what I'm going to do.

So, what did I do with this? Well, first, find good parameters, one for render samples and another for height scale. Render time grows mostly linearly with render samples, so I just searched for the one before detail stopped appearing; the value I found was 120 samples. When we left off I was using 10 instead of 5 for height scale, but it looks too exaggerated on hills (but it looks AWESOME in mountains like the Mount Blanc/Monte Bianco! See below), so I tried to pinpoint a good balance. For me it's 8, maybe 7.

Why get these values right? Because like I mentioned before, a single 1x1°, 3601x5137px tile takes some 40m in my laptop at 100 samples, so the more tuned the better. One nice way to quickly test is to lower the samples or use the --render-scale option of the script to reduce the size of the output. Note that because you reduce both dimensions at the same time, the final render (and the time that takes) is actually the square of this factor: 50% is actually 25% (because 0.50 * 0.50 = 0.25).

So, without further addo, here's my script. If you find it useful but want more power, open issues or PRs, everything is welcome.

https://github.com/StyXman/blender_hilllshading ⁵

Try to use the main branch; develop is considered unstable and can be broken.

A couple of images of the shadows applied to my style as teaser, both using only 20 samples and x10 height scale:

Dhaulagiri:

Mont Blanc/Monte Bianco:

Man, I love the fact that the tail of the Giacchiaio del Miage is in shadows, but the rest is not; or how Monte Bianco/Mont Blanc's shadow reaches across the valley to the base of la Tête d'Arp. But also notice the bad data close to la Mer de Glace.

Ok, TBH here, I'm very much used to ipython's console, it's really closer to the plain python one. No tab completion, so lots of calls to dir() and a few help()s. ↩
I couldn't find it again. Mastodon posts are not searchable by default, which I understand is good for privacy, but on the other hand the current clients don't store anything locally, so you can't even search what you already saw. I have several semi-ranting posts about this and I would show them to you, but they got lost on Mastodon. See what I mean? ↩
So you have an idea, this took me a whole week of free time to finish, including but not in the text, my old nemesis, terracing effect. This thing is brittle. ↩
Yeah, maybe the API is mostly designed for this. ↩
My site generator keeps breaking. This is the second time I have to publicly admit this. Maybe next weekend I'll gather steam and replace it with nikola. ↩

Blender hillshading and mapnik

Marcos Dione — Sat, 22 Oct 2022 23:55:01 GMT

How did it start? Well, it actually started a long time ago, maybe a year ago, most probably more. I got myself tumbling down the beautiful rabbit hole of Blender based mapping. The idea is very simple: if you have a DEM, you can build a 3D representation of the terrain and use a renderer to build your map. To me the most striking thing about those maps were not their 3D-ness (which to some it's starting to be tiresome , and I agree), but the shadows. I've been pursuing the best shadow for a while, and this seemed like the perfect fit.

So, like I said, one year ago or so I took "the" Blender relief tutorial and run with it. I got to the point where I could reproduce it with a 1x1, 3600x3600px DEM from mapzen, but when I tried to automate it, I found out that Blender has a python console where it prints out the commands that are equivalent to the actions you make in the UI, but the resulting script was too horrible to my eyes and run out of breath (another of those cases of the perfect being the enemy of the good).

Then a few days ago I read that first link and got some steam build up. In fact, it was two passages in it that lit up the fire:

Most of these use a tool called Blender, an extremely powerful open-source tool for all kinds of 3D modeling and rendering. A few cartographers use other tools, such as Aerialod, or R‘s Rayshader plugin.

R! I can easily automate this!

If we stick to a very zoomed-out map, or if we have a really huge computer running Blender, we could try to do a hillshade for the entire world, and then slice that up for our map tiles. But there’s no way we could do this at a high-enough resolution so you could zoom all the way in, as in the sample tiles above.

Challenge accepted! (Spoiler: I'm not there yet).

I tried Rayshader. I wanna be honest: it's easy, quick, but I didn't like the results. It seemed like no matter how high you put the sun, it always drew very long shadows. So despite its pragmaticism, I left it on a side.

So I picked up what I did in the past and tried to apply it to a map. I re-rendered everything and applied it to my style. The outcome was encouraging:

To start with, I probably did that render with not enough render passes, so the output looks grainy. Second, the material color is too bright, so the height tints are washed out. Still, we can see the big shadow cast over the valley some 3200m below the Mont Blanc/Monte Bianco.

This proved to be a good place to test the method, because of the great difference between the valley and the peak casting the shadow over it, and that lead me to think: are there more extreme places in the world? An easy bet is yes, and the place to look for them was the Himalayas. The Aconcagua could have been a good contender, but the valley at its SE is some 4550m already. Surprisingly, the way I searched for a good place was to use existing maps with the usual hill shade technique, looking for big dark spots, specially those wide in the NW-SE direction. I first found my spot in the Tukuche Peak, that looms some 4350m above the Gandaki River, and then the nearby Dhaulagiri, that's even some 1250m higher, clocking at 8167m. Here's how they look (not the big structure in the upper left, but the bigger grey [8000m+] blob to the East of it; the river snakes in the right all the way down):

I had to add 3 more color bands to my style and reshuffle them because I never rendered any 8k'er before, so the colors were haphazardly concocted for the rendering and are not definitive. At least it lets you behold the grandiosity of that rock jutting through thousands and thousands of meters with very steep sides.

Time to get real. I usually render regions were I'll be going, and next time it's the Upper Segre Valley, so I rendered N42E001-2 in one go. That's 7201x4884px after reprojecting to WebMercator (and compensating as described in the second link!), so some 35Mpx. Blender took some 44m+ on a 4.5yo medium-high spec'ed laptop at 200 render samples, which means that I can continue to render small regions this way, but that for the moment I won't be applying this technique to the whole Europe.

Up to here I was just applying the same style in QGIS, which has been an indispensable tool to develop this style. But trying these TIFFs in mapnik for the first time needed an extra step. Well, two, in fact. Blender does not save the TIFFs georeferenced, so you have to copy the data from the original DEM. For that, use gdal_edit.py -a_srs ... -a_ullr ... with the right ESPG and the data from the output of gdalinfo. Next, for some reson, it always use 16bits integers, even when explicitly saying to use 8. This little snippet takes care of that:

import imageio

image = imageio.imread('pirinoak-blender-10x.tif')
image = image / 256
image = image.astype('uint8')
imageio.imwrite('pirinoak-blender-10x-8bits.tif', image)

Thank $DEITY (and developers!) for good libraries.

The first thing I noticed was that we have been lied by maps (again!) for a long time. Most hill shading algos use a 45° high sun (the direction does not matter much). But if you think about it, how many mountains have sides 45°+ steep? According to a (real, not like me!) cartographer friend, for continental Argentina it's less than 1% at 30arcsecs of resolution (note that SRTM is 1arcsec). Still, some shadows are there, and they help us (and we get used to that) to recognize slope direction. And now we're asking a raytracing program to calculate real shadows? The result I initially got was underwhelming, even when I was already asking Blender to exaggerate height by 5x!:

So, I bit the bullet and went all in with 10x:

Much better, but not definitive. I still have to render Dhaulagiri again, and at least some region I already know well by having being there a lot. Here's how that region looks in my style:

Now some notes about "the" Blender relief tutorial. I followed it to the letter, but with my experience I had to make some changes. One you already know, using a displacement scale of 10x instead of 0.3. I have no exact idea why his initial rendering were so spiky, but I suspect that the DEM grid unit was not meters.

Second, since that first Mount Blanc/Monte Bianco render, we know the color is too bright. I lowered it to 0.6 (and later I found that that's what he actually suggests at the end of the plane section) and then compared the grey in a plain (#b5b5b5) to what GDAL outputs and compensated using a simple coefficient. The final value is 0.402.

Third, I was having issues rendering: I was getting a lot of terracing. After a long chat with Viktor_smg from blender.chat/support they figured out that the sRGB color space in the Image Texture is broken and that I should use XYZ instead. This meant installing Blender by hand instead of relying on the one in Debian Unstable because it's too old and does not have it. They also gave me pointers about how to automate it.

Last, you can't apply this technique DEM by DEM because you want the shadows from the neighbouring tiles to spill over the current one. That link shows how to render the tile and its 8 neighbouring ones, but I think that you can optimize it in two ways: First, since shadows come from the NW, just add the tiles that lie in that general direction. Second, no shadow would cast over 10s of kilometers. You could even get away with just adding a smaller band around the tile.

That's it for now. The next step is to automate this an publish that. $DEITY knows when that will happen.