Note: this is a translation of an old post. I decided to translate it because now I'm looking for a SysAdmin position (tell your friends!) and I would like this post to show how I work.

Last Saturday I received an email from one of the guys from work with the subject «urgennnnnnnnt: heeeeeeeeeelp»[sic]. He says he was idling on Friday night when his machine stopped emiting sound via the soundcard and then it behaved erratically. When he tried rebooting it, it didn't boot anymore. «It says something about disk not bootable...».

Monday morning I go to work and go to see the machine. Precisely, it said something about «disk not bootable». I boot with a USB key with GRML and I find that the disk has no partitions.

Panic.

The guy is doing a PostDoc in something astronomical (literally) and all his work is in that machine. No backups, as usual, so I prepare myself to rescue the partitions.

In that same USB key I have a system with parted. I boot with it and I try using parted's rescue tool. Nothing. I ask the guy how the disk was partitioned, etc. He tells me that he only installed Kubuntu clicking 'Next'. Kubuntu by default creates a swap partition and an ext3 partition for / and that's it, which made what was coming relatively easy.

I reboot in GRML and I use hexdump -C /dev/sda | more to see the disk's content. This is not the first time that I juggle with partitions and MBRs, but last time I did it, I used a tool that is now discontinued (the tool was called DiskEdit, included in The Norton Utilities), which had special edit modes for MBRs, FATs, and a lot of useful things... in MS universe.

First I confirm that, yes, the first sector is a MBR (at least it has the 0x55aa signature at the end), and that the whole partition table is empty, but in the second sector of the disk there seems to be a copy. I take pen and paper, write down what I found, but it turns out not only I have half the data, the partition I thought I found was too small.

So I decide to look for the partition by hand. To do that I needed to find out first how does the ext3 kernel code know wether a partition is ext3 or not. I knew it would be some kind of magic signature, but I had no idea which. So I installed the sources for 2.6.29 in my laptop and started to look at ext3's code. After going around a lot, including reading the code that is excuted when you mount a filesystem of type ext3, where we can see that it uses a magic signature[3] and the structure of the ext3 superblock, where we can see the magic's offset is 0x38.

So the problem of finding an ext3 partition is reduced to the problem of finding 0x53ef (damn little endian) at a sector's offset 0x38 in the disk. Luckily more has a find tool, so I sit down to search every occurrence of 53 ef, hoping that the address at the left ends in 30 and that they would be the 9th and 10th bytes in the line (damn 0 based offsets).

A few 'next' after, I get my first candidate. It looks good, because I was also comparing my findings with a similar dump from my USB key (which I have formatted as ext2, and luckily ext2 and ext3 share those structures), and also I spot something that looks like a uuid.

This candidate's address is 0x80731038. I substract 0x38 and I get the address 0x80731000, a nice round number for a superblock. Converted to decimal that's 2.155.024.384, some 2GiB from the disk's begginning. Looks really good! The swap partition could be before the root one, and could have that size.

I use fdisk /dev/sda to see the (still empty) partition table. It says there's 16.065 sectors per cylinder, times 512 bytes per sector, equals 8.225.280 bytes per cylinder. Almost all distros (actually I think all of them) partition disks at cylinder boundaries[1], so if the sector I found is right at the beginning of a cylinder...

I divide 2.155.024.384/8.225.280=...

(suspense pause)[2]

262.000124494...

¡Damn! I almost had it... Hmm, how much is the factional part? (262.000124494-262)*8.225.280=... ¡1024! ¿Is it that...?

I run strace debugfs -R show_super_stats /dev/sdb1 (the partition in my USB key) and I see that it actually seeks 1024 bytes within the partition for reading the superblock!

This is it. With 262 in my head, I fire fdisk /dev/sda and I create two partitions: swap in cylinders 1-261 and root from cylinder 262 till the end. I save, cross my fingers and I run debugfs -R show_super_stats /dev/sda1. It fails! What's wrong? I reboot and I try again, just in case the kernel did not re-read correctly the partition table. It fails again. WTF?

Ah, duh, it's sda2, where do I have my head... Ok, debugfs -R show_super_stats /dev/sda2... it works, the sonofabitch works! I can't believe it. I risk it: fsck -n /dev/sda2. «Filesystem is clean». Damn, I try harder: fsck -n -f /dev/sda2...

Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/sda2 etc etc...

It's fine! But the MBR doesn't have GURB installed, so I do the usual GRUB reinstall process, I reboot...

It boots like nothing has happened, and it finishes in a beautiful login. Satisifed, I pat myself in the back, pack my things and I start my weekend.

sysadmin rescue


[1] ... wasting some 8MiB between the MBR and the first partition.

[2] The sharp ones reading this will notice that this can not give an integer by no means.

[3] Reiser magics are funny. Looks like he started the fad that now AdOlEsCeNtS use.

Posted Thu 07 Apr 2011 11:49:04 PM CEST Tags: tags/rescue
12

At work I was tasked to develop three different lines of work. Each one seemed to be auto-contained and with an specific and compact obective. Of course, for each one of these projects, I set up a VCS repo. The chosen one here at work is Mercurial.

But then, the fourth task proved to be the integration of the three previous tasks in one demo that would show them off and then evolve into an application. At fisrt I just relied on setting up a fourth repo, adding to it only the new files, and symlinking the files I needed from the other repos. It was hacky, but it allowed me to continue the development quickly, and as long as I kept making commits in the four different repos, all was peachy.

Today, after the demo was presented, it occured to me that this setup wasn't easy to share with the rest of the team. So far I was working alone, but now that this project will get more and more developers, sharing was crucial.

So I asked in #mercurial and the short answer was:

11:13 < hstuart> StucKman, you can pull --force and merge the unrelated heads together, but it'll be a two-way merge and there'll potentially be a lot of conflict resolution if they have diverged in the same files

As the four repos had disjoint file set, this seemed enough for me. So, the first step was to set up a fifth empty clean repo:

$ hg init

No surprises there. Now I pulled from the four lines of history from other directories:

$ hg pull --force ../soap/
pulling from ../soap/
requesting all changes
adding changesets
adding manifests
adding file changes
added 7 changesets with 9 changes to 3 files
(run 'hg update' to get a working copy)

$ hg pull --force ../stomp/
pulling from ../stomp/
searching for changes
warning: repository is unrelated
adding changesets
adding manifests
adding file changes
added 43 changesets with 49 changes to 8 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)

$ hg pull --force ../phidgets/
pulling from ../phidgets/
searching for changes
warning: repository is unrelated
adding changesets
adding manifests
adding file changes
added 29 changesets with 40 changes to 7 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)

$ hg pull --force ../smartimmo/
pulling from ../smartimmo/
searching for changes
warning: repository is unrelated
adding changesets
adding manifests
adding file changes
added 26 changesets with 58 changes to 20 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)

So far so good. I checked with hgview what I had: four lines of history with no common point. Do you see that each pull made a new head? Here, look:

$ hg heads
changeset:   104:a923c43e33fa
tag:         tip
user:        Marcos Dione <mdione@sophia.inrira.fr>
date:        Fri Dec 17 11:04:24 2010 +0100
summary:     * soft-coded image filepaths (patch by Manuel).

changeset:   78:1a3faf6e6e8a
branch:      single-ifkit
user:        Marcos Dione <mdione@sophia.inrira.fr>
date:        Thu Dec 16 16:12:40 2010 +0100
summary:     * wrong message for up-> nobody.

changeset:   74:53c2942a8548
user:        Marcos Dione <mdione@grulic.org.ar>
date:        Wed Dec 01 14:58:53 2010 +0100
summary:     * s/state-update/light-state-update/ in event name, so it doesn´t clash with the same event in the temp controller.

changeset:   49:9480820604fe
user:        Marcos Dione <mdione@sophia.inrira.fr>
date:        Thu Dec 16 16:13:04 2010 +0100
summary:     + si-heater-switch.

changeset:   6:be775f2b6af5
user:        Marcos Dione <mdione@sophia.inrira.fr>
date:        Fri Jul 09 11:43:58 2010 +0200
summary:     * cleanup.

Notice how the tip is set to the tip of the last repo I pulled from. If you're very 'detaillist' (I'm not sure that word exists in English; it does in Spanish), take that in account. Me, I didn't care that much. So the next step was to merge them:

$ hg merge --force 6
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don´t forget to commit)
$ hg ci -m "|\ soap history."

The first one went fine, but the second one presented a small problem:

$ hg merge --force 49
merging smart-immo.scm
merging smart-immo.scm failed!
7 files updated, 0 files merged, 0 files removed, 1 files unresolved
use 'hg resolve' to retry unresolved file merges or 'hg update -C' to abandon

So I went back, before the merge, moved the file, and continued:

$ hg update -C 105
1 files updated, 0 files merged, 7 files removed, 0 files unresolved
$ hg mv smart-immo.scm smart-immo-soap.scm
$ hg ci -m "* rename to avoid file clashes."
$ hg merge --force 49
8 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don´t forget to commit)
$ hg ci -m "|\ obix/stomp history."
$ hg merge --force 74
6 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don´t forget to commit)
$ hg ci -m "|\ smartimmo demo history."

I still had something left: one of the hsitories had a branch which wasn't default but was tip, so I had to merge that too. I don't like how this is done in Mercurial, but I know the moves:

$ hg update -C single-ifkit
2 files updated, 0 files merged, 31 files removed, 0 files unresolved
$ hg ci --close -m "_ closing before merging."
$ hg update -C default
33 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg merge single-ifkit
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don´t forget to commit)
$ hg ci -m "|\ single-ifkit."
$ hg heads
changeset:   110:bfc9eb117a76
tag:         tip
parent:      108:d53b4b9a3ffc
parent:      109:b41570e653c9
user:        Marcos Dione <mdione@sophia.inrira.fr>
date:        Fri Dec 17 11:34:26 2010 +0100
summary:     |\ single-ifkit.

Tadaaa! Yes, I would like a hg merge --close <head> command that did all this dance for me, but as I said, as I know the hoops I have to jump through, and as long they're not in flames, I can live with it. The only thing I miss from other VCS is Bazaar's --show-diff commit option, which lets me review the diff while editing the commit message.

One thing that I figured out out of this: histories in Mercurial are acyclic directed graphs, yes, but they don't need to have only one root. Nice.

mercurial

Posted Fri 17 Dec 2010 01:44:33 PM CET Tags:

Note: I began to write this post a long time ago, then I thought it was too much a rant, and left it there. Today, while procrastinating a lot from work, I read it again and decided it wasn't so bad after all. So, here it is.

Today I sat down to write some lines of code for satyr. I has been more than one month since I wrote any line of code in it, which is a shame, but I really wasn't in the mood. For that, I decided to queue some songs so the music was not very much eclectic. Just to be not very boring, I put Slash's solo album. During my small breaks from the code (I do them from time to time; seems like I can't be focused on anything for too long) I was editing the tags of the songs and reading the Wikipedia's page about the album. When I got to the tracklisting I noticed that there were no more and no less than 9, yes, 9 different editions.

This is not the first time I see this; normally I had seen alternative editions, specially japanese editions, most of the time with one more or different track, but it never really bothered me. I really don't know why, but this time I think it's plain abuse.

Combining the other 8 editions from the one I have there are 5 more songs, plus an english version of one of those (as it is a song in the 2, yes, 2 japanese versions, I can only assume the original is in japanese) and a DVD track. And of course none of the editions has them all; the biggest one is the canadian deluxe, with only 3 of those. So, no matter which disc one buys, there is no way to have them all.

Of course, besides buying several discs. How many? Hold your socks, pants and jaw: The japanese deluxe edition, the iTunes version, the canadian deluxe edition and the Monster energy drink edition; and then you get twice the same song (the english version of 'Sahara'). Yes, count them: 4. Not to mention that two of them are only sold in two countries not only far away from one to another, but also faraway from where I'm living, and you can't buy them online and have them delivered in your house.

I think the message is obvious already: those [expeletive] from the record labels are out to get your money more than ever. Just look at the list of different labels involved in this opportunity and you'll find the usual suspects: EMI, Universal Music, Roadrunner Records and Sony Music. What a bunch of greedy bastards that we already know for their greediness and stupidity:

  • Sony and its copy protection scandal.
  • Universal has at least 4 dubious actions.
  • EMI, which used to have DRM in their sales through iTunes, but not anymore. Also they tried to buy Warner and convert two greedy bastards in one huge greedy bastard, but got bought by Terra Firma instead. They laid off ~2k jobs.
  • Roadrunner is part of Warner, which blocks videos in YouTube even when they're not officially WMG content. And they have more.

Another example of how things are wrong is the album 'Live at The Greek', a live album recorded by The Black Crowes and Jimmy Page, the guitarrist from Led Zeppelin. In this album we can only find the songs they played that were not from the TBC repertorie 'due to contractual problems with their [TBC's] record comapny'. Of course, most of what's left are songs from Led Zeppelin, which makes it a fantastic live/cover album for Led Zeppelin songs, but then I can't hear TBC songs with legend Jimmy Page as first/second guitar, something never ever again is going to happen. I simply can't. Ok, I can, and you know how? Because someone recorded the concert with a video camera, and then this hero/heroine put it online, and I downloaded it. Yes, completely ilegal.

I simply can't wait 'till this greedy bastards dissapear once and for all, and culture goes back to flow more freely among us, as it should be. To finish, I give you the translation of a quote I have among my random fortunes. Unluckily the original article where this comes from is not available online anymore:

La razón por la que la industria de medios es tan amiga del término “piratería” es que inmediatamente evoca una imagen negativa, de violencia y saqueo, en la que ellos son las víctimas que se ven privadas de “su propiedad”. [...] en una conferencia que diera la abogada de la Federación Internacional de la Industria Fonográfica, [...] escuchamos cómo [este] abogado se alarmaba del hecho de que “la sociedad se está apropiando de la cultura”.

The reason why the media industry likes so much the term “piracy” is because it evokes a negative image, of violence and sacking, where they appear as the victims who are deprived of ”their property”. [...] in a conference given by the lawyer of the International Federation of the Phonographic Industry, [...] we hear how [this] lawyer is alarmed of the fact that ”society is appropiating the culture”.


rants

Posted Thu 16 Dec 2010 12:50:29 AM CET Tags:
Posted Wed 06 Apr 2011 08:29:31 PM CEST
08
Posted Wed 06 Apr 2011 08:29:31 PM CEST
01

El segundo día me amaneció a las 13. Hoy tocaba terminar con el Apache, que incuía apenas los tracs y el dotproject. Ambos implicaban upgrades.

Los trac no me hicieron renegar mucho, pues está muy bien documentado y hasta fue scripteable. Básicamente era un salto de sqlite2 a sqlite3, un trac-admin ... upgrade seguido de un trac-admin ... resync.

Con lo único con lo que renegué fue que a pesar del upgrade fue exitoso no podía entrar. En los logs encontraba esto:

(9)Bad file descriptor: Could not open password file: (null)

Google al rescate me dijo que había que apagar esa directiva que había tenido que modificar el día anterior:

<span class="createlink">AuthzLDAPAuthoritative</span> off

También me salió esto:

Failed to load the <span class="createlink">AuthzSVNAccessFile</span>: The character 'o' in rule '@except' is not allowed in authz rules

Eso era porque en un archivo de configuración del repo (conf.svnaccess) tenía los permisos de sólo lectura como ro en vez de r.

El dotproject me enfrentó a un viejo archienemigo: mysql. La verdad que no se a queinacarajos se le ocurre que es una excelente idea poner la configuración de acceso y permisos de una base de datos dentro de la base misma. Por un lado eso termina siendo un archivo binario no versionable y por otro obliga al sysadmin a aprender SQL (cosa que sé, pero no manejo fluidamente ni me interesa saberlo; otro de los motivos por los que amo los ORM's). Y además esta configuración termina en /var y no en etc. postgresql, en cambio, es mucho más inteligente. Y viva el SQL independiente del motor. Lástima nadie lo usa...

Bien, sólo tuve que hacer un dump del mysql anterior (chroot mediante), crear la base en el nuevo y hacer un load. Fantástico. Luego una lucha trabado con el sistema de permisos antesmencionado. Luego apuntar un browser a https://server/dotproject/install. En ese minisitio tuve primero que configurarlo (como DP no es un paquete en Debian, lo instalé de fuentes; la configuración queda en un archivo en include/config.php; ojo que las otras opciones es nukear las bases), luego volver a entrar a dicha URL, momento en el cual detecta las bases viejas y da la opción de upgradearlas. Anduvo sin problemas y ahora disfrutamos de un DP más nuevo. Yeepee!

sysadmin debian sarge etch apache trac svn mysql dotproject

Posted Wed 27 Jan 2010 11:55:55 PM CET

In one of my previous post I mentioned that my blog was brokenito. Actually my blog is just a bunch of markdown files that I compile in a blog with ikiwiki, and I store it in a Bazaar repo.

A month ago I bought a new hardisk that I installed in my notebook, replacing the one that used to have the sources of my blog. I reinstalled Debian Sid from scratch[1] and just grabbed everything from the backups (yes, I have weekly backups. How many can say that? :)

The problem was this: the backups included the Bazaar working copies/branches, except for the (normally empty) directory .bzr/repository/upload/. This directory is populated with a temporary file each time you do a commit, but bzr doesn't try to create it if it doesn't exist. I think they assume it's a given because it's created when you do a bzr init.

So here are two bugs: first my backup system (just a bash script that runs rsync) should store empty directories (fixed) and I think bzr should create the dir if it's not there. I will try to make a patch and submmit a wish in their bugtracker.

This oneliner should fix all my restored repos:

find . -name .bzr | while read repo; do mkdir -vp $repo/repository/upload/; done

sysadmin bazaar


[1] Somehow I feel the new instalation faster than the previous one, specially when installing new software or updates. I think this might be related to the high fragmentation that the old system might have. I should explore this.

Posted Fri 19 Mar 2010 07:10:22 PM CET Tags:

Tal vez ya lo leyeron en otro lado, pero bueno: el otro día fue el día de los tutoriales de Kubuntu. Básicamente fueron tutoriales por IRC. Los logs los pueden encontrar en el wiki de Kubuntu. En particular hay 3 que me parecen bastante piolas:

Como para salir masomenos andando están muy piolas.

bazaar pykde

Posted Wed 27 Jan 2010 11:55:54 PM CET Tags:

Siguiendo el post anterior, y en particular con bazaar, un par de links más.

Empecemos por el hecho de que bazaar acaba de lanzar su versión 1.0. Notable que entonces yo encuentre esto:

$ dpkg -l bazaar
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Installed/Config-files/Unpacked/Failed-config/Half-installed
|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
||/ Name    Version    Description
+++-=======-==========-========================================================
ii  bazaar  1.4.2-5.3  arch-based distributed revision control system

La explicación según la comunidad:

14:56 < <span class="createlink">StucKman</span>> is debian's version numbering wrong?
14:57 < <span class="createlink">MattCampbell</span>> The package name is bzr
14:58 < sabdfl> <span class="createlink">StucKman</span>: bzr was bazaar-ng
14:59 < sabdfl> there was a project called tla
14:59 < sabdfl> some folks at canonical had a branch of that, which they called bazaar
14:59 < sabdfl> that's what you are looking at v1.4.2 of
15:00 < sabdfl> bzr was a skunkworks, from scratch clean set of ideas
15:00 < sabdfl> and when most of the canonical guys embraced that, they brought the name along

Del sitio de bazaar, recomiendo el minitutorial, un pdf con una quick reference, obviamente la extensa documentación, y muy particularmente los workflows.

Por último, acá hay un link de cómo usar bazaar para administrar /etc, el cual al momento de escribir este post parece estar caído.

bazaar

Posted Wed 27 Jan 2010 11:55:54 PM CET Tags:

Acaba de terminar la edición 2009 de pyCamp. Esta vez vinieron cerca de 40 personas, lo cual hizo que hubiera más proyectos dando vueltas y mas gente en los proyectos. Fueron 4 días fantásticos llenos de ideas, código, reuniones, juegos, algo de alcohol y mucho mas. A diferencia del año pasado, esta vez vienieron algunos audaces con familia, no sé cómo les habrá ido.

Este año estuve mucho mas enganchado. El primer día hicimos un schedule cuasi definitivo y en el momento se me ocurrió hacer cosas con Fuse y Python. Cuando tocó el slot, di una charla de cómo funciona Fuse y algunas puntas de cómo implementar file systems con él. Al final del evento yo había terminado el wrapper que venía haciendo hace unas semanas (ok, ok, falta statvfs) y perrito se hizo un filesystem para acceder los iPod. Lucio me hizo prometer ver cómo combinar Fuse async con Twisted. También le estuve explicando ctypes al Polako, con lo que creo que terminé de entender el módulo y me ayudó a entender algunas cosas que había hecho para el wrapper.

También estuve en el diseño y (re)implementación del bot de irc. En apenas 2 días y medio ya tenemos el core y unos cuantos plugins, y hay varios desarrolladores haciendo mas. Sólo faltan implementar pedezos de infraestructura, sobre todo la parte de bases de datos, pero me veo metiendo un par de plugins mas y ponerla en producción muy muy pronto (en relaidad perrito le va a dar hosting). También fue una oportunidad para (re)aprender Twisted, y enterarse de cosas como que no podés hacer asincrónico un proceso sincrónico, y de aprender de boca de Guillo cómo usar bzr para laburar entre los 6 u 8 que metíamos código.

También estuve renegando los dos primeros días con el applet de batería de KDE4. Terminé encontrando (un bug en Solid)[https://bugs.kde.org/show_bug.cgi?id=187600] y aprendiendo detalles sobre Hal, D-Bus, algunos bastante oscuros y bizarros. Al mismo tiempo estuve viendo cómo se comportan los algoritmos de recarga de batería y de estimación de los tiempos de descarga y de descarga. Resulta que cuando está terminando de cargar se empieza a estirar el tiempo y los últimos 5 minutos pueden termiar siendo 20.

Estuvo genial poder conocer más gente y de volver a ver algunas caras conocidas (hace rato que no estaba en un evento de alguna comunidad). Entre los nuevos encontré a gente de Kde-ar como Leo u otros jugando con PyQt. Me encantó volver a sentir que programaba, ver unos proyectos arrancar y otros continuar a velocidades de la hostia, con features apareciendo como hongos y bugs desapareciendo como... bueno, no es una buena fecha para hablar de desapariciones :|

El último sprint estuvo genial; monitoreen la lista y/o el canal para enterarse de los resultados ;-)

python twisted bazaar pyar kde

Posted Wed 27 Jan 2010 11:55:54 PM CET Tags:
Posted Wed 27 Jan 2010 11:55:55 PM CET
06
Posted Wed 27 Jan 2010 11:55:55 PM CET

A couple of days ago Marcelo Fernández wrote a simple image viewer in PyGTK. It's less than 200 lines long[1], and I thought that it would be nice to compare how the same app would be written in PyKDE4. But then I though that it would not be fair, as KDE is a whole desktop environment and GTK is 'only' a widget library, so I did it in PyQt4 instead.

To make this even more fair, I hadn't had a good look at the code itself, I only run it to see what it looks like: a window with only the shown image in it, both scrollbars, no menu or statusbar, and no external file, so I assume he builds the ui 'by hand'. He mentions these features:

  • Pan the image with the mouse.
  • F1 to F5 handle the zoom from 'fit to window', 25%, 50%, 75% and 100%.
  • Zooming with the mouse wheel doesn't work.

Here's my take:

#! /usr/bin/python
# -*- coding: utf-8 -*-

# OurManInToulon - Example image viewer in PyQt4
# Marcos Dione <mdione@grulic.org.ar> - http://grulicueva.homelinux.net/~mdione/glob/

# TODO:
#     * add licence! (GPLv2 or later)

from PyQt4.QtGui import QApplication, QMainWindow, QGraphicsView, QGraphicsScene
from PyQt4.QtGui import QPixmap, QGraphicsPixmapItem, QAction, QKeySequence
import sys

class OMITGraphicsView (QGraphicsView):
    def __init__ (self, pixmap, scene, parent, *args):
        QGraphicsView.__init__ (self, scene)
        self.zoomLevel= 1.0
        self.win= parent
        self.img= pixmap
        self.setupActions ()

    def setupActions (self):
        # a factory to the right!
        zoomfit= QAction (self)
        zoomfit.setShortcuts ([QKeySequence.fromString ('F1')])
        zoomfit.triggered.connect (self.zoomfit)
        self.addAction (zoomfit)

        zoom25= QAction (self)
        zoom25.setShortcuts ([QKeySequence.fromString ('F2')])
        zoom25.triggered.connect (self.zoom25)
        self.addAction (zoom25)

        zoom50= QAction (self)
        zoom50.setShortcuts ([QKeySequence.fromString ('F3')])
        zoom50.triggered.connect (self.zoom50)
        self.addAction (zoom50)

        zoom75= QAction (self)
        zoom75.setShortcuts ([QKeySequence.fromString ('F4')])
        zoom75.triggered.connect (self.zoom75)
        self.addAction (zoom75)

        zoom100= QAction (self)
        zoom100.setShortcuts ([QKeySequence.fromString ('F5')])
        zoom100.triggered.connect (self.zoom100)
        self.addAction (zoom100)

    def zoomfit (self, *ignore):
        winSize= self.size ()
        imgSize= self.img.size ()
        print winSize, imgSize
        hZoom= 1.0*winSize.width  ()/imgSize.width  ()
        vZoom= 1.0*winSize.height ()/imgSize.height ()
        zoomLevel= min (hZoom, vZoom)
        print zoomLevel
        self.zoomTo (zoomLevel)

    def zoom25 (self, *ignore):
        self.zoomTo (0.25)

    def zoom50 (self, *ignore):
        self.zoomTo (0.5)

    def zoom75 (self, *ignore):
        self.zoomTo (0.75)

    def zoom100 (self, *ignore):
        self.zoomTo (1.0)

    def zoomTo (self, zoomLevel):
        scale= zoomLevel/self.zoomLevel
        print "scaling", scale
        self.scale (scale, scale)
        self.zoomLevel= zoomLevel

if __name__=='__main__':
    # this code is enough for loading an image and show it!
    app= QApplication (sys.argv)
    win= QMainWindow ()

    pixmap= QPixmap (sys.argv[1])
    qgpi= QGraphicsPixmapItem (pixmap)
    scene= QGraphicsScene ()
    scene.addItem (qgpi)

    view= OMITGraphicsView (pixmap, scene, win)
    view.setDragMode (QGraphicsView.ScrollHandDrag)
    view.show()

    app.exec_ ()
    # up to here!

# end

Things to note:

  • The code for loading, showing the image and pan support is only 13 lines of Python code, including 3 imports. The resulting app is also able to handle vector graphics, but of course I didn't exploit that, I just added a QPixmap/QGraphicsPixmapItem pair.
  • Zooming is implemented via QGraphicsView.scale(), which is accumulative (scaling twice to 0.5 actually scales to 0.25 of the original size), so I have to keep the zoom level all the time. There should be a zoom() interface!
  • The code for calculating the scale level is not very good: scaling between 75% and 50% or 25% produces scales of 0.666 and 0.333, which I think at the end of the day will accumulate a lot of error.
  • For the same reason, zoomToFit() has to do some magic. I also got hit by the integer division of Python (I was getting zoom factors of 0) so I had to add 1.0* to the claculations. It's good that this is fixed in Python2.6/3.0.
  • The size reported by the QMainWindow was any vegetable (it said 640x480 when it actually was 960x600), so I used the QGraphicsView instead. WTF?
  • For some strange reason zoomToFit() scales the image a little bigger than it should, so a scrollbar appears in the direction of the constraining dimension.
  • Less that 100 lines! Even if setupActions() could surely be improved.
  • In Marcelo's favor I should mention that he writes docstrings for most of his methods both in english and spanish (yes, of course I read his code after I finished mine). I barely put a couple of comments, but doing the same should add 10 more lines, tops. Also, I don't want to convert this into a who-has-it-smaller contest (the code, I mean :).
  • It took me approx 3 hours, with no previous knowledge of how to do it and no internet connection, so no asking around. I just used the «Qt Reference Documentation», going to the «Gropued Classes» page and to the «Graphics View Classes» from there.
  • It doesn't zoom with the mouse wheel either.
  • The default colors of ikiwiki's format plugin are at most sucky, but better than nothing.

omit pykde python


[1] Unluckly he didn't declared which license it has, so I'm not sure if I really can do this. I GPL'ed mine.

Posted Wed 27 Jan 2010 11:55:54 PM CET Tags:
Posted Wed 27 Jan 2010 11:55:55 PM CET
05
Posted Wed 27 Jan 2010 11:55:55 PM CET
09
Posted Wed 27 Jan 2010 11:55:55 PM CET

Después de la debacle de la semana pasada (de la que aún quedan algunas secuelas que ya comentaré) y teniendo ya planeada una upgradeada del desrver de Sarge a Etch (cosa que venía haciendo en un Xen hasta que me fui de vacaciones), decidimos hacerlo de una.

Empezamos en otra máquina instalando un Etch prístino. Esto implicó (re)instalar todo el soft que ya estaba corriendo en el servidor (que aún seguía corriendo). Lo único que no instalamos de nuevo aún es el zope/plone; tal vez lo hagamos directamente con lo que llaman un buildout.

Una desición personal, sobre todo basada en que en este entorno casi todos somos root, fue instalar algo que mantuviera en un rcs el contenido de /etc. Hace un par de años me senté a intentar un wrapper para svn que guardara la metadata como properties de los archivos llamado sylvan. El sistema era mas o menos usable, pero por suerte el genio de Joey Hess le ocurrió el mismo problema y salió con etckeeper. ectkeeper no está en Etch, por lo que instalé ese paquete y metastore bajándolos del repo de Sid y mercurial del repo de backports. Como no estoy seguro que el nnotito de aptitude sepa manejar este tipo de repos [para el caso creo que dselect tampoco] no puse backports entre los deblines.

El siguiente paso fue mergear la configuración actual del server con lo que me dejó esta instalación. Claramente no era cuestión de tirar el /etc viejo encima del otro y que se hagan agua los helados. Para ello usé xxdiff (o también podría haber usado meld o diff3 para emacs), pudiendo seleccionar qué pedazos quería específicamente. Un poco de edición y estábamos casi listos.

El penúltimo paso fue popular el nuevo LDAP. pare ello alcanzó un slapcat -l en el server; borrar los registros que ya están en el server nuevo (el root del directorio y el admin); luego un slapadd -l en el nuevo; y luego probar con un par de ldapsearchs. Tambien copiar los certificados y cambiarles el owner a openldap.

Luego tocó hacer andar PAM y nss contra dicho LDAP. La configuración estaba "as is", incluyendo los certificados. Luego de algo así como una hora encontré que la parte de TLS no estaba andando, así que hube de desactivarla, para lo cual śolo hubo que comentar el ssl start_tls y ya. Mas adelante veré cómo reactivarla.

Ya listos (cerca de las 12 de la noche) apagué el server, instalé el disco y a bootear. Y acá es cuando comienza el verdadero baile.

Por suerte el mail no me trajo verdaderos problemas. Sólo tuve que ser muy cuidadoso, pues nosotros bajamos losmails de nuestra verdadero servidor por fetchmail. Por algún motivo se me escapó en el merge de la configuración del postfix la parte que dice que use MailDir en el home del usuario, por lo que estuve renegando otro tanto. ssh también fallaba, pero era porque se me había escapado un PasswordAuthentication no.

El último paso de la noche era (re)configurar el Apache para que anduvieran los repos svn vía DAV que autenticaban contra el LDAP. Al día siguiente vendría la parte de reactivar otros servicios del Apache. No hubo que renegar mucho, sólo cambiaron la forma en que le decía que autentique contra LDAP y agregarle una z a AuthzLDAPAuthoritative:

# <span class="createlink">AuthLDAPEnabled</span> On
<span class="createlink">AuthBasicProvider</span> ldap
<span class="createlink">AuthzLDAPAuthoritative</span> on

13 horas después de iniciar mi día laboral (3 de la matina) me retiré tambaleando a mi casa.

sysadmin ldap debian sarge etch pam apache svn

Posted Wed 27 Jan 2010 11:55:54 PM CET Tags:

El segundo día me amaneció a las 13. Hoy tocaba terminar con el Apache, que incuía apenas los tracs y el dotproject. Ambos implicaban upgrades.

Los trac no me hicieron renegar mucho, pues está muy bien documentado y hasta fue scripteable. Básicamente era un salto de sqlite2 a sqlite3, un trac-admin ... upgrade seguido de un trac-admin ... resync.

Con lo único con lo que renegué fue que a pesar del upgrade fue exitoso no podía entrar. En los logs encontraba esto:

(9)Bad file descriptor: Could not open password file: (null)

Google al rescate me dijo que había que apagar esa directiva que había tenido que modificar el día anterior:

<span class="createlink">AuthzLDAPAuthoritative</span> off

También me salió esto:

Failed to load the <span class="createlink">AuthzSVNAccessFile</span>: The character 'o' in rule '@except' is not allowed in authz rules

Eso era porque en un archivo de configuración del repo (conf.svnaccess) tenía los permisos de sólo lectura como ro en vez de r.

El dotproject me enfrentó a un viejo archienemigo: mysql. La verdad que no se a queinacarajos se le ocurre que es una excelente idea poner la configuración de acceso y permisos de una base de datos dentro de la base misma. Por un lado eso termina siendo un archivo binario no versionable y por otro obliga al sysadmin a aprender SQL (cosa que sé, pero no manejo fluidamente ni me interesa saberlo; otro de los motivos por los que amo los ORM's). Y además esta configuración termina en /var y no en etc. postgresql, en cambio, es mucho más inteligente. Y viva el SQL independiente del motor. Lástima nadie lo usa...

Bien, sólo tuve que hacer un dump del mysql anterior (chroot mediante), crear la base en el nuevo y hacer un load. Fantástico. Luego una lucha trabado con el sistema de permisos antesmencionado. Luego apuntar un browser a https://server/dotproject/install. En ese minisitio tuve primero que configurarlo (como DP no es un paquete en Debian, lo instalé de fuentes; la configuración queda en un archivo en include/config.php; ojo que las otras opciones es nukear las bases), luego volver a entrar a dicha URL, momento en el cual detecta las bases viejas y da la opción de upgradearlas. Anduvo sin problemas y ahora disfrutamos de un DP más nuevo. Yeepee!

sysadmin debian sarge etch apache trac svn mysql dotproject

Posted Wed 27 Jan 2010 11:55:55 PM CET
04
Posted Wed 27 Jan 2010 11:55:55 PM CET

Después de la debacle de la semana pasada (de la que aún quedan algunas secuelas que ya comentaré) y teniendo ya planeada una upgradeada del desrver de Sarge a Etch (cosa que venía haciendo en un Xen hasta que me fui de vacaciones), decidimos hacerlo de una.

Empezamos en otra máquina instalando un Etch prístino. Esto implicó (re)instalar todo el soft que ya estaba corriendo en el servidor (que aún seguía corriendo). Lo único que no instalamos de nuevo aún es el zope/plone; tal vez lo hagamos directamente con lo que llaman un buildout.

Una desición personal, sobre todo basada en que en este entorno casi todos somos root, fue instalar algo que mantuviera en un rcs el contenido de /etc. Hace un par de años me senté a intentar un wrapper para svn que guardara la metadata como properties de los archivos llamado sylvan. El sistema era mas o menos usable, pero por suerte el genio de Joey Hess le ocurrió el mismo problema y salió con etckeeper. ectkeeper no está en Etch, por lo que instalé ese paquete y metastore bajándolos del repo de Sid y mercurial del repo de backports. Como no estoy seguro que el nnotito de aptitude sepa manejar este tipo de repos [para el caso creo que dselect tampoco] no puse backports entre los deblines.

El siguiente paso fue mergear la configuración actual del server con lo que me dejó esta instalación. Claramente no era cuestión de tirar el /etc viejo encima del otro y que se hagan agua los helados. Para ello usé xxdiff (o también podría haber usado meld o diff3 para emacs), pudiendo seleccionar qué pedazos quería específicamente. Un poco de edición y estábamos casi listos.

El penúltimo paso fue popular el nuevo LDAP. pare ello alcanzó un slapcat -l en el server; borrar los registros que ya están en el server nuevo (el root del directorio y el admin); luego un slapadd -l en el nuevo; y luego probar con un par de ldapsearchs. Tambien copiar los certificados y cambiarles el owner a openldap.

Luego tocó hacer andar PAM y nss contra dicho LDAP. La configuración estaba "as is", incluyendo los certificados. Luego de algo así como una hora encontré que la parte de TLS no estaba andando, así que hube de desactivarla, para lo cual śolo hubo que comentar el ssl start_tls y ya. Mas adelante veré cómo reactivarla.

Ya listos (cerca de las 12 de la noche) apagué el server, instalé el disco y a bootear. Y acá es cuando comienza el verdadero baile.

Por suerte el mail no me trajo verdaderos problemas. Sólo tuve que ser muy cuidadoso, pues nosotros bajamos losmails de nuestra verdadero servidor por fetchmail. Por algún motivo se me escapó en el merge de la configuración del postfix la parte que dice que use MailDir en el home del usuario, por lo que estuve renegando otro tanto. ssh también fallaba, pero era porque se me había escapado un PasswordAuthentication no.

El último paso de la noche era (re)configurar el Apache para que anduvieran los repos svn vía DAV que autenticaban contra el LDAP. Al día siguiente vendría la parte de reactivar otros servicios del Apache. No hubo que renegar mucho, sólo cambiaron la forma en que le decía que autentique contra LDAP y agregarle una z a AuthzLDAPAuthoritative:

# <span class="createlink">AuthLDAPEnabled</span> On
<span class="createlink">AuthBasicProvider</span> ldap
<span class="createlink">AuthzLDAPAuthoritative</span> on

13 horas después de iniciar mi día laboral (3 de la matina) me retiré tambaleando a mi casa.

sysadmin ldap debian sarge etch pam apache svn

Posted Wed 27 Jan 2010 11:55:54 PM CET Tags:

El segundo día me amaneció a las 13. Hoy tocaba terminar con el Apache, que incuía apenas los tracs y el dotproject. Ambos implicaban upgrades.

Los trac no me hicieron renegar mucho, pues está muy bien documentado y hasta fue scripteable. Básicamente era un salto de sqlite2 a sqlite3, un trac-admin ... upgrade seguido de un trac-admin ... resync.

Con lo único con lo que renegué fue que a pesar del upgrade fue exitoso no podía entrar. En los logs encontraba esto:

(9)Bad file descriptor: Could not open password file: (null)

Google al rescate me dijo que había que apagar esa directiva que había tenido que modificar el día anterior:

<span class="createlink">AuthzLDAPAuthoritative</span> off

También me salió esto:

Failed to load the <span class="createlink">AuthzSVNAccessFile</span>: The character 'o' in rule '@except' is not allowed in authz rules

Eso era porque en un archivo de configuración del repo (conf.svnaccess) tenía los permisos de sólo lectura como ro en vez de r.

El dotproject me enfrentó a un viejo archienemigo: mysql. La verdad que no se a queinacarajos se le ocurre que es una excelente idea poner la configuración de acceso y permisos de una base de datos dentro de la base misma. Por un lado eso termina siendo un archivo binario no versionable y por otro obliga al sysadmin a aprender SQL (cosa que sé, pero no manejo fluidamente ni me interesa saberlo; otro de los motivos por los que amo los ORM's). Y además esta configuración termina en /var y no en etc. postgresql, en cambio, es mucho más inteligente. Y viva el SQL independiente del motor. Lástima nadie lo usa...

Bien, sólo tuve que hacer un dump del mysql anterior (chroot mediante), crear la base en el nuevo y hacer un load. Fantástico. Luego una lucha trabado con el sistema de permisos antesmencionado. Luego apuntar un browser a https://server/dotproject/install. En ese minisitio tuve primero que configurarlo (como DP no es un paquete en Debian, lo instalé de fuentes; la configuración queda en un archivo en include/config.php; ojo que las otras opciones es nukear las bases), luego volver a entrar a dicha URL, momento en el cual detecta las bases viejas y da la opción de upgradearlas. Anduvo sin problemas y ahora disfrutamos de un DP más nuevo. Yeepee!

sysadmin debian sarge etch apache trac svn mysql dotproject

Posted Wed 27 Jan 2010 11:55:55 PM CET
08
Posted Wed 27 Jan 2010 11:55:55 PM CET
Posted Wed 06 Apr 2011 08:29:31 PM CEST
12

Ever since it came out in November 2009 I wanted a Nokia N900. Many reasons made me buy one, some reasons made me wait until this month. Maemo, it's OS, is almost exactly what I wanted, a real GNU/busybox/Linux. I can do a lot of stuff with it because it's, I think, the closest to a mini computer with an integrated GPS, camera, cellular network modem, WiFi and can act both as an USB master or a slave. I don't know about the N950 or the N9, but really this is a tinkerer's dream.

So far I haven't done much with it, most of my personal time is being invested in something else, but I've been using it mainly as an in-car computer and a WiFi-to-LAN router. The latter will be the subject to another post, but I want to concentrate on the former in this post.

As an in-car computer right now I want two things: navigation and audio player. The audio player is done; so far the media player that comes in the... phone? computer? I'll say computer. So far the media player that comes with the computer is enough for me. Maybe sometime I'll do a satyr skin for Maemo and port the beast to it, just for the heck of it.

Also it has the OVI maps, the default app for navigation, which from what I have heard, is very good, but it needs a data connection to work. I don't want to buy a data plan yet, and also there are places where there is no good data coverage. You might not even get enough signal to make a call or even send an SMS. So I want something that can cache maps locally in the computer.

The answer is actually somewhat complex: marble, monav and OpenStreetMap. Marble is the KDE map viewing application. It can show several types of maps, even from several celestial bodies. It can talk to a GPS to record and save tracks, and to monav, which is a routing application that uses OSM data. Of course, Marble can use OSM's tiles. And the good thing about both is that they can use data cached on the computer, which means no data link needed, at least not all the time.

Installing them in Maemo is as simple as selecting them in the App manager (although I suggest to install and use Faster Application Manager, which is no so far from a mix between a smartphone store and, let's say, aptitude). Both are in the extras repo.

Once they're installed, the next logical step is to donwload monav's data. For that you open marble; from its menu you select «Routing», then «Configure». There you can select Monav, and «Configure». There you can download routing maps per country. Download all the ones you want, but remember it takes space and some time to uncompress.

As for the maps themselves, you can select the tileset you want (so far the common Mapnik and OSMaRenderer are available). Marble will start downloading and caching tiles from OSM's servers. You can even use «Download Region...» to download the tiles for a certain region or route in an interval of zooms. Becareful about this, because OSM's servers will ban you for a little while if you pull too much info.

So now I have an almost complete offline map and routing solution. I still have to investigate how to add voices (so far it's mute). I also found a couple of problems with this setup. The most "simple" to solve are UI ones: the navigation instructions have too much text and too small icons, and while having the navigation widget on, you lose the zoom controls. Luckily my dock leaves the volume/zoom hardware buttons free, so I can zoom with them (who needs multitouch :).

But the most prominent one is the map itself. Both Mapnik and OSMaRenderer are too confusing for in-car navigation, they have too much detail, and minor streets are not clearly distinguished from the blocks themselves. Mapnik in particular has a very low contrast, which also makes it hard to read in plain daylight.

Enter CloudMade. I still don't get much what CloudMade is exactly for, but one thing I know: it's very easy to create tilesets from OSM data. First I browsed the styles already in the site and I found one called «Almost Ordinance Survey 25k». I took it and modified it heavily. I named it «daylight in-car navigation».

Then I found out that it wouldn't be easy to use the tiles with marble. At least I got to find the right settings. Next chapter will be about using OSMaRenderer locally.

openstreetmap marble maemo

Posted Fri 30 Dec 2011 08:50:49 PM CET Tags:
Posted Fri 30 Dec 2011 08:54:26 PM CET

Yesterday[1] I wrote: «Then I found out that it wouldn't be easy to use the tiles [generated by CloudMade] with marble». How wrong was I.

Here you can read how to access the tiles in CM's servers. The only thing you need is an API key, which you can get once you registered (it practically begs you to get one :). Another piece of the puzzle is marble itself. I was specting to copy OSM's service description file and bend it and twist it until it worked with CM's tiles. But things are, luckily, much more simple.

In marble you simply do «File», «Create a new map...», then select «Online map providing indexed tiles», which clearly describes CM. Next, you have to provide and URL template, which you can simply derive from the doc in CM's site. Mine's like this:

http://tile.cloudmade.com/5f806ad32bb44b38a464020fa2223193/51084/256/{zoomLevel}/{x}/{y}.png

You, of course, will have to change the API key and the style number for yours. One thing I found is that the @2x doesn't seem to work with marble; without it it works just fine. Maybe it's a User-Agent thing. Then you keep answering what marble asks you in the wizard. By the end of it, you'll have marble working with your tiles! It's amazing.

One extra hack was to add support for hillshading. These are tiles that are alpha-blended with OSM's, wich gives you the impression of viewing a 3D map (bah, at least in the covered regions with mountains). The service files (.dgml files) are actually XML files quite easy to edit if you defocus and just look at the big picture ot if. So I just simply opened both files (OSM's is in /usr/share/kde4/apps/marble/data/maps/earth/openstreetmap/openstreetmap.dgml, the one you created in $HOME/.local/share/marble/maps/earth/foo/foo.dgml), copied the texture tag that describes hillshading, pasted it in my .dgml file, saved, reopened marble and... voilà! Even better, as hillshading is actually another tile server, marble can share it for OSM's, OSMaRenderer and yours.

The final result is simply amazing. You can see a screenshot here.

More on my use of marble and OSM pretty soon.


[1] When I say "yesterday", I actually mean "almost 7 months ago". That's what took me to fix my glob and come back to writing, finishing this post.


openstreetmap marble

Posted Wed 25 Jul 2012 03:38:31 PM CEST Tags:

Ever since it came out in November 2009 I wanted a Nokia N900. Many reasons made me buy one, some reasons made me wait until this month. Maemo, it's OS, is almost exactly what I wanted, a real GNU/busybox/Linux. I can do a lot of stuff with it because it's, I think, the closest to a mini computer with an integrated GPS, camera, cellular network modem, WiFi and can act both as an USB master or a slave. I don't know about the N950 or the N9, but really this is a tinkerer's dream.

So far I haven't done much with it, most of my personal time is being invested in something else, but I've been using it mainly as an in-car computer and a WiFi-to-LAN router. The latter will be the subject to another post, but I want to concentrate on the former in this post.

As an in-car computer right now I want two things: navigation and audio player. The audio player is done; so far the media player that comes in the... phone? computer? I'll say computer. So far the media player that comes with the computer is enough for me. Maybe sometime I'll do a satyr skin for Maemo and port the beast to it, just for the heck of it.

Also it has the OVI maps, the default app for navigation, which from what I have heard, is very good, but it needs a data connection to work. I don't want to buy a data plan yet, and also there are places where there is no good data coverage. You might not even get enough signal to make a call or even send an SMS. So I want something that can cache maps locally in the computer.

The answer is actually somewhat complex: marble, monav and OpenStreetMap. Marble is the KDE map viewing application. It can show several types of maps, even from several celestial bodies. It can talk to a GPS to record and save tracks, and to monav, which is a routing application that uses OSM data. Of course, Marble can use OSM's tiles. And the good thing about both is that they can use data cached on the computer, which means no data link needed, at least not all the time.

Installing them in Maemo is as simple as selecting them in the App manager (although I suggest to install and use Faster Application Manager, which is no so far from a mix between a smartphone store and, let's say, aptitude). Both are in the extras repo.

Once they're installed, the next logical step is to donwload monav's data. For that you open marble; from its menu you select «Routing», then «Configure». There you can select Monav, and «Configure». There you can download routing maps per country. Download all the ones you want, but remember it takes space and some time to uncompress.

As for the maps themselves, you can select the tileset you want (so far the common Mapnik and OSMaRenderer are available). Marble will start downloading and caching tiles from OSM's servers. You can even use «Download Region...» to download the tiles for a certain region or route in an interval of zooms. Becareful about this, because OSM's servers will ban you for a little while if you pull too much info.

So now I have an almost complete offline map and routing solution. I still have to investigate how to add voices (so far it's mute). I also found a couple of problems with this setup. The most "simple" to solve are UI ones: the navigation instructions have too much text and too small icons, and while having the navigation widget on, you lose the zoom controls. Luckily my dock leaves the volume/zoom hardware buttons free, so I can zoom with them (who needs multitouch :).

But the most prominent one is the map itself. Both Mapnik and OSMaRenderer are too confusing for in-car navigation, they have too much detail, and minor streets are not clearly distinguished from the blocks themselves. Mapnik in particular has a very low contrast, which also makes it hard to read in plain daylight.

Enter CloudMade. I still don't get much what CloudMade is exactly for, but one thing I know: it's very easy to create tilesets from OSM data. First I browsed the styles already in the site and I found one called «Almost Ordinance Survey 25k». I took it and modified it heavily. I named it «daylight in-car navigation».

Then I found out that it wouldn't be easy to use the tiles with marble. At least I got to find the right settings. Next chapter will be about using OSMaRenderer locally.

openstreetmap marble maemo

Posted Fri 30 Dec 2011 08:50:49 PM CET Tags:
Posted Fri 30 Dec 2011 08:54:26 PM CET

Ever since it came out in November 2009 I wanted a Nokia N900. Many reasons made me buy one, some reasons made me wait until this month. Maemo, it's OS, is almost exactly what I wanted, a real GNU/busybox/Linux. I can do a lot of stuff with it because it's, I think, the closest to a mini computer with an integrated GPS, camera, cellular network modem, WiFi and can act both as an USB master or a slave. I don't know about the N950 or the N9, but really this is a tinkerer's dream.

So far I haven't done much with it, most of my personal time is being invested in something else, but I've been using it mainly as an in-car computer and a WiFi-to-LAN router. The latter will be the subject to another post, but I want to concentrate on the former in this post.

As an in-car computer right now I want two things: navigation and audio player. The audio player is done; so far the media player that comes in the... phone? computer? I'll say computer. So far the media player that comes with the computer is enough for me. Maybe sometime I'll do a satyr skin for Maemo and port the beast to it, just for the heck of it.

Also it has the OVI maps, the default app for navigation, which from what I have heard, is very good, but it needs a data connection to work. I don't want to buy a data plan yet, and also there are places where there is no good data coverage. You might not even get enough signal to make a call or even send an SMS. So I want something that can cache maps locally in the computer.

The answer is actually somewhat complex: marble, monav and OpenStreetMap. Marble is the KDE map viewing application. It can show several types of maps, even from several celestial bodies. It can talk to a GPS to record and save tracks, and to monav, which is a routing application that uses OSM data. Of course, Marble can use OSM's tiles. And the good thing about both is that they can use data cached on the computer, which means no data link needed, at least not all the time.

Installing them in Maemo is as simple as selecting them in the App manager (although I suggest to install and use Faster Application Manager, which is no so far from a mix between a smartphone store and, let's say, aptitude). Both are in the extras repo.

Once they're installed, the next logical step is to donwload monav's data. For that you open marble; from its menu you select «Routing», then «Configure». There you can select Monav, and «Configure». There you can download routing maps per country. Download all the ones you want, but remember it takes space and some time to uncompress.

As for the maps themselves, you can select the tileset you want (so far the common Mapnik and OSMaRenderer are available). Marble will start downloading and caching tiles from OSM's servers. You can even use «Download Region...» to download the tiles for a certain region or route in an interval of zooms. Becareful about this, because OSM's servers will ban you for a little while if you pull too much info.

So now I have an almost complete offline map and routing solution. I still have to investigate how to add voices (so far it's mute). I also found a couple of problems with this setup. The most "simple" to solve are UI ones: the navigation instructions have too much text and too small icons, and while having the navigation widget on, you lose the zoom controls. Luckily my dock leaves the volume/zoom hardware buttons free, so I can zoom with them (who needs multitouch :).

But the most prominent one is the map itself. Both Mapnik and OSMaRenderer are too confusing for in-car navigation, they have too much detail, and minor streets are not clearly distinguished from the blocks themselves. Mapnik in particular has a very low contrast, which also makes it hard to read in plain daylight.

Enter CloudMade. I still don't get much what CloudMade is exactly for, but one thing I know: it's very easy to create tilesets from OSM data. First I browsed the styles already in the site and I found one called «Almost Ordinance Survey 25k». I took it and modified it heavily. I named it «daylight in-car navigation».

Then I found out that it wouldn't be easy to use the tiles with marble. At least I got to find the right settings. Next chapter will be about using OSMaRenderer locally.

openstreetmap marble maemo

Posted Fri 30 Dec 2011 08:50:49 PM CET Tags:
Posted Thu 01 Dec 2011 09:49:09 PM CET
Posted Wed 06 Apr 2011 08:29:31 PM CEST
03

Last year and a half I was working in research. This position was about, among other things, to port a programming language (two, actually, Hop and Bigloo) to the Antroid platform. I already had written something about it, but this time I want to show my high level impressions of the platform. What follows is part of a report I wrote at the end of that job, which includes the things I wanted to say.

The Android port can be viewed as four separate sub-tasks. Hop is software developed in the Scheme language; more particularly, it must be compiled with the Bigloo Scheme compiler, which in turn uses gcc for the final compilation. That means that we also needed to port Bigloo first to the platform, not because we were planning to use it in the platform, but because we need the Bigloo runtime libraries ported to Android, as Hop and any other program compiled with Bigloo uses them. The other three subtasks, which are discussed later, are porting Hop itself; developing libraries to access devices and other features present in the platform; and, we'll see later the reasons, make the port work with threads.

When we started to investigate how to port native code to the platform we found that there wasn't much support. At fisrt the only documentation we could find was blog posts of people trying to do it by hand. They were using the compiler provided in Android's source code to compile static binaries that could be run on the platform. Because Bigloo uses dinamic libraries to implement platform dependent code and modules, we aimed to find a way to compile things dinamically. After 3 or 4 weeks we found a wrapper written in Ruby that managed all the details of calling gcc with the proper arguments. With this we should be able to port anything that uses gcc as the compiler, just like Bigloo does. At the same time, the first version of Android's NDK (Native Development Kit) appeared, but it wasn't easy to integrate in our build system.

(Note: Actually I think most of the problems we faced doing this port stem from this. The NDK forces you to write a new set of Makefiles, but our hand-made build system and build hierarchy made such an effort quite big. Also, that mean supporting a parallel build system, while it should not be so crazy to spect a cleaner way to integrate the toolchain into an existing build system, not only in hand-made like in this case, but also the most common ones, like autotools, cmake, etc.)

Even having the proper compiler, we found several obstacles related to the platform itself. First of all, Bigloo relies heavily on the C and pthread libraries to implement lowlevel functionality. Bigloo can use both glibc, GNU's implementation, or µlibc, an implementation aimed for embedded aplications. Bigloo also relies on Boehm's Garbage Collector (GC) for its memory management. The C library implementation in Android is not the glibc or the µlibc, but an implementation developed by Google for the platform, called Bionic. This version of the C library is tailored to the platform's need, with little to no regards to native application development.

The first problem we found is that GC compiled fine with Bionic, but the apllications that used GC did not link: there was a missing symbol that normally is defined in the libc, but that Bionic did not. We tried cooperating with the GC developers, we tried inspecting a Mono port to Android, given that this project also uses GC, trying to find a solution that could be useful for everyone, but at the end we just patched our sources to fake such symbol with a value that remotely made sense.

We also found that Bionic's implementation of pthreads is not only incomplete, but also has some glitches. For instance, in our build system, we test the existence of a function like everybody else: we compile a small test program wich uses it. With this method we found at least one function that is declared but never defined. That means that Bionic declares that the function exists, but then it never implements it. Another example is the declaration and definition of a function, but the lack of definition of constants normally used when calling this function.

Also, because most of the tests also must be executed to inform about the peculiarities of each implementation, we had to change our build system to be able to execute the produced binaries in the Android emulator.

Google also decided to implement their own set of tools, again, trimmed down to the needs of the platform, instead of using old and proven versions, like Busybox. This means that some tools behave differently, with no documentation about it, so we mostly had to work around this differences everytime a new one apperared.

All in all, we spent two and a half months just getting Bigloo to run in Android, dismissing the problem that Boehm's GC, using its own build system, detected that the compiler declared to not support threads, and refused to compile with threads enabled. This meant that Bigloo itself could not be compiled with pthreads support.

With this caveat in mind, we tackled the second subtask, porting Hop itself. This still raised problems with the peculiarities of the platform. We quickly found that the dinamic linker wasn't honoring the LD_LIBRARY_PATH environment variable, which we were trying to use to tell the system where to find the dynamic libraries.

The Android platform installs new software using a package manager. The package manager creates a directory in the SD card that it's only writable by the applilcation being installed. Within this directory the installer puts the libraries declared in the package. Bigloo, besides the dinamic libraries, requieres some aditional files that initialize the global objects. This files are not extracted by the installer, so we had to make a frontend in Java that opens the package and extract them by hand. But the installer creates the directory for the libraries in such a way that the application later cannot write in it.

Also, we found that the dinamic linker works for libraries linked at runtime, but does not for dlopen()'ing them, so we also had to rewrite a great part of our build system for both Bigloo and Hop to produce static libraries and binaries. This also needed disabling the dynamic loading of libraries, and with them, their initialization, so we had to initialize them by hand.

To add more unsuspected work, the Android package builder, provided with the SDK, ignores hidden files, which Bigloo uses to map Scheme module names to dynamic libraries. We had to work around this feature in the unpacking algorithm.

Then we moved to improve the friendliness of the frontend. So far, we could install Hop in the platform, either in a phone or in the emulator, but we could only run it in the emulator, because we were using a shell that runs as root on the emulator, but that runs as a user in a real device. This user, for the reasons given above, cannot even get into Hop's install dir. Even when Android has a component interface that allows applications to use components from other apps, none of the terminal apps we found at that time declared the terminal itself as a reusable component. We decided to use the code from the most popular one, which was based on a demo available on Android's source code, but not installed in actual devices. We had to copy the source code and trimm it down to our needs.

Having a more or less usable Hop package for Android, we decided to try and fix the issue we mentioned before: GC didn't compile with threads enabled. This means that we can't use the pthreads library, which is very useful for Hop. Hop uses threads to attend several requests at the same time. Bigloo implements two threads APIs, one based on pthreads and another which implements fair threads. Hop is able to use 5 different request schedulers, but works better with the one based on pthreads.

For these reasons we decided to focus in getting GC to use threads with the Android platform. GC's build system tests the existence of a threading platform checking the thread model declared by gcc. The gcc version provided with Android's SDK declares to have a 'single thread model', but we couldn't find what does this mean in terms of the code produced by gcc or how this could affect to GC's execution.

(Note: we didn't manage to make GC compile with threads.)

With a threadless Hop running, we had to add code to the server so we could talk between the server and the frontend while at the same time it is attending the requests from a web client. After several attempts to attack this problem, we decided that the best solution was to make this interface another service served by Hop. This meant less modifications to Hop itself, but a bigger one to the frontend we already had.

During these changes we found out a problem with JNI. The terminal component we imported into our code uses a small C library for executing the application inside (normaly a shell, in the original code, but Hop in our case) which is accessed from Java using JNI. The original Term application exported this class as com.android.term.Exec, but our copy exported it as fr.inria.hop.Exec. Even with this namespace difference JNI got confused and tried to use the Exec class from the original Term app. This is just another example how the platform is hard to work with. We found that the community support is more centered around Java and that very few people know about JNI, the NDK or any other native related technologies. We couldn't find an answer to this problem, so we worked around this by renaming the class.

So that's it. I can provide all the technical details for most the assertions I postulated above, but that would make this post unreadbal for its length. If you have any question about them, just conact me.

android

Posted Sat 26 Mar 2011 12:14:11 AM CET Tags:
Posted Wed 06 Apr 2011 08:29:31 PM CEST
05

This wiki has OpenID enabled.

OpenID is a decentralized authentication mechanism that allows you to have one login that you can use on a growing number of websites.

If you have an account with some of the larger web service providers, you might already have an OpenID. Directory of OpenID providers

To sign in to this wiki using OpenID, just enter it in the OpenID field in the signin form. You do not need to give this wiki a password or go through any registration process when using OpenID.


It's also possible to make a page in the wiki usable as an OpenID url, by delegating it to an openid server. Here's an example of how to do that:

[[!meta  openid="http://yourid.myopenid.com/"
       server="http://www.myopenid.com/server"]]
Posted Sat 08 May 2010 04:30:42 AM CEST

Use this template to insert a note into a page. The note will be styled to float to the right of other text on the page. This template has one parameter:

  • `text` - the text to display in the note

Posted Thu 06 May 2010 04:47:37 AM CEST

Use this template to create a popup window that is displayed when the mouse is over part of the page. This template has two parameters:

  • `mouseover` - This is the text or other content that triggers the popup.
  • `popup` - This should be the content of the popup window. It can be anything, even images or a whole little wiki page, but should not be too large for good usability.

Note that browsers that do not support the CSS will display the popup inline in the page, inside square brackets. []

Posted Thu 06 May 2010 04:47:37 AM CEST
Posted Wed 06 Apr 2011 08:29:31 PM CEST
05

Directives are similar to a WikiLink in form, except they begin with ! and may contain parameters. The general form is:

[[!directive  param="value" param="value"]]

This gets expanded before the rest of the page is processed, and can be used to transform the page in various ways.

The quotes around values can be omitted if the value is a simple word. Also, some directives may use parameters without values, for example:

[[!tag  foo]]

A directive does not need to all be on one line, it can be wrapped to multiple lines if you like:

[[!directive  foo="baldersnatch"
bar="supercalifragilisticexpialidocious" baz=11]]

Also, multiple lines of quoted text can be used for a value. To allow quote marks inside the quoted text, delimit the block of text with triple-double-quotes or triple-single-quotes:

[[!directive  text="""
1. "foo"
2. "bar"
3. "baz"
""" othertext='''
1. 'quux'
2. "foo"
''']]

If you want to put text with triple quotes into a parameter value, you can use perl-style here-doc syntax, even nesting it like this:

[[!directive  text=<<OUTER
[[!otherdirective <<INNER
inner text
INNER]]
outer text
OUTER]]

ikiwiki also has an older syntax for directives, which requires a space in directives to distinguish them from wikilinks. This syntax has several disadvantages: it requires a space after directives with no parameters (such as pagecount ), and it prohibits spaces in wikilinks. ikiwiki now provides the !-prefixed syntax shown above as default. However, ikiwiki still supports wikis using the older syntax, if the prefix_directives option is disabled.

Posted Fri 13 May 2011 06:06:00 PM CEST
Posted Wed 06 Apr 2011 08:29:31 PM CEST
11

On November 26, the Qt/KDE Debian team (qkd from now on) released an experimental packaging of KDE SC 4.7. This means a substantial and very much awaited upgrade from the actual version available in sid, 4.6.5 . From what I've been told, it has taken so long because there have been a lot of changes in source code structure (KDE has recently restructured its source modularization) and in packaging itself, and even with recent aditions to the group, the resources in terms of developers is still low. In any case, thanks to them for getting this version out, I really appreciate your hard work.

But because of this lack of resources, they only got out packages for the amd64 architecture, and I'm running i386 at home, so I decided to try and compile it by myself. This is the log of the attempt.

First, I added deb-src http://qt-kde.debian.net/debian experimental-snapshots main to my sources.list and got the code for kde4libs, the package I think is the root of the whole repo[1][2]:

mdione@mustang:~/src/system/debian$ apt-get source kde4libs
Reading package lists... Done
Building dependency tree
Reading state information... Done
NOTICE: 'kde4libs' packaging is maintained in the 'Git' version control system at:
git://git.debian.org/pkg-kde/kde-sc/kde4libs.git
Need to get 12.4 MB of source archives.
Get:1 http://qt-kde.debian.net/debian/ experimental-snapshots/main kde4libs 4:4.7.2-0r3 (dsc) [4,883 B]
Get:2 http://qt-kde.debian.net/debian/ experimental-snapshots/main kde4libs 4:4.7.2-0r3 (tar) [12.1 MB]
Get:3 http://qt-kde.debian.net/debian/ experimental-snapshots/main kde4libs 4:4.7.2-0r3 (diff) [334 kB]
Fetched 12.4 MB in 1min 54s (109 kB/s)
gpgv: Signature made Sat 22 Oct 2011 02:29:45 AM CEST using RSA key ID 73A85F31
gpgv: Can't check signature: public key not found
dpkg-source: warning: failed to verify signature on ./kde4libs_4.7.2-0r3.dsc
dpkg-source: info: extracting kde4libs in kde4libs-4.7.2
dpkg-source: info: unpacking kde4libs_4.7.2.orig.tar.bz2
dpkg-source: info: unpacking kde4libs_4.7.2-0r3.debian.tar.gz
dpkg-source: info: applying kconf_update_migrate_from_kde3_icon_theme.diff
dpkg-source: info: applying add_debian_build_type.diff
dpkg-source: info: applying disable_usr_lib_install_rpath.diff
dpkg-source: info: applying make_libkdeinit4_private.diff
dpkg-source: info: applying default_kde4_xdg_menu_prefix.diff
dpkg-source: info: applying qt4_designer_plugins_path.diff
dpkg-source: info: applying hardcode_ptm_device.diff
dpkg-source: info: applying kfreebsd_support.diff
dpkg-source: info: applying debian_menu.diff
dpkg-source: info: applying findservicebydesktoppath_try_realfilepath.diff
dpkg-source: info: applying findqt4_optional_x11_pthread.diff
dpkg-source: info: applying use_dejavu_as_default_font.diff
dpkg-source: info: applying hack_in_etc_kde4_in_kstandarddirs.diff
dpkg-source: info: applying ld_exclude_libs_qtuitools.diff
dpkg-source: info: applying konsole_kfreebsd_fix.diff
dpkg-source: info: applying hurd_support.diff
dpkg-source: info: applying kfileshare_kdesu_fileshareset.diff
dpkg-source: info: applying relax_plugin_kde_version_check.diff
dpkg-source: info: applying add_dlrestrictions_support.diff
dpkg-source: info: applying findpythonlibrary_layout_deb_on_debian.diff
dpkg-source: info: applying ktar_header_checksum_fix.diff
dpkg-source: info: applying ktar_longlink_length_in_bytes.diff
dpkg-source: info: applying nepomuk_unicode.diff

Nice, now get the build-deps:

mdione@mustang:~/src/system/debian$ sudo apt-get build-dep kde4libs
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Build-Depends dependency for kde4libs cannot be satisfied because candidate version of package shared-desktop-ontologies can't satisfy version requirements

Dang, I have an old shared-desktop-ontologies (v. 0.6.x in sid), and peeking in qkd's repo I find no new version of it. The answer must be in experimental, so I add deb http://ftp.nl.debian.org/debian/ experimental main to my sources.list and run apt-get update. Now it must be just a matter of:

mdione@mustang:~/src/system/debian$ sudo apt-get install -t experimental shared-desktop-ontologies
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
  shared-desktop-ontologies
1 upgraded, 0 newly installed, 0 to remove and 103 not upgraded.
Need to get 129 kB of archives.
After this operation, 4,096 B of additional disk space will be used.
Get:1 http://ftp.nl.debian.org/debian/ experimental/main shared-desktop-ontologies all 0.8.0-1 [129 kB]
Fetched 129 kB in 1s (92.7 kB/s)
Retrieving bug reports... Done
Parsing Found/Fixed information... Done
Reading changelogs... Done
apt-listchanges: Mailing root: apt-listchanges: changelogs for mustang
(Reading database ... 218251 files and directories currently installed.)
Preparing to replace shared-desktop-ontologies 0.6.0-1 (using .../shared-desktop-ontologies_0.8.0-1_all.deb) ...
Unpacking replacement shared-desktop-ontologies ...
Setting up shared-desktop-ontologies (0.8.0-1) ...

And I try again:

mdione@mustang:~/src/system/debian$ sudo apt-get build-dep kde4libs
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  hspell libacl1-dev libattica-dev libattr1-dev libdbusmenu-qt-dev libdlrestrictions-dev libpolkit-qt-1-dev libqca2-dev libstreamanalyzer-dev libstreams-dev
  libutempter-dev pkg-kde-tools
0 upgraded, 12 newly installed, 0 to remove and 8 not upgraded.
Need to get 1,170 kB of archives.
After this operation, 3,551 kB of additional disk space will be used.
Do you want to continue [Y/n]?
[...]

We're set! now to fire the compilation itself. The method I use must not be the best one, but I remember it by heart and works for me :)

mdione@mustang:~/src/system/debian$ cd kde4libs-4.7.2/
mdione@mustang:~/src/system/debian/kde4libs-4.7.2$ nice -n 19 fakeroot debian/rules binary

Notice I use nice to not hog too much my computer and fakeroot. I don't remember why I must use it :| , but I remember I must. Later, asking in #debian-devel I was pointed to dpkg-buildpackage, to which you can give the -j# option, which makes the package to be compiled with # processes in parallel (actually it only works if the compilation is make based, as is in this case), so from now on I'll use that.

The resulting packages are these:

kdelibs5-data_4.7.2-0r3_all.deb
kdelibs5-dbg_4.7.2-0r3_i386.deb
kdelibs5-dev_4.7.2-0r3_i386.deb
kdelibs5-plugins_4.7.2-0r3_i386.deb
kdelibs-bin_4.7.2-0r3_i386.deb
kdoctools_4.7.2-0r3_i386.deb
libkcmutils4_4.7.2-0r3_i386.deb
libkde3support4_4.7.2-0r3_i386.deb
libkdeclarative5_4.7.2-0r3_i386.deb
libkdecore5_4.7.2-0r3_i386.deb
libkdesu5_4.7.2-0r3_i386.deb
libkdeui5_4.7.2-0r3_i386.deb
libkdewebkit5_4.7.2-0r3_i386.deb
libkdnssd4_4.7.2-0r3_i386.deb
libkemoticons4_4.7.2-0r3_i386.deb
libkfile4_4.7.2-0r3_i386.deb
libkhtml5_4.7.2-0r3_i386.deb
libkidletime4_4.7.2-0r3_i386.deb
libkimproxy4_4.7.2-0r3_i386.deb
libkio5_4.7.2-0r3_i386.deb
libkjsapi4_4.7.2-0r3_i386.deb
libkjsembed4_4.7.2-0r3_i386.deb
libkmediaplayer4_4.7.2-0r3_i386.deb
libknewstuff2-4_4.7.2-0r3_i386.deb
libknewstuff3-4_4.7.2-0r3_i386.deb
libknotifyconfig4_4.7.2-0r3_i386.deb
libkntlm4_4.7.2-0r3_i386.deb
libkparts4_4.7.2-0r3_i386.deb
libkprintutils4_4.7.2-0r3_i386.deb
libkpty4_4.7.2-0r3_i386.deb
libkrosscore4_4.7.2-0r3_i386.deb
libkrossui4_4.7.2-0r3_i386.deb
libktexteditor4_4.7.2-0r3_i386.deb
libkunitconversion4_4.7.2-0r3_i386.deb
libkutils4_4.7.2-0r3_i386.deb
libnepomuk4_4.7.2-0r3_i386.deb
libnepomukquery4a_4.7.2-0r3_i386.deb
libnepomukutils4_4.7.2-0r3_i386.deb
libplasma3_4.7.2-0r3_i386.deb
libsolid4_4.7.2-0r3_i386.deb
libthreadweaver4_4.7.2-0r3_i386.deb

Next package is kadebase. Again, we find the missing deps with:

mdione@mustang:~/src/system/debian$ sudo apt-get build-dep kdebase
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Build-Depends dependency for kdebase cannot be satisfied because candidate version of package kde-sc-dev-latest can't satisfy version requirements

The message is simlar to the one about shared-desktop-ontologies, but I have not even the slightest idea which one of the packages above is the one responsible for it, so I'll just install everything, as most probably they're build-deps for most of the remaining packages (it's kde5libs, after all):

mdione@mustang:~/src/system/debian$ sudo dpkg -i *.deb
(Reading database ... 218481 files and directories currently installed.)
[...]

After that I try again, but still the same thing. Poking around I find that a package with that name exists (it's the first time I met him, enchanté), so I get the source in the usual way:

mdione@mustang:~/src/system/debian$ apt-get source meta-kde
Reading package lists... Done
Building dependency tree
Reading state information... Done
Need to get 14.0 kB of source archives.
Get:1 http://qt-kde.debian.net/debian/ experimental-snapshots/main meta-kde 5:71~pre15 (dsc) [2,098 B]
Get:2 http://qt-kde.debian.net/debian/ experimental-snapshots/main meta-kde 5:71~pre15 (tar) [11.9 kB]
Fetched 14.0 kB in 0s (49.3 kB/s)
gpgv: Signature made Mon 21 Nov 2011 09:38:12 PM CET using RSA key ID 73A85F31
gpgv: Can't check signature: public key not found
dpkg-source: warning: failed to verify signature on ./meta-kde_71~pre15.dsc
dpkg-source: info: extracting meta-kde in meta-kde-71~pre15
dpkg-source: info: unpacking meta-kde_71~pre15.tar.gz

And to compile:

mdione@mustang:~/src/system/debian$ ( cd meta-kde-71~pre15 && nice -n 19 dpkg-buildpackage -j3 -b -us -uc )
dpkg-buildpackage: source package meta-kde
dpkg-buildpackage: source version 5:71~pre15
dpkg-buildpackage: source changed by Debian Qt/KDE Maintainers <debian-qt-kde@lists.debian.org>
dpkg-buildpackage: host architecture i386
[...]

That generates several packages, but I just install the one I want:

mdione@mustang:~/src/system/debian$ sudo dpkg -i kde-sc-dev-latest_4.7.2+5.71~pre15_all.deb
Selecting previously unselected package kde-sc-dev-latest.
(Reading database ... 218469 files and directories currently installed.)
Unpacking kde-sc-dev-latest (from kde-sc-dev-latest_4.7.2+5.71~pre15_all.deb) ...
Setting up kde-sc-dev-latest (4:4.7.2+5.71~pre15) ...

Now the build-deps again:

mdione@mustang:~/src/system/debian$ sudo apt-get build-dep kdebase
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  libkatepartinterfaces4
The following NEW packages will be installed:
  libqimageblitz-dev libtidy-dev
0 upgraded, 2 newly installed, 1 to remove and 8 not upgraded.
Need to get 205 kB of archives.
After this operation, 1,932 kB disk space will be freed.
Do you want to continue [Y/n]?
[...]

You'll probably won't notice (as I didn't) that it has removed a package, libkatepartinterfaces. This is because in my system I have the old version of it and there seems to be some kind of conflict somewhere. I just cross my fingers it won't break much[3].

Now get the sources and compile:

mdione@mustang:~/src/system/debian$ apt-get source kdebase
Reading package lists... Done
Building dependency tree
Reading state information... Done
NOTICE: 'kdebase' packaging is maintained in the 'Git' version control system at:
git://git.debian.org/pkg-kde/kde-sc/kdebase.git
Need to get 2,721 kB of source archives.
Get:1 http://qt-kde.debian.net/debian/ experimental-snapshots/main kdebase 4:4.7.1-0r2 (dsc) [3,217 B]
Get:2 http://qt-kde.debian.net/debian/ experimental-snapshots/main kdebase 4:4.7.1-0r2 (tar) [2,685 kB]
Get:3 http://qt-kde.debian.net/debian/ experimental-snapshots/main kdebase 4:4.7.1-0r2 (diff) [33.1 kB]
Fetched 2,721 kB in 26s (105 kB/s)
gpgv: Signature made Tue 27 Sep 2011 09:43:53 PM CEST using RSA key ID 73A85F31
gpgv: Can't check signature: public key not found
dpkg-source: warning: failed to verify signature on ./kdebase_4.7.1-0r2.dsc
dpkg-source: info: extracting kdebase in kdebase-4.7.1
dpkg-source: info: unpacking kdebase_4.7.1.orig.tar.bz2
dpkg-source: info: unpacking kdebase_4.7.1-0r2.debian.tar.gz
dpkg-source: info: applying enable_debianabimanager.diff
dpkg-source: info: applying enable_dlrestrictions.diff

mdione@mustang:~/src/system/debian$ ( cd kdebase-4.7.1 && nice -n 19 dpkg-buildpackage -j3 -b -us -uc )
[...]

Installing the resulting packages is another history, as some of them depend on kde-runtime, so it's better if you compile kde-runtime before[4]. Trying to install its build-deps, you realize taht you actually need to compile soprano first.

You get the idea. Now I'll just show the generated packages I installed by hand after each compilation, but only those that suited my needs:

soprano:

mdione@mustang:~/src/system/debian$ sudo dpkg -i \
soprano-daemon_2.7.3+dfsg.1-0r0_i386.deb \
libsoprano4_2.7.3+dfsg.1-0r0_i386.deb \
libsoprano-dev_2.7.3+dfsg.1-0r0_i386.deb

kde-runtime:

mdione@mustang:~/src/system/debian$ sudo dpkg -i \
kdebase-runtime_4.7.2-0r3_all.deb \
khelpcenter4_4.7.2-0r3_i386.deb \
kde-config-phonon-xine_4.7.2-0r3_i386.deb \
kde-runtime-data_4.7.2-0r3_all.deb \
kde-runtime_4.7.2-0r3_i386.deb \
plasma-scriptengine-javascript_4.7.2-0r3_i386.deb

While installing kde-runtime-data I had to remove kdebase-runtime-data:

[...]
Selecting previously unselected package kde-runtime-data.
dpkg: regarding kde-runtime-data_4.7.2-0r3_all.deb containing kde-runtime-data:
 kde-runtime-data breaks kdebase-runtime-data (<< 4:4.7.2)
  kdebase-runtime-data (version 4:4.6.5-1) is present and installed.
[...]

mdione@mustang:~/src/system/debian$ sudo dpkg --remove kdebase-runtime-data
(Reading database ... 218612 files and directories currently installed.)
Removing kdebase-runtime-data ...
[...]

kdebase:

mdione@mustang:~/src/system/debian$ sudo dpkg -i \
dolphin_4.7.1-0r2_i386.deb \
kdebase-bin_4.7.1-0r2_i386.deb \
kdebase-data_4.7.1-0r2_all.deb \
kdepasswd_4.7.1-0r2_i386.deb \
konq-plugins_4.7.1-0r2_i386.deb \
konqueror_4.7.1-0r2_i386.deb \
konqueror-nsplugins_4.7.1-0r2_i386.deb \
libkonq5abi1_4.7.1-0r2_i386.deb \
libkonq-common_4.7.1-0r2_i386.deb \
plasma-widget-folderview_4.7.1-0r2_i386.deb

While trying to get akonadi I got this error message:

Failed to fetch http://qt-kde.debian.net/debian/pool/main/a/akonadi/akonadi_1.6.2-0r1.dsc  Hash Sum mismatch

Checking the MD5, SHA1 and SHA256 checksums and comparing to the ones in the .dsc file revealed no difference. lisandro told me to use dget instead:

mdione@mustang:~/src/system/debian$ dget http://qt-kde.debian.net/debian/pool/main/a/akonadi/akonadi_1.6.2-0r1.dsc
dget: retrieving http://qt-kde.debian.net/debian/pool/main/a/akonadi/akonadi_1.6.2-0r1.dsc
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2620  100  2620    0     0   6842      0 --:--:-- --:--:-- --:--:--  8161
dget: using existing akonadi_1.6.2.orig.tar.bz2
dget: using existing akonadi_1.6.2-0r1.debian.tar.gz
akonadi_1.6.2-0r1.dsc:
dscverify: akonadi_1.6.2-0r1.dsc failed signature check:
gpg: Signature made Thu 03 Nov 2011 08:55:52 AM CET using RSA key ID 73A85F31
gpg: Can't check signature: public key not found
Validation FAILED!!

Unpacking by hand:

mdione@mustang:~/src/system/debian$ dpkg-source -x akonadi_1.6.2-0r1.dsc
gpgv: Signature made Thu 03 Nov 2011 08:55:52 AM CET using RSA key ID 73A85F31
gpgv: Can't check signature: public key not found
dpkg-source: warning: failed to verify signature on ./akonadi_1.6.2-0r1.dsc
dpkg-source: info: extracting akonadi in akonadi-1.6.2
dpkg-source: info: unpacking akonadi_1.6.2.orig.tar.bz2
dpkg-source: info: unpacking akonadi_1.6.2-0r1.debian.tar.gz
dpkg-source: info: applying x11_not_required.diff

akonadi:

mdione@mustang:~/src/system/debian$ sudo dpkg -i \
akonadi-server_1.6.2-0r1_i386.deb \
akonadi-backend-sqlite_1.6.2-0r1_i386.deb \
libakonadi-dev_1.6.2-0r1_i386.deb \
libakonadiprotocolinternals1_1.6.2-0r1_i386.deb

Here I cheated: I really didn't want to spend 10 minutes of iterative attempts to find the minimal set of lib packages to install:

libkdepim:

mdione@mustang:~/src/system/debian$ sudo dpkg -i \
kdepimlibs5-dev_4.7.2-0r1_i386.deb \
kdepimlibs-kio-plugins_4.7.2-0r1_i386.deb \
lib*.deb

kde-wallpapers:

mdione@mustang:~/src/system/debian$ sudo dpkg -i \
kde-wallpapers-default_4.7.2-0r0_all.deb

kde-workspace:

sudo dpkg -i --auto-deconfigure \
systemsettings_4.7.2-0r7_i386.deb \
plasma-desktop_4.7.2-0r7_i386.deb \
plasma-scriptengine-python_4.7.2-0r7_all.deb \
plasma-widgets-workspace_4.7.2-0r7_i386.deb \
plasma-dataengines-workspace_4.7.2-0r7_i386.deb \
klipper_4.7.2-0r7_i386.deb \
kdm_4.7.2-0r7_i386.deb \
kde-workspace-bin_4.7.2-0r7_i386.deb \
kde-workspace-data_4.7.2-0r7_all.deb \
kde-window-manager_4.7.2-0r7_i386.deb \
kdebase-workspace_4.7.2-0r7_all.deb \
kdebase-workspace-bin_4.7.2-0r7_all.deb \
libkworkspace4_4.7.2-0r7_i386.deb \
kde-workspace-kgreet-plugins_4.7.2-0r7_i386.deb \
kde-style-oxygen_4.7.2-0r7_i386.deb \
libkdecorations4_4.7.2-0r7_i386.deb \
libkephal4abi1_4.7.2-0r7_i386.deb \
libkwineffects1abi2_4.7.2-0r7_i386.deb \
libkworkspace4_4.7.2-0r7_i386.deb \
libplasmagenericshell4_4.7.2-0r7_i386.deb \
libtaskmanager4abi2_4.7.2-0r7_i386.deb \
libplasmaclock4abi2_4.7.2-0r7_i386.deb \
libksgrd4_4.7.2-0r7_i386.deb \
libplasma-geolocation-interface4_4.7.2-0r7_i386.deb \
libsolidcontrol4abi2_4.7.2-0r7_i386.deb \
libprocesscore4abi1_4.7.2-0r7_i386.deb \
libweather-ion6_4.7.2-0r7_i386.deb \
libkscreensaver5_4.7.2-0r7_i386.deb \
libprocessui4a_4.7.2-0r7_i386.deb \
libsolidcontrolifaces4abi2_4.7.2-0r7_i386.deb \
kde-workspace_4.7.2-0r7_all.deb \
ksysguard_4.7.2-0r7_i386.deb \
freespacenotifier_4.7.2-0r7_i386.deb \
libksignalplotter4_4.7.2-0r7_i386.deb \ 
ksysguardd_4.7.2-0r7_i386.deb

I'll just finish this post here. The rest is just more of the same. The final list of packages a compile in right order is:

kde4libs
meta-kde
soprano
kde-runtime
kdebase
akonadi
prison
kdepimlibs
kde-wallpapers
kde-workspace

All in all, this took some 10 hours of finding deps, compiling and installing, and 6.4 GiB between original .tar.gz files, compilation dirs and generated packages.


[1] It's nice that I don't need to compile libqt4*, those take ages[5].

[2] Later lisandro from qkd pointed me to the dependency graph that confirms my guess.

[3] I did all this while running a KDE SC 4.6.5 session, and even writing this post in kate. Thanks to Linux and Debian not much actually broke, I only lost the ability to browse with konqueror after I installed the first batch of packages because KIO could not instantiate an HTTP/HTTPS ioslave anymore.

[4] Note that the dependency graph two notes above is actually a build dependency graph.

[5] Not that this compilation was fast. kdelibs without parallelism took some 3h and kdepimlibs and kde-workspace are two huge beasts.


debian kde

Posted Sun 27 Nov 2011 11:19:05 PM CET Tags:
Posted Wed 06 Apr 2011 08:29:31 PM CEST
10

A few years back I made a map of Córdoba's downtown by hand, using inkscape, to be used by people coming to a free software conference. You can watch the results here, but I can assure you that if it weren't for the mostly squarish/triangularish shape of Córdoba's blocks, it would have been very hard to do it.

Today someone asked me for the .svg file to do something similar. I searched and searched through my files, but couldn't find the originals. Then I remembered one of my current projects, which I'll mention bellow, which is based on OpenStreetMaps's data. So I told him to find the region he wanted and export it to PNG or something with the 'export' tab. Then, thoughts kept coming, and I even managed to remember MapOsMatic, which lets you select a region in OSM's maps and generate not only a nice printable map in several formats, but also a street index for the back of the same map (even as .csv, so you could have a .png of the map for your site and populate a database to look for streets). Even more, I generated a map that covers a slightly bigger area than my humble attempt from 4 years ago. So, there.

The project I'm toying now with is to make a bird's view map of the region I live now. This map will have only the towns and the main interconencting ways between them, much like maps like this one[1][2]. For that I firts tried using josm to download the data from the region, filter out the small routes, and work from there, but the region is quite big, and OSM's servers sensibly refuse to dump so much data at once. Asking in freenode's #openstreetmap[3] channel, they told me I could use XAPI to filter out only the data I wanted. So that's what I did: downloaded all roads from highways down to secondary roads, and rivers. While I was at it, I also downloaded the coastlines, because I'm close to the sea and there are also several lakes around. So, without further ado, here are the links I used:

wget -S -O routes.osm 'http://www.informationfreeway.org/api/0.6/way[highway=motorway|motorway_link|trunk|primary|secondary][bbox=5.5,43,8.5,45.5]'
wget -S -O routes.osm 'http://www.informationfreeway.org/api/0.6/way[waterway=river][bbox=5.5,43,8.5,45.5]'
wget -S -O routes.osm 'http://www.informationfreeway.org/api/0.6/way[natural=coastline][bbox=5.5,43,8.5,45.5]'

Notes: I also wanted waterway=riverbank, but for some reason the server never answered such a request. This does not include town names, hmmm... I could also have included historic=* and tourism=*. I might as well. There might be some follow ups to this project.


[1] RANT: the webmaster of the official turism site decided that is a good idea to put most of the contents in a flash applet, so I can't actually link to the map I wanted. Instead, here's a map from a neighbouring province, San Luis.

[2] If you follow the link, yes, you'll get a 'site not found' error. That's because it's actually a curse to that site's webmaster 'encoded' into an url.

[3] If you try to get into #osm, you're redirected to #joomla. Go fugure.


openstreetmap

Posted Fri 29 Oct 2010 08:57:24 PM CEST Tags:
Posted Fri 29 Oct 2010 08:57:24 PM CEST

To select a set of pages, such as pages that are locked, pages whose commit emails you want subscribe to, or pages to combine into a blog, the wiki uses a PageSpec. This is an expression that matches a set of pages.

The simplest PageSpec is a simple list of pages. For example, this matches any of the three listed pages:

foo or bar or baz

More often you will want to match any pages that have a particular thing in their name. You can do this using a glob pattern. "*" stands for any part of a page name, and "?" for any single letter of a page name. So this matches all pages about music, and any SubPages of the SandBox, but does not match the SandBox itself:

*music* or <a href="../../../sandbox/">SandBox</a>/*

You can also prefix an item with "!" to skip pages that match it. So to match all pages except for Discussion pages and the SandBox:

* and !SandBox and !*/Discussion

Some more elaborate limits can be added to what matches using these functions:

  • "glob(someglob)" - matches pages and other files that match the given glob. Just writing the glob by itself is actually a shorthand for this function.
  • "page(glob)" - like glob(), but only matches pages, not other files
  • "link(page)" - matches only pages that link to a given page (or glob)
  • "tagged(tag)" - matches pages that are tagged or link to the given tag (or tags matched by a glob)
  • "backlink(page)" - matches only pages that a given page links to
  • "creation_month(month)" - matches only files created on the given month number
  • "creation_day(mday)" - or day of the month
  • "creation_year(year)" - or year
  • "created_after(page)" - matches only files created after the given page was created
  • "created_before(page)" - matches only files created before the given page was created
  • "internal(glob)" - like glob(), but matches even internal-use pages that globs do not usually match.
  • "title(glob)", "author(glob)", "authorurl(glob)", "license(glob)", "copyright(glob)", "guid(glob)"
    • match pages that have the given metadata, matching the specified glob.
  • "user(username)" - tests whether a modification is being made by a user with the specified username. If openid is enabled, an openid can also be put here. Glob patterns can be used in the username. For example, to match all openid users, use user(*://*)
  • "admin()" - tests whether a modification is being made by one of the wiki admins.
  • "ip(address)" - tests whether a modification is being made from the specified IP address.
  • "comment(glob)" - matches comments to a page matching the glob.
  • "comment_pending(glob)" - matches unmoderated, pending comments.
  • "postcomment(glob)" - matches only when comments are being posted to a page matching the specified glob

For example, to match all pages in a blog that link to the page about music and were written in 2005:

blog/* and link(music) and creation_year(2005)

Note the use of "and" in the above example, that means that only pages that match each of the three expressions match the whole. Use "and" when you want to combine expression like that; "or" when it's enough for a page to match one expression. Note that it doesn't make sense to say "index and SandBox", since no page can match both expressions.

More complex expressions can also be created, by using parentheses for grouping. For example, to match pages in a blog that are tagged with either of two tags, use:

blog/* and (tagged(foo) or tagged(bar))

Note that page names in PageSpecs are matched against the absolute filenames of the pages in the wiki, so a pagespec "foo" used on page "a/b" will not match a page named "a/foo" or "a/b/foo". To match relative to the directory of the page containing the pagespec, you can use "./". For example, "./foo" on page "a/b" matches page "a/foo".

To indicate the name of the page the PageSpec is used in, you can use a single dot. For example, link(.) matches all the pages linking to the page containing the PageSpec.

Posted Thu 21 Oct 2010 12:53:50 AM CEST
Posted Sat 09 Oct 2010 01:12:46 AM CEST
Posted Wed 06 Apr 2011 08:29:31 PM CEST