After several months thinking about it, and just two requests, I finally decided to publish satyr's code. I decided to use github because I already switched to satyr from hg to git, mainly for testing and understanding it. I think I can live with hg, althought branch management in git seems to be given more thought and a good implementation.

So, without further ado: satyr in github

Remember, it's still a test software, by no means polished or ready for human consumption, and with very low development force. Still, I think it has some nice features, like interchangeable skins and a well defined backend, D-Bus support, quick tag edition, reasonable collection managment, and, thanks to Phonon, almost-gapless playback and things like «stop after playing the current this file» (but not «after any given file» yet).

In Debian Sid it mostly works only with the GStreamer backend; I haven't tried the xine one and I know VLC does not emit a signal needed for queueing the next song, so you have to press «next» after each song. AFAIK this is fixed upstream.

satyr pykde python

satyr-0.3.2 "I should install my own food" is out. The Changelog is not very impressive:

• We can save the list of queued Songs on exit and load it at startup.
• The queue position is not presented when editing the title of a Song that is queued.
• Setting the track number works now (was a bug).
• Fixed a bug in setup.py.

but the last one is somewhat important (Thanks Chipaca). Also, 2 months ago, I made the satyr-0.3.1 "freudian slip" release, when user 'dglent' from http://kde-apps.org/ found a packaging bug. It was not only a bugfixing revision, it also included new features:

• nowPlaying(), exported via dbus.
• 'now playing' plugin for irssi.
• Changes in tags is copied to selected cells (in the same column). This allows 'Massive tag writing'.
• Fixed "NoneType has no attribute 'row'" bug, but this is just a workaround.
• Forgot to install complex.ui.

Now go get it!

satyr pykde python

So far both the default and complex skins in satyr were (ab)using the selection in their respectives views to highlight the current song and, in the case of the complex skin, that meant that you couldn't select cells. This could be (and will be) handy for editing several tags at the same time or for grouping songs in an Album. I decided that this should not longer be this way.

It was really simple: I simple reenabled selection, the extended version (Shift and Control work as you grew to spect). Now the way to highlight the current song is to simply change the background and foreground colors for the cells of that row. How? Simple, extending the QPlayListModel.data() method to provide these properties and using QApplication.palette() to get colors that would stand out from the normal application colors (even the selection) and would still make the text readable:

    def data (self, modelIndex, role):
        if modelIndex.isValid () and modelIndex.row ()<self.aggr.count:
            [...]
            # check the row so we highlight the whole row and not just the cell
            elif role==Qt.BackgroundRole and modelIndex.row ()==self.parent_.modelIndex.row ():
                # highlight the current song
                # must return a QBrush
                data= QVariant (QApplication.palette ().dark ())

            elif role==Qt.ForegroundRole and modelIndex.row ()==self.parent_.modelIndex.row ():
                # highlight the current song
                # must return a QBrush
                data= QVariant (QApplication.palette ().brightText ())
            [...]

And that's almost it... because there's always a but. Whenever the current song changes, there's a little lag between this happens (you can hear the new song being played) and the highlighting updates. This is because I update that self.parent_.modelIndex you see up there, but the QPLM.data() method is not called until the next repaint, which doesn't happen inmediately. So, we must force that repaint. How? Again, it's simple: we already emited the dataChanged() signal in the method setData() when the data could be successfully changed. Now we do the same just so the view updates the highlighting:

    def showSong (self, index):
        [...]
        # FIXME? yes, this could be moved to the model (too many self.appModel's)
        start= self.appModel.index (oldModelIndex.row (), 0)
        end=   self.appModel.index (oldModelIndex.row (), columns)
        self.appModel.dataChanged.emit (start, end)

        start= self.appModel.index (self.modelIndex.row (), 0)
        end=   self.appModel.index (self.modelIndex.row (), columns)
        self.appModel.dataChanged.emit (start, end)
        [...]

satyr pykde python

Tal vez ya lo leyeron en otro lado, pero bueno: el otro día fue el día de los tutoriales de Kubuntu. Básicamente fueron tutoriales por IRC. Los logs los pueden encontrar en el wiki de Kubuntu. En particular hay 3 que me parecen bastante piolas:

• Cómo usar bazaar (con Launchpad, esa bonga de project server cerrado que usan los *buntu guys).

• Empaquetando para Kubuntu (obvio, sirve para Debian también).

• Tutorial de PyKDE4, el cual lamebtablemente es muy corto.

Como para salir masomenos andando están muy piolas.

bazaar pykde

I certainly hope this is the last post in the Phonon-and-badly-encoded/mixed-encodings-filenames saga, but I know is just wishful thinking: as all encoding-related problems they never really dissapear, it's just that you hadn't hit the right wrong stone yet. In any case, I fixed all my later problems wherever they where, and now I can answer this question: how to play files whose filenames are badly encoded and/or have mixed encodings, all this in Phonon?

Right now the answer is: you have to provide a properly encoded QUrl. How, you might ask, can I get one of those? Are they selled in the same odly-looking places where you can buy cigarretes, or even marihuana[1]? The answer, luckly, is way more simple.

Putting together all the code I've been showing about Python, PyQt4/PyKDE4 and Phonon recently, it comes down to this[5]:

# path is a str()
qba= QByteArray (path)
# the exceptions are not needed,
# but is cleaner if you print the outcome of this
qu= QUrl.fromEncoded (qba.toPercentEncoding ("/ "))
# this is needed by the gstreamer backend[3],
# and the xine backend doesn't complain
qu.setScheme ('file')

... and that's it. You can now create a MediaSource with this qu.

There are a couple of ideas that I want to express as conclusion to all this:

• In an ideal world these things should not happen. But this is one of the lesser problems with this non-ideal world, so bear with it.
• Paths should not be stored in QStrings, even if they can (and they do) store this kind of pathnames, because if you try to 'encode' its contents (in the Unicode sense; that is, convert it to an encoding like UTF-8[4]) you get farts or barks at best. Yes, you always have constData() but from QString's class reference there is no warranty that this will keep being the case[6].
• In fact, QString's class reference says at some point: «[one case] where QByteArray is appropriate are when you need to store raw binary data...», and as I already wrote, «[t]his would be the case for paths; you need the bytes».
• QFile and QDir can only be created from QStrings. I'm not sure if, given all I wrote, that's right.

The good news is that satyr now can play any file that the backends can whatever their filename-as-string-of-bytes is, I'm a little bit happier about it, I got another contribution to KDE and might even have to close a lot of bugs!

satyr pykde python phonon

[1] That question is only legal in Nederlands[2] and very few others cities in the planet.

[2] Actually is not legal. See this wikipedia article.

[3] I might pull up my sleeves again and fix that.

[4] You might have already know this, but if you not: you cannot print Unicode, because Unicode is not and encoding. You have to encode it first. Hence, the toLatin1(), toUtf8() and similar QString methods, and also the inverse from*().

[5] Of course the equivalent C++ code also works, with path being a char *.

[6] And in the case of PyQt4, that method is not even available. But I already globed about it.

kReiSSy (se pronuncia como la palabra "crazy" del inglés) es uno de mis proyectos más ambiciosos en este momento (tengo uno mayor, pero está en el freezer; ya hablaré de él). en resumen, puedo decir que kReiSSy es un lector de feeds (rss, atom, quéno[1]) alimentado a gofios, o como dicen ahora, en esteroides. otra descripción puede ser: es un concentrador de información externa con categorzación basada en tags.

¿qué features lo hacen tan pulenta? veamos:

• tagging[2] de posts (eso lo hace cualquiera).
• almacenamiento local de posts (en una base de datos, no borra ninguno).
• browser integrado. acá empecé a irme al carajo.
• tagging de páginas "leídas por ahí", que las convierte en first class citizens del programa.

¿qué significan estos dos últimos fatures? que puedo estar leyendo un post, seguir un link, de ahí a otro, y otro, todo en el browser integrado, y así hasta que encuentro otra página que nada que ver por dónde empecé, y entonces puedo tagear dicha página. esto hace dos cosas automáticas: guarda la página en la base de datos como si fuera un post más, y me permite luego buscar dicha página por tag.

ahora, me fui de boca un poco al decir "browser integrado" (esta es la sección "proyect status"). por ahora el browser nos permite ir hacia adelante, siguiendo links, pero no para atrás. es decir, le faltan todos los botones de navegación. también se podría marcar un post como "leer después", pero no hay código que lo haga. el filtrado por varios feeds o varios tags está roto/no anda (esto se debe o a un bug en SQLAlchemy o a mi inoperancia). aún no graba bien su sesión, cosa de volver en el mismo estado en que lo dejamos. y hace un par de chanchadas con los archivos, asume un par de paths y negradas así.

¿y en qué está hecha semejante bestia? python, obvio, mi lenguaje de cabecera desde hace unos añitos ya. pero no python puro, sino con varios agregados.

uno ya lo mencioné, SQLAlchemy, un ORM bastante potente, pues no fuerza muchas cosas. para usarlo como un ORM clásico (una clase por tabla) hay que hacer un par de giladas, pero nada grave. y permite hacer queries con SQL a lo macho, aunque no me llevo muy bien con eso. sqlite por debajo.

feedparser y beautifulsoup, un lector de múltiples tipos de feeds, y un html scrapper.

y la vedette de todos, y el motivo por el que empecé este proyecto: PyKDE. soy un usuario y fanático de kde desde que usé un redhat5.2 allá por el '98 o así. conozco bastante la infraestructura que hay por debajo, he leído varias veces cachos de código en busca de solucionar algún bug que me mordió, algunas veces hasta logré repararlo y todo. si bien su look no es muy bonito, la infraestructura que hay debajo es impresionante.

a tal punto que hacer este programa me resultó muy sencillo hasta ahora, pues sólo me concentré en mi funcionalidad. la parte de mostrado de html u otro tipo de archivos se lo dejé a KDE:

    mime= KMimeType.findByURL(url, 0, False, False)
    mimeType= mime.name ()
    if mimeType=='application/octet-stream':
        mimeType= KIO.NetAccess.mimetype (url, self);
    else:
    # asumo que es html
        mimeType= "text/html"

    ptr= KTrader.self().query(mimeType, "'KParts/ReadOnlyPart' in <span class="createlink">ServiceTypes</span>")[0]
    part= createReadOnlyPart (ptr.library (), tab, ptr.name ())

esto hace la fantástica magia de fijarse qué MimeType es el link (dado por url) y luego KTrader me entrega un KPart que sabe mostrar ese MimeType. simplemente la embebo en un tab y ya. juzguen ustedes.

ok, suficiente por ahora. ya estaré hablando de éste y otros proyectos.

[1] no es una traducción literal del "whatnot" en inglés, sino una reimplementación en castellano de la misma idea.

[2] uso muchos términos en inglés que ni me gasto en traducir. deal with it.

kreissy python pykde

In my last post I said «The next step is to make my Player class to export its methods via DBus and that's it!». Well, tell you what: is not that easy. If you try to inherit from QObject and dbus.service.Object you get this error:

In [3]: class Klass (QtCore.QObject, dbus.service.Object): pass
<span class="createlink">TypeError</span>: Error when calling the metaclass bases
    metaclass conflict: the metaclass of a derived class must be a
    (non-strict) subclass of the metaclasses of all its bases

This occurs when both ancestors have their own metaclasses. Unluckily Python doesn't resolve it for you. The answer is to create a intermediate metaclass which inherits from both metaclasses (which we can obtain with type()) and make it the metaclass of our class. In code:

class <span class="createlink">MetaPlayer</span> (type (QObject), type (dbus.service.Object)):
    """Dummy metaclass that allows us to inherit from both QObject and
    d.s.Object"""
    pass

class Player (QObject, dbus.service.Object):
    __metaclass__= <span class="createlink">MetaPlayer</span>
    [...]

Is that it now? Can I go and do my code? Unfortunately no. See this:

qdbus org.kde.satyr
/
/player
Cannot introspect object /player at org.kde.satyr:
org.freedesktop.DBus.Python.KeyError (Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.5/dbus/service.py", line 702, in _message_cb
    retval = candidate_method(self, *args, **keywords)
  File "/usr/lib/pymodules/python2.5/dbus/service.py", line 759, in Introspect
    interfaces = self._dbus_class_table[self.__class__.__module__ + '.' + self.__class__.__name__]
<span class="createlink">KeyError</span>: '__main__.Player'
)

This is the class dbus.service.Object complaining something else. It's getting late here and I'm tired, so I'll continue tomorrow.

dbus python pykde satyr

More than two months ago I globed about QStrings and paths. The problem was this: my app accepts paths via command line, which are processed via KCmdLineOptions; which in turn converts everything to QStrings. What I wanted were paths, which are more like QByteArrays, not QStrings (because the latter have internally an unicode representation; more on that later). Including PyQt4 in the equation forced me to resort to QByteArray to get the path as a str instead of using QString.constData() (PyQt4 doesn't export that function). But that's only the beginning of the problem.

Take for instance this situation. I have a music collection that I've been building for years now (more that 10, I think). In the old times of this collection the filenames were encoded in iso-8859-1. Then the future came and converted all my machines to utf-8. But only the software; the filesystems were in one way or another inherited from system to system, from machine to machine. So I ended with a mixture of utf and iso filenames, to the point where I have a file whose filename is in iso, but the directory where it is is in utf. Yes, I know, it is a mess. But if I take any decent media player, I can play the file allright. That's because the filesystem knows nothing of encodings (otherwise it would reject badly encoded filenames).

I just spent last saturday making sure that satyr only stored filepaths in strs, not unicodes or QStrings. It took concentration, but having just a bunch of classes and only 3 or 4 points where the filepaths are managed it wasn't that difficult. Still, it took a day. But then, as I mentioned in that post, Phonon the is not able to play such files... or so I thought.

If you run satyr after executing export PHONON_XINE_DEBUG=1 you'll see a lot of Phonon debug info in the console (not that there is another way to run satyr right now anyways). Among all that info you'll see lines such as these two:

void Phonon::Xine::XineStream::setMrl(const QByteArray&, Phonon::Xine::XineStream::StateForNewMrl) ...
bool Phonon::Xine::XineStream::xineOpen(Phonon::State) xine_open succeeded for m_mrl = ...

If you're sharp enough (I'm not; sandsmark from #phonon had to tell me) you'll note the mention of MRL's. MRL's are xine's URL for media. As any URL, they can (and most of the time must) encode 'strange' characters with the so-called "percent encoding". This means that no matter what encodings the different parts of a filepath is in, I just add file:// at the beginning and then I can safely encode it scaping non-ascii characters to %xx representations... or that's what the theory says. One thing to note is that the file:// part must not be scaped; xine complains that the file does not exist in that case.

Looking for help in Qt's classes one can find QUrl and the already known QByteArray. I can call QByteArray.toPercentEnconding() from my str and feed that to QUrl.fromPercentEncoding() (which strangely returns a QString, which is exactly what we're avoiding) or QUrl.fromEncoded(). But then the first function encodes too much, replacing :// with %3A%2F%2F. No fun.

Ok, let's try creating a QByteArray with only the file:// and then append() the toPercentEncoding() of the path only. It works:

<span class="createlink">PyQt4</span>.QtCore.QByteArray('file://%2Fhome%2Fmdione...%2F%C3%9Altimo%20bondi%20a%20Finisterre%2F07-%20La%20peque%F1a%20novia%20del%20carioca.wav')

But then calling QUrl.fromEncoded() gives:

<span class="createlink">PyQt4</span>.QtCore.QUrl("file://xn--/home/mdione.../ltimo bondi a finisterre/07- la pequea novia del carioca-wkmz60758d.wav")

The URL got somehow puny-encoded, which of course xine doesn't recognize for local files.

Another option is to create an empty QUrl, call setEncodedUrl() with the ParsingMode to QUrl.StrictMode so we avoid 50 lines of code that start here[1] that try to escape everything all over again (and I already had some double-or-even-triple-enconding nightmares parsing RSS/Atom feeds last year, thank you), but we get puny-encoded again (maybe it is 'pwny-encoded'?).

Last resort: backtrack to the point were we created only one QByteArray with the path and call toPercentEncoding(); feed that to the method setEncodedPath() of an empty QUrl. Then we add the last piece calling setScheme('file') and we're ready! Of course we're not:

<span class="createlink">PyQt4</span>.QtCore.QByteArray('file:%2Fhome%2Fmdione...%2F%C3%9Altimo%20bondi%20a%20Finisterre%2F07-%20La%20peque%F1a%20novia%20del%20carioca.wav')

Notice the lack of the two // after file:? xine doesn't like it; hence, I don't either.

Ok, this post got too long. I hope I can resolve this soon, I already spent too much time on it. At least a good part of it was expaining it, so others don't have to suffer the same as I did.

BTW, satyr will shortly be released, whether I fix this bug or not.

satyr pykde phonon

[1] Look at the size of that file! 6k lines to handle URL's! Who would say it was so difficult... Once more I'm remembered of how lucky I am to have this libraries at the tips of my fingers, yay!

I seem to have fixed the bug I mentioned in my last post. This is what I had:

class <span class="createlink">MetaPlayer</span> (type (QObject), type (dbus.service.Object)):
    """Dummy metaclass that allows us to inherit from both QObject and
    d.s.Object"""
    pass

class Player (QObject, dbus.service.Object):
    __metaclass__= <span class="createlink">MetaPlayer</span>
    [...]

Notice that MetaPlayer doesn't have a explicit __init__() method; one would spect that Python would take are of that. Here's the fixing code:

<span class="createlink">MetaQObject</span>= type (QObject)
<span class="createlink">MetaObject</span>= type (dbus.service.Object)

class <span class="createlink">MetaPlayer</span> (MetaQObject, <span class="createlink">MetaObject</span>):
    """Dummy metaclass that allows us to inherit from both QObject and d.s.Object"""
    def __init__(cls, name, bases, dct):
        <span class="createlink">MetaObject</span>.__init__ (cls, name, bases, dct)
        <span class="createlink">MetaQObject</span>.__init__ (cls, name, bases, dct)

I really don't understand why I have to be so explicit. Maybe it's because the metaclass for dbus.service.Object, dbus.service.InterfaceType, inherits from the type type[1]; this type is a new style class[2], but doesn't inherits from object. Thus, I think, the inherited __init__() methods are not called automatically.

In any case, now I can mix QObject and dbus.service.Object, and it works fine. For instance, this call works:

$ qdbus org.kde.satyr /player quit

dbus python pykde

[1] the type type is of type type! here:

In [1]: type (type)
Out[1]: <type 'type'>

[2] its type is not instance but type, as mentioned above.

Update before even publishing: most of the numbers in the initial writing were almost doubled. The problem was that distutils left a build directory when I tried either to install or to package satyr, I don't remember which, so the files found by the find commands below were mostly duplicated. I had to remove the directory and run all the commands again!

I wanted to know some things about satyr's code, in particular some statistics about its lines of code. A first approach:

$ find . -name '*.py' -o -name '*.ui' | xargs wc -l | grep total
  2397 total

Hmm, that's a lot, I don't remember wrinting so many lines. Beh, the comments, let's take them out:

$ find . -name '*.py' -o -name '*.ui' | xargs egrep -v '^#' | wc -l
2136

What about empty lines?:

$ find . -name '*.py' -o -name '*.ui' | xargs egrep -v '^(#.*| *)$' | wc -l
1764

Meeh, I didn't take out all the comment lines, only those lines starting with #, which are mainly the license lines on each source file. I have to also count the comments in the middle of the code:

$ find . -name '*.py' -o -name '*.ui' | xargs egrep -v '^( *#.*| *)$' | wc -l
1475

And how much of those lines are actual code and not from some xml file describing user interface?:

$ find . -name '*.py' | xargs egrep -v '^( *#.*| *)$' | wc -l
1124

How much code means its 3 current skins?:

$ find satyr/skins/ -name '*.py' | xargs egrep -v '^( *#.*| *)$' | wc -l
341

How much in the most complex one?

$ egrep -v '^( *#.*| *)$' satyr/skins/complex.py | wc -l
182

All this numbers tell something: ~300 empty lines means that my code is not very tight. I already knew this: I like to break functions in secuential blocks of code, each one accomplishing a somehow atomic step towards the problem the function tries to solve. Almost 300 comment lines means my code is very well commented, even if a sixth of those comments are BUGs, TODOs, FIXMEs or HINTs:

$ find . -name '*.py' | xargs egrep '^ *# (TODO|BUG|FIXME|HINT)' | wc -l
56

Wow, more than I thought. Now, 1100 lines of actual code for all that satyr accomplishes today (playing almost any existing audio file format, progressive collection scanning, lazy tag reading and also tag writing, 3 skins[4], handle several Collections, searching, random, stop after, saving some state, picking up new songs added to the collection via the filesystem, queuing songs[1], dbus interface[2], handle any encodings in filenames[3]... phew! and some minor features more!) I think is quite impressive.

Of course, doing all that in athousandsomething lines of code would be nearly impossible without PyQt4/Qt4, PyKDE4/KDE (something) 4[7], Tagpy/Taglib and finally Python itself. It's really nice to have such a nice framework to work with, really.

satyr pykde python

[1] No user interface for this yet; shame on me.

[2] ... which toguether with kalarm and qdbus make my alarm clock.

[3] Almost all the support is in Phonon itself.

[4] [5] Less than 800 if we don't count skins.

[5] Yes, I add more footnotes as I readproof the posts :)

[6] I skipped [6] :)

[7] After the rebranding I don't know which is the proper name for the libraries, because I'm writing this post while very much offline, and TheDot does not publish the whole articles via rss, which I hate.