Note: this is a translation of an old post. I decided to translate it because now I'm looking for a SysAdmin position (tell your friends!) and I would like this post to show how I work.

Last Saturday I received an email from one of the guys from work with the subject «urgennnnnnnnt: heeeeeeeeeelp»[sic]. He says he was idling on Friday night when his machine stopped emiting sound via the soundcard and then it behaved erratically. When he tried rebooting it, it didn't boot anymore. «It says something about disk not bootable...».

Monday morning I go to work and go to see the machine. Precisely, it said something about «disk not bootable». I boot with a USB key with GRML and I find that the disk has no partitions.

Panic.

The guy is doing a PostDoc in something astronomical (literally) and all his work is in that machine. No backups, as usual, so I prepare myself to rescue the partitions.

In that same USB key I have a system with parted. I boot with it and I try using parted's rescue tool. Nothing. I ask the guy how the disk was partitioned, etc. He tells me that he only installed Kubuntu clicking 'Next'. Kubuntu by default creates a swap partition and an ext3 partition for / and that's it, which made what was coming relatively easy.

I reboot in GRML and I use hexdump -C /dev/sda | more to see the disk's content. This is not the first time that I juggle with partitions and MBRs, but last time I did it, I used a tool that is now discontinued (the tool was called DiskEdit, included in The Norton Utilities), which had special edit modes for MBRs, FATs, and a lot of useful things... in MS universe.

First I confirm that, yes, the first sector is a MBR (at least it has the 0x55aa signature at the end), and that the whole partition table is empty, but in the second sector of the disk there seems to be a copy. I take pen and paper, write down what I found, but it turns out not only I have half the data, the partition I thought I found was too small.

So I decide to look for the partition by hand. To do that I needed to find out first how does the ext3 kernel code know wether a partition is ext3 or not. I knew it would be some kind of magic signature, but I had no idea which. So I installed the sources for 2.6.29 in my laptop and started to look at ext3's code. After going around a lot, including reading the code that is excuted when you mount a filesystem of type ext3, where we can see that it uses a magic signature[3] and the structure of the ext3 superblock, where we can see the magic's offset is 0x38.

So the problem of finding an ext3 partition is reduced to the problem of finding 0x53ef (damn little endian) at a sector's offset 0x38 in the disk. Luckily more has a find tool, so I sit down to search every occurrence of 53 ef, hoping that the address at the left ends in 30 and that they would be the 9th and 10th bytes in the line (damn 0 based offsets).

A few 'next' after, I get my first candidate. It looks good, because I was also comparing my findings with a similar dump from my USB key (which I have formatted as ext2, and luckily ext2 and ext3 share those structures), and also I spot something that looks like a uuid.

This candidate's address is 0x80731038. I substract 0x38 and I get the address 0x80731000, a nice round number for a superblock. Converted to decimal that's 2.155.024.384, some 2GiB from the disk's begginning. Looks really good! The swap partition could be before the root one, and could have that size.

I use fdisk /dev/sda to see the (still empty) partition table. It says there's 16.065 sectors per cylinder, times 512 bytes per sector, equals 8.225.280 bytes per cylinder. Almost all distros (actually I think all of them) partition disks at cylinder boundaries[1], so if the sector I found is right at the beginning of a cylinder...

I divide 2.155.024.384/8.225.280=...

(suspense pause)[2]

262.000124494...

¡Damn! I almost had it... Hmm, how much is the factional part? (262.000124494-262)*8.225.280=... ¡1024! ¿Is it that...?

I run strace debugfs -R show_super_stats /dev/sdb1 (the partition in my USB key) and I see that it actually seeks 1024 bytes within the partition for reading the superblock!

This is it. With 262 in my head, I fire fdisk /dev/sda and I create two partitions: swap in cylinders 1-261 and root from cylinder 262 till the end. I save, cross my fingers and I run debugfs -R show_super_stats /dev/sda1. It fails! What's wrong? I reboot and I try again, just in case the kernel did not re-read correctly the partition table. It fails again. WTF?

Ah, duh, it's sda2, where do I have my head... Ok, debugfs -R show_super_stats /dev/sda2... it works, the sonofabitch works! I can't believe it. I risk it: fsck -n /dev/sda2. «Filesystem is clean». Damn, I try harder: fsck -n -f /dev/sda2...

Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/sda2 etc etc...

It's fine! But the MBR doesn't have GURB installed, so I do the usual GRUB reinstall process, I reboot...

It boots like nothing has happened, and it finishes in a beautiful login. Satisifed, I pat myself in the back, pack my things and I start my weekend.

sysadmin rescue


[1] ... wasting some 8MiB between the MBR and the first partition.

[2] The sharp ones reading this will notice that this can not give an integer by no means.

[3] Reiser magics are funny. Looks like he started the fad that now AdOlEsCeNtS use.

Posted Thu 07 Apr 2011 11:46:15 PM CEST Tags:
01
Posted Wed 06 Apr 2011 08:28:18 PM CEST
03

Last year and a half I was working in research. This position was about, among other things, to port a programming language (two, actually, Hop and Bigloo) to the Antroid platform. I already had written something about it, but this time I want to show my high level impressions of the platform. What follows is part of a report I wrote at the end of that job, which includes the things I wanted to say.

The Android port can be viewed as four separate sub-tasks. Hop is software developed in the Scheme language; more particularly, it must be compiled with the Bigloo Scheme compiler, which in turn uses gcc for the final compilation. That means that we also needed to port Bigloo first to the platform, not because we were planning to use it in the platform, but because we need the Bigloo runtime libraries ported to Android, as Hop and any other program compiled with Bigloo uses them. The other three subtasks, which are discussed later, are porting Hop itself; developing libraries to access devices and other features present in the platform; and, we'll see later the reasons, make the port work with threads.

When we started to investigate how to port native code to the platform we found that there wasn't much support. At fisrt the only documentation we could find was blog posts of people trying to do it by hand. They were using the compiler provided in Android's source code to compile static binaries that could be run on the platform. Because Bigloo uses dinamic libraries to implement platform dependent code and modules, we aimed to find a way to compile things dinamically. After 3 or 4 weeks we found a wrapper written in Ruby that managed all the details of calling gcc with the proper arguments. With this we should be able to port anything that uses gcc as the compiler, just like Bigloo does. At the same time, the first version of Android's NDK (Native Development Kit) appeared, but it wasn't easy to integrate in our build system.

(Note: Actually I think most of the problems we faced doing this port stem from this. The NDK forces you to write a new set of Makefiles, but our hand-made build system and build hierarchy made such an effort quite big. Also, that mean supporting a parallel build system, while it should not be so crazy to spect a cleaner way to integrate the toolchain into an existing build system, not only in hand-made like in this case, but also the most common ones, like autotools, cmake, etc.)

Even having the proper compiler, we found several obstacles related to the platform itself. First of all, Bigloo relies heavily on the C and pthread libraries to implement lowlevel functionality. Bigloo can use both glibc, GNU's implementation, or µlibc, an implementation aimed for embedded aplications. Bigloo also relies on Boehm's Garbage Collector (GC) for its memory management. The C library implementation in Android is not the glibc or the µlibc, but an implementation developed by Google for the platform, called Bionic. This version of the C library is tailored to the platform's need, with little to no regards to native application development.

The first problem we found is that GC compiled fine with Bionic, but the apllications that used GC did not link: there was a missing symbol that normally is defined in the libc, but that Bionic did not. We tried cooperating with the GC developers, we tried inspecting a Mono port to Android, given that this project also uses GC, trying to find a solution that could be useful for everyone, but at the end we just patched our sources to fake such symbol with a value that remotely made sense.

We also found that Bionic's implementation of pthreads is not only incomplete, but also has some glitches. For instance, in our build system, we test the existence of a function like everybody else: we compile a small test program wich uses it. With this method we found at least one function that is declared but never defined. That means that Bionic declares that the function exists, but then it never implements it. Another example is the declaration and definition of a function, but the lack of definition of constants normally used when calling this function.

Also, because most of the tests also must be executed to inform about the peculiarities of each implementation, we had to change our build system to be able to execute the produced binaries in the Android emulator.

Google also decided to implement their own set of tools, again, trimmed down to the needs of the platform, instead of using old and proven versions, like Busybox. This means that some tools behave differently, with no documentation about it, so we mostly had to work around this differences everytime a new one apperared.

All in all, we spent two and a half months just getting Bigloo to run in Android, dismissing the problem that Boehm's GC, using its own build system, detected that the compiler declared to not support threads, and refused to compile with threads enabled. This meant that Bigloo itself could not be compiled with pthreads support.

With this caveat in mind, we tackled the second subtask, porting Hop itself. This still raised problems with the peculiarities of the platform. We quickly found that the dinamic linker wasn't honoring the LD_LIBRARY_PATH environment variable, which we were trying to use to tell the system where to find the dynamic libraries.

The Android platform installs new software using a package manager. The package manager creates a directory in the SD card that it's only writable by the applilcation being installed. Within this directory the installer puts the libraries declared in the package. Bigloo, besides the dinamic libraries, requieres some aditional files that initialize the global objects. This files are not extracted by the installer, so we had to make a frontend in Java that opens the package and extract them by hand. But the installer creates the directory for the libraries in such a way that the application later cannot write in it.

Also, we found that the dinamic linker works for libraries linked at runtime, but does not for dlopen()'ing them, so we also had to rewrite a great part of our build system for both Bigloo and Hop to produce static libraries and binaries. This also needed disabling the dynamic loading of libraries, and with them, their initialization, so we had to initialize them by hand.

To add more unsuspected work, the Android package builder, provided with the SDK, ignores hidden files, which Bigloo uses to map Scheme module names to dynamic libraries. We had to work around this feature in the unpacking algorithm.

Then we moved to improve the friendliness of the frontend. So far, we could install Hop in the platform, either in a phone or in the emulator, but we could only run it in the emulator, because we were using a shell that runs as root on the emulator, but that runs as a user in a real device. This user, for the reasons given above, cannot even get into Hop's install dir. Even when Android has a component interface that allows applications to use components from other apps, none of the terminal apps we found at that time declared the terminal itself as a reusable component. We decided to use the code from the most popular one, which was based on a demo available on Android's source code, but not installed in actual devices. We had to copy the source code and trimm it down to our needs.

Having a more or less usable Hop package for Android, we decided to try and fix the issue we mentioned before: GC didn't compile with threads enabled. This means that we can't use the pthreads library, which is very useful for Hop. Hop uses threads to attend several requests at the same time. Bigloo implements two threads APIs, one based on pthreads and another which implements fair threads. Hop is able to use 5 different request schedulers, but works better with the one based on pthreads.

For these reasons we decided to focus in getting GC to use threads with the Android platform. GC's build system tests the existence of a threading platform checking the thread model declared by gcc. The gcc version provided with Android's SDK declares to have a 'single thread model', but we couldn't find what does this mean in terms of the code produced by gcc or how this could affect to GC's execution.

(Note: we didn't manage to make GC compile with threads.)

With a threadless Hop running, we had to add code to the server so we could talk between the server and the frontend while at the same time it is attending the requests from a web client. After several attempts to attack this problem, we decided that the best solution was to make this interface another service served by Hop. This meant less modifications to Hop itself, but a bigger one to the frontend we already had.

During these changes we found out a problem with JNI. The terminal component we imported into our code uses a small C library for executing the application inside (normaly a shell, in the original code, but Hop in our case) which is accessed from Java using JNI. The original Term application exported this class as com.android.term.Exec, but our copy exported it as fr.inria.hop.Exec. Even with this namespace difference JNI got confused and tried to use the Exec class from the original Term app. This is just another example how the platform is hard to work with. We found that the community support is more centered around Java and that very few people know about JNI, the NDK or any other native related technologies. We couldn't find an answer to this problem, so we worked around this by renaming the class.

So that's it. I can provide all the technical details for most the assertions I postulated above, but that would make this post unreadbal for its length. If you have any question about them, just conact me.

android

Posted Sat 26 Mar 2011 12:12:39 AM CET Tags:
Posted Wed 06 Apr 2011 08:28:18 PM CEST
10
Posted Wed 06 Apr 2011 08:28:18 PM CEST

[[!inline preprocessing loop detected on archives/2011 at depth 74]]

Posted Wed 06 Apr 2011 08:28:18 PM CEST
11

On November 26, the Qt/KDE Debian team (qkd from now on) released an experimental packaging of KDE SC 4.7. This means a substantial and very much awaited upgrade from the actual version available in sid, 4.6.5 . From what I've been told, it has taken so long because there have been a lot of changes in source code structure (KDE has recently restructured its source modularization) and in packaging itself, and even with recent aditions to the group, the resources in terms of developers is still low. In any case, thanks to them for getting this version out, I really appreciate your hard work.

But because of this lack of resources, they only got out packages for the amd64 architecture, and I'm running i386 at home, so I decided to try and compile it by myself. This is the log of the attempt.

First, I added deb-src http://qt-kde.debian.net/debian experimental-snapshots main to my sources.list and got the code for kde4libs, the package I think is the root of the whole repo[1][2]:

mdione@mustang:~/src/system/debian$ apt-get source kde4libs
Reading package lists... Done
Building dependency tree
Reading state information... Done
NOTICE: 'kde4libs' packaging is maintained in the 'Git' version control system at:
git://git.debian.org/pkg-kde/kde-sc/kde4libs.git
Need to get 12.4 MB of source archives.
Get:1 http://qt-kde.debian.net/debian/ experimental-snapshots/main kde4libs 4:4.7.2-0r3 (dsc) [4,883 B]
Get:2 http://qt-kde.debian.net/debian/ experimental-snapshots/main kde4libs 4:4.7.2-0r3 (tar) [12.1 MB]
Get:3 http://qt-kde.debian.net/debian/ experimental-snapshots/main kde4libs 4:4.7.2-0r3 (diff) [334 kB]
Fetched 12.4 MB in 1min 54s (109 kB/s)
gpgv: Signature made Sat 22 Oct 2011 02:29:45 AM CEST using RSA key ID 73A85F31
gpgv: Can't check signature: public key not found
dpkg-source: warning: failed to verify signature on ./kde4libs_4.7.2-0r3.dsc
dpkg-source: info: extracting kde4libs in kde4libs-4.7.2
dpkg-source: info: unpacking kde4libs_4.7.2.orig.tar.bz2
dpkg-source: info: unpacking kde4libs_4.7.2-0r3.debian.tar.gz
dpkg-source: info: applying kconf_update_migrate_from_kde3_icon_theme.diff
dpkg-source: info: applying add_debian_build_type.diff
dpkg-source: info: applying disable_usr_lib_install_rpath.diff
dpkg-source: info: applying make_libkdeinit4_private.diff
dpkg-source: info: applying default_kde4_xdg_menu_prefix.diff
dpkg-source: info: applying qt4_designer_plugins_path.diff
dpkg-source: info: applying hardcode_ptm_device.diff
dpkg-source: info: applying kfreebsd_support.diff
dpkg-source: info: applying debian_menu.diff
dpkg-source: info: applying findservicebydesktoppath_try_realfilepath.diff
dpkg-source: info: applying findqt4_optional_x11_pthread.diff
dpkg-source: info: applying use_dejavu_as_default_font.diff
dpkg-source: info: applying hack_in_etc_kde4_in_kstandarddirs.diff
dpkg-source: info: applying ld_exclude_libs_qtuitools.diff
dpkg-source: info: applying konsole_kfreebsd_fix.diff
dpkg-source: info: applying hurd_support.diff
dpkg-source: info: applying kfileshare_kdesu_fileshareset.diff
dpkg-source: info: applying relax_plugin_kde_version_check.diff
dpkg-source: info: applying add_dlrestrictions_support.diff
dpkg-source: info: applying findpythonlibrary_layout_deb_on_debian.diff
dpkg-source: info: applying ktar_header_checksum_fix.diff
dpkg-source: info: applying ktar_longlink_length_in_bytes.diff
dpkg-source: info: applying nepomuk_unicode.diff

Nice, now get the build-deps:

mdione@mustang:~/src/system/debian$ sudo apt-get build-dep kde4libs
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Build-Depends dependency for kde4libs cannot be satisfied because candidate version of package shared-desktop-ontologies can't satisfy version requirements

Dang, I have an old shared-desktop-ontologies (v. 0.6.x in sid), and peeking in qkd's repo I find no new version of it. The answer must be in experimental, so I add deb http://ftp.nl.debian.org/debian/ experimental main to my sources.list and run apt-get update. Now it must be just a matter of:

mdione@mustang:~/src/system/debian$ sudo apt-get install -t experimental shared-desktop-ontologies
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
  shared-desktop-ontologies
1 upgraded, 0 newly installed, 0 to remove and 103 not upgraded.
Need to get 129 kB of archives.
After this operation, 4,096 B of additional disk space will be used.
Get:1 http://ftp.nl.debian.org/debian/ experimental/main shared-desktop-ontologies all 0.8.0-1 [129 kB]
Fetched 129 kB in 1s (92.7 kB/s)
Retrieving bug reports... Done
Parsing Found/Fixed information... Done
Reading changelogs... Done
apt-listchanges: Mailing root: apt-listchanges: changelogs for mustang
(Reading database ... 218251 files and directories currently installed.)
Preparing to replace shared-desktop-ontologies 0.6.0-1 (using .../shared-desktop-ontologies_0.8.0-1_all.deb) ...
Unpacking replacement shared-desktop-ontologies ...
Setting up shared-desktop-ontologies (0.8.0-1) ...

And I try again:

mdione@mustang:~/src/system/debian$ sudo apt-get build-dep kde4libs
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  hspell libacl1-dev libattica-dev libattr1-dev libdbusmenu-qt-dev libdlrestrictions-dev libpolkit-qt-1-dev libqca2-dev libstreamanalyzer-dev libstreams-dev
  libutempter-dev pkg-kde-tools
0 upgraded, 12 newly installed, 0 to remove and 8 not upgraded.
Need to get 1,170 kB of archives.
After this operation, 3,551 kB of additional disk space will be used.
Do you want to continue [Y/n]?
[...]

We're set! now to fire the compilation itself. The method I use must not be the best one, but I remember it by heart and works for me :)

mdione@mustang:~/src/system/debian$ cd kde4libs-4.7.2/
mdione@mustang:~/src/system/debian/kde4libs-4.7.2$ nice -n 19 fakeroot debian/rules binary

Notice I use nice to not hog too much my computer and fakeroot. I don't remember why I must use it :| , but I remember I must. Later, asking in #debian-devel I was pointed to dpkg-buildpackage, to which you can give the -j# option, which makes the package to be compiled with # processes in parallel (actually it only works if the compilation is make based, as is in this case), so from now on I'll use that.

The resulting packages are these:

kdelibs5-data_4.7.2-0r3_all.deb
kdelibs5-dbg_4.7.2-0r3_i386.deb
kdelibs5-dev_4.7.2-0r3_i386.deb
kdelibs5-plugins_4.7.2-0r3_i386.deb
kdelibs-bin_4.7.2-0r3_i386.deb
kdoctools_4.7.2-0r3_i386.deb
libkcmutils4_4.7.2-0r3_i386.deb
libkde3support4_4.7.2-0r3_i386.deb
libkdeclarative5_4.7.2-0r3_i386.deb
libkdecore5_4.7.2-0r3_i386.deb
libkdesu5_4.7.2-0r3_i386.deb
libkdeui5_4.7.2-0r3_i386.deb
libkdewebkit5_4.7.2-0r3_i386.deb
libkdnssd4_4.7.2-0r3_i386.deb
libkemoticons4_4.7.2-0r3_i386.deb
libkfile4_4.7.2-0r3_i386.deb
libkhtml5_4.7.2-0r3_i386.deb
libkidletime4_4.7.2-0r3_i386.deb
libkimproxy4_4.7.2-0r3_i386.deb
libkio5_4.7.2-0r3_i386.deb
libkjsapi4_4.7.2-0r3_i386.deb
libkjsembed4_4.7.2-0r3_i386.deb
libkmediaplayer4_4.7.2-0r3_i386.deb
libknewstuff2-4_4.7.2-0r3_i386.deb
libknewstuff3-4_4.7.2-0r3_i386.deb
libknotifyconfig4_4.7.2-0r3_i386.deb
libkntlm4_4.7.2-0r3_i386.deb
libkparts4_4.7.2-0r3_i386.deb
libkprintutils4_4.7.2-0r3_i386.deb
libkpty4_4.7.2-0r3_i386.deb
libkrosscore4_4.7.2-0r3_i386.deb
libkrossui4_4.7.2-0r3_i386.deb
libktexteditor4_4.7.2-0r3_i386.deb
libkunitconversion4_4.7.2-0r3_i386.deb
libkutils4_4.7.2-0r3_i386.deb
libnepomuk4_4.7.2-0r3_i386.deb
libnepomukquery4a_4.7.2-0r3_i386.deb
libnepomukutils4_4.7.2-0r3_i386.deb
libplasma3_4.7.2-0r3_i386.deb
libsolid4_4.7.2-0r3_i386.deb
libthreadweaver4_4.7.2-0r3_i386.deb

Next package is kadebase. Again, we find the missing deps with:

mdione@mustang:~/src/system/debian$ sudo apt-get build-dep kdebase
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Build-Depends dependency for kdebase cannot be satisfied because candidate version of package kde-sc-dev-latest can't satisfy version requirements

The message is simlar to the one about shared-desktop-ontologies, but I have not even the slightest idea which one of the packages above is the one responsible for it, so I'll just install everything, as most probably they're build-deps for most of the remaining packages (it's kde5libs, after all):

mdione@mustang:~/src/system/debian$ sudo dpkg -i *.deb
(Reading database ... 218481 files and directories currently installed.)
[...]

After that I try again, but still the same thing. Poking around I find that a package with that name exists (it's the first time I met him, enchanté), so I get the source in the usual way:

mdione@mustang:~/src/system/debian$ apt-get source meta-kde
Reading package lists... Done
Building dependency tree
Reading state information... Done
Need to get 14.0 kB of source archives.
Get:1 http://qt-kde.debian.net/debian/ experimental-snapshots/main meta-kde 5:71~pre15 (dsc) [2,098 B]
Get:2 http://qt-kde.debian.net/debian/ experimental-snapshots/main meta-kde 5:71~pre15 (tar) [11.9 kB]
Fetched 14.0 kB in 0s (49.3 kB/s)
gpgv: Signature made Mon 21 Nov 2011 09:38:12 PM CET using RSA key ID 73A85F31
gpgv: Can't check signature: public key not found
dpkg-source: warning: failed to verify signature on ./meta-kde_71~pre15.dsc
dpkg-source: info: extracting meta-kde in meta-kde-71~pre15
dpkg-source: info: unpacking meta-kde_71~pre15.tar.gz

And to compile:

mdione@mustang:~/src/system/debian$ ( cd meta-kde-71~pre15 && nice -n 19 dpkg-buildpackage -j3 -b -us -uc )
dpkg-buildpackage: source package meta-kde
dpkg-buildpackage: source version 5:71~pre15
dpkg-buildpackage: source changed by Debian Qt/KDE Maintainers <debian-qt-kde@lists.debian.org>
dpkg-buildpackage: host architecture i386
[...]

That generates several packages, but I just install the one I want:

mdione@mustang:~/src/system/debian$ sudo dpkg -i kde-sc-dev-latest_4.7.2+5.71~pre15_all.deb
Selecting previously unselected package kde-sc-dev-latest.
(Reading database ... 218469 files and directories currently installed.)
Unpacking kde-sc-dev-latest (from kde-sc-dev-latest_4.7.2+5.71~pre15_all.deb) ...
Setting up kde-sc-dev-latest (4:4.7.2+5.71~pre15) ...

Now the build-deps again:

mdione@mustang:~/src/system/debian$ sudo apt-get build-dep kdebase
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  libkatepartinterfaces4
The following NEW packages will be installed:
  libqimageblitz-dev libtidy-dev
0 upgraded, 2 newly installed, 1 to remove and 8 not upgraded.
Need to get 205 kB of archives.
After this operation, 1,932 kB disk space will be freed.
Do you want to continue [Y/n]?
[...]

You'll probably won't notice (as I didn't) that it has removed a package, libkatepartinterfaces. This is because in my system I have the old version of it and there seems to be some kind of conflict somewhere. I just cross my fingers it won't break much[3].

Now get the sources and compile:

mdione@mustang:~/src/system/debian$ apt-get source kdebase
Reading package lists... Done
Building dependency tree
Reading state information... Done
NOTICE: 'kdebase' packaging is maintained in the 'Git' version control system at:
git://git.debian.org/pkg-kde/kde-sc/kdebase.git
Need to get 2,721 kB of source archives.
Get:1 http://qt-kde.debian.net/debian/ experimental-snapshots/main kdebase 4:4.7.1-0r2 (dsc) [3,217 B]
Get:2 http://qt-kde.debian.net/debian/ experimental-snapshots/main kdebase 4:4.7.1-0r2 (tar) [2,685 kB]
Get:3 http://qt-kde.debian.net/debian/ experimental-snapshots/main kdebase 4:4.7.1-0r2 (diff) [33.1 kB]
Fetched 2,721 kB in 26s (105 kB/s)
gpgv: Signature made Tue 27 Sep 2011 09:43:53 PM CEST using RSA key ID 73A85F31
gpgv: Can't check signature: public key not found
dpkg-source: warning: failed to verify signature on ./kdebase_4.7.1-0r2.dsc
dpkg-source: info: extracting kdebase in kdebase-4.7.1
dpkg-source: info: unpacking kdebase_4.7.1.orig.tar.bz2
dpkg-source: info: unpacking kdebase_4.7.1-0r2.debian.tar.gz
dpkg-source: info: applying enable_debianabimanager.diff
dpkg-source: info: applying enable_dlrestrictions.diff

mdione@mustang:~/src/system/debian$ ( cd kdebase-4.7.1 && nice -n 19 dpkg-buildpackage -j3 -b -us -uc )
[...]

Installing the resulting packages is another history, as some of them depend on kde-runtime, so it's better if you compile kde-runtime before[4]. Trying to install its build-deps, you realize taht you actually need to compile soprano first.

You get the idea. Now I'll just show the generated packages I installed by hand after each compilation, but only those that suited my needs:

soprano:

mdione@mustang:~/src/system/debian$ sudo dpkg -i \
soprano-daemon_2.7.3+dfsg.1-0r0_i386.deb \
libsoprano4_2.7.3+dfsg.1-0r0_i386.deb \
libsoprano-dev_2.7.3+dfsg.1-0r0_i386.deb

kde-runtime:

mdione@mustang:~/src/system/debian$ sudo dpkg -i \
kdebase-runtime_4.7.2-0r3_all.deb \
khelpcenter4_4.7.2-0r3_i386.deb \
kde-config-phonon-xine_4.7.2-0r3_i386.deb \
kde-runtime-data_4.7.2-0r3_all.deb \
kde-runtime_4.7.2-0r3_i386.deb \
plasma-scriptengine-javascript_4.7.2-0r3_i386.deb

While installing kde-runtime-data I had to remove kdebase-runtime-data:

[...]
Selecting previously unselected package kde-runtime-data.
dpkg: regarding kde-runtime-data_4.7.2-0r3_all.deb containing kde-runtime-data:
 kde-runtime-data breaks kdebase-runtime-data (<< 4:4.7.2)
  kdebase-runtime-data (version 4:4.6.5-1) is present and installed.
[...]

mdione@mustang:~/src/system/debian$ sudo dpkg --remove kdebase-runtime-data
(Reading database ... 218612 files and directories currently installed.)
Removing kdebase-runtime-data ...
[...]

kdebase:

mdione@mustang:~/src/system/debian$ sudo dpkg -i \
dolphin_4.7.1-0r2_i386.deb \
kdebase-bin_4.7.1-0r2_i386.deb \
kdebase-data_4.7.1-0r2_all.deb \
kdepasswd_4.7.1-0r2_i386.deb \
konq-plugins_4.7.1-0r2_i386.deb \
konqueror_4.7.1-0r2_i386.deb \
konqueror-nsplugins_4.7.1-0r2_i386.deb \
libkonq5abi1_4.7.1-0r2_i386.deb \
libkonq-common_4.7.1-0r2_i386.deb \
plasma-widget-folderview_4.7.1-0r2_i386.deb

While trying to get akonadi I got this error message:

Failed to fetch http://qt-kde.debian.net/debian/pool/main/a/akonadi/akonadi_1.6.2-0r1.dsc  Hash Sum mismatch

Checking the MD5, SHA1 and SHA256 checksums and comparing to the ones in the .dsc file revealed no difference. lisandro told me to use dget instead:

mdione@mustang:~/src/system/debian$ dget http://qt-kde.debian.net/debian/pool/main/a/akonadi/akonadi_1.6.2-0r1.dsc
dget: retrieving http://qt-kde.debian.net/debian/pool/main/a/akonadi/akonadi_1.6.2-0r1.dsc
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2620  100  2620    0     0   6842      0 --:--:-- --:--:-- --:--:--  8161
dget: using existing akonadi_1.6.2.orig.tar.bz2
dget: using existing akonadi_1.6.2-0r1.debian.tar.gz
akonadi_1.6.2-0r1.dsc:
dscverify: akonadi_1.6.2-0r1.dsc failed signature check:
gpg: Signature made Thu 03 Nov 2011 08:55:52 AM CET using RSA key ID 73A85F31
gpg: Can't check signature: public key not found
Validation FAILED!!

Unpacking by hand:

mdione@mustang:~/src/system/debian$ dpkg-source -x akonadi_1.6.2-0r1.dsc
gpgv: Signature made Thu 03 Nov 2011 08:55:52 AM CET using RSA key ID 73A85F31
gpgv: Can't check signature: public key not found
dpkg-source: warning: failed to verify signature on ./akonadi_1.6.2-0r1.dsc
dpkg-source: info: extracting akonadi in akonadi-1.6.2
dpkg-source: info: unpacking akonadi_1.6.2.orig.tar.bz2
dpkg-source: info: unpacking akonadi_1.6.2-0r1.debian.tar.gz
dpkg-source: info: applying x11_not_required.diff

akonadi:

mdione@mustang:~/src/system/debian$ sudo dpkg -i \
akonadi-server_1.6.2-0r1_i386.deb \
akonadi-backend-sqlite_1.6.2-0r1_i386.deb \
libakonadi-dev_1.6.2-0r1_i386.deb \
libakonadiprotocolinternals1_1.6.2-0r1_i386.deb

Here I cheated: I really didn't want to spend 10 minutes of iterative attempts to find the minimal set of lib packages to install:

libkdepim:

mdione@mustang:~/src/system/debian$ sudo dpkg -i \
kdepimlibs5-dev_4.7.2-0r1_i386.deb \
kdepimlibs-kio-plugins_4.7.2-0r1_i386.deb \
lib*.deb

kde-wallpapers:

mdione@mustang:~/src/system/debian$ sudo dpkg -i \
kde-wallpapers-default_4.7.2-0r0_all.deb

kde-workspace:

sudo dpkg -i --auto-deconfigure \
systemsettings_4.7.2-0r7_i386.deb \
plasma-desktop_4.7.2-0r7_i386.deb \
plasma-scriptengine-python_4.7.2-0r7_all.deb \
plasma-widgets-workspace_4.7.2-0r7_i386.deb \
plasma-dataengines-workspace_4.7.2-0r7_i386.deb \
klipper_4.7.2-0r7_i386.deb \
kdm_4.7.2-0r7_i386.deb \
kde-workspace-bin_4.7.2-0r7_i386.deb \
kde-workspace-data_4.7.2-0r7_all.deb \
kde-window-manager_4.7.2-0r7_i386.deb \
kdebase-workspace_4.7.2-0r7_all.deb \
kdebase-workspace-bin_4.7.2-0r7_all.deb \
libkworkspace4_4.7.2-0r7_i386.deb \
kde-workspace-kgreet-plugins_4.7.2-0r7_i386.deb \
kde-style-oxygen_4.7.2-0r7_i386.deb \
libkdecorations4_4.7.2-0r7_i386.deb \
libkephal4abi1_4.7.2-0r7_i386.deb \
libkwineffects1abi2_4.7.2-0r7_i386.deb \
libkworkspace4_4.7.2-0r7_i386.deb \
libplasmagenericshell4_4.7.2-0r7_i386.deb \
libtaskmanager4abi2_4.7.2-0r7_i386.deb \
libplasmaclock4abi2_4.7.2-0r7_i386.deb \
libksgrd4_4.7.2-0r7_i386.deb \
libplasma-geolocation-interface4_4.7.2-0r7_i386.deb \
libsolidcontrol4abi2_4.7.2-0r7_i386.deb \
libprocesscore4abi1_4.7.2-0r7_i386.deb \
libweather-ion6_4.7.2-0r7_i386.deb \
libkscreensaver5_4.7.2-0r7_i386.deb \
libprocessui4a_4.7.2-0r7_i386.deb \
libsolidcontrolifaces4abi2_4.7.2-0r7_i386.deb \
kde-workspace_4.7.2-0r7_all.deb \
ksysguard_4.7.2-0r7_i386.deb \
freespacenotifier_4.7.2-0r7_i386.deb \
libksignalplotter4_4.7.2-0r7_i386.deb \ 
ksysguardd_4.7.2-0r7_i386.deb

I'll just finish this post here. The rest is just more of the same. The final list of packages a compile in right order is:

kde4libs
meta-kde
soprano
kde-runtime
kdebase
akonadi
prison
kdepimlibs
kde-wallpapers
kde-workspace

All in all, this took some 10 hours of finding deps, compiling and installing, and 6.4 GiB between original .tar.gz files, compilation dirs and generated packages.


[1] It's nice that I don't need to compile libqt4*, those take ages[5].

[2] Later lisandro from qkd pointed me to the dependency graph that confirms my guess.

[3] I did all this while running a KDE SC 4.6.5 session, and even writing this post in kate. Thanks to Linux and Debian not much actually broke, I only lost the ability to browse with konqueror after I installed the first batch of packages because KIO could not instantiate an HTTP/HTTPS ioslave anymore.

[4] Note that the dependency graph two notes above is actually a build dependency graph.

[5] Not that this compilation was fast. kdelibs without parallelism took some 3h and kdepimlibs and kde-workspace are two huge beasts.


debian kde

Posted Sun 27 Nov 2011 11:18:21 PM CET Tags:
Posted Wed 06 Apr 2011 08:28:18 PM CEST
08
Posted Wed 06 Apr 2011 08:28:18 PM CEST
06
Posted Wed 06 Apr 2011 08:28:18 PM CEST
07
Posted Wed 06 Apr 2011 08:28:18 PM CEST
05
Posted Wed 06 Apr 2011 08:28:18 PM CEST