More than a year ago I bought a FairPhone2 because that's what a geek with social responsible inclinations (and some hardware hacking) does. It came with Android Marshmallow (aka 6), but last December I bit the bullet and upgraded to Nougat (7.1). Also, as any megacorporation paranoid geek would do, I don't have a Google account (even when 90%+ of my mails ends up in their humongous belly, but who uses mail nowadays anyways...), so I have been using it with F-Droid and Yalp Store.

The upgrade went smoothly, and almost right after it I was poking aroung the Yalp Store when I saw several system updates, including the Android System WebView. This component is the one responsible of showing web content in your apps, and, believe me, you use it more than you think. The new Android came with version 67.0.3396.87, and Yalp Store was offering v71.0.3578.99, so I didn't think about it and installed the further upgrade, along with most of the apps that I knew were not installed through F-Droid[1]. There's also the fact that since Nougat, ASWB is deprecated in favor of embracing Chrome, but I have it disabled in my phone, just like most of the Google Apps (including Google Play Services).

The issue came when I tried to launch the official Selfoss reader. The list of articles worked fine, but trying to read one made the app crash. Even worse were the two homebanking apps I have: they didn't even show their main screen.

Thanks to a small troubleshooting session with jochensp in the #fairphone ICR channel, we found out that in fact it was a ASWB problem (hint: use adb logcat). Once more I had to use one of those don't-know-how-shady-it-is APK mirror sites (I used APKMirror, if you're curious, but don't blame me if the soft you install from there comes with all kinds of troyans).

The first thing I tried was to downgrade to the original version, so I downloaded the closest one I found (they didn't have the exact version, which makes me wonder how ofthen do they scan apps for upgrades), but downgrades don't work, even with adb install -r -d. For some reason, the same site offered a newer version than Yalp Store (72.0.3626.53, which I just found out it's a beta version!), so I upgraded (manual download + install) and that fixed it!

[1] There's an issue where Yalp Store tries to manage the apps installed via F-Droid by offering the versions available in Google Play, but most if the time it doesn't work because F-Droid recompiles everything and I think the keys are different. I hadn't compared if the versions offered by YS are really newer than those in FD).

android fairphone

Posted jue 24 ene 2019 10:50:08 CET Tags: android

Disclaimer: rough language ahead.

So we bought an Acer Iconia A200. Reasons are not important, and I was hesitating about an Android device. But features are good, I can't indefenetely deny that Android is the option to Apple. A clear case of choosing the least of all bads.

Today it came. I did the clasic deboxing photo shooting, and that was that. I opted out from all the options that were presented to me that meant telling google what I was doing or where, got the wifi working, upgraded to 4.0.3, and started testing.

First things first, play with the widgets. There's a 4 square panel one that uses the top 2 panels for showing the hour, the lower right one for the date (which is not a shortcut to the agenda... WTF?) and the lower left one has a small sun-and-clouds-and-sky image that suggested a weather applet. So I hit it and it tells me that because I opted out from using wifi spots as alternative ways to find the device on the surface of our unique world planet, I can't simply use the applet. I mean, there is no way to tell the applet, "hey, you know what? for no particular reason I want to know the weather in, let's say, Saint-Symphorien-du-Mahun, France". I can either enable the "network Location Service" (or something like that, I set the device in spanish) or not use it.

So I forget about it. Next step, install a real map application. For that, of course, I had to yield a Google account. I have one, or several, laying around, so I use one of those to let Google Play install things. That's already shitty. But then what does it do? It also uses that account for configuring the gmail client. Who the fuck told it to do it? I was expecting to set it up to my girlfiend's account. I hope I will be able to do it later. Do these people ever thought that a tablet could be used by more than one person?[1] Also, account sync got activated. I had to go to the settings and turn everything off. Still, the gmail app keeps being linked to that Google account.

At some point, too, a multimedia server went on. WTF from? Who set it up? Why is it enabled by default? Who decides these things? Ok, this is getting close to a rant. I see a tag comming. It also comes with an app that shows in the "task list" (there is no concept of app in Android) which is actually a "apps you last used up to certain value of «last»" or something like that; the app is called VirusScan. When I try to activate it, it gives me back to the home screen. Go figure.

So back to Google Play. I had already skiped the yielding of credit card info. Now it's time to opt out of being spammed about news and offers from Google Play. I think I see a pattern here, but at least some things are being asked, "can I, pretty pretty please?". Most of the time, my answer will be "No!", sometimes followed by "Go to your bedroom, no dinner or dessert!".

I select Maverick, which I remember from the time I had a Motorola Droid from work. I choose it because it can handle offline maps. This tablet has a GPS, but it does not have GPRS. The rights it asks for are a little permissive (for instance, taking photos and vodeos), but I hope it's just a matter of not using those features and that's it.

I fire it up. I choose OSM maps and it works just fine. The guide for the offline maps is nice, not fancy, but it doesn't have to be. It only needs an µSD, VFAT formatted, which I don't have (I lost a 8Gb recently with a lot of wine labels from wines that I liked. It's hard to live in a country where wines sold by features completely orthogonal to what you're used to). Makes me wonder why I bought a 32GiB device. I'll have to fix that soon.

For some reason I try the image viewer. Of course it went to the picassa account associated to that Google account. That was really a bad move. Even the calendar is tied to that account. I'll have to ckeck how to reset that shit. Probably I'll have to reset it to shipping state. Not that I did a lot in it. Also, I missed some privacy settings, like the backup in Google servers, which at least deactivating them erases that data from Google servers... or so they say >:-)

Android apart, the tablet is nice. Not very heavy, looks good, responsive, can act as USB master, has internal GPS as I already said, and whatever else. I will be actively looking for an app that lets me embed another two, so I can use it as a GPS and media center in the car, which is the primary goal that I had for my non-charging-from-USB-anymore N900 :(

So lasts test: The GPS didn't got a good signal in my balcony (My god it's cold outside. What's the teperature? I don't know[2], the weather applet does not work, the second one I put in the home screen reset itself to NYC [!??!] and I hadn't try the third I installed), but at least it found it fast. The first guess put me some 80 meters from my actual position, but it slowly got it right.

Now audio: let's see how well it plays MP3s, OGGs and FLACs. I connect it via USB and simultaneously I get two suprises: a) the tablet does not seem to charge via USB. Now, sincerely, WTF? I corroborate this by plugging it to a wall USB charger. Nothing. I should had suspected something when I saw at first that the wallwart has a cilindrical connector. And b) it announces via MTP device, not an USB mass storage as others do, with an option of PTP for the photos. Amarok can handle MTP devices, but then Amarok tries to bee too clever and handling a big-ish collection with lots of missing data and inconsistencies is a nightmare, so I skip it. mtpfs is somehow broken in Debian Sid, and I don't have the willpower to compile kio-mtp.

So that's it: I have a nice 11" tablet with a operating system I knew I hate, which in spite it has GPS can't charge via USB and is not even a USB mass storage, when it can act as a USB master. Probably the bottom line is: I should have investigated better the device before buying it. If only my N900 would charge via USB again[5]...

[1] Ok, yes, it hs support for several Google accounts, or at least Gmail ones.

[2] It's 15°, according to the weather applet in my notebook. How difficult would it be to install GNU/Linux on this beauty[3]?

[3] Just kidding[4].

[4] Actully, not so much. I mean, the device runs an Google/Linux system.

[5] For EUR 300 I probably could buy a not refurbished-but-sold-as-new one. But then, I'm not the only intended user for this device.

rants android google

Posted sáb 23 mar 2013 23:47:50 CET Tags: android

Last year and a half I was working in research. This position was about, among other things, to port a programming language (two, actually, Hop and Bigloo) to the Antroid platform. I already had written something about it, but this time I want to show my high level impressions of the platform. What follows is part of a report I wrote at the end of that job, which includes the things I wanted to say.

The Android port can be viewed as four separate sub-tasks. Hop is software developed in the Scheme language; more particularly, it must be compiled with the Bigloo Scheme compiler, which in turn uses gcc for the final compilation. That means that we also needed to port Bigloo first to the platform, not because we were planning to use it in the platform, but because we need the Bigloo runtime libraries ported to Android, as Hop and any other program compiled with Bigloo uses them. The other three subtasks, which are discussed later, are porting Hop itself; developing libraries to access devices and other features present in the platform; and, we'll see later the reasons, make the port work with threads.

When we started to investigate how to port native code to the platform we found that there wasn't much support. At fisrt the only documentation we could find was blog posts of people trying to do it by hand. They were using the compiler provided in Android's source code to compile static binaries that could be run on the platform. Because Bigloo uses dinamic libraries to implement platform dependent code and modules, we aimed to find a way to compile things dinamically. After 3 or 4 weeks we found a wrapper written in Ruby that managed all the details of calling gcc with the proper arguments. With this we should be able to port anything that uses gcc as the compiler, just like Bigloo does. At the same time, the first version of Android's NDK (Native Development Kit) appeared, but it wasn't easy to integrate in our build system.

(Note: Actually I think most of the problems we faced doing this port stem from this. The NDK forces you to write a new set of Makefiles, but our hand-made build system and build hierarchy made such an effort quite big. Also, that mean supporting a parallel build system, while it should not be so crazy to spect a cleaner way to integrate the toolchain into an existing build system, not only in hand-made like in this case, but also the most common ones, like autotools, cmake, etc.)

Even having the proper compiler, we found several obstacles related to the platform itself. First of all, Bigloo relies heavily on the C and pthread libraries to implement lowlevel functionality. Bigloo can use both glibc, GNU's implementation, or µlibc, an implementation aimed for embedded aplications. Bigloo also relies on Boehm's Garbage Collector (GC) for its memory management. The C library implementation in Android is not the glibc or the µlibc, but an implementation developed by Google for the platform, called Bionic. This version of the C library is tailored to the platform's need, with little to no regards to native application development.

The first problem we found is that GC compiled fine with Bionic, but the apllications that used GC did not link: there was a missing symbol that normally is defined in the libc, but that Bionic did not. We tried cooperating with the GC developers, we tried inspecting a Mono port to Android, given that this project also uses GC, trying to find a solution that could be useful for everyone, but at the end we just patched our sources to fake such symbol with a value that remotely made sense.

We also found that Bionic's implementation of pthreads is not only incomplete, but also has some glitches. For instance, in our build system, we test the existence of a function like everybody else: we compile a small test program wich uses it. With this method we found at least one function that is declared but never defined. That means that Bionic declares that the function exists, but then it never implements it. Another example is the declaration and definition of a function, but the lack of definition of constants normally used when calling this function.

Also, because most of the tests also must be executed to inform about the peculiarities of each implementation, we had to change our build system to be able to execute the produced binaries in the Android emulator.

Google also decided to implement their own set of tools, again, trimmed down to the needs of the platform, instead of using old and proven versions, like Busybox. This means that some tools behave differently, with no documentation about it, so we mostly had to work around this differences everytime a new one apperared.

All in all, we spent two and a half months just getting Bigloo to run in Android, dismissing the problem that Boehm's GC, using its own build system, detected that the compiler declared to not support threads, and refused to compile with threads enabled. This meant that Bigloo itself could not be compiled with pthreads support.

With this caveat in mind, we tackled the second subtask, porting Hop itself. This still raised problems with the peculiarities of the platform. We quickly found that the dinamic linker wasn't honoring the LD_LIBRARY_PATH environment variable, which we were trying to use to tell the system where to find the dynamic libraries.

The Android platform installs new software using a package manager. The package manager creates a directory in the SD card that it's only writable by the applilcation being installed. Within this directory the installer puts the libraries declared in the package. Bigloo, besides the dinamic libraries, requieres some aditional files that initialize the global objects. This files are not extracted by the installer, so we had to make a frontend in Java that opens the package and extract them by hand. But the installer creates the directory for the libraries in such a way that the application later cannot write in it.

Also, we found that the dinamic linker works for libraries linked at runtime, but does not for dlopen()'ing them, so we also had to rewrite a great part of our build system for both Bigloo and Hop to produce static libraries and binaries. This also needed disabling the dynamic loading of libraries, and with them, their initialization, so we had to initialize them by hand.

To add more unsuspected work, the Android package builder, provided with the SDK, ignores hidden files, which Bigloo uses to map Scheme module names to dynamic libraries. We had to work around this feature in the unpacking algorithm.

Then we moved to improve the friendliness of the frontend. So far, we could install Hop in the platform, either in a phone or in the emulator, but we could only run it in the emulator, because we were using a shell that runs as root on the emulator, but that runs as a user in a real device. This user, for the reasons given above, cannot even get into Hop's install dir. Even when Android has a component interface that allows applications to use components from other apps, none of the terminal apps we found at that time declared the terminal itself as a reusable component. We decided to use the code from the most popular one, which was based on a demo available on Android's source code, but not installed in actual devices. We had to copy the source code and trimm it down to our needs.

Having a more or less usable Hop package for Android, we decided to try and fix the issue we mentioned before: GC didn't compile with threads enabled. This means that we can't use the pthreads library, which is very useful for Hop. Hop uses threads to attend several requests at the same time. Bigloo implements two threads APIs, one based on pthreads and another which implements fair threads. Hop is able to use 5 different request schedulers, but works better with the one based on pthreads.

For these reasons we decided to focus in getting GC to use threads with the Android platform. GC's build system tests the existence of a threading platform checking the thread model declared by gcc. The gcc version provided with Android's SDK declares to have a 'single thread model', but we couldn't find what does this mean in terms of the code produced by gcc or how this could affect to GC's execution.

(Note: we didn't manage to make GC compile with threads.)

With a threadless Hop running, we had to add code to the server so we could talk between the server and the frontend while at the same time it is attending the requests from a web client. After several attempts to attack this problem, we decided that the best solution was to make this interface another service served by Hop. This meant less modifications to Hop itself, but a bigger one to the frontend we already had.

During these changes we found out a problem with JNI. The terminal component we imported into our code uses a small C library for executing the application inside (normaly a shell, in the original code, but Hop in our case) which is accessed from Java using JNI. The original Term application exported this class as, but our copy exported it as fr.inria.hop.Exec. Even with this namespace difference JNI got confused and tried to use the Exec class from the original Term app. This is just another example how the platform is hard to work with. We found that the community support is more centered around Java and that very few people know about JNI, the NDK or any other native related technologies. We couldn't find an answer to this problem, so we worked around this by renaming the class.

So that's it. I can provide all the technical details for most the assertions I postulated above, but that would make this post unreadbal for its length. If you have any question about them, just conact me.


Posted sáb 26 mar 2011 00:12:39 CET Tags: android

In my daily work (the one that pays the bills) I've been burdened with the responsability to port two projects to the Android platform: hop, a programming language for the web 2.0/application server writen in Scheme. For that I would need to also port bigloo, a Scheme compiler. The goal is to be able to run hop apps in the phone itself.

I really didn't know much about the platform then, only that probably it would be a fun platform to have in your phone, being GNU/Linux based, open source and all. There was also the promise to play with these phones; being somewhat a technophile, that was definetely a plus. With a smile in my face, I cracked may hands, sat down and started... reading.

The first thing one learns when reading about developing for the platform is that the language is Java. There's a complete SDK for developing them. If you want to port some C/C++ library, you can do so with the NDK, and that's pretty much it. Now, from my point of view, this is very restricting: you cannot choose your own programming language. Even if I like the language a bit (but not its std library or the way that the language is driven), I prefer Python or whatever suits better the problem. But then, I understand Android/Google/OHA needs to be control freaks with the apps running on the phone, because there are limited resources and you cannot leave an application open forever. So, they devised their component lifecycle, which reminds me of OSGi, made it in Java, made all the tools and that was it.

But in my case I needed a native port, and that's for two reasons. First, even if bigloo can produce Java bytecode, and even when I've been told that Dalvik, their JVM implementation, can run Java bytecode[1], the result is not fast enough, being a both bigloo and the JVM in turn problem. Certainly bigloo's Java output might not be the best, but also the JVMs we tried were too heavy[2]. The second and most important reason is that if we did a Java port, we would be tied to the component lifecycle I mentioned above. We need a hop daemon running in order to be able to run its apps. If this deamon goes off by the platform's request[3], there's no way to awake it again when a (web) client tries to make a request[4], so the request would fail and would make hop useless. Hence, a native port[5].

I have to say in everybody's defense that this is my first contact with toolchains and cross compiling.

Given that native porting is not supported, I went off searching. The first posts talked about static linking. This guy even pointed to a first wrapper for the precompiled toolchain and gave some details on some CFLAGS to set. But static linking was not an option for us. hop uses bigloo's runtime libs as dynlibs[6].

Then I hit Rob Savoye's efforts to port gnash. There I started to fully understand the Android platform: «while the C compiler is ok, the C++ compiler is crippled. No iostream support in libstdc++, no locales, etc... Then to make it more fun they use their own libc called Bionic, which is trimmed down for embedded devices». He has more details on C++ compilation because he needed it (we don't... yet).

So I turned to what I knew, if it only was its name: emdebian. I was till a litle hazy about what cross compiling meant, so I started to make questions. Thing is, emdebian is coupled with Debian (it only makes sense), so the first step would have been creating a Bionic package. Of course, this was way beyond my requirements and abilities. In that channel I was also pointed to a post by Harald Welte, which in turn links to a talk called "Android Mythbusters", where we learn also that the linking is not standard [s.6], some hacks in the device support [s.7], hardcoded hotplug [s.8], no tslib support [s.12], hardcoded limits [s.13], and a lot more of calamities, including bad community support [s.18], and its peak, the CyanogenMod-gate.

All that analisys is nice, but I was still stuck with no porting system. My boss suggested scratchbox, but just like emdebian it only has support for (e)glibc and uClibc. It is possible to add support for other platforms, but their script didn't work and my feeble attempts led me no closer to a solution.

Then with a strike of luck I found Joel Reymont's Android Notes, who not only explains a little more the platform, but also has a example Makefile for compiling executables for Android. And then, just a few minutes later, Takuya Murakami's wrapper, which, together with the prebuilt toolchain in Android's source code, made most of it. I finally got a simple Hello World! dynamic executable running in the emulator.

From here all was mostly downhill. The last stone in the path was a linking problem. bigloo uses Boehm's gc, a renowed garbage collector used by several proyects, including mono and ocalm. gc needs a symbol called __stack_base__ defined, which, from what I could find looking, used to be defined in crt0.o, one of the parts that the compiler normally attaches to executables to make them so. The problem is that this file is not in the SDK or Android's source code/prebuilt toolchain. To be fair, that file is not to be found in a fairly new distro. What I found was a reverse engineered crt0.S [7], but it doesn't define the symbol I needed, and in any case I already got running executables in the emulator. In the other hand, naoya_t seems to have hit the same problem, but he writes in japanese and I couldn't get hold of anyone to translate it for me[8].

Long story short, I made a hack to make the linking happy. bigloo finished compiling, I pushed it to the emulator and... it worked!

Some notes about this port: with a little bit of luck it will be officially announced next week, even if it works with a hack. Ivan Maidanski assures me that it should not be nessesary, so probably there's an error on our side when configuring gc; but then again, I spent more that 2 weeks with that problem alone and I followed all the suggestions they gave us[9], so I'm pretty sure is something in the middle. Also, mono has been ported to Android, and even if I read references to the NDK, I've been assured that it's a native port. If they could port it without hacks, so should we[10].

Lastly, some notes about the crippled GNU/Linux running behing the Android platform: besides all the low level stuff mentioned by others cited above, there are no cp or tee; ls does not support the -R option, mkdir does not support the -p option, and even when one of these tools don't recognize an option, instead of barking, it happily accepts it as another file to treat. so, mkdir -p bongs/bonga creates a -p dir and barfs on bongs/bonga. What it is there, yes, are dd and strace! Not very surprisingly, there's no /etc/passwd, but then the system somehow knows several others users besides root and shell, so I guess they're hardcoded somewhere. It's really impressive the effort they put to cripple this thing. Most probably they're using a tool like BusyBox[11], and if so, they are only saving some directory entries in the case of cp or tee and wasting time in the other cases.

All in all, I'm dissapointed by the platform. I understand that what we're trying to do is not supported, and that alone tells me that I don't want an Android device until this has changed, if it ever does. Probably the fact that Android is (now) open source and that the Cyanogengate led to the formation of «the Open Android Alliance (not to be confused with the Open Handset Alliance) an organization whose stated goal is to distribute "a Flavor of Android that is fully customizable and does not rely on Google or other copyrights» gives the platform some hope.

Meanwile I will prefer Maemo (who at least looks more open), specially now that Qt will be the default toolkit. I'll also keep an eye on Bada, Samsung's new platform. There are not many details about this last one, except that they have their own toolkit in C++, and from the API's documentation, it looks like it's a very thorough one.


[1] but then the WikiPedia page says you have to convert the .class file in a .dex file using a tool called dx, and that «It uses its own bytecode, not Java bytecode».

[2] but then Dalvik is supposed to be fast enough for the platform.

[3] «If an activity is paused [not having the focus] or stopped [run at some time but obscured completely], the system can drop it from memory either by asking it to finish (calling its finish() method), or simply killing its process».

[4] because instead of communicating via Android's component system, the communication is via a simple TCP socket.

[5] but probably we'll have to make at least a Java wrapper to start it. I'm even pondering making a "hop apps" app which would be a WebKit widget and a hop server runnning in the background.

[6] or in any case I would need to compile bigloo first dynamically to get the libs to compile hop staticly.

[7] that blog has some more details about porting to Android.

[8] a chineese friend could read some, but his translation didn't make more sense than [google's]( layout=1&eotf=1& 2F1246689974&sl=ja&tl=en). babelfish just timed out on that page.

[9] the ones that made sense, anyways.

[10] when I was looking for anyone involved in mono's irc channel, I was told to look in the code. A simple “noone involved is here, try reaching them by mail” would have sufficed. Way to go, mono community!

[11] and if they're not, then I don't know what's going on in their heads.

Posted mié 24 feb 2010 23:09:44 CET Tags: android

It took me several days to figure it out. I read tons of pages of people (trying?) to do the same thing, and among them I found an answer that works. This is the short, 'to the foot'[1] version. Later will come a longer post talking more in deep about it.

You just need two things: Android's source code, which comes with a precompiled toolchain, and this wrapper made in Ruby. With these two think you can then simply export CC=/usr/local/bin/droid-gcc and then compile your app with it. You could also download the Android SDK to test the result.

Now, a couple of comments. Android's source code is around 5GiB, so be patient with it. When I was trying to get it their git server wasn't working properly, so it took me like 4 days to do it. I tried to download only the toolchain source code and work from there, but I couldn't make it work. Also, not everything seems to be portable. I had problems porting Boehm's gc, but then the androidmono project did it, so I'll have to take a look at it. But for now that's it.


[1] changos, it's difficult to translate some expressions. This one is related to the magic that some football players can make, which is deliver a ball right 'to the foot' of a teammate, specially when is a long pass.

Posted mié 27 ene 2010 23:55:55 CET Tags: android