Porting Android to a new device

Porting Android to a new device

by Peter McDermott

Executive summary

If you're having trouble estimating the time and complexity of porting Google's Android mobile platform to your embedded device, our experience porting Android to a Nokia N810 Internet Tablet may provide you with useful information.

Android overview

Android is a mobile phone software platform created by Google. It is based on the Linux kernel, a number of software libraries written in 'C,' a Java-like virtual machine for running application code, and a full suite of mobile phone applications.

Android costs nothing and the source code is freely available. Its license terms (see the Apache license link at the end) are commercial-friendly, saying, in essence, "Do what you like with it, just don't sue us if something goes wrong." The one exception to this rule is the Linux kernel, which is licensed under the GNU Public License. Because of this, manufacturers must release their device's Linux kernel source code after product shipment.

Taiwan's HTC shipped the first Android phone, the G1, in October 2008. Other mobile manufacturers, including Motorola, LG, and Samsung are all reportedly creating Android-based phones. Additionally, from NthCode's home base in China, we see many startups and smaller manufacturers leveraging Android in an attempt to gain a foothold in the global smartphone market.


T-Mobile G1, the first Android phone

The Android platform provides all the services one would expect in a mobile phone, including phone calling, networking, messaging, and multimedia. Our analysis of the source code so far is that these services are complete and built on top of an excellent architecture, but do not implement every last feature found in modern mobile phones. This is the 1.0 release, after all.

For application developers, Android builds on top of the popular open-source Eclipse integrated development environment. Developers can use Eclipse to write Java applications, which are then compiled to Java bytecode before being translated for running on Android's Dalvik Virtual Machine (VM).

Google claims their Dalvik VM runs more efficiently in terms of speed and memory tradeoffs than Java -- slyly suggesting that Google wrote their own VM for performance reasons. However, while we haven't seen supporting data, we suppose Google really created Dalvik to escape from needing to cede control or pay a license fee to Sun, the creator and steward of the Java platform.

By translating from Java to Dalvik bytecode, Google is able to leverage the mature tools and large existing community of Java developers. The downside is that developers will need to take the additional step of running the Google translation tool on their compiled Java code before their application will run on an Android phone (Google automates this step in their Android Eclipse plug-in, however).


Android emulator screenshot

Google also provides an Android emulator. The emulator is a virtualized ARM microprocessor that runs the same system code -- and almost exactly the same Linux kernel -- that runs on a device. Google provides an Eclipse add-on so that Android programs can be compiled and debugged in Eclipse while running in their emulator. The Android emulator is as complete an emulator as we have ever seen -- just make sure your developers have enough RAM in their PCs to be able to run Eclipse and the emulator together.

So what's in it for Google?

Google has given mobile phone manufacturers everything they want -- a free, open, mobile operating system -- and, in return, only asked that the devices they create can run Android applications. We can only speculate that Google expects to eventually earn more from consumers using Google services on Android devices than it costs Google to develop and maintain Android.

Overview of the Nokia N810

We work with embedded Linux systems at NthCode, so we thought it would be a fun challenge (and would help with a project we're working on) if we could port Android to an existing Linux device.

Luckily, we already had two Nokia N810 Internet Tablets in the office. The N810 is a handheld mobile device that comes with a high-resolution 4.1-inch touch screen and a slide-out QWERTY keyboard. The N810 has a 400 MHz Texas Instruments OMAP 2420 CPU, 128MB RAM, and WiFi networking. The one thing the N810 can't do is make phone calls -- this is why Nokia markets it as an Internet Tablet that can surf the web, rather than a phone. At the time of this writing, the N810 costs about USD $425.

An active community of enthusiasts enjoys hacking the N810's Linux-based software. While some of those enthusiasts had already ported a pre-release version of the Android SDK to the N810, we discovered that no one had yet ported the Android 1.0 release to it. So we decided to see how long it would take us, and what we could learn in the process.

Development steps

As mentioned earlier, the Android system software runs on top of a Linux kernel. This Linux kernel provides services to applications -- such as file access, process scheduling, and inter-process communication.

Google made a number of modifications to the standard Linux kernel for Android. Likewise, Nokia modified the standard Linux kernel to support their hardware, such as the keypad, touch screen, and file system.

We quickly discovered that Nokia's N810 changes are made against an earlier 2.6.21 Linux kernel. And, unlike earlier Android prereleases, the Android 1.0 modifications are made against a later 2.6.25 Linux kernel. We investigated and discovered that between these two versions, a year had elapsed and the Linux community had made thousands of changes to kernel source code.

So, to make both Nokia's and Google's modifications work together on the N810 we either needed to port the N810 changes to work on the newer 2.6.25 Linux kernel, or port the Android changes to the N810's earlier 2.6.21 Linux kernel.

What did Google change in the kernel?

We checked the differences between the Android kernel and the standard Linux kernel and found that Google had changed 75 files and added an additional 88. We have prepared an annotated list of changed files at the end of this document, and a brief summary here.

• Goldfish -- 44 Files -- The Android emulator runs a virtual CPU that Google calls Goldfish. Goldfish executes ARM926T instructions and has hooks for input and output -- such as reading key presses from or displaying video output in the emulator.
  
  These interfaces are implemented in files specific to the Goldfish emulator and will not be compiled into a kernel that runs on real devices. So we safely ignored these files in our work.
  
• YAFFS2 -- 35 Files -- Unlike PCs, which store files on disks, mobile phones store files in sold-state flash memory chips. The HTC G1 uses NAND flash, a type of flash memory that is becoming more popular due to its combination of high density and low cost.
  
  YAFFS2 is an acronym for "Yet Another Flash File System, 2nd edition." It provides a high-performance interface between the Linux kernel and NAND flash devices. YAFFS2 was already freely available for Linux. However, it is not part of the standard 2.6.25 Linux kernel, and so Google added it to Android.
  
• Bluetooth -- 10 files -- Google made changes to 10 files in the Bluetooth communications stack. These changes fix apparent bugs related to Bluetooth headsets, and add Bluetooth debugging and access control functions.
  
• Scheduler -- 5 files -- The Android kernel also contains slight changes to the CPU process scheduler and time-keeping algorithms. We don't know the history of these changes, and the impact was not evident based on a cursory examination.
  
• New Android Functionality -- 28 files -- In addition to bug fixes and other small changes, Android contains a number of new subsystems that are worth mentioning here, including the following:
  • IPC Binder -- The IPC Binder is an Inter-Process Communication (IPC) mechanism. It allows processes to provide services to other processes via a set of higher-level APIs than are available in standard Linux. An Internet search indicated that the Binder concept originated at Be, Inc., and then made its way into Palm's software, before Google wrote a new Binder for Android.
    
  • Low Memory Killer -- Android adds a low-memory killer that, each time it's called, scans the list of running Linux processes, and kills one. It was not clear in our cursory examination why Android adds a low-memory killer on top of the already existing one in the standard Linux kernel.
    
  • Ashmem -- Ashmem is an Anonymous SHared MEMory system that adds interfaces so processes can share named blocks of memory. As an example, the system could use Ashmem to store icons, which multiple processes could then access when drawing their UI. The advantage of Ashmem over traditional Linux shared memory is that it provides a means for the kernel to reclaim these shared memory blocks if they are not currently in use. If a process then tries to access a shared memory block the kernel has freed, it will receive an error, and will then need to reallocate the block and reload the data.
    
  • RAM Console and Log Device -- To aid in debugging, Android adds the ability to store kernel log messages to a RAM buffer. Additionally, Android adds a separate logging module so that user processes can read and write user log messages.
    
  • Android Debug Bridge -- Debugging embedded devices can best be described as challenging. To make debugging easier, Google created the Android Debug Bridge (ADB), which is a protocol that runs over a USB link between a hardware device running Android and a developer writing applications on a desktop PC.
    
    Android also adds a new real-time clock, switch support, and timed GPIO support. We list the impacted files for these new modules at the end of this document.
• Power Management -- 5 files -- Power management is one of the most difficult pieces to get right in mobile devices, so we split it out into a group separate from the other pieces. It's interesting to note that Google added a new power management system to Linux, rather than reuse what already existed. We list the impacted files at the end of this document.
  
• Miscellaneous Changes -- 36 files -- In addition to the above, we found a number of changes that could best be described as, 'Miscellaneous.' Among other things, these changes include additional debugging support, keypad light controls, and management of TCP networking.
  
• NetFilter -- 0 files -- Finally, our change list showed Netfilter as having 22 changed files. However, examination showed the only difference was the capitalization of the filenames (xt_DSCP.c vs. xc_dscp.c). The contents of the files were all identical. So we ignored these files in our port.

Port the N810 changes forward or the Android changes backward?

As you can see from the above, Android makes significant changes to the standard Linux kernel. Additionally, Google's system software checks to make certain the kernel version is 2.6.25 or newer. This led us to wonder if Android requires features in the standard 2.6.25 Linux kernel that are not in earlier releases.

On the other hand, the N810 kernel changes were only to add hardware support. Nokia did not change core operating system services or add new interfaces.

We decided that we would have a higher chance of success by porting the N810 changes forward to the 2.6.25 Linux kernel.

So, without further ado...

Step 1: Port the N810 kernel changes to the 2.6.25 Linux kernel

This work took one developer seven days to complete, and included the following steps:

• Downloaded the latest Linux-OMAP-2.6 kernel from Montavista
• Selected the 2.6.25 version from the kernel download (used 'git reset --hard v2.6.25-omap1'))
• Patched the 2.6.25 kernel with the following patches from the N810's 2.6.21 kernel:
  • N810 Multi Media Card (MMC) driver
  • N810 I2C serial bus controller chip driver
  • N810 touch screen driver
  • N810 keyboard mapping
  • N810 audio and video DSP driver
  • N810 DMA driver
  • N810 board revision patch

After a number of software build/flash/test/debug cycles, our new kernel was able to boot the N810's default software and function well.

One issue we did have is that the kernel driver for the CX3110 WiFi chip is provided as a binary-only kernel module, not source code that we could recompile on the new kernel. This meant that we were not able to make our new kernel support WiFi on the N810.

This is an important note for those trying to do similar ports: if you have binary-only drivers, make sure they've been compiled against the kernel baseline that you plan to use.

Step 2: Add the Android kernel patches to the 2.6.25 N810 Linux kernel

With our new N810 Linux kernel up and running, we could now apply the Android patches. This was mostly straightforward and took one developer two and a half days of applying patches, flashing the kernel, and verifying the patches worked.

If you are using a kernel baseline different from the one in the Android SDK, you should plan to spend more time and resources to make it work correctly.

Luckily, the Android and N810 patches do not touch any of the same kernel parts. This made merging these two change sets much easier.

Step 3: Bring-up Android on our new kernel

After two weeks of patching and testing, we were finally ready to try to boot the full Android system. Software development would not be any fun if everything worked the first time, so, luckily, there were issues (that is a joke).

Here is the list of what we worked through:

• The N810's jffs2 file system lacked a feature Android needs in the mmap system call
• So we switched to using the ext2 file system, but it reported data corruption
• So we switched to using the ext3 file system, and the corruption went away
• The system hung while booting, and while we were trying to debug, it would reboot
• We discovered the watchdog timer was rebooting the system, so we disabled it
• After that, the system sometimes successfully booted, but the screen was blank
• We could see graphics after we applied a double-buffered video patch to the N810
• After this, we could sometimes boot to the home screen, but the system often crashed
• We recopied some system libraries to the N810, and the crashes went away
• We enabled the touch screen by changing a power setting in the system software
• We enabled the keypad by fixing a keymap file in the system software

And, finally, afer 19 days, it worked. The Android robot would show while powering up, and then we could see the main menu and use the applications.


Android 1.0 booting on the N810
(Click to play)

Step 4: Play Around

This, of course was the pay off. If you get this far, we recommend that you take a photo and email it to your nerdiest friends and colleagues to show off your accomplishment.


Android 1.0 running on Nokia's N810 Internet tablet
(Click to enlarge)

Next steps

We want to be able to create our own Android applications and run them on the N810. To do that, we will need to add support for the Android Debug Bridge, fix an intermittent system crash, and see if we can enable battery charging when the power supply is plugged in. Additionally, our application needs Internet access, so we either need to make the WiFi chip work, or enable USB Ethernet to send network data through the USB cable to a host PC, which can then route the N810's data to the Internet.

Finishing up

Before we're done, we will release the changes we made back to the Android and N810 communities so that they can take advantage of the work we have done and have a bit of fun.

Things to consider in your project

If you're porting Android to a new device, we recommend you consider the following points:

First, this is a just a milestone along the way to any kind of shipping product. We still have work to do before the N810 will be good enough for us to develop custom applications. Likewise, the N810 isn't a phone, so we didn't have to spend time debugging the call stack. And then there are the pieces that don't work -- power management, WiFi, etc. -- that could all take more time to make work than we spent on this initial port. So the hard work is ahead, not behind.

Second, the N810 is a shipped commercial product that has gone through Nokia's strenuous quality control processes and is well loved by lots of hackers. We have worked on other projects with flakey hardware and no vendor support (yeah, that was a huge mistake -- and it was made even worse by doing it fixed-bid). If you are porting Android to your own hardware, be sure to factor time and resources into dealing with hardware risks. We highly recommend that you make sure there are software people assigned to making the hardware drivers work.

Third, figure out which baseline kernel version you will use at the start. This way, you can verify or add tasks for making sure your hardware has the kernel drivers developed for that version of the Linux kernel. And since an outside vendor often provides binary drivers, make certain that you will have those drivers compiled against your Linux kernel when you need them.

Fourth, use the right tools and processes -- preferably the same ones that the Linux and Android developers use. We've worked with big companies who import open-source software into closed-source development models. While this makes it easier to get started, these teams later pay a price when they are not able to easily merge new versions of open-source software into their code base. Additionally, if your developers want to use Windows at all, we recommend you push them out of their cocoon and into the Linux world. A highly effective team would be one running Linux on their desktop, git for source control, and vim or emacs to edit 'C' code.

Finally, assemble the right team. While we're new to Android, we've been hacking embedded Linux systems for a long time. So check your teams' experience and plan accordingly.

Saying thanks

First, I'd like to thank Tang Yongjun, the NthCode developer who actually ported Android to the N810. I don't know that I would have been as patient as he was.

He and I would both like to also thank PenguinBait, B-man, and QWERTY-12 (yes, awesome names), who made the original port of the pre-release Android SDK to the N810. We were able to follow many of their steps to make our port of the Android 1.0 release easier.

Finally, this is a living document. If you spotted a mistake, or think others would benefit some piece of piece of information we don't have here, please email me so I can consider the change.

Posted in: linux mobile