Building an embedded Linux system emulator

Building an embedded Linux system emulator
by Gilad Ben-Yossef

One of the hallmarks of embedded system programming is working with specialized hardware. Unfortunately, embedded system developers do not always have the luxury to develop and test their code on the actual hardware they target. Often, the hardware is developed in tandem with the system software and therefore it it is not available for much of the embedded system software development cycle.

While one can develop and test much of our code on a PC running Linux, such a PC is a very different environment from the target board. More often then not, the target board is not even of the same architecture as the PC. A solution to this problem can be obtained by using an emulator - a software tool that executes software code of our target platform in a virtual machine that is running on our development host, and running our system software in it.

The following article describes how to build an embedded Linux system running inside an emulator which can be used to develop, test and debug target code even without access to target hardware.


The components

To build our emulator we will need the following components:

• Hardware emulator (we'll use Qemu)
• Minimal Linux root file system containing a C library and Busybox
• The Linux kernel

Installing Qemu

Created by Fabrice Ballard, Qemu is an open source machine emulator supporting seven target architectures, including x86, MIPS, ARM, and PowerPC. As first step, we will download and install the emulator. Depending on the Linux distribution you use on your workstation, you might be able to use the native package management system of the distribution to do so.

For Debian, Ubuntu and derivatives:

$ sudo apt-get install qemu

For Fedora and derivatives (as root):

# yum install qemu

For other distributions lacking a Qemu package, or for those wishing to obtain the very latest package (note that the "i386" label refers to the host running the emulator, and not the target platform):

$ wget http://bellard.org/qemu/qemu-0.9.1-i386.tar.gz $ cd / $ sudo tar zxvf qemu-0.9.1-i386.tar.gz

Or, as root:

# tar zxvf qemu-0.9.1-i386.tar.gz

Alternatively, you can download the sources and build the emulator from scratch. This has the added advantage that you can later adapt the emulator to more accurately reflect your actual hardware:

$ wget http://bellard.org/qemu/qemu-0.9.1.tar.gz $ tar zxvf qemu-0.9.1.tar.gz $ cd qemu-0.9.1/ $ ./configure $ make $ sudo make install

Or, as root:

# make install

Kernel and file system images

The Qemu emulator we have just installed provides a virtual machine mimicking our target hardware. To actually get Linux running on this virtual machine, however, we will need to download an image of the Linux kernel and a suitable root file system image for our target architecture.

Luckily, the Qemu project provides test images for several architectures that can be used to get a fast start with Qemu as an embedded Linux system emulator. Go to the Qemu project download page and choose one of the Qemu test disk images suitable for your embedded platform and download it to your Linux host (in this example we use ARM):

$ wget http://bellard.org/qemu/arm-test-0.2.tar.gz

Now extract the image:

$ tar zxvf arm-test-0.2.tar.gz $ cd arm-test

Booting Linux on the emulator

Start up Qemu with the following command line, adjusting the architecture name, kernel file name, and root file system image name according to your specific architecture (again, we use ARM in this example):

$ qemu-system-arm -kernel zImage.integrator \ -initrd arm_root.img -tftp / -redir tcp:9999::9999

The above command line starts Qemu in system emulation mode, booting into the kernel image zImage.integrator while loading into the virtual machine RAM the arm_root.img file system, and instructing Qemu to make your entire host root file system available for access via TFTP from the emulated machine (more on this ahead).

You should now be seeing a window similar to the following in which the emulated LCD display of the board is shown:


Qemu screenshot
(Click to enlarge)

You can log-in with the user "root" -- no password is required.


Transferring files to and from the host

The emulator and file system are set up to automatically configure a virtual Ethernet interface in the virtual machine with an internal IP. Through that virtual network interface, the emulator is set up to enable transferring of files to and from the host machine file system using the TFTP protocol.

For example, the following command will copy the file "/home/gby/hello_emu" from the host file system to the current directory inside the emulator:

$ tftp -g -r /home/gby/hello_emu -l hello_emu 10.0.2.2

The following command will copy the file "/root/test.log" from the emulator to the host file system directory "/home/gby/":

$ tftp -p -l/root/test.log -r /home/gby/test.log 10.0.2.2

In addition, you can use the "wget" comment to transfer files using the FTP and HTTP protocol to the emulator from any compatible server accessible in the network:

$ wget http://codefidence.com/some/file

Qemu supports numerous other way to interact with the host and it's environment, including bridged virtual network interfaces (as opposed to the default NAT used in the example above). Bridged virtual network interfaces enable:

• Using NFS to communicate with the host
• Remote debugging from the host
• VLAN support
• Exposing the host file system as a FAT file system
• Mounting disk, flash, or CDROM images from the host file system
• Using USB devices connected to the host

For more information on these advanced options, please refer to the Qemu user manual.


Debugging user applications

Using the GNU debugger GDBserver agent, we can debug applications running inside the emulator using the GDB debugger on the host. To do this, first use one of the methods outlined above to copy the "gdbserver" executable to the emulator. Note that you will need a gdbserver executable that was built to run on the target architecture (such as ARM, in the example above), and not on that of the host!

Also note that since the test images do not contain debugging symbols for the system libraries, you will only be able to debug statically compiled applications using them. This limitation can be removed by building your own kernel and file system image (see below for more information on this topic).

$ tftp -g -r /home/gby/src/gdb/gdb/gdbserver/gdberver \ -l gdbserver 10.0.2.2

Next, assign the gdbserver binary execute permissions:

$ chmod u+x gdbserver

Now, run the gdbserver agent, instructing it to use port 9999 (which we previously redirected to the emulator, when we launched qemu-system-arm from the command-line) to listen for connections from the debugger:

$ gdbserver 0.0.0.0:9999 /bin/myprog

Or, if you wish to attach to an already running program, use:

$ gdbserver 0.0.0.0:9999 --attach 1234

Finally, run the GDB debugger on your host and instruct it to connect to the host local port 9999:

$ arm-linux-gdb target/bin/myprog GNU gdb 6.6-debian Copyright (C) 2006 Free Software Foundation, Inc. ... (gdb) set solib-absulote-prefix /dev/null (gdb) set solib-search-path target/lib/ (gdb) target remote 127.0.0.1:9999

Debugging the kernel

Using the Qemu emulator to debug kernel code is quite straight forward, as Qemu incorporates a minimal GDB agent as part of the emulator itself. To debug the Linux kernel running inside the emulator, add the "-s" parameter to the command line used to start Qemu:

$ qemu-system-arm -kernel zImage.integrator \ -initrd arm_root.img -tftp / -redir tcp:9999::9999 -s

Now when the emulator starts, it will wait for a debugger connection on the default port "1234" (or a different port specific with the "-p" option), before proceeding with the boot. Once the emulator has started, you can debug the Linux kernel running inside it, using GDB on the host:

$ arm-linux-gdb linux/vmlinux GNU gdb 6.6-debian Copyright (C) 2006 Free Software Foundation, Inc. ... (gdb) target remote 127.0.0.1:1234

You can use GDB as you normally would. For example, type "cont" to launch the kernel:

(gdb) cont

Building your own kernel and file system images

So far we have seen how to use the Qemu emulator with the test kernel and file system images that are available on the Qemu site. To make full use of the emulator, we can create our own custom kernel and file system images that will better reflect the real target we are trying to develop for.

First, query Qemu regarding which boards it can emulate for your chosen architecture. Replace "arm" in the example above with one of: mips, x86_64, ppc, or sparc. For i386, simply use "qemu" as the command:

$ qemu-system-arm -M \?

Choose the board that most closely resembles your real target environment. Note that you can add support to Qemu of your specific true board. This requires some programming though, and we shall not cover it in this tutorial.

The creation of a kernel and file system for our emulated target is no different then doing the same task for real hardware. In fact, many tools are freely available to accomplish this task. In this example, we shall use the Buildroot framework. Buildroot is a set of make files and patches that simplify the generation of a cross-compilation tool chain and root file system for your target Linux system, using the uClibc C library.

First, we shall download the latest Buildroot release from the project web site and extract it:

$ wget http://buildroot.uclibc.org/downloads/snapshots/buildroot-snapshot.tar.bz2 $ tar jxvf buildroot-snapshot.tar.bz2 $ cd buildroot/

Next, let's configure Buildroot for our chosen target board:

$ make menuconfig

You will be presented with a menu enabling you to pick your architecture, sub-architecture, specific board to build for. Other options include GCC and uClibc versions, and related details. For each menu choice in the configuration tool, you can find associated help information describing the purpose of the entry.

At minimum, the following configuration options needs to be set:

• Target Architecture option -- choose your target architecture (e.g., arm.)
• Target Architecture Variant option -- Chose a specific model of the architecture (e.g., arm926t).
• Target options menu -- If the target board you wish to emulate (that is supported by Qemu) is listed, turn on support for that board (e.g., enable the "ARM Ltd. Device Support" menu, and inside it choose the "Integrator arm926" option).
• Toolchain menu -- Turn on "Build gdb server for the Target" option, and if you would like to test C++ programs on the emulator, also the "C++ cross-compiler support" option.
• Target filesystem options menu -- Enable the "cpio the root filesystem" option, and choose the "gzip" compression method. You may also request the file system image to be copied to a specified directory once it is generated.
• Kernel menu -- Choose the "linux (Advanced configuration)" option, and pick one of the offered Linux kernel versions of the list offered. Also, select the "zImage" binary format. Here, you can also specify a directory to copy the generated kernel to.
  
  In addition, you will need to supply a proper Linux kernel configuration file. Note that you can extract the kernel configuration file used to generate the kernel supplied as part of the test images, by issuing the following command from inside the emulator:
  
  

  $ zcat /proc/config.gz > linux.config

  
  Alternatively, Linux provides specific kernel configuration for optimal use with Qemu for some architectures. Run the following command to inspect the default kernel configuration included in a specific Linux kernel version:
  
  

  $ make help

When you're done configuring Buildroot, exit the configuration utility (making sure to OK saving the changes) and type: "make". Buildroot will now download all required sources, and build your new kernel and file system image for you. You should now be able to run the emulator using the kernel and file system image you have just created. Use the file name and path of the zImage binary as a parameter to Qemu's "-kernel" option, and the file name and path of the file system image with Qemu's "-initrd" parameter, like so:

$ qemu-system-arm -kernel zImage \ -initrd rootfs.arm.cpio.gz -tftp / -redir tcp:9999::9999 -s

As we have shown, the Qemu emulator provides a fairly simple way to develop, debug, and test Linux kernels, drivers, and applications for a variety of embedded architectures, even when no actual hardware is available. More information about the software used in this article can be found on the qemu, gdb, and Buildroot websites.

Read More

Porting Android to a new device

Porting Android to a new device

by Peter McDermott

Executive summary

If you're having trouble estimating the time and complexity of porting Google's Android mobile platform to your embedded device, our experience porting Android to a Nokia N810 Internet Tablet may provide you with useful information.

Android overview

Android is a mobile phone software platform created by Google. It is based on the Linux kernel, a number of software libraries written in 'C,' a Java-like virtual machine for running application code, and a full suite of mobile phone applications.

Android costs nothing and the source code is freely available. Its license terms (see the Apache license link at the end) are commercial-friendly, saying, in essence, "Do what you like with it, just don't sue us if something goes wrong." The one exception to this rule is the Linux kernel, which is licensed under the GNU Public License. Because of this, manufacturers must release their device's Linux kernel source code after product shipment.

Taiwan's HTC shipped the first Android phone, the G1, in October 2008. Other mobile manufacturers, including Motorola, LG, and Samsung are all reportedly creating Android-based phones. Additionally, from NthCode's home base in China, we see many startups and smaller manufacturers leveraging Android in an attempt to gain a foothold in the global smartphone market.


T-Mobile G1, the first Android phone

The Android platform provides all the services one would expect in a mobile phone, including phone calling, networking, messaging, and multimedia. Our analysis of the source code so far is that these services are complete and built on top of an excellent architecture, but do not implement every last feature found in modern mobile phones. This is the 1.0 release, after all.

For application developers, Android builds on top of the popular open-source Eclipse integrated development environment. Developers can use Eclipse to write Java applications, which are then compiled to Java bytecode before being translated for running on Android's Dalvik Virtual Machine (VM).

Google claims their Dalvik VM runs more efficiently in terms of speed and memory tradeoffs than Java -- slyly suggesting that Google wrote their own VM for performance reasons. However, while we haven't seen supporting data, we suppose Google really created Dalvik to escape from needing to cede control or pay a license fee to Sun, the creator and steward of the Java platform.

By translating from Java to Dalvik bytecode, Google is able to leverage the mature tools and large existing community of Java developers. The downside is that developers will need to take the additional step of running the Google translation tool on their compiled Java code before their application will run on an Android phone (Google automates this step in their Android Eclipse plug-in, however).


Android emulator screenshot

Google also provides an Android emulator. The emulator is a virtualized ARM microprocessor that runs the same system code -- and almost exactly the same Linux kernel -- that runs on a device. Google provides an Eclipse add-on so that Android programs can be compiled and debugged in Eclipse while running in their emulator. The Android emulator is as complete an emulator as we have ever seen -- just make sure your developers have enough RAM in their PCs to be able to run Eclipse and the emulator together.

So what's in it for Google?

Google has given mobile phone manufacturers everything they want -- a free, open, mobile operating system -- and, in return, only asked that the devices they create can run Android applications. We can only speculate that Google expects to eventually earn more from consumers using Google services on Android devices than it costs Google to develop and maintain Android.

Overview of the Nokia N810

We work with embedded Linux systems at NthCode, so we thought it would be a fun challenge (and would help with a project we're working on) if we could port Android to an existing Linux device.

Luckily, we already had two Nokia N810 Internet Tablets in the office. The N810 is a handheld mobile device that comes with a high-resolution 4.1-inch touch screen and a slide-out QWERTY keyboard. The N810 has a 400 MHz Texas Instruments OMAP 2420 CPU, 128MB RAM, and WiFi networking. The one thing the N810 can't do is make phone calls -- this is why Nokia markets it as an Internet Tablet that can surf the web, rather than a phone. At the time of this writing, the N810 costs about USD $425.

An active community of enthusiasts enjoys hacking the N810's Linux-based software. While some of those enthusiasts had already ported a pre-release version of the Android SDK to the N810, we discovered that no one had yet ported the Android 1.0 release to it. So we decided to see how long it would take us, and what we could learn in the process.

Development steps

As mentioned earlier, the Android system software runs on top of a Linux kernel. This Linux kernel provides services to applications -- such as file access, process scheduling, and inter-process communication.

Google made a number of modifications to the standard Linux kernel for Android. Likewise, Nokia modified the standard Linux kernel to support their hardware, such as the keypad, touch screen, and file system.

We quickly discovered that Nokia's N810 changes are made against an earlier 2.6.21 Linux kernel. And, unlike earlier Android prereleases, the Android 1.0 modifications are made against a later 2.6.25 Linux kernel. We investigated and discovered that between these two versions, a year had elapsed and the Linux community had made thousands of changes to kernel source code.

So, to make both Nokia's and Google's modifications work together on the N810 we either needed to port the N810 changes to work on the newer 2.6.25 Linux kernel, or port the Android changes to the N810's earlier 2.6.21 Linux kernel.

What did Google change in the kernel?

We checked the differences between the Android kernel and the standard Linux kernel and found that Google had changed 75 files and added an additional 88. We have prepared an annotated list of changed files at the end of this document, and a brief summary here.

• Goldfish -- 44 Files -- The Android emulator runs a virtual CPU that Google calls Goldfish. Goldfish executes ARM926T instructions and has hooks for input and output -- such as reading key presses from or displaying video output in the emulator.
  
  These interfaces are implemented in files specific to the Goldfish emulator and will not be compiled into a kernel that runs on real devices. So we safely ignored these files in our work.
  
• YAFFS2 -- 35 Files -- Unlike PCs, which store files on disks, mobile phones store files in sold-state flash memory chips. The HTC G1 uses NAND flash, a type of flash memory that is becoming more popular due to its combination of high density and low cost.
  
  YAFFS2 is an acronym for "Yet Another Flash File System, 2nd edition." It provides a high-performance interface between the Linux kernel and NAND flash devices. YAFFS2 was already freely available for Linux. However, it is not part of the standard 2.6.25 Linux kernel, and so Google added it to Android.
  
• Bluetooth -- 10 files -- Google made changes to 10 files in the Bluetooth communications stack. These changes fix apparent bugs related to Bluetooth headsets, and add Bluetooth debugging and access control functions.
  
• Scheduler -- 5 files -- The Android kernel also contains slight changes to the CPU process scheduler and time-keeping algorithms. We don't know the history of these changes, and the impact was not evident based on a cursory examination.
  
• New Android Functionality -- 28 files -- In addition to bug fixes and other small changes, Android contains a number of new subsystems that are worth mentioning here, including the following:
  • IPC Binder -- The IPC Binder is an Inter-Process Communication (IPC) mechanism. It allows processes to provide services to other processes via a set of higher-level APIs than are available in standard Linux. An Internet search indicated that the Binder concept originated at Be, Inc., and then made its way into Palm's software, before Google wrote a new Binder for Android.
    
  • Low Memory Killer -- Android adds a low-memory killer that, each time it's called, scans the list of running Linux processes, and kills one. It was not clear in our cursory examination why Android adds a low-memory killer on top of the already existing one in the standard Linux kernel.
    
  • Ashmem -- Ashmem is an Anonymous SHared MEMory system that adds interfaces so processes can share named blocks of memory. As an example, the system could use Ashmem to store icons, which multiple processes could then access when drawing their UI. The advantage of Ashmem over traditional Linux shared memory is that it provides a means for the kernel to reclaim these shared memory blocks if they are not currently in use. If a process then tries to access a shared memory block the kernel has freed, it will receive an error, and will then need to reallocate the block and reload the data.
    
  • RAM Console and Log Device -- To aid in debugging, Android adds the ability to store kernel log messages to a RAM buffer. Additionally, Android adds a separate logging module so that user processes can read and write user log messages.
    
  • Android Debug Bridge -- Debugging embedded devices can best be described as challenging. To make debugging easier, Google created the Android Debug Bridge (ADB), which is a protocol that runs over a USB link between a hardware device running Android and a developer writing applications on a desktop PC.
    
    Android also adds a new real-time clock, switch support, and timed GPIO support. We list the impacted files for these new modules at the end of this document.
• Power Management -- 5 files -- Power management is one of the most difficult pieces to get right in mobile devices, so we split it out into a group separate from the other pieces. It's interesting to note that Google added a new power management system to Linux, rather than reuse what already existed. We list the impacted files at the end of this document.
  
• Miscellaneous Changes -- 36 files -- In addition to the above, we found a number of changes that could best be described as, 'Miscellaneous.' Among other things, these changes include additional debugging support, keypad light controls, and management of TCP networking.
  
• NetFilter -- 0 files -- Finally, our change list showed Netfilter as having 22 changed files. However, examination showed the only difference was the capitalization of the filenames (xt_DSCP.c vs. xc_dscp.c). The contents of the files were all identical. So we ignored these files in our port.

Port the N810 changes forward or the Android changes backward?

As you can see from the above, Android makes significant changes to the standard Linux kernel. Additionally, Google's system software checks to make certain the kernel version is 2.6.25 or newer. This led us to wonder if Android requires features in the standard 2.6.25 Linux kernel that are not in earlier releases.

On the other hand, the N810 kernel changes were only to add hardware support. Nokia did not change core operating system services or add new interfaces.

We decided that we would have a higher chance of success by porting the N810 changes forward to the 2.6.25 Linux kernel.

So, without further ado...

Step 1: Port the N810 kernel changes to the 2.6.25 Linux kernel

This work took one developer seven days to complete, and included the following steps:

• Downloaded the latest Linux-OMAP-2.6 kernel from Montavista
• Selected the 2.6.25 version from the kernel download (used 'git reset --hard v2.6.25-omap1'))
• Patched the 2.6.25 kernel with the following patches from the N810's 2.6.21 kernel:
  • N810 Multi Media Card (MMC) driver
  • N810 I2C serial bus controller chip driver
  • N810 touch screen driver
  • N810 keyboard mapping
  • N810 audio and video DSP driver
  • N810 DMA driver
  • N810 board revision patch

After a number of software build/flash/test/debug cycles, our new kernel was able to boot the N810's default software and function well.

One issue we did have is that the kernel driver for the CX3110 WiFi chip is provided as a binary-only kernel module, not source code that we could recompile on the new kernel. This meant that we were not able to make our new kernel support WiFi on the N810.

This is an important note for those trying to do similar ports: if you have binary-only drivers, make sure they've been compiled against the kernel baseline that you plan to use.

Step 2: Add the Android kernel patches to the 2.6.25 N810 Linux kernel

With our new N810 Linux kernel up and running, we could now apply the Android patches. This was mostly straightforward and took one developer two and a half days of applying patches, flashing the kernel, and verifying the patches worked.

If you are using a kernel baseline different from the one in the Android SDK, you should plan to spend more time and resources to make it work correctly.

Luckily, the Android and N810 patches do not touch any of the same kernel parts. This made merging these two change sets much easier.

Step 3: Bring-up Android on our new kernel

After two weeks of patching and testing, we were finally ready to try to boot the full Android system. Software development would not be any fun if everything worked the first time, so, luckily, there were issues (that is a joke).

Here is the list of what we worked through:

• The N810's jffs2 file system lacked a feature Android needs in the mmap system call
• So we switched to using the ext2 file system, but it reported data corruption
• So we switched to using the ext3 file system, and the corruption went away
• The system hung while booting, and while we were trying to debug, it would reboot
• We discovered the watchdog timer was rebooting the system, so we disabled it
• After that, the system sometimes successfully booted, but the screen was blank
• We could see graphics after we applied a double-buffered video patch to the N810
• After this, we could sometimes boot to the home screen, but the system often crashed
• We recopied some system libraries to the N810, and the crashes went away
• We enabled the touch screen by changing a power setting in the system software
• We enabled the keypad by fixing a keymap file in the system software

And, finally, afer 19 days, it worked. The Android robot would show while powering up, and then we could see the main menu and use the applications.


Android 1.0 booting on the N810
(Click to play)

Step 4: Play Around

This, of course was the pay off. If you get this far, we recommend that you take a photo and email it to your nerdiest friends and colleagues to show off your accomplishment.


Android 1.0 running on Nokia's N810 Internet tablet
(Click to enlarge)

Next steps

We want to be able to create our own Android applications and run them on the N810. To do that, we will need to add support for the Android Debug Bridge, fix an intermittent system crash, and see if we can enable battery charging when the power supply is plugged in. Additionally, our application needs Internet access, so we either need to make the WiFi chip work, or enable USB Ethernet to send network data through the USB cable to a host PC, which can then route the N810's data to the Internet.

Finishing up

Before we're done, we will release the changes we made back to the Android and N810 communities so that they can take advantage of the work we have done and have a bit of fun.

Things to consider in your project

If you're porting Android to a new device, we recommend you consider the following points:

First, this is a just a milestone along the way to any kind of shipping product. We still have work to do before the N810 will be good enough for us to develop custom applications. Likewise, the N810 isn't a phone, so we didn't have to spend time debugging the call stack. And then there are the pieces that don't work -- power management, WiFi, etc. -- that could all take more time to make work than we spent on this initial port. So the hard work is ahead, not behind.

Second, the N810 is a shipped commercial product that has gone through Nokia's strenuous quality control processes and is well loved by lots of hackers. We have worked on other projects with flakey hardware and no vendor support (yeah, that was a huge mistake -- and it was made even worse by doing it fixed-bid). If you are porting Android to your own hardware, be sure to factor time and resources into dealing with hardware risks. We highly recommend that you make sure there are software people assigned to making the hardware drivers work.

Third, figure out which baseline kernel version you will use at the start. This way, you can verify or add tasks for making sure your hardware has the kernel drivers developed for that version of the Linux kernel. And since an outside vendor often provides binary drivers, make certain that you will have those drivers compiled against your Linux kernel when you need them.

Fourth, use the right tools and processes -- preferably the same ones that the Linux and Android developers use. We've worked with big companies who import open-source software into closed-source development models. While this makes it easier to get started, these teams later pay a price when they are not able to easily merge new versions of open-source software into their code base. Additionally, if your developers want to use Windows at all, we recommend you push them out of their cocoon and into the Linux world. A highly effective team would be one running Linux on their desktop, git for source control, and vim or emacs to edit 'C' code.

Finally, assemble the right team. While we're new to Android, we've been hacking embedded Linux systems for a long time. So check your teams' experience and plan accordingly.

Saying thanks

First, I'd like to thank Tang Yongjun, the NthCode developer who actually ported Android to the N810. I don't know that I would have been as patient as he was.

He and I would both like to also thank PenguinBait, B-man, and QWERTY-12 (yes, awesome names), who made the original port of the pre-release Android SDK to the N810. We were able to follow many of their steps to make our port of the Android 1.0 release easier.

Finally, this is a living document. If you spotted a mistake, or think others would benefit some piece of piece of information we don't have here, please email me so I can consider the change.

Read More

Simple Embedded Linux System (part 5)

Introduction

This is the fifth article in a series demonstrating the fundamental aspects of constructing embedded systems.

We discuss some issues about the end of a project and then consider alternative tools and methods for building systems.

The whole series of articles assumes a basic understanding of a Linux-based operating system. While discussing concepts and general approaches these concepts are demonstrated with extensive practical examples. All the practical examples are based upon a Debian- or Ubuntu-based distribution.

If it's not broken, don't fix it

Although the targeting of the ARM system in the previous article produced some issues, the system did work and was usable. In a real project the issues raised might simply not be worth addressing. If the solution arrived at meets all the requirements from the project brief it is self-evident no further development is required.

The other side of the argument is that the project brief may not be complete. During the development of a solution factors which were not originally foreseen often arise, the project planning must be flexible enough to integrate these updates without compromising the project.

This point is made explicitly as experience shows embedded system projects suffer from either over or under compensation. Projects like these fail because fundamental issues arise which the project brief cannot reflect or because the schedule has to be repeatedly extended to accommodate a ephemeral specification. A workable compromise must be struck.

An especially common reason the brief may be changed is the question of continued development and support. Experience shows that once a system reaches a state where it becomes usable and may have fulfilled its original brief the potential to extend and improve it causes feature creep. If a project is to succeed this tendency must be controlled.

The best way to handle the "feature creep" issue is to plan for it. This statement may seem obvious but it is often omitted, and these articles are partly an attempt to reduce common mistakes. Planning might be as simple as a requirement in the brief that development is conducted in a way which will allow updates in the future. This means a project can be completed after having met its original requirements and then a new project created to extend the feature set.

Embedded developers seem stubbornly myopic about the issue of ongoing support. There seems to be a view that at the end of a project everything will be archived and no further development will be required. For any project complex enough to employ a Linux kernel and userspace this is rarely true. Given the large NRE of any embedded system it is almost always more cost effective to refine an existing product, perhaps updating it with new technologies as they become available than to start afresh. A successful, well planned, project enables the one that follows it to succeed more easily.

Is there another way?

So far in this series we have used the approach of taking pre-compiled executables and libraries from a host system and constructing a suitably arranged file system image. This has run into the issue that the build and target system have differing requirements which has undesirable effects on development.

Another issue which has not yet been considered is that the resulting systems tend to be larger than necessary. This is because the executables and libraries are built for a generic operating system which must provide the full user-space API. If we were to build only the libraries with options specifically required for our system the dependencies would be fewer and the result smaller.

Finally, the host-based approach requires the target system be capable of running a complete operating system either on the real hardware or under emulation. This may not be practical if the target system is heavily customised.

The solution to these issues, which is employed by many projects, is to use cross-compilation of the entire system from source. Cross-compilation is a technique where a host PC runs a compiler which generates output executables for a different architecture. For our ARM web kiosk example, this compiler would execute on the x86 machine and generate ARM executables.

Building an entire operating system from source is a daunting task even for a seasoned professional familiar with the process. To indicate the scale of the task we will return to our very first example and outline the process of building a simple busybox environment.

To build busybox you require a C library and a compiler. The compiler must be built against the selected C library. The most common choices of C library are GLibc which is feature complete and as a result very large, or uClibc which has less functionality but is much smaller. Building the cross compiler with the appropriate C library requires extensive configuration and may take several hours to compile. Once you have a working cross compiler you may configure busybox; there are over a thousand configuration items in busybox, of course you don't need to set all of them but deciding what needs to be enabled is a challenging task in itself. Assuming the busybox compile has been successful the binary and C library must be deployed in a file system as in our previous examples.

The above description is of course a gross simplification of the process as each of the selected components may have version interactions and need additional patches applying to produce a workable solution. Perhaps the reader will begin to appreciate why these articles started with the most straightforward approach and waited to the fifth before introducing such a intimidating concept.

All is not lost! Because of the complexity of building these systems from source, developers responded by automating the process. These automated process bear a superficial resemblance to the mkbusyfs.sh script we have used previously but perform a huge number of operations.

The most common OS builders in use at this time are buildroot and Openembedded. Neither are particularly easy to use but do reduce a nearly impossible task to something more practical.

The buildroot tool is a collection of Makefiles and script which is controlled by a configuration file. Buildroot is generally used for smaller systems with a limited interface, it is actively maintained and supports several architectures. The configuration file is generated using the kconfig system (the system the Linux kernel uses) which allows for easy manipulation of settings. Once configured the system may be built with a single make command, the initial build may take several hours as the entire cross compiler tool chain must be built. Subsequent builds should be incremental and only rebuild those components that have been modified.

The Openembedded system uses the bitbake tool to generate binary packages, the resulting system is generated by installing these packages into a target directory. The systems Openembedded builds tend to be much larger (e.g. PDAs and netbook devices) have a graphical interface and have rich user interaction. The initial build may take many hours but subsequent builds only rebuild and install the packages required so are much faster.

Other tools exist for building systems; in fact there are a substantial number. The two highlighted above have been selected as examples purely because of their popularity and general flexibility. Many of the other build tools available are domain specific, i.e. they target a single device or area. One such tool is OpenWrt, which builds systems specifically for networking applications.

It may be desirable to use one of these more targeted tools depending on a projects requirements. The selection should not be made without a good deal of research as the tool will have a major influence on the outcome of a project.

A simple system with buildroot

To demonstrate the buildroot system we shall use it to build a simple busybox system for our ARM system.

First the buildroot tool must be obtained. The stable 2009.02 release was downloaded and unpacked. An configuration was generated using make menuconfig, and the ARM target was configured along with a CPU type of 920t (the type of the S3C2440 SoC) and an EABI build.



Configuring buildroot
(Click to enlarge)

The target options were changed to make the project name "simple".



Selecting target options
(Click to enlarge)

The softfloat option was selected in the tool chain configuration.



Tool chain configuration
(Click to enlarge)

The minimal target skeleton option was selected. this causes the target system to use the mdev system to generate device nodes. We have used mdev in the previous examples.



Package selection for the target configuration
(Click to enlarge)

Finally the target file system options were selected.



Target file system configuration
(Click to enlarge)

All the other options were left at their default values. Once configured, make O=/buildroot/build was used to start the compilation process. The O option causes the build to be performed outside the source tree; this is desirable for repeatability as it does not alter the source.

Fifty minutes later the build completed and a compressed cpio archive was produced as before. The file /buildroot/build/binaries/simple/rootfs.arm.cpio.gz was copied to the TFTP server and started from the boot-loader. the image was loaded and kernel execution commenced. The system failed to start with a somewhat unhelpful message.

Kernel panic - not syncing: Attempted to kill init

This is both unfortunate (our example did not work first time) and illuminating (we get to explain how to debug the issue). In a previous article it was mentioned that the first process the kernel starts, the init process, has a special constraints within the system different to those of other user processes. One of these constraints is the init process may never exit, if it does the kernel will panic and you will receive the above message.

As our system only actually contains the busybox program which is providing all the commands for our system (including init) we have a fairly simple problem to solve -- why is busybox exiting? Experience shows the easiest way to test such a binary is to run it on an already operational system.

Fortunately we have the ideal candidate, the NFS-based Debian system we used to build the web kiosk application. This OS was started, the rootfs.arm.cpio.gz unpacked into a directory and the chroot command used to start a shell inside the target.

$ mkdir simple $ cd simple $ sudo cpio -i ../rootfs.arm.cpio.gz $ cd .. $ sudo chroot simple /bin/sh SIGILL $

It seems the busybox binary has been built containing instructions which the CPU cannot execute. This can happen on ARM systems when the compiler has built with the wrong architecture target. The CPU in question supports version 4t of the instruction set and is capable of correct EABI operation; however the compiler appears to have selected the version 5 instruction set instead. This is unexpected as the CPU type was clearly set to 920t within the buildroot configuration implying version 4t.

Investigating the buildroot make files lead us to discover that the CPU type selection only configures the CPU the compiler will optimise for, not the instruction set version. The buildroot configuration script (.config) was manualy edited and the variable BR2_GCC_TARGET_ARCH set to armv4t.

BR2_ARM_TYPE="ARM920T" # BR2_ARM_OABI is not set BR2_ARM_EABI=y BR2_ARCH="arm" BR2_ENDIAN="LITTLE" BR2_GCC_TARGET_TUNE="arm920t" BR2_GCC_TARGET_ARCH="armv4t" BR2_GCC_TARGET_ABI="aapcs-linux" # # Target options #

The output build directory was cleared (excepting the downloaded source archives) and the build repeated.

The system was once again started from the boot-loader. This time it appeared to start successfully and a login was presented on the video console. Unfortunately none of the kernel modules required for USB HID devices are present in the system and no login was presented on the serial console. Because of this the system as it stands cannot be interacted with.

A serial login can be added simply by editing /buildroot/build/project_build_arm/simple/root/etc/inittab and adding a line for the S3C2440 SoC first serial port.

s3c2410_serial0::respawn:/sbin/getty -L s3c2410_serial0 115200 vt100

If root should be able to login on the serial port the /buildroot/build/project_build_arm/simple/root/etc/securetty file must have s3c2410_serial0 added to it.

The system was rebuilt and the image booted and a successful login made.

Welcome to Buildroot simple login: default $ ls -l / drwxrwxr-x 2 root root 0 Dec 31 1969 bin drwxr-xr-x 2 root root 0 Dec 31 1969 config drwxr-xr-x 3 root root 13280 Mar 6 19:49 dev drwxr-xr-x 4 root root 0 Mar 6 19:49 etc drwxrwxrwx 3 root root 0 Dec 31 1969 home lrwxrwxrwx 1 root root 9 Dec 31 1969 init -> sbin/init drwxr-xr-x 2 root root 0 Dec 31 1969 lib lrwxrwxrwx 1 root root 11 Dec 31 1969 linuxrc -> bin/busybox dr-xr-xr-x 43 root root 0 Dec 31 1969 proc drwxr-x--- 2 root root 0 Dec 31 1969 root drwxrwxr-x 2 root root 0 Dec 31 1969 sbin drwxr-xr-x 13 root root 0 Dec 31 1969 sys drwxrwxrwt 2 root root 120 Mar 6 19:49 tmp drwxr-xr-x 5 root root 0 Dec 31 1969 usr drwxr-xr-x 3 root root 0 Dec 31 1969 var $

As can be seen from this example the buildroot system does reduce the effort of building a system from source, but also requires a great deal of domain-specific knowledge to fix issues when they arise. The illegal instruction issue took several hours to debug and fix. The final solution was relatively simple but the process to find it was involved.

The issue with configuration files being incomplete for our target hardware was simple to fix but we would need some way to ensure this is automated for future builds (by editing the generic target skeleton). This problem did however highlight the benefit of the buildroot system in that the rebuild to include the updated files took seconds.

What's next?

This article has covered how the development processes might be refined and introduced a new build strategy. The new strategy also has (differing) issues which have to be considered.

In the next article we examine issues related to moving from simple volatile RAM-based systems to deploying on non-volatile storage media.

Read More

Simple Embedded Linux System (part 4)

Introduction

This is the fourth article in a series demonstrating the fundamental aspects of constructing embedded systems.

This article covers the targeting of the web kiosk system from the previous article to a specific ARM based hardware platform.

This document, and indeed the whole series of articles, assumes a basic understanding of a Linux-based operating system. While discussing concepts and general approaches these concepts are demonstrated with extensive practical examples. All the practical examples are based upon a Debian- or Ubuntu-based distribution.

Scaling the design

The previous articles have not required any additional hardware; indeed, by using the QEMU emulator, we have been able to keep all the examples confined to the realm of software.

It is rare however for an embedded system to have no limits on its hardware resources. In fact, it is common for a project brief to severely limit the available hardware platform, generally through cost and size, but increasingly through power usage. The cost and size limits are self evident however the power limitation bears a closer examination.

Embedded systems like the web kiosk example are generally in a fixed location which presents the opportunity to have a full mains power connection. In this situation why might power consumption be an issue?

The primary emerging reason is that running electrical cabling for power to these systems limits their location and may have high cost implications. If, instead, the systems can be powered entirely from a single data cable, using low voltages, they become much more flexible.

A second consideration which has gained greater significance in recent times is that of environmental impact. Appliance applications where the unit is powered for extended periods of time, and must always be available, by their very nature have a large environmental impact. A PC might consume 100W on average, over a day that builds up to 2.4KWh or some 876KWh per year, which equates to roughly 400kg of coal or 377kg of carbon dioxide released into the atmosphere (conversion factors from DEFRA). If we produce a system which can operate from just 12W, we reduce the yearly figure to just 105kWh or 45kg of carbon dioxide. A side effect of this is, obviously, reducing the electricity bill by over 80% which makes economic sense.


Figure 1: Web kiosks
(Click to enlarge)
The obvious power supply method for a networked system is to use Ethernet cabling employing the IEEE 802.3 Power-over-Ethernet (PoE) standard. This however introduces a maximum available power of 12 Watts (24W for the much less common IEEE 802.3at standard). Running a standard PC and LCD monitor from 12W is simply not possible with current available technologies.

If the project brief calls for an integrated display we may plausibly use a directly connected LCD panel in a fashion similar to a laptop. By using a directly connected panel, the refresh rate may be lowered significantly, which could reduce power draw. A direct connection also removes the need for power sapping line drivers which are required for standard monitors to connect over long cables. This approach might let us reduce the display power requirements to around 10W. This only leaves 2W budget for the computer, clearly a PC of any description cannot operate within such a power budget and an alternative solution must be sought.

The most common low-power computing platforms are currently based upon ARM System On Chip (SoC) devices. These devices operate at a few hundred megahertz instead of the gigahertz speeds found in PCs and consume a fraction of the power. There are numerous SoCs available, in a huge array of hardware solutions, and selecting the correct one for a given application is something to be discussed in a future article. One important factor in selecting a suitable system for developing a Linux solution is it must have kernel support. A useful reference for what systems are supported in current ARM Linux kernels is the KAutobuild project.

For the purposes of this article a Simtec Electronics thin client system with a 400MHz Samsung SoC module has been selected. This system provides reasonable CPU performance coupled with a GPU capable of driving large screens. This hardware would be capable of being directly connected to an LCD panel as discussed previously, and has sufficient power supply flexibility to have PoE functionality added if required.

An important idea which should be highlighted here is that, for many embedded applications, suitable hardware is often already available. Embedded projects often start with a faulty premise that they will need bespoke hardware solutions specifically for their application.

Often existing products can be modified or extended to produce completely acceptable results at a fraction of the cost. It is especially important to be realistic about the volumes of a product. A product which will have small production volumes will by definition have a higher per unit cost when the Non Recurring Engineering (NRE) cost is amortized over all the units produced. Thus, although purchasing and customizing an existing product may have a relatively high per unit cost, the NRE will be much lower.

To make this clearer let us use a real example. For a hundred units the thin client system selected costs GBP 200 per system (approximately $300 U.S.) or GBP 20,000 pounds total. A fully bespoke design from Simtec Electronics for a system of equivalent complexity would cost approximately GBP 40,000 and have a manufactured per unit cost around the GBP 150 mark for small runs. For a hundred unit run this means the bespoke approach has a unit cost of GBP 550; indeed the GBP200 per unit cost is not arrived at until almost 300 units have been produced. This is a trivial example and includes no NRE cost for required software, such as a boot-loader and a Linux kernel port, which the pre-built system already includes, and no budget for contingencies or specification changes.

The practical result of this is that, for fewer than a thousand units, the "customized off-the-shelf" approach nearly always makes economic sense.

Converting the example

The method we have employed so far is to use a shell script to copy binaries from a host system and generate an initramfs cpio archive to boot. Just because we have changed architecture there is absolutely no reason not to continue in this way.

Because the board has no local storage attached (there is provision for such; we are simply not using it) the instructions for bootstrapping the system with a Debian armel installation accessed via NFS were followed. This produced a system we can develop on in exactly the same way as our x86 host previously. Because Debian provides support for numerous architectures, this approach can be used just about anywhere that a Linux kernel, with the appropriate options, can be started.

The actions to build the browser were identical to those in the previous article. Indeed because no graphical environment was installed, the NetSurf browser could be run directly from the NFS mounted system.

The configuration script for the ARM web kiosk system is very similar to the original. The main difference is in the number of modules required. It is drastically reduced because a specifically-configured pre-built kernel was used. This kernel has the drivers built-in for the limited peripheral set found on the development boards made by Simtec Electronics.

The webkiosk-arm.gz and the kernel were placed on a TFTP server and the ABLE boot-loader was used to start the system.

> (tftpboot)vmlinuz-2.6.26-simtec1-s3c24xx-eabi initrd=(tftpboot)webkiosk-arm.gz \ root=/dev/ram0 console=ttySAC0

The system started and displayed the web browser as expected. The drawback to the host-based approach has become evident at this point however. Compiling the web browser on the ARM box took some seventeen minutes, preparing the cpio took another four minutes, shutting down the NFS system and booting the output took another six minutes.

From these timings it can be seen the edit, build and test cycle using this approach is over half an hour which, as we learned in the second article of the series, will have a large impact on the developer's productivity. We might make some improvement on that time by not having to rebuild the entire browser every time and by using a faster boot medium than TFTP (the ABLE boot-loader supports HTTP, for example).

Because the system was targeted at a Simtec Electronics integrated module it is possible to run the system on other baseboards in the range. To test this the image was started on a DePicture touchscreen tablet. The system started as expected and was able to navigate the web as normal with the touchscreen.


Figure 2. DePicture system showing the NetSurf welcome page
(Click to enlarge)

Improving the process

The primary issue with the approach illustrated here is that of development speed. The hardware system was selected for its ability to meet our project requirements, these are different to those necessary to construct the system.

The hardware selected traded CPU performance for a reduction in power consumption. The processor in PCs in general use at this time has around ten times the CPU power and five times the memory performance of the ARM SoC.

We could of course simply build on a more powerful ARM host and deploy the output on the target. This approach has the merit that it is straightforward to implement and the substantial drawback that a compatible host may not be available. Even if a compatible system is available they must be purchased for each developer which adds cost and may not shorten the development cycle by an amount large enough to justify the expense.

A commonly-used solution to this issue is to harness the power of a full PC system to generate output for the target. We have already used the QEMU emulator to test our output images, it can also emulate targets other than an x86 PC. Although some of the performance of the host processor will be lost to the emulation, the resulting system will generally still perform better than the native ARM hardware. This gives us the desired build performance without the expense of purchasing additional physical hardware. It also means we do not have to reboot the build system to test the results.

One issue worth mentioning here is that the more powerful the host PC the faster the emulation will be and the shorter the development cycle. If the host PC is not up to the task this approach might actually result in the opposite of the desired result and slow development.

Our bootstrapping article outlines how to create a disc image suitable for use with the emulator. A Debian Lenny system for the armel architecture should be prepared and started with QEMU and the system used to generate images.

The only small caveat is that the chosen target hardware system must be supported by QEMU. Unfortunately it is somewhat challenging to get new hardware support accepted into the QEMU project. At this time the Simtec Electronics boards are not supported in the pre-packaged QEMU, patches and sources necessary to create a QEMU that supports Simtec Electronics boards are available but must be compiled and installed separately.

What's next?

This fourth article outlines some of the reasons to use hardware more suited to a set of requirements. It demonstrates the relative ease with which a host-based build approach can be used to generate a working system for a small ARM system. Finally it shows some of the drawbacks of this approach and how they might be addressed.

In the next article we shall examine more issues surrounding the completion of a project and examine an alternative approach to building systems.

source from http://www.linuxfordevices.com.
by Vincent Sanders and Daniel Silverstone.

Read More

Simple Embedded Linux System (part 3)

Introduction

This is the third article in a series demonstrating the fundamental aspects of constructing embedded systems using Linux. This article covers the construction of a web browser with a command shell on the serial console. It assumes a basic understanding of a Linux-based operating system. Practical examples listed here are based upon a Debian- or Ubuntu-based distribution.

Pushing the limits

So far in this series we have used binaries from the host system or from pre-packaged software. We will now expand this to the inclusion of software built from source. This involves only a little more effort than the previous examples in terms of configuration script complexity, but there is an increasing burden of attempting to support additional methods of user interaction.

The primary cause of this increase in complexity comes from the additional kernel driver modules required to make the input and output devices operational. We saw in the previous web server example that drivers were required for each network card we wanted to support. For supporting two network cards (PCI NE2000 for QEMU and e1000e for a ThinkPad laptop) we added three modules.

In this project we need the modules necessary to support the input event system and the drivers to run the hardware. To support a minimal useful set, for mice and keyboards attached via USB HID and PS/2, the count runs to some eleven modules. This is further exacerbated by the need to have output using the framebuffer. To support the intelfb driver (for a Thinkpad laptop) and the cirrusfb driver (for QEMU) requires an additional twelve modules.

As these numbers demonstrate, a relatively small increase in supported interfaces rapidly increases the number of drivers required. We are also rapidly approaching the limits of what static module insertion is capable of and would need to start dynamically loading modules according to what hardware is present.

Selecting a browser

To have a web kiosk application we obviously need a browser. We could attempt to run Firefox or a Webkit browser using the X windows system but the dependencies for X and such a browser would make our resulting system huge which is undesirable.

One possibility for improving the situation might be to use a graphics library other than X, such as DirectFB with toolkit support. This appears appealing at first glance, but still requires a large number of libraries and a lot of software which is not frequently tested and hence will probably contain numerous issues we have to solve.

Another possibility is a browser which outputs directly to the Linux framebuffer. This possibility appeals as there is no need for a large toolkit and graphics library and browsers of this type tend to be smaller than their X based siblings.

After some searching, three candidate browsers that run directly on the Linux framebuffer were found:

• Zen -- A small browser which implements only basic layout. It removed itself from serious consideration as its author clearly states that all development has ceased.
  
• Links -- The Links browser is relatively small and supports HTML 4; however, its lack of CSS support and various build issues reduce its appeal.
  
• NetSurf -- The NetSurf browser has several build targets, including GTK, so it might have been a contender for the DirectFB and GTK type approach. In addition, however, it supports several framebuffer type display options including the Linux framebuffer. It is in current development and has support for HTML 5 and CSS although it lacks JavaScript support.

Based on the available choices, NetSurf using the Linux framebuffer front-end was selected.

Building the browser

Building the NetSurf browser is suprisingly simple. Since the Linux framebuffer port has not been included in an official release, it must be built from the project's Subversion repository. This is not generally recommended for embedded systems, but there is little other option at this time.

First the NetSurf development trunk should be checked out using Subversion:

$ svn co svn://svn.netsurf-browser.org/trunk/netsurf

Within the checkout (in the netsurf/ directory) there is a document called Docs/BUILDING-Framebuffer which describes the steps necessary to build the browser.

The summary of operations is to install and build the library dependencies, place a configuration makefile fragment in the NetSurf directory and run make TARGET=framebuffer.

$ sudo apt-get install build-essential libcurl3-dev libxml2-dev $ sudo apt-get install libmng-dev librsvg2-dev lemon $ sudo apt-get install re2c libfreetype6-dev ttf-bitstream-vera $ svn co svn://svn.netsurf-browser.org/trunk/libnsbmp $ sudo make -C libnsbmp install $ svn co svn://svn.netsurf-browser.org/trunk/libnsgif $ sudo make -C libnsgif install $ svn://svn.netsurf-browser.org/trunk/libparserutils $ sudo make -C libparserutils install $ svn://svn.netsurf-browser.org/trunk/hubbub $ sudo make -C hubbub install $ cd netsurf

The Makefile.config.override should contain the lines:

NETSURF_FB_FONTLIB=freetype NETSURF_FB_FRONTEND=linux

This configures the use of the Linux framebuffer frontend and the FreeType 2 library for font handling. Once the compile has completed it should produce an nsfb-linux binary.

Putting it all together

We will be using the same mkbusyfs.sh script from our previous examples; indeed you can continue using the previous installation and simply add configurations as the series progresses.

The configuration script for the web kiosk system is straightforward. The only especially interesting item is the number of kernel modules which, as already discussed, has grown considerably.

The mkbusyfs.sh tool should be used to generate the webkiosk.gz which can then be tested with QEMU. The QEMU command line is slightly different to previous examples as it must redirect the console to a serial port so the video hardware can be used for the graphical framebuffer.

qemu -kernel ./vmlinuz-2.6.26-1-686 -initrd webkiosk.gz \ -append "root=/dev/ram console=ttyS0" -net nic -net user /dev/zero

Several issues were experienced with QEMU and only after resorting to installing the latest version from the QEMU Subversion repository was graphical output obtained. As can be seen, the output is still not correct. It was not determined whether this was another issue with the emulated video card, or with the browser.


Figure 1 -- QEMU video display
(Click to enlarge)

The pre built Kernel and generated output for an x86 system are available.

Putting it on a real system

It was decided to test the system on a real machine. A Lenovo ThinkPad T61 was available. This machine has an Intel i915 chipset which is supported by the intelfb driver resulting in a usable graphical framebuffer.

As in the first article, syslinux is used to make a USB stick bootable. The syslinux.cfg used was:

default webkiosk timeout 100 prompt 1 label webkiosk kernel VMLINUZ append initrd=WEBKIOSK root=/dev/ram video=intelfb vga=0x317/pre>

The laptop was booted from the USB stick, and the web browser started as expected with the correct colors.

What's next?

This third example extends the ideas from the previous two articles and produces a useful embedded system. It also demonstrates that as the ways a user interacts with the system become more diverse the system's complexity rises. Finally there is also an indication that as the target system's hardware becomes diverse (e.g. multiple possible video cards), the number of drivers must also rise and unexpected behavior is more likely to be experienced -- as demonstrated by the odd colors from the QEMU video card emulation.

The next step will be to apply the web kiosk example to a specific embedded hardware platform and compare how this alters the system's construction and usage.

source from http://www.linuxfordevices.com.
by Vincent Sanders and Daniel Silverstone.

Read More

Simple Embedded Linux System (part2)

Introduction

This is the second article in a series demonstrating the fundamental aspects of constructing embedded systems.

In this article, we cover the construction of a simple web server with a command shell on the console.

This article, and indeed the whole series, assumes a basic understanding of a Linux-based operating system. While discussing concepts and general approaches these concepts are demonstrated with extensive practical examples. All the practical examples are based upon a Debian- or Ubuntu-based distribution.

Automated, reproducible, reliable building

One of the common pitfalls in building embedded systems is the tendency towards too much manual involvement in the build process. There seems to to be a misconception that because embedded systems are built, deployed and rarely updated that building them by hand saves time that would otherwise be spent automating the process.

Making the build process automatic and repeatable should be viewed as a critical part of the project. This enables the software engineers developing a product to have as short an edit, build and test cycle as possible. The desirability of a short development process should be self-evident, in that an engineer who can perform only one or two tests a day can only hope to debug and fix a small number of issues, where one who can perform a hundred tests can find and fix a far larger number of issues.

Another common mistake is not keeping the whole project in a Revision Control System (RCS). The benefits of revision control on any project have become increasingly evident and a wide selection of extremely powerful systems exist. The Subversion (svn) system is extremely popular for centralised RCS, while Bazaar (bzr) and GIT have become common for distributed RCS. Regardless of the model and tools chosen, revision control should never be omitted from a project.

For larger projects a centralized "build manager" is often desirable. This is a piece of software which builds the current project from the revision control system on a regular basis. Some projects rebuild on every commit, which may not be practical where a build takes an extended period of time. In such cases a system which rebuilds as often as it can, perhaps including numerous commits, should be employed. The results of these builds should be made available, and the developers informed as soon as possible of failures. This ensures the project is always in a state where it might be branched ready for formal testing and release.

A good article discussing these ideas further is Daily Builds Are Your Friend, by Joel Spolsky. Although this article refers to application development specifically, its analysis is valid on the larger project scale. Several other articles on best-practice for building software projects exist.

Scripting builds and common tasks

In the previous article we constructed an initramfs cpio-based system using the binaries of the host system. The steps were performed manually, and were we to continue with that approach any increase in complexity would rapidly make it impractical.

To solve this issue we turn to the Unix system’s scripting tools. A shell script to perform the build actions makes the build easily repeatable and saves continually re-typing a lot of commands.

The mkbusyfs.sh [available here] script is an automation of the steps performed in the previous article.

In addition to the basic setup the script adds functions to copy executables and their library dependencies, configure a DHCP client and copy kernel modules from the host to the target. These functions are straightforward and self-contained and their operation should be obvious.

The script is written to be reusable for a number of projects by including a second “application” script. This enables us to reuse the base functionality in future articles. The scripts are provided under a BSD-style licence and may be taken and modified as desired.

To implement the simple system illustrated in the previous article, the simple.sh [available here] configuration script can be used.

To keep things neat these scripts should be placed in a directory (these examples assume that it will be called mkbusyfs) alongside where the output should be generated.

The simple system would be generated using the command ./mkbusyfs.sh simple and output would be placed in in the parent directory.

$ pwd /home/dev/mkbusyfs $ ./mkbusyfs.sh simple Building simple simple specific Creating CPIO ../simple.gz $ ls .. mkbusyfs simple simple.gz $

Creating a web server's file system

The first thing we must consider is which web server to install, this is of course dependant on our project requirements.

One choice might be the Apache Web Server, however this would be impractical for all but the largest embedded system. Apache's executables, library dependencies and other system dependencies are relatively large. A more suitable alternative would be thttpd which is a few hundred kilobytes and has very few dependencies.

First we need to create a mkbusyfs configuration script.

#!/bin/sh # web server application specific mkbusyfs shell fragment USE_DHCPC=y EXTRA_LIBS="ld-linux.so.2 libnss_dns.so.2" #KERNEL_VER=2.6.26-1-686 KERNEL_MODULES="kernel/drivers/net/ne2k-pci.ko \ kernel/drivers/net/8390.ko \ kernel/drivers/net/e1000e/e1000e.ko" application_specific() { DESTDIR=$1 }

This configuration enables the DHCP client (which will require an init script, available here) and lists the kernel modules to be copied into the output. These are required drivers for network cards.

The project could be built and tested at this point if desired. It should initialise a system and acquire an IP address using DHCP.

Next the thttpd binary needs to be acquired. Instead of simply copying this from the host file system we can acquire the deb package from the package mirror, unpack it and extract the items we require without needing to install the package on the host. This does require the devscripts package to be installed but means no superuser privileges are required to build the system.

#!/bin/sh # web server application specific mkbusyfs shell fragment USE_DHCPC=y EXTRA_LIBS="ld-linux.so.2 libnss_dns.so.2" KERNEL_VER=2.6.26-1-686 KERNEL_MODULES="kernel/drivers/net/ne2k-pci.ko \ kernel/drivers/net/8390.ko \ kernel/drivers/net/e1000e/e1000e.ko" application_specific() { DESTDIR=$1 CURDIR=$(pwd) mkdir -p /tmp/thttpd/thttpd cd /tmp/thttpd dget thttpd dpkg -x thttpd*.deb thttpd cd ${CURDIR} add_program /tmp/thttpd/thttpd/usr/sbin/thttpd /usr/sbin/thttpd rm -rf /tmp/thttpd }

If we were going to add a second package it might be worth extracting this package acquisition and unpacking into a function. Such judgements are of course arbitrary, but if something is done more than once it is often sensible to create a helper function as then, if a change must be made, all the affected uses will be updated.

The next step is to add the configuration to start the web server and add some basic content. The complete script can be downloaded here.

The only slight difference here is that an additional library and a /etc/passwd file was required so the web server could change to execute as the www-data user.

When the output is generated it may be tested using QEMU again. The command line to run QEMU is slightly different as it needs to enable a NIC and redirect the emulated systems port 80 to the host's port 8080. This allows the web server to be accessed from the host using a web browser and the URL http://localhost:8080/.

$ qemu -kernel ./vmlinuz-2.6.26-1-686 -initrd webserver.gz \ -append "root=/dev/ram" -net nic -net user \ -redir tcp:8080:10.0.2.15:80 /dev/zero

The pre-built Kernel and generated output for an x86 system are provided here and here.

What's next?

This second step demonstrates the ideas of automation and repeatability and shows how the basic environment constructed in the previous article can be expanded to produce a system capable of interacting with a user.

The next step is to use the concepts presented here and expand them by introducing a more complex application built from source and discussing some limitations of real hardware and issues that arise from it.


source from http://www.linuxfordevices.com
by Vincent Sanders and Daniel Silverstone

Read More

Simple Embedded Linux System (part 1)

Introduction

Constructing an embedded system with Linux is often seen as a complex undertaking. This article is the first in a series which will show the fundamental aspects of constructing such systems and enable the reader to apply this knowledge to their specific situation.

This first article covers the construction of the most basic system possible, which will provide a command shell on the console. Along with the rest of the series, it assumes a basic understanding of a Linux-based operating system. While discussing concepts and general approaches, these concepts are demonstrated with extensive practical examples. All the practical examples are based upon a Debian- or Ubuntu-based distribution.

What is an embedded system?

The term "Embedded System" has been applied to such a large number of systems that its meaning has become somewhat ill-defined. The term has been applied to everything from 4-bit microcontroller systems to huge industrial control systems.

The context in which we are using the term here is to refer to systems where the user is limited to a specific range of interaction with a limited number of applications (typically one). Thus, from the whole spectrum of applications which a general purpose computer can run, a very narrow selection is made by the creator of the embedded system software.

It should be realized that the limits of interaction with a system may involve hardware as well as software. For example, if a system is limited to a keypad with only the digits 0 to 9, user interaction will be more constrained than if the user had access to a full 102-key keyboard.

In addition to the limiting of user interaction, there may also be limits on the system resources available. Such limits are typically imposed by a system's cost, size, or environment. However, wherever possible, these limits should be arrived at with as much knowledge of the system requirements as possible. Many projects fail unnecessarily because an arbitrary limit has been set which makes a workable solution unachievable. An example of this would be the selection of a system's main memory size before the application's memory requirements have been determined.

What do you want to achieve?

A project must have a clearly defined goal.

This may be viewed as a statement of the obvious, but it bears repeating as for some unfortunately inexplicable reason, embedded systems seem to suffer from poorly-defined goals.

An "embedded" project, like any other, should have a clear statement of what must be achieved to be declared a success. The project brief must contain all the requirements, as well as a list of "desirable properties." It is essential that the two should not be confused; e.g., if the product must fit inside a 100mm by 80mm enclosure, that is a requirement. However, a statement that the lowest cost should be achieved is a desirable item, whereas a fixed upper cost would be a requirement.

If information necessary to formulate a requirement is not known, then it should be kept as a "desirable item" couched in terms of that unknown information. It may be possible that once that information is determined, a requirement can be added.

It is, again, self-evident that any project plan must be flexible enough to cope with changes to requirements, but it must be appreciated that such changes may have a huge impact on the whole project and, indeed, may invalidate central decisions which have already been made.

General IT project management is outside the scope of this article. Fortunately there exist many good references on this topic.

Requirements which might be added to a project brief based on the assumptions of this article are:

• The system software will be based upon a Linux kernel.
• The system software will use standard Unix-like tools and layout.

The implications of these statements mean the chosen hardware should have a Linux kernel port available, and must have sufficient resources to run the chosen programs.

Another important consideration is what kind of OS the project warrants. For example, if you have a project requirement of in-field updates, then you may want to use a full OS with package management, such as Debian GNU/Linux or Fedora. Such a requirement would, however, imply a need for a non-flash-based storage medium such as a hard disc for storing the OS, as these kinds of systems are typically very large (even in minimal installations), and not designed with the constraints of flash-based storage in mind. However, given that additional wrinkle, using an extant operating system can reduce software development costs significantly.

Anatomy of a Linux-based system

Much has been written on how Linux-based systems are put together; however a brief review is in order, to ensure that basic concepts are understood.

To be strictly correct the term "Linux" refers only to the kernel. Various arguments have been made as to whether the kernel constitutes an operating system (OS) in its entirety, or whether the term should refer to the whole assemblage of software that makes up the system. We use the latter interpretation here.

The general steps when any modern computer is turned on or reset is:

• The CPU (or designated boot CPU on multi-core/processor systems) initializes its internal hardware state, loads microcode etc.
  
• The CPU commences execution of the initial boot code, e.g., the BIOS on x86 or the boot-loader on ARM.
  
• The boot code loads and executes the kernel. However, it is worth noting that x86 systems generally use the BIOS to load an intermediate loader such as GRUB or syslinux, which then fetches and starts the kernel.
  
• The kernel configures the hardware and executes the init process.
  
• The init process executes other processes to get all the required software running.

The kernel's role in the system is to provide a generic interface to programs, and arbitrate access to resources. Each program running on the system is called a process. Each operates as if it were the only process running. The kernel completely insulates a program from the implementation details of physical memory layout, peripheral access, networking, etc.

The first process executed is special in that it is not expected to exit, and is expected to perform some basic housekeeping tasks to keep a system running. Except in very specific circumstances, this process is provided by a program named /sbin/init. The init process typically starts a shell script at boot to execute additional programs.

Some projects have chosen to run their primary application as the init process. While this is possible, it is not recommended, as such a program is exceptionally difficult to debug and control. A programming bug in the application halts the system, and there is no way to debug the issue.

One feature of almost all Unix-like systems is the shell, an interactive command parser. Most common shells have the Bourne shell syntax.

A simple beginning

We shall now consider creating a minimal system. The approach taken here requires no additional hardware beyond the host PC, and the absolute minimum of additional software.

As already mentioned, these examples assume a Debian or Ubuntu host system. To use the QEMU emulator for testing, the host system must be supported by QEMU as a target. An example where this might not be the case is where the target system is x86-64, which QEMU does not support.

To ease construction of the examples, we will use the kernel's initramfs support. An initramfs is a gzip-compressed cpio archive of a file system. It is unpacked into a RAM disk at kernel initialization. A slight difference to normal system start-up is that while the first process executed must still be called init, it must be in the root of the file system. We will use the /init script to create some symbolic links and device nodes before executing the more-typical /sbin/init program.

This example system will use a program called Busybox, which provides a large number of utilities in a single executable, including a shell and an init process. Busybox is used extensively to build embedded systems of many types.

The busybox-static package is required to obtain pre-built copy of the Busybox binary and the qemu package is required to test the constructed images. These may be obtained by executing:

$ sudo apt-get install busybox-static qemu

As mentioned, our initramfs-based approach requires a small /init script. This configures some basic device nodes and directories, mounts the special /sys and /proc file systems, and starts the processing of hotplug events using mdev.

#!/bin/sh # Create all the busybox symbolic links /bin/busybox --install -s # Create base directories [ -d /dev ] || mkdir -m 0755 /dev [ -d /root ] || mkdir --mode=0700 /root [ -d /sys ] || mkdir /sys [ -d /proc ] || mkdir /proc [ -d /tmp ] || mkdir /tmp mkdir -p /var/lock # Mount essential filesystems mount -t sysfs none /sys -onodev,noexec,nosuid mount -t proc none /proc -onodev,noexec,nosuid # Create essential filesystem nodes mknod /dev/zero c 1 5 mknod /dev/null c 1 3 mknod /dev/tty c 5 0 mknod /dev/console c 5 1 mknod /dev/ptmx c 5 2 mknod /dev/tty0 c 4 0 mknod /dev/tty1 c 4 1 echo "/sbin/mdev" > /proc/sys/kernel/hotplug echo "Creating devices" /sbin/mdev -s exec /sbin/init

To construct the cpio archive, the following commands should be executed in a shell. Note, however, that INITSCRIPT should be replaced with the location of the above script.

$ mkdir simple $ cd simple $ mkdir -p bin sbin usr/bin usr/sbin $ cp /bin/busybox bin/busybox $ ln -s busybox bin/sh $ cp INITSCRIPT init $ chmod a+x init $ find . | cpio --quiet -o -H newc | gzip >../simple.gz $ cd ..

To test the constructed image use a command like:

$ qemu -kernel /boot/vmlinuz-2.6.26-1-686 -initrd simple.gz \ -append "root=/dev/ram" /dev/zero

This should present a QEMU window where the OS you just constructed boots and displays the message "Please press Enter to activate this console." Press enter and you should be presented with an interactive shell from which you can experiment with the commands Busybox provides. This environment is executing entirely from a RAM disc and is completely volatile. As such, any changes you make will not persist when the emulator is stopped.

Booting a real system

Starting the image under emulation proves the image ought to work on a real system, but there is no substitute for testing on real hardware. The syslinux package allows us to construct bootable systems for standard PCs on DOS-formatted storage.

A suitable medium should be chosen to boot from, e.g., a DOS-formatted floppy disk or a DOS-formatted USB stick. The DOS partition of the USB stick must be marked bootable. Some USB sticks might need repartitioning and reformatting with the Linux tools in order to work correctly.

The syslinux program should be run on the device /dev/fd0 for a floppy disk, or something similar to /dev/sdx1 for a USB stick. Care must be taken, as selecting the wrong device name might overwrite your host system's hard drive.

The target device should then be mounted and the kernel and the simple.gz file copied on.

The syslinux loader can be configured using a file called syslinux.cfg which would look something like:

default simple timeout 100 prompt 1 label simple kernel vmlinuz append initrd=simple root=/dev/ram

The complete command sequence to perform these actions, substituting file locations as appropriate, is:

$ sudo syslinux -s /dev/sdd1 $ sudo mount -t vfat -o shortname=mixed /dev/sdd1 /mnt/ $ cd /mnt $ sudo cp /boot/vmlinuz-2.6.26-1-686 VMLINUZ $ sudo cp simple.gz SIMPLE $ sudo cp syslinux.cfg SYSLINUX.CFG $ cd /mnt $ sudo umount /mnt

The device may now be removed and booted on an appropriate PC. The PC should boot the image and present a prompt exactly the same way the emulator did.

What's next?

This first step, while simple, provides a complete OS, and demonstrates that constructing an embedded system can be a straightforward process.

The next step is to expand this simple example to encompass a specific application, which will be covered in the next article.




source from http://www.linuxfordevices.com.
by Vincent Sanders and Daniel Silverstone.

Read More