TECHNOLOGY IN SYSTEMS
The Yocto Project: Corralling Linux for Embedded
The Yocto Project: Challenges and Opportunities
Complexity in processors and systems leads to complexity in software, especially with a vast open source entity like Linux. The Yocto Project helps developers navigate the path through complexity, hardware compatibility, source dependencies and open source license issues.
BY CHRISTOPHER HALLINAN, MENTOR GRAPHICS
Page 1 of 1
In just over four years since its inception, the Yocto Project has become the de facto standard build system for custom embedded Linux distributions. Virtually every major semiconductor manufacturer and embedded operating system vendor has joined the project, as well as several device and board manufacturers and independent software vendors. Many other organizations large and small are using the project as participants. Several commercial Linux distributions are Yocto Project Compatible, the official badge of Yocto Project compliance (see sidebar Yocto Project Compliance Program. p. 22). The Yocto Project is sponsored by, and is a collaborative project of, the Linux Foundation (Figure 1).
The Yocto Project is an open source collaboration project that provides templates, tools, and methods to help developers create custom Linux-based systems.
Looking back at the history of open source projects, the level of commercial participation within the relatively young Yocto Project is nearly unprecedented. This can be seen as a testament to the utility of the Yocto Project. Some of the commercial products that have earned Yocto Project compatibility are currently listed on the Yocto Project website.
Why Such Rapid Success? A Technical Overview
The Yocto Project is an open source collaboration project that provides templates, tools and methods to help developers create custom Linux-based systems for embedded products—regardless of the hardware architecture. The Yocto Project provides a solid foundation upon which to build customized embedded Linux platforms. This type of collaboration brings together not only member and participating organizations, but also a large collection of open source projects that can be leveraged to build custom embedded solutions.
The Yocto Project integrates several open source projects to create a framework for building custom embedded Linux distributions. The foundation of this framework is called Poky, which is composed of technology from two upstream projects: OpenEmbedded and the BitBake build engine. The OpenEmbedded project hosts a large collection of build instructions, a subset of which forms the oe-core layer within Poky. BitBake is the build engine responsible for interpreting the build instructions, collectively called metadata, contained within Poky’s oe-core and meta-yocto layers.
Poky is both a build system and a reference embedded Linux distribution. Figure 2 illustrates the elements that make up Poky. Additional layers of metadata can be stacked on top of Poky including BSP layers, middleware software layers and customer-provided application layers. It’s worth noting that the Yocto Project documentation found within the Poky collection is surprisingly very good, given that open source software has a reputation for poor or non-existent documentation.
The various elements that comprise Poky.
The most common type of metadata is a recipe. A recipe can be thought of as a set of instructions for building a software package, especially within a cross-development environment, targeting a specific embedded hardware platform on a particular architecture (Figure 3). A recipe is a file containing build instructions (metadata) or more frequently a set of files in a similar fashion to a C source file that specifies #include files. The Yocto Project contains recipes for many hundreds of software packages that facilitate building these packages in a cross-development environment.
Simplified Yocto Project build.
In addition to software packages, the Yocto Project also contains recipes for building the Linux kernel for a variety of common reference boards such as the Beagleboard series based on processors from Texas Instruments, and the Sabre series based on processors from Freescale Semiconductor. It also has recipes for building a cross-toolchain and standard runtime library (C/C++), although most commercial OS vendors that market a product based on the Yocto Project ship with a commercial cross-toolchain, pre-integrated with a baseline Yocto Project reference distribution and build system.
Designing with Embedded Linux: Challenges
Saying that developing an embedded Linux distribution is non-trivial is like saying one hundred million lines of code is a lot. Of course it is! Software complexity is spiraling rapidly due in part to the advances in hardware platforms and system-on-chip (SoC) processors. Many of today’s embedded devices are targeted at the “Internet of Things” and contain SoCs with integrated peripherals such as advanced graphics engines supporting OpenGL/ES, SD/MMC and Flash storage, Wi-Fi, Bluetooth, Ethernet, USB host and device interfaces, and often contain hardware accelerated video and audio codec engines.
Building embedded Linux systems involves integrating hundreds of disparate software packages from scores of open source and commercial software repositories scattered around the globe. Moreover, a custom hardware platform requires a compatible Linux kernel ported to that specific hardware platform, and most likely a bootloader also customized to the hardware. A cross-toolchain tuned and optimized for a given architecture is also required.
Package version selection can be the most challenging aspects of integrating many unique and unrelated software packages. Indeed, many if not most of these software packages are developed and maintained by disconnected communities, and therefore are released with independent and often unpredictable cadences. Software packages that depend on other underlying software packages (for example, applications that depend on underlying system libraries) must be version matched for correct operation. While most developers would never break backward compatibility intentionally, incompatibilities occur with striking regularity between “related” packages.
Such incompatibilities are most evident in software packages that contain many dependencies, such as graphical and user-interface programs. These often have long dependency chains, requiring font libraries, widget libraries, device libraries (e.g., USB) and other system libraries such as network utilities and system configuration utilities. Consider mplayer for example, an open source media player. The Yocto Project build instructions for mplayer enumerates 21 dependencies (mostly libraries) required to build many of these libraries, which have dependencies themselves. All of these libraries and their dependencies must be tested and confirmed to work together, even though they are largely developed independently, on different release cadences and without formal cooperation.
Developers and the organizations they work for need to be aware of open source licensing issues. This is another significant challenge for manufacturers of embedded Linux devices. A typical embedded Linux distribution contains upward of one hundred or more different open source licenses. Take a look at an Android or other smartphone for an example of the hundreds of pages of license text found on these devices. Some organizations have policies preventing the deployment of certain open source license types, and this also presents a challenge. The Yocto Project has tools and facilities to help manage the legal challenge of deploying open source software.
Probably the most visible addition to the Yocto Project in the last six months has come in the form of additional member companies and project participants. Freescale Semiconductor joined as a gold level member. LG, Dell, and Renesas and OS Systems also joined as silver members. Six new entities registered as Yocto Project Participants in the last few months. A dozen new products or open source projects have registered and received approval by the Yocto Project Advisory Board in the last year. These include a GENIVI baseline, providing a reference foundation for an automotive Linux platform, as well as Linux products from Enea, Wind River, Intel and Mentor Graphics, among others. The Yocto Project website maintains a list of members, participants and products registered under the Yocto Project branding program.
Toaster: One of the most interesting new technical features of the Yocto Project is the “Toaster” appliance. Toaster was due out with Yocto Project 1.6 in April of 2014. Toaster provides the developer with Web-based viewing tools to look deeply into the build. Using Toaster, one can discover why packages were built, examine dependency chains and list the packages in a given image. One of the most useful features of Toaster is the ability to view each package’s metadata, and discover what files were responsible for modifying each metadata variable. Prior to Toaster, this was a tedious chore.
Hob: While not exactly new, Hob continues to attract developer attention within the Yocto Project. Hob is a graphical user interface for BitBake. Hob is most useful for a newbie to the Yocto Project to be able to make rudimentary changes and modifications to the project configuration. For example, Hob can be used to select what image type to build (graphical versus non-graphical, for example) and to select what BSP to build. Hob can also be used to add functionality to the base image. For example, using Hob, one can easily add software packages to the file system image. Hob has many configuration menus that allow the developer to select from a list of items, when it may not be apparent to the newbie what the choices are.
The Yocto Project has achieved traction in the industry because it enables a wide selection of software to be easily assembled into a coherent system. It will continue to be an important source of technology to both do-it-yourselfers as well as commercial vendors of hardware, software, operating systems and tools.