Hypervisors and Virtualization for Multicore

Facing the Needs of Today’s Connected Embedded Devices

With the growth of the Internet of Things, some connected devices will be small, dedicated devices while others will have to combine real-time operation with full OS functionality and secure connectivity. Multicore processors with the proper underlying software offer big opportunities.


  • Page 1 of 1
    Bookmark and Share

Article Media

The embedded device market is exploding, with the “Internet of Things” predicting tens of billions of connected embedded devices being online by 2015. Many of these embedded devices will be relatively low complexity sensor functions, but others, especially when connected to the Internet or directly to humans, will take on personalities that become more like the computers and mobile devices used in the corporate and enterprise world. This means that they need to run more full-functioned operating systems, with connectivity and popular human interfaces; they will require more processing power, including multicore chips; and they are likely to be exposed to the same types of cyber attacks that we are facing in our corporate and personal computers.

The problem is that our traditional network and endpoint protection mechanisms are struggling today to protect our corporate IT infrastructure and computers, and adding billions of additional systems will exacerbate the problem dramatically. So, the reality is that we should build extra security and protection into these embedded systems before we connect them. 

The advantage that the embedded industry has over the traditional computer world is that the devices are generally being built for a specific purpose, and security and protection could be a design consideration for these systems. Providers of the traditional embedded software platform, the real-time operating system, are already offering additional security functionality to help protect the connected embedded devices. Often, however, as embedded developers use processors that are more like traditional computer and mobile systems (e.g., x86, ARM), they will often use operating systems from that world such as Windows, Linux or Android, to offer a familiar look and feel or connectivity options. At this point, security becomes more difficult to design into the operating system itself, and other protection or isolation mechanisms need to be used to help protect against cyber-attacks.

There is a technology available today that allows embedded developers to build security into their systems, regardless of which operating system is used. Originally designed and developed to meet the exacting security needs of the DoD, a separation kernel is a technology that provides isolation of devices and memory on a microprocessor-based system. It is particularly well suited to embedded devices as it is typically a very small and efficient kernel implementation offering real-time properties as well as isolation. In DoD systems, the separation kernel was used to help provide domain isolation for applications running at different security levels on a single hardware system, and protected domains from seeing each other’s data, network and applications.

What makes the separation kernel technology really useful for modern embedded systems is the addition of virtualization support using a hypervisor (see sidebar “Hypervisor Types”). This hypervisor runs on top of the separation kernel and gives each isolated domain the opportunity to run whatever operating systems and applications are required. This is a key feature for adding protection to traditional computer and mobile operating systems, without having to make changes to the OS itself. The OS runs in a protected domain, which can have restricted or no access to other parts of the system. Because the separation kernel/hypervisor sits below all the operating systems, it has a privileged position and can control the resources that each domain has access to. So, certain devices, areas of memory and network access can be restricted in a fine-grained manner.

The hypervisor essentially provides each of the secure domains a virtual motherboard (vMB) for the “guest” operating system that is running in that domain. The separation kernel interacts with the hypervisor to define what the vMB looks like. The types of parameters that can be configured for each vMB include memory allocation; available physical and virtual devices; how much or how many of the processors are allocated; and which other domains can be communicated with. This can be very useful in helping to isolate any cyber attacks as they enter the embedded device, restricting the view that they have of the overall system, and hence reducing the attack surface available to them. Figure 1 shows an example of a virtualized embedded system, where the type-0 hypervisor has split the system into two isolated virtual motherboards, and has split processors and memory between the 2 vMBs, allocated some of the physical devices directly to vMB 1 (with no access to vMB 2) and then offered device sharing for the network and hard disk, so both vMBs see their own virtual versions of the network and share a virtualized hard disk.

Figure 1
Type-0 hypervisor offers two secure virtual motherboards to guest operating systems.

Adding virtualization to an embedded system is now particularly interesting as more embedded processors contain two or more cores, and using virtualization is a good way of segmenting a multicore system in an efficient way, and offers consolidation of multiple physical systems onto a single multicore system. A flexible separation kernel/hypervisor solution will allow the embedded developer to look at the requirements of each guest OS and allocate the appropriate number of processors. This could involve sharing processors across multiple OSs, dedicating a single core to a single OS, and allowing OSs that support symmetric multiprocessing to access multiple cores. Figure 2 shows an example of how each of these schemes could be implemented on a single system

Figure 2
A flexible scheme for provisioning a multicore system using virtualization.

This flexible provisioning of the physical system also adds another benefit in the ability to select the right operating system for particular parts of the system, especially if real-time capabilities and computer OS capabilities are both required. Normally this issue would require adding real-time features and determinism to a computer OS. That is very difficult, even if the source code is available. Alternatively, all the facilities of a computer OS could be added to an RTOS, which again is a huge amount of work. With a separation kernel/hypervisor, the real-time component of the system can be handled by an RTOS running in one domain, and the computer/GUI part handled by a general purpose OS in another domain, with no compromise in performance and functionality and very little effort as neither OS has to be modified. 

With the wide availability of multicore embedded processors, it is very feasible to run multiple operating systems in a single system, providing the underlying separation kernel can provision the processor core accordingly, and also be able to facilitate inter-domain communication where required. Examples of embedded applications for this type of configuration include networked medical devices, where a user interface is required for the caregiver. The device could be networked to provide information from the device to either remote doctors with a networked user interface or to remote storage systems. There are also industrial control embedded systems where the device needs to be connected both to the control network as well as to the management network. An example of a mixed domain system on a multicore processor is shown in Figure 3.

Figure 3
Multi-domain, multi-OS, multicore system using a separation kernel/hypervisor.

Another key factor of connected embedded devices is network domain isolation, because many embedded devices need to be connected to a proprietary internal network, and potentially also to the Internet for remote access. Although encrypted secure communications is the traditional method of communicating sensitive information over the Internet, it is really aimed at protecting the information being transmitted rather than the computer/device that is attached to it. As embedded systems use more traditional computer OSs and then link to the open Internet, they are every bit as vulnerable to stealthy cyber threats as our normal computers. However, unlike our normal computers, these embedded devices could be controlling our critical infrastructure, and when infected, could pose a dramatic security breach, even affecting nation states. 

The latest super stealth cyber weapons include rootkits and bootkits that infect the system below the OS, residing on hard disks, in devices or in memory, which wait for instructions from a command and control center communicating over the Internet before actually starting the attack. Traditional network and endpoint protections are quite poor at detecting them, and general-purpose operating systems are very susceptible to them. As a result, our new connected embedded systems are vulnerable to them. If there was a way to detect these attacks as they occur, when they are in the early stage of infection, then our infrastructure would be more resilient to attacks than it is currently.

The separation kernel/hypervisor has the privileged position of being under the OS and controlling access to the hardware devices, so it actually has a chance of detecting these new stealthy cyber attacks if the right intelligence is built into the implementation. The LynxSecure separation kernel/hypervisor from LynuxWorks is now armed with a rootkit detection feature that has been designed to detect and alert when the rootkit has entered the system and is looking for a place to hide itself. This feature monitors key areas of the system (disc, memory or devices), and when those areas are accessed by an application, the hypervisor reports this activity immediately to either a management or forensic system to investigate this potentially malicious threat and make decisions on what actions to take. This action can include asking the hypervisor to shut down the guest OS to prevent the attack going any further while a remedy is found. In another case, the hypervisor could actually make repairs to the affected parts of the system, and then restart the system, thus cleaning it. This early warning system is critical to the defense of our connected embedded devices, as they become targets for the next wave of cyber threats.

In conclusion, the propagation of connected embedded systems, as predicted as part of the “Internet of Things,” is opening up a potential security hole for cyber criminals and cyber terrorists to take advantage of. By using a military-grade secure separation kernel/hypervisor, these threats could be mitigated and detected before the attack has really started. 

SIDEBAR: Hypervisor Types

Traditional hypervisors are widely used in IT environments where they can allow different operating systems and applications to run on the same server or endpoint. The two common types of hypervisor are type-2 and type-1. (Sidebar Figure)

Sidebar Figure

  • The type-2 hypervisors run directly on the native operating system and typically allow the user to run an operating system or applications that do not usually run on the native OS. A good example of this is running Windows OS on a Mac using a hypervisor. These type-2 hypervisors are not really suitable for embedded systems, as they do not have real-time properties, and more importantly they are very large as they are running Native OS+hypervisor+guest OS+ applications.
  • The type-1 hypervisors are often called “bare metal” and would seem to be more appropriate to be used in embedded systems as the hypervisor runs directly on the hardware. However, the type-1 hypervisor generally has a “helper” OS built into it, typically a variant of Linux, which helps to manage the hardware and devices for the hypervisor. The upshot of this is that the type-1 is almost as large as the type-2, but with better performance. This type of hypervisor is typically found in data centers and runs on server class processors.
  • Both type-2 and type-1 hypervisors are not typically built with security in mind, and both have large attack vectors for malicious code as they are running either on or with a large OS, which contains attack points such as device drivers.
  • The separation kernel/hypervisor solution is much better suited to securing embedded systems and can really be seen as a type-0 hypervisor. The separation kernel provides the security and real-time characteristics, and as it doesn’t use either a native or helper OS, its size and attack points are minimized dramatically.


LynuxWorks, San Jose, CA. (408) 979-3900. [].