Testing Devices for the IoT

Simulating Your IoT design; Will Your Large-Scale Deployment Work in Practice?

Simulation can be a thorough and efficient way to test an IoT network that physically will consists of large numbers of nodes that communicate wirelessly from device to Cloud.


  • Page 1 of 1
    Bookmark and Share

Article Media

Developing and testing Internet of Things applications and systems are big challenges, simply because the systems are big – they contain a lot of units. It is difficult to get hundreds of nodes into the software development lab for testing, and it is also difficult to provide all those nodes with interesting and realistic inputs.  When developing software that will run on hundreds or even thousands of IoT nodes, just how do you test that software in a practical manner? Simulation is a very good answer.

The IoT systems that are being built today often follow a three-tiered architecture as shown in Figure 1. There are many small nodes that connect to each other and to gateways using wireless mesh networks, and the gateways then connect to a management server or the Cloud. The small nodes can be sensors like temperature sensors, electricity meters, cameras, light switches, or actuators like thermostats, lights, and door locks. The gateways or concentrators handle the connection to the outside world, and ensure security. The back-end server, which is often in the Cloud, deals with the business and control aspects of the IoT system.

Figure 1
Simulation of a large network

To test this type of system, you want to have the wireless nodes spread out over a large area so that not all are in contact with each other, which requires using entire buildings or campuses as the “lab.” Setting up and maintaining such a network is a significant amount of work, with labor costs quickly dwarfing the cost of the nodes themselves.

In a simulator, as shown in Figure 1, setting up a large network is really easy. You just write a program to virtually deploy and spread out the nodes over the virtual space you need, and then model the wireless reachability between the nodes. Instead of manually handling hundreds of physical items, you manage a single script or program. Using a simulation solution like Wind River Simics to build this simulation, we simulate the hardware of each node, with processors, memory, timers, LEDs, wireless radio, and everything else that is needed. The simulated nodes run the real operating system and target applications, using the same binaries as would run on the real hardware. The different types of nodes are faithfully simulated, and run within the same simulation setup.

Figure 1
Simulation of a large network

Simulating the entire IoT system in this fashion allows you to test all aspects of the software, including things such as the wireless communications stack and how it deals with network problems, the sensing and actuator code and how it works with the environment, and the sleep modes and wake-up intervals on the nodes and how well they conserve power. Other software functions that could be also tested include the reporting function from sensors to gateways and on to server, the middleware that manages network nodes and updates software on the nodes.  This includes OTA updates, along with the security of the gateways and the nodes and the scalability of the data management system as the number of nodes goes up.

One particular aspect of an IoT system test that is a very good fit for simulation is testing system and software behavior as the system is scaled up. As shown in Figure 2 , simulation provides the ability to build systems of any size – from quite small to very large. This means that the behavior of the system can be tested on a whole range of scales, from small unit tests or subsystem tests, all the way up to the largest setups imaginable. Often, each system scale will reveal different issues in the system. It is not just about the very largest setups, but also about making sure things work efficiently at intermediate system sizes too.

Figure 2
Scaling up the network provides to ability to simulate large networks

Figure 2 also shows simulation of the environment that the IoT system operates in. Each sensor node will typically interface to a simulation of the world surrounding it – so that it has some data to send back to the gateway and server. An IoT node without a world around it is not all that useful. System testing will involve varying the simulated radio network conditions. In a simulator, it is trivial to impose particular signal strengths between any pair of nodes, and to implement rules that randomly drop packets as signal strength goes down. The configuration can be varied during a test, to check how nodes behave when conditions change, such as when a train passes across the line of sight between two nodes and interrupts radio communications for a short while. Best of all, such tests are precisely controllable and repeatable, unlike in the real world where trying to impose radio conditions is difficult at best.

Figure 2
Scaling up the network provides to ability to simulate large networks

Testing can also scale out horizontally, as shown in Figure 3. It is easy to build many variants of networks to test the software with different ways to deploy the same number of nodes. Different balances between gateways and sensor nodes can be tested, as well as different network topologies. Figure 3 also shows how simulation lets you run many different tests in parallel, which makes the total time to run a set of tests much shorter than if they had to be run serially on hardware.

Figure 3
Parallel simulation of different tests

But can it really work in practice to simulate hundreds or thousands of nodes on a single host computer? The answer is yes. IoT sensor nodes typically have a very low duty cycle. The sensors do not sense the world continuously, but rather, wake up regularly to take a sample and report it. Each sample run might take a second or just a few milliseconds, and then the system can be idle for minutes or even hours. This saves power and makes it possible to have nodes deployed in the real world for extended periods of time without having to service them to change batteries.

Thus, there is a large amount of idle time in the system, idle time that can be exploited to accelerate the simulation by using hypersimulation. Rather than playing out idle time cycle by cycle, a simulation solution like Simics jumps straight to the next interesting event that would wake up a sleeping node. That means that a system that is mostly idle can be simulated many times faster than real time, which is a property that is exploited in large IoT simulations. I actually did this myself a decade ago, when we ran 1000 IoT nodes on a single-core Windows XP 32-bit host faster than they would have in the real world! At the time, that seemed insane, but today it sounds like business as usual. 

In the end, physical labs are needed to perform final testing on your system. You have to test what you ship and ship what you test. However, using simulation to augment the physical test lab to cover more test cases and run more test variants is necessary to ensure that quality is maintained and that the system is robust across a wide variety of situations. With simulation, you will be able to build a better IoT system in a better way.

Wind River
Alameda, CA
(510) 748-4100