Data acquisition devices have come a long way, but since the first devices were created, one thing has remained the same: they have limited resources. In such embedded devices high performance is paramount, and small data acquisition platforms need to handle the data throughput at high rates, manage data relationships and process the information with minimal processing power so that more is available for the actual acquisition application.

A common misconception when hearing the term “database” is the idea that the database is synonymous with enterprise-wise relational databases or inefficient and slow personal databases. Managing data in embedded devices is a different matter than managing data on PCs and servers. Enterprise databases are not designed for embedded devices, and their subsequent adaptations result in large memory footprints, often entirely too large to fit into the target device. Enterprise databases do not translate well into embedded devices, and attempting to adapt open source implementations presents its own special set of headaches. The embedded database is another type of database that is dramatically different from the well-known enterprise databases.

When it comes to choosing the right embedded database for data acquisition devices, there are three main types: flat-file, relational and network models. Which is the best one to use for a particular job will depend on factors such as type and amount of data to be processed, as well as how frequently it will be used.

Flat-File and Relational Models

Essentially, the flat-file model is a set of strings in one or more files that can be parsed to get at the information they store. It is decent in storing simple lists and data values, and can get more complicated when trying to model more complex data. Modeling complex data creates a new set of headaches. One of the main issues with using flat files for even a semi-active database is the tendency to corruption. Generally, a flat-file database does not have a locking mechanism, which prevents data from being corrupted when multiple threads may simultaneously try to write to the database. Even when the device is designed for multiple threads, it is possible for two or more threads to cause a “race condition,” which could be prevented with a locking mechanism. A race condition may force the device to stall indefinitely, which is detrimental especially in an embedded device that normally does not reset easily. In a data acquisition device where reliability of data is also a priority, flat-file databases may not be the best choice.

The relational model stores data in tables composed of columns and rows. When data from more than one table is needed, a joint operation relates these different data using a duplicate column from each table (Figure 1). While the relational model is flexible, performance is limited by the need to create new tables to hold results from relational operations, and storing redundant columns. Even when designed efficiently, there are several sources of overhead. The overhead comes in duplication of data, in helping maintain database integrity, and a need for a foreign key to help maintain relationships in the relational database. The overhead results in excess in file size and extra I/O needed to perform basic database operation. Such overhead is especially expensive in resource-constrained devices.

Most developers are familiar with the relational database model, such as those from Oracle, Informix, Sybase, etc. Alternative data model architecture is more appropriate for resource-constrained devices such as data acquisition devices.

The Network Model

The network model is conceived as a flexible way of representing objects and their relationships. The network model predates the relational model and can be viewed as a superset. This implies that anything expressed in the relational model can be expressed in the network model, even SQL support. The main advantage is the way the relationships are modeled.

A primary distinction to the relational data model is that the network model allows designers to describe relationships between records using “sets,” where pointers are used to relate objects directly and navigate between them (Figure 2). When compared to the relational model, the network model is faster, more reliable, uses disk space more efficiently and is better at expressing complex database designs.