Modular Systems for Industrial Automation
Matching I/O and Processor Performance Removes Data Processing Bottlenecks
PCIe 3.0 and attention to signal integrity prove high-speed VPX for connected embedded designs.
BY VINCENT CHUFFART, KONTRON
Page 1 of 1
Offering application-enabling data transfer performance, VPX VITA 46 is one of the first modular computer open standards to define a connector and backplane infrastructure that allows data transfers at rates in excess of one gigabit per physical channel. With this advance, VPX systems are able to implement the same state-of-the-art interconnect technologies that currently exist in server and consumer PC arenas. As a result, VPX users greatly benefit from the same level of performance for input/output, graphics and computer data exchanges, while the ratio of I/O throughput to CPU processing speed remains coherent.
These values address key design requirements in the growing realm of connected embedded applications, which often require non-stop, high-speed performance in rugged, automated settings. With high data rates over VPX, designers have access to a long-term deployment path for high-speed data transfer applications. At the same time, Generation 3 Peripheral Component Interconnect Express (PCIe 3.0) is emerging as a leading serial link technology for VPX, relying on point-to-point connections to manage high bandwidth traffic. This design approach is particularly important in proving VPX for high level sensor processing common to connected embedded applications.
Processing High Data Rates in VPX Platforms
The VPX specification acknowledges that high-performance data processing applications must not be bottlenecked with limited I/O performance. After having successfully addressed the 5 Gbit/s level for PCIe 2.0, VPX is now ready to adopt the higher data rates required by the latest version of two fundamental protocols including PCIe 3.0 at 8 Gbits/s and Ethernet at 10 Gbits/s. This affords a design advantage in moving away from parallel bus architectures, and lets systems instead rely on higher performance switch fabric options. Switch-based fabrics allow seamless implementation of all routes to and from boards dynamically—creating the bandwidth required for any application’s data flow, all within the same hardware. For designers of connected embedded systems, this enables a new breed of unparalleled applications for high-performance data processing platforms.
Apart from VITA 46, OpenVPX VITA 65 (built on top of the VPX norm) is the subsystem level standard specifying how VPX modules and backplanes interoperate, in which protocols and with what pin assignments. The current ANSI ratified OpenVPX version is “ANSI/VITA 65-2010 (R2012) OpenVPX System Specification” approved in 2012. This R2012 version of VITA 65 defines a nomenclature for daughter modules and backplanes up to a rate of 6.25 Gbaud, which is sufficient to run protocols such as Serial RapidIO and PCIe Gen2. The next revision currently under consideration includes support for PCIe 3.0 and 10G Ethernet up to a rate of 10 Gbits/s.
The rise of VPX in industrial deployments has been a long time coming, and overall VPX revenues were projected by industry analysts to match VME by 2012. This shift is related primarily to military design and hasn’t evolved as expected based on cuts and ongoing uncertainty in military program spending. New data from IMS Research shows that 2016 is a more likely timeframe for a VME to VPX crossover, with other market sectors picking up on VPX technology as well. Members of the VITA Standards Organization are reporting VPX design wins that illustrate the potential of this platform for industrial arenas.
Where an existing distributed application might exchange information on Gigabit Ethernet, for example as implemented on most VME or CompactPCI platforms, VPX platforms can implement the TCP/IP protocol over the PCIe 3.0 infrastructure. This simple method taps into high-speed PCIe 3.0 bandwidth for data transfers by selecting a different IP address to connect to the other boards, handled by VXFabric, Kontron’s open infrastructure, which implements efficient inter-board communication at hardware speed. No software coding changes are required, protecting applications from obsolescence as well as the complex, low-level details of the current generation of PCIe silicon management.
Building Value with Switch Fabric Options
Switched fabric bridges current gigabit Ethernet on the backplane (in industrial platforms such as VME, cPCI, VPX) and the next data plane generation of 10G and 40G Ethernet. Industrial applications seeking a cost-effective performance jump can today access 10G and 40G performance in compact VPX-based systems. Featuring low power consumption and harsh environment capabilities, rugged VPX and PCIe 3.0 work together to address all fast and low latency peer-to peer inter computer node communication within a chassis. This holds promise for enabling significant advances in image processing—for example 3D reconstruction applications such as radar and sonar—while also reducing system footprint.
PCIe 3.0 is the enabler here, facilitating point-to-point connections between boards as a means of managing high bandwidth traffic. Backplane routing is specific to each application, and is used to create connections between boards that match the application’s data flows. PCIe 3.0 switches allow systems to combine different data types in a single converged pathway. For example, data (compute, communication or storage) is created and consumed as PCIe on each of the slots in the rack, delivering efficiency both in hardware architectural and software usage. Proving this in defense electronics, Kontron has developed a set of VPX modules that take advantage of these high data rates over the VPX backplane. The PCIe 3.0 protocol operating at 8 Gbits/s and the Ethernet 10GBase-KR links were successfully qualified at extended temperature environments (Figure 1).
Built around PLX ExpressLane PCIe 3.0 switches, Kontron’s VX3042 and VX3044 Intel Core i7-based single-board computers (SBCs) routinely achieve 5.6 gigabytes per second (Gbyte/s) in data throughput between any boards in a VPX rack.
The availability inside a VPX chassis of a high number of switched 10 Gbit/s Ethernet links is changing the way parallel processing can be organized. A flat CPU board interconnect could be easily set up using standard software architecture. In a real-world scenario proven by a Kontron-designed reference system called StarVX, more than 20 CPU boards could compute in parallel, exchanging data at Gigabyte speed without any bandwidth limitation due to the target board position (Figure 2).
None of the VITA 46 or VITA 65 standards specify the requirement for high speed interoperability of modules and backplanes; this includes timing, eye openness, jitter budget and more general electrical channel characteristics. Rather they point to the electrical requirements inside the existing base protocol standards such as PCI Express or Ethernet 802.3, and also to the VITA 68 standard, currently a draft.
Addressing Signal Integrity Challenges with 10 Gbit/s
Designing each channel—or single differential copper pair—requires attention to a variety of factors to ensure ideal signal integrity for high-speed I/O processing in a rugged VPX system. For example, stubs can limit channel performance; higher I/O speeds demand stubs to be virtually nonexistent in order to avoid influencing signal integrity. Stubs are short conductive tracks that don’t belong to the main signal path. Actually, any protrusion from the smooth path of the signal transmission line is considered a stub and can have a direct negative impact on signal processing. At 10 Gbits/s, with signal traveling in copper at close to half the speed of the light, each bit transiting over the backplane is separated from the next bit by less than 2 cm. For example, short stubs of 5 mm would typically reflect part of the signal back to main path and to the transmitter, thereby affecting half of the length of a following transitioning bit, in addition to the losses resulting in the main direction forward.
It sounds simple, yet stubs are everywhere. The connector contact itself, depending on its technology, could be a source of stubs, for example a rear VPX connector with populated contacts. Even a metalized via drilled in the PCB, enabling transition from one layer to another, could create a significant stub. The backplane itself, typically 5 mm thick when it is designed to host front and rear VPX connectors, will also exhibit stubs at the VPX connector via unless they are removed by using techniques such as blind via or back-drilling (Figure 3).
These views illustrate how design techniques such as blind and back-drilled vias can eliminate signal integrity issues caused by stubs.
Impedance mismatch inside vias—or the transitions between boards—must remain consistent; differential impedance of pairs carrying data must be 100 ohms +/- 10% according the IEEE 802.3 standard. This is a critical design detail, as impedance discontinuities along the signal path create partial signal reflections and ringing, resulting in degradation of attenuation and return loss parameters. However, inside the via of VPX connectors, both at the module connector and the backplane connector, differential impedance is generally much lower and runs in the range of 80 ohms and below.
Good signal design practices will also focus on managing crosstalk, PCB losses and length mismatch. Appropriate gaps and reference planes manage crosstalk, and the right PCB materials help avoid high attenuation that could lead to high Inter Symbol Interference (ISI), a phenomenon defined as distortion due to differing attenuation of bit sequences. Most protocols do not require a precise differential pair length adjustment. However, the plus and minus components of the pair should be the same. Length mismatch directly impacts the eye width, and accordingly, the bit error rate, and requires greater compensation when it occurs inside the connector itself.
Proving Performance for Industrial Embedded Applications
Connected embedded systems must frequently be designed to withstand the toughest environmental conditions: extended temperature ranges (-40° to +85°C), humidity, vibrations and power fluctuations. Even heat build-up and energy absorption have to stay within narrow ranges, for example in remote deployments or harsh manufacturing settings, where only passive cooling technologies are permitted. Rugged VPX fits the bill; it supports PCIe 3.0’s high-speed connections in harsh environments, and VXFabric readily bridges the gap between this disruptive technology and currently deployed applications exchanging data on gigabit Ethernet. Further, TCP/IP applications run unmodified on proven, ruggedized VPX platforms, protecting software investments over long-term deployments.
10 Gbit/s baud rates are starting to be deployed over the VPX infrastructure, which allows standard protocols such as Ethernet and PCIe to run at scalable speeds defined by the number of lanes forming a port. Carrying multi-gigahertz signals, VPX connectors and backplanes are enabling systems where boards no longer share available bandwidth and are capable of one or more dedicated 10 Gigabit connections via Ethernet or PCIe. Processing rates are backward compatible with speeds of the previous generation protocol, and can be activated only where necessary in the critical portion of the data path. The CPU to I/O ratio is kept in the same range as non-embedded computer equipment, based on cost-effective, standard software architectures. Proven with reference systems and defense electronics deployments, this combination of technologies provides significant opportunity and advantage for designers—supporting innovation and allowing access to a new breed of applications for high-performance data processing platforms.