RTC Magazine

Following the debut of PCI Express-based graphics applications in 2004, a wide array of product categories have adopted the technology. Server and storage systems, for example, have taken advantage of PCI Express (PCIe) bandwidth and scalability. But while PCIe technology had already been expected to expand to other market segments, no one anticipated just how widespread its deployment would be in areas such as communications, embedded systems, home entertainment and consumer electronics, for which a steady stream of products are now rolling out. Driving this rapid adoption across product categories is the ever-increasing demand for bandwidth, designers’ preference for pain-free migration from the conventional PCI architecture, and an abundance of various PCIe devices becoming available.

PCIe technology was created back in 2002 when major system OEMs realized that existing bus-based interconnect technologies such as conventional PCI and PCI-X had reached their peak. What resulted was a 2.5 Gbit/s serial interface with several advantages over conventional bus-based interconnect technologies. Table 1 outlines these advantages.

Since releasing that first PCIe specification in 2002, the PCI Special Interest Group (PCI-SIG) has introduced several enhancements to the PCIe base specifications to propel its use in a surprisingly wide range of additional applications. The following is a brief description of these enhancements:

1. Gen2 PHY Layer: A new physical interface for 5 Gbit/s links (Rev. 0.9 released). This allows high-performance computing to scale to the next level of performance. The systems equipped with Gen2 devices are expected to be released by mid-2007.

2. Cable Specification: The specification to connect systems through PCIe cable (Rev 0.9 released). This allows systems with PCIe ports to be connected together within the range of nine meters. Pre-standard cables have been shipping for almost six months.

3. I/O Virtualization (IOV) Spec: The specification for implementation of PCIe in multi-host and shared-I/O environments such as blade servers (Rev 0.5 released). This will reduce the cost of ownership and management for blade servers as the I/O’s resources will be shared amongst different CPUs. The spec is expected to be completed in the second half of 2007.

From servers to video/graphics applications to embedded systems, PCIe technology fits the needs of an ever-increasing number of industry segments—some highly anticipated, while others less so. In these applications, it provides scalable bandwidth between the CPU and I/Os in servers; matches the SAS/SATA and Fibre Channel (FC) data rates in storage; provides high-speed links to control or packet processors in communication; links real-time audio/video processors in consumer applications; and facilitates high-bandwidth serial links in many embedded applications.

Rackmount and Blade Servers

Most servers would fall in one of two classes—I/O servers and compute servers. I/O servers tend to have one to two CPUs and many I/O slots, or devices that connect to I/O resources, such as storage and communication devices. On the other hand, the compute servers have a lot of processing power (four or more CPUs) with fewer I/O resources. Typically, servers come in a 19-inch-wide 1-4U high rackmount casing and a comparatively new form-factor called the “blade server.” Blade servers are becoming increasingly popular as they offer cost savings over rackmount servers as well as lower power consumption, smaller form-factor and ease of management.

Servers started the transition to PCIe to take advantage of the bandwidth, scalability, broad ecosystems and ubiquity of this technology. The servers shipping today offer some PCIe slots, and while the majority of the slots are still PCI-X, we’re seeing a steady trend toward less PCI-X and more PCIe as chipset vendors eliminate PCI-X interfaces from the chipset.

Most chipsets on the market today offer a fixed number (three or four) of PCIe ports. However, servers used in I/O-intensive applications such as storage require more ports. PCIe ports on server motherboards can be increased by using off-the-shelf PCIe switches, with various lane and port counts, from vendors such as PLX Technology. Figure 1 illustrates the use of a PCIe switch to create more high-speed slots in a server application.