TECHNOLOGY IN SYSTEMS
Advances in ATCA
Boosting ATCA Hardware Platform Management for Power Subsystems
A new set of extensions to the AdvancedTCA specification includes additional means for power management and supports a new generation of sensors for more comprehensive monitoring and control of power consumption.
BY MARK OVERGAARD, PIGEON POINT SYSTEMS
Page 1 of 1
More than ten years after its adoption on the last day of 2002, AdvancedTCA will get a significant set of extensions later this year in PICMG 3.7, the AdvancedTCA Base Extensions specification, now nearing completion within PICMG. Among other things, these extensions enable larger and more powerful ATCA shelves, such as a single shelf with 16 double-wide boards (each board drawing up to 800 watts!), with 8 of them installed on each side of a dual-sided shelf.
A subset of these extensions relate to hardware platform management for the power-related aspects of these shelves, which potentially consume more than 12 kW each. Many of the extensions described here could be added via software changes to the hardware platform management layer of some currently shipping ATCA platforms.
One area of these management enhancements concerns “power geography,” rounding out the capability for shelves and boards to self-describe their power characteristics to enable more accurate operational tracking and diagnosis. These power geography enhancements also include coverage for AC-powered shelves, where the base ATCA specification covers only DC-powered shelves.
Another extension area is enhancing the sensor facilities of the Intelligent Platform Management Interface (IPMI) so that sensor values larger than 8 bits can be represented. One key goal is to enable integration of the wide ranging sensors defined by PMBus, an open protocol based on I2C for communicating with power conversion and other power-related devices. PMBus-compatible physical sensors are already being implemented in ATCA power subsystems from the bulk AC power supplies at the shelf level to the -48V power input modules on individual ATCA boards and the 12V point of load controllers that provide power for specific board subsystems.
Leveraging the Power Geography
Figure 1 shows the power subsystem of an example PICMG 3.7 shelf that includes four double-wide board slots. In each of those slots, a blue connector—called a Zone 1 connector in ATCA—on the backplane mates with a corresponding connector on the board to provide management and power connections for that board. Each board spans two slots, but connects to the backplane in only one of them. A slot position without a Zone 1 connector, for instance slot ØA1, is called a void slot. This example is assumed to show just one side (side A) of a dual-sided PICMG 3.7 shelf; the slot numbers include an “A” to indicate that.
Power architecture for example PICMG 3.7 shelf.
There are four -48V feeds in the example shelf, two from each of the Power Entry Modules (PEMs); each feed
supplies one of the two power inputs on each board. The PEMs themselves are each powered by three AC Power Supply Units (PSUs).
In a shelf built in accordance with the PICMG 3.7 enhancements, all elements shown here are fully self-describing, including which PSUs are connected to which PEMs, and which feeds from those PEMs are connected to which board inputs. With current ATCA, in contrast, AC PSUs are not addressed at all and the power path from a specific PEM output to a specific board input in a particular slot is not self-describing. The missing information can complicate power management and problem diagnosis in a shelf.
Figure 2 shows another extension in PICMG 3.7 where new or augmented data structures record the voltage ranges for which boards and shelves are designed and within which they are expected to operate in a particular installation. The design voltage ranges are established at the factory and the expected voltage ranges are configured when a shelf is installed and connected to a power source. In PICMG 3.7, it is possible to build shelves and boards that support a more restricted voltage range than ATCA. For example, designing for the enterprise voltage range (-43V to -57V) can allow cost savings compared to supporting the full ATCA legacy voltage range (-39V to -72V).
Expected and design voltage ranges in PICMG 3.7 enable detection and management of mixed voltage range configurations.
What happens if a board designed for a restricted range (such as the enterprise voltage range supported by the board installed in slot A4 in this example) is installed in a shelf that implements a wider voltage range? With the information available in PICMG 3.7, a shelf manager could choose not to enable power to such a board at all, to avoid risking an unacceptable operating environment for the board.
What if the example shelf shown in the figure is installed with no power source for PEM B? The expected voltages for the two feeds from that PEM would all be configured as zero. A PICMG 3.7 board can use a new Intelligent Platform Management Interface (IPMI) command to query the shelf manager for the expected voltages on its feeds. The shelf manager uses power geography data to identify the correct feeds and retrieves the expected voltages that have been configured for those feeds. In this case, when a board in the shelf detects zero volts on an input feed that has an expected voltage of zero, there is no need to raise an alert.
Extended IPMI Sensors in Hardware Platform Management
IPMI sensors that provide numeric sensor values are based on an 8-bit raw data value and a set of coefficients that can be applied to that value by a client application. This produces a processed value with appropriate scaling to reflect physical measurement units such as volts or amperes. For some physical measurements (including the temperatures
typically measured in an ATCA shelf), 8 bits of raw data is sufficient for management purposes.
In the power area, however, there is often a need to size data values for worst-case conditions while maintaining accuracy for finer granularity assessments. The physical devices used to monitor power-related properties in some currently shipping ATCA shelves capture greater than 8-bit data for voltage, current and power measurements. However, that data has to be reduced to 8-bit data for reporting via IPMI. For example, the 16-bit hardware sensor in some currently shipping PEMs can measure power in 0.2W increments, but when reduced to 8-bit data the IPMI sensor can only report power in 51W increments, thus sacrificing potentially important accuracy.
The PMBus architecture, which was designed for and by power monitoring specialists, defines two fundamental raw value formats. The PMBus direct format uses 16-bit signed raw values while the linear format uses an 11-bit mantissa and a 5-bit binary scale factor— both signed integers. Like IPMI, PMBus defines a set of coefficients and a conversion formula based on those coefficients to calculate processed values of sensor measurements. Furthermore, the PMBus coefficients and conversion formula, while similar to IPMI’s, are sufficiently different that PMBus-based sensors cannot be accurately represented with the existing IPMI coefficients and conversion formula.
Figure 3 shows the main levels of ATCA power subsystems, from the onboard power domains (such as one supporting a particular CPU socket), through overall board-level power, to the PEMs and PSUs that supply feeds in the overall shelf. As noted in the figure, there are existing power monitoring devices and products at all these levels that either: 1) support greater than 8-bit data without using PMBus or 2) implement PMBus and therefore trigger a need to accurately represent the measurements collected by PMBus-based sensors.
Availability of > 8-bit or PMBus-based raw sensor values for devices used in various atca power subsystems.
Extended IPMI Sensors in PICMG 3.7
Based on the needs described above, PICMG 3.7 defines facilities for extended sensors that support raw data values that are two to eight bytes in size, versus the one-byte limit for raw data values in normal IPMI sensors. For any given sensor, the size of raw values is fixed within that range, as appropriate for the measurements that particular sensor needs to represent. For instance, an extended IPMI sensor representing a physical PMBus current sensor that returns raw data in the 16-bit direct format could be configured for two-byte raw data. The coefficients for an extended IPMI sensor are represented in the same fixed size as the raw data.
In addition, an extended IPMI sensor can be configured to use either the normal IPMI conversion formula (with extended size coefficients) or the PMBus conversion formula. The combination of larger raw data values and the option to use the PMBus conversion formula allows PMBus sensor values to be fully and accurately represented by PICMG 3.7’s extended IPMI sensors.
With the PMBus linear format, specific raw values in a stream of sensor measurements could potentially have any values for the 11-bit mantissa and 5-bit scale factor fields. When such raw values need to be converted as extended IPMI raw values, the scaling factor needs to be applied at that time. Given a binary scaling factor ranging from 16 bits of negative scaling to 15 bits of positive scaling, a scaled 11-bit mantissa can require up to 42 bits. The 11-bit mantissa could be shifted left by 15 bits (with a positive scaling factor of 01111b) or right by 16 bits (with a negative scaling factor of 10000b). Therefore the raw value storage must be at least 42 bits in size. As a result, an extended IPMI sensor based on a PMBus linear format data needs to be configured for at least six-byte raw data, rounding up from 42 bits to the next 8-bit boundary of 48 bits.
The measurement of energy consumption is one area where the extended IPMI sensor facility is especially important. An energy consumption measurement can be considered the sum of a series of instantaneous power measurements over a known period of time. The longer the measurement time, the larger this sum can be, with a corresponding increase in the number of bytes needed to represent it. With the 64-bit raw sensor values supported in extended IPMI sensors, it should be feasible to implement realistic energy consumption measurements with reasonable assumptions on how frequently a client application needs to retrieve an accumulated energy consumption total to ensure that a rollover of the 64-bit integer sum is not missed, thus avoiding an incorrect energy reading. Client
applications can use floating point computation to maintain arbitrarily large sums with acceptable accuracy. Most microcontrollers used as local IPMI management controllers do not have floating point support in hardware and IPMI assumes that only integer arithmetic is supported.
The Service Availability Forum’s Hardware Platform Interface (HPI) is widely, but not universally, used as the foundation of system management applications above an ATCA shelf manager. At the HPI level, sensor values are represented natively as either already processed 64-bit integers or floating point numbers. Any processing implied by IPMI or PMBus conversion formulas and coefficients is applied in the HPI implementation layer below the application code. When that implementation layer is augmented to support the extended IPMI sensors here, existing HPI applications can use the resulting sensor values without needing any changes.
Pigeon Point Systems