RTC Magazine

Sometime soon the computer system in an industrial machine responsible for capping a popular cola, or welding the frame on an automobile, or controlling the operation of a city’s power generation, will go haywire, bringing the works to a halt. An IT technician will be dispatched to analyze the problem and after going back to the shop for parts, he’ll have the machine running again, albeit a couple of hours or more later. Site visits are the bane of CIOs everywhere. They’re costly in IT man-hours and corporate productivity. For those IT organizations charged with the care and feeding of computer-controlled industrial machines, there is something to be learned from your corporate IT counterparts about advancements in remote management of those systems. And for you IT pros in the corporate realm, whose dominion is the white-collared world of notebook PCs in neat cubicles, pay attention—there’s something to be learned from blue-collar IT.

To help guide IT practitioners at industrial and manufacturing organizations, research firm Wipro Product Strategy and Architecture took a look at the costs associated with industrial PC and embedded systems management. Wipro’s “Reducing Industrial PC and Embedded System Support Costs with Intel vPro Technology” involved interviews with companies in automotive and automotive parts, communications, electronics manufacturing, food production, high-tech manufacturing, industrial machine manufacturing, power generation and utilities, semiconductor fabrication and water disposal and environment. In particular, it analyzed the impact of remote management of industrial PCs, computer numerical controllers (CNCs), motion controllers (MCs) and human machine interfaces (HMIs) controlled by systems based on Intel Core vPro processors (Figure 1).

Intel vPro technology on Intel Core processors is familiar to corporate IT as it comes on all of Intel’s top processors for business PCs. However, it might come as a surprise to these IT managers just how prevalent Intel vPro is on computer systems controlling industrial machines and how the advantages gained on the factory floor might benefit them as well.

Intel vPro comprises a package of business PC hardware and software technologies that provides IT workers with high levels of computing performance, but equally important are more remote capabilities that lead to easier, more cost-effective management and maintenance of computer fleets. It is a familiar technology to a broad number of IT professionals because it goes a step further than other remote management technologies with its out-of-band capabilities, which enable it to remotely access a computer even if it is powered off, or the OS and hard disk are inoperable, to manage, diagnose, repair, inventory and patch their PCs without stepping outside their offices.

To make its findings more relevant to IT professionals, Wipro used the data compiled from interviews with 23 firms in Germany and Japan to create the “average” industrial firm. That typical organization contains 134 HMIs, MCs, CNCs and industrial PCs, which are refreshed on a five-year schedule with a variety of models from different manufacturers. They are deployed across 30 facilities and about 80 percent are connected to the corporate network. They are regularly inventoried, patched and audited. Depending on the system type, the groups of systems experience 2 to fifteen security incidents a year. External equipment or service providers care for forty-three percent of the equipment (Table 1).

The Wipro study backs up what we all know: the cost of servicing computerized industrial machines and resolving problems faster to reduce downtime are our two biggest concerns, far outpacing still important issues, such as flexibility, extended lifecycle, asset visibility and scalability.

“It cost how much!?!”

Intel vPro remote management provides a number of cost-cutters. Many more application and hardware failures can be handled remotely because of vPro’s out-of-band capabilities. Even if a part requiring a visit is needed, remote diagnosis can determine that in advance, enabling the technician to take the part with him, saving a second trip. Security incidents that brought an industrial machine to its knees could also be repaired remotely through the out-of-band connection. Finally, the efficiency gained from reducing the overall complexity of systems makes a significant impact on the bottom line.

Virtually all of the companies interviewed have plans to reduce power consumption. Though motivation varies—green PR, government regulation, internal cost cutting—the solutions were the same. We noted earlier that the Wipro model company let its systems run 19 hours daily. However, while drawing power, they are only in production 14 of those hours. According to Wipro, over their five-year life spans, each of those systems will cost the company $18,913 while hanging around a figurative water cooler. One likely reason for leaving them on is to allow remote maintenance, if necessary. For those companies that use them, Intel vPro’s out-of-band capabilities enable them to remotely shut those systems off, either automatically or manually, at quitting time, and if remote maintenance is needed, turn them on only long enough to complete it.

According to Wipro’s research, by replacing its computer-controlled machines on its current refresh schedule with Intel vPro-based systems, the Model Company will return net present value (NPV, the accounting method for determining the profitability of a project) of $485,000 or $1,414 per system to the corporate coffers. HMIs and industrial PCs see the biggest reward—$1,956 and $1,831 per system, respectively, because the average company deploys a wider variety of models from multiple vendors in these areas (Figure 2).

In many companies, security concerns have traditionally demanded an “air gap” in the network between parts of plants, and the corporate and external worlds. While still common today, Wipro anticipates this architecture will continue to erode as security and access technologies evolve, bringing the cost savings of remote management to these areas. Pressure from the corner offices for real-time production and efficiency data, and from plant managers to simplify their operations is driving this transformation. The System Defense features of vPro can play a role in this migration by helping block incoming viruses and other threats, and by isolating infected systems to stop further spread.  In addition, because vPro is hardware-based, factory workers can’t inadvertently or intentionally meddle with the system.

“Fix it fast!”

Unlike their corporate counterparts, factory IT professionals often manage systems in secretive, inaccessible and dangerous environments. Hey, it’s a factory out there! Obviously, the more comprehensive the remote management, the quicker and less costly a problem can be attended to in a clean room, power generation plant or smelter. One of the participants in the Wipro study, a power generation plant, provides a good example of where Intel AMT can provide the needed virtual access.

The power plant has 400 HMIs, many situated in very difficult to reach spots. The firm already employs a number of good IT practices to reduce costs and speed up repairs, such as using a single vendor and refreshing with one model annually over its five-year cycle. However, that doesn’t get a human any closer to some of the plant’s HMIs. By increasing their remote reach into the machines with Intel vPro, IT no longer needs physical access, speeding up repairs. In addition, the plant saved an NPV of $451,000 ($1,408 per system) after implementation costs were deducted.  Most of the savings no longer required IT technicians to manually handle application failures, security patches and audits.  While not included in the computation, going remote also eliminated system downtime.

The figure from the study that will chill any CIO is the average cost of $15,650 per hour in lost production when a computer fails and the machine is out of service. The cost of downtime varies widely, depending on product value, point in the production process and availability of a back-up machine. Wipro therefore made conservative calculations, assuming that only 10 percent of problems will actually bring a production line to a halt. Still, that lost opportunity cost equates to $24 million (Table 2). vPro, of course, can’t completely prevent downtime, but previous studies demonstrate it is capable of keeping systems online 60 percent of the time. So, with Intel vPro-based systems, the Model Company would prevent $14.3 million in lost revenue.

 “What do you mean it needs a 165-volt power outlet?”

One final consideration that was apparently overlooked by 70 percent of the firms interviewed for the study, is the benefits associated with standards-based management tools. Only 30 percent were taking advantage of standards-based solutions. Others were either using equipment vendors’ solutions or internally developed tools. Looking at the variety of vendors many companies are using, the former and likely the latter lead to unnecessary management complexity and costs.

Another advantage of building around vPro processors is the availability of continuously advancing offerings from the broad ecosystem of standards-based software and hardware providers. Their products are largely optimized for vPro platforms. There also tends to be lower development and programming costs in the construction of industrial computing environments. 

Intel
Santa Clara, CA.
[www.intel.com].

Wipro
Bangalore, India.
+91 (80) 28440011.
[www.wipro.com].