Standard approach to computing spells doom for data centers

Mike Dillard was the first chairman of the CIA's Information Policy Board, serving during the time that Bob Gates was the director. This board set many of the IT governance policies and processes that are still active in the intelligence community and federal government. He now is president of Criterion HPS, Inc.

Last summer, I visited a critical operations center in the federal government and noticed that many of the analysts were shutting down their desktop PCs. I was shocked to see this in the middle of the afternoon at a 24-hour center and even more shocked by the reason.

It turned out that the building did not have enough power to simultaneously cool the building and keep the computers running on a hot summer afternoon. So, as a matter of course, the employees power down all but the most important systems. This was not an out building somewhere, but a major government office complex inside the beltway with thousands of employees.

Hardly a day goes by that I don’t hear from colleagues that they have run out of power, space, and cooling. Virtually every conversation circles back to: How did we get here? Why is this happening? And what can I do about it?

I began to think back on our decisions and strategies over the past 20 years and wondered if we really failed to equip the next generation of chief information officers with the infrastructure needed to support today’s business applications? What emerged from this review is that we have left a legacy of policies and attitudes that have become institutional hand-cuffs to really dealing with today’s space, power and cooling issues.

During the early decades of the computer revolution (and especially during the 1960s and ’70s), we all had easy hardware decisions to make. We just bought large mainframes from IBM and occasionally from other vendors. Once in operation, they had so much computing power that we ran them at a “pegged rate” of around 50 percent. And we forced all software and applications to run on those platforms. It made for a very simple, yet costly, system that lasted for an entire generation.

In the ’90s, we were hit with a barrage of new server platforms from what seemed like an endless array of manufactures. At the same time, we saw an explosion of new types of programming languages and techniques. As our computing environments became chaotic and maintenance costs kept climbing, most CIOs and IT executives decided to put in controls.

At the CIA, one of the first actions of the Information Policy Board, which I chaired, was the creation of an approved products list and the installation of a common operating environment. And it worked for a while – maintenance costs were reduced by 8 percent and we were better able to control personnel and training costs.

These actions were good at the time, but this mentality is now killing our data centers and networks in terms of power, space, and cooling. What has evolved are computing centers “standardized” on platforms, many of which are running at less than 20 percent efficiency. Servers or computing platforms are not tuned for the applications they run and servers are not purchased for efficiency. Most CIOs today don’t even measure or know the total efficiency of their computing environments.

I visited another large government agency that is standardizing on servers from a well-known manufacturer. The agency made this decision based on purchase cost and comfort, and with very little consideration given to power efficiency and the requirements of today’s computing environment and applications. By some estimates those servers could run at less than 25 percent capacity. To make matters even worse, they selected a common storage system that further exacerbates the problem.

As a result of this incredibly shortsighted vision, the agency has locked into a very inefficient system for the next decade at a time when electrical power is going to grow more limited and expensive. And they have locked the software engineers into old architectures that don’t take advantage of the rapidly emerging, highly modular and customizable platforms. These new platforms are extremely dense, use very little power, and leverage multicore technologies.

We can no longer afford to make such poorly thought-out decisions that result in running inefficient, underutilized, under-tuned generic platforms that are power and space hogs. And to standardize on these is even more preposterous. No power grid in the country will be able to support the computing required for the massive amounts of processing and data crunching that is coming.

A new paradigm is emerging, born of necessity. Early adopters are already moving in this direction. This new paradigm assumes that applications, operation systems, memory, processing, storage and I/O must be matched for efficiency. Benchmarking on this new paradigm is underway. As I analyze these results, it is clear they indicate that highly dense computing that is focused on tuning the application to the compute platforms can often perform at 100 times the efficiency and often perform 10 times faster than conventional systems. In one case, there was a considerable reduction in heat generated to do the same task.

New measures of performance are emerging to help information technology executives. For example, measuring performance by processing-threads-per-watt and other metrics are needed to fully understand just how much more efficiently we can operate.

We can no longer afford the excess of standardized environments. We have to be more efficient. We need to adopt new computing approaches. We must learn to tune our systems.

We have built up our IT infrastructures around a computing concept that is now killing us. We have a unique opportunity to improve operations and at the same time pave the way for green. And we don’t have to wait. These technologies are here now and can be deployed immediately