And then there were few

The founding fathers had a good idea when they adopted a consolidation strategy for government operations, instituting a federal administration for those tasks best centralized rather than leaving everything up to the states. Faced with the need to provide higher levels of service, tight budgets and staff retirements, agencies at all levels of government are adopting a similar approach to their information technology operations.'With the advent of the Internet, the best of both the mainframe and distributed-computing worlds merged,' said Scott Keough, director of enterprise solutions at government IT provider GTSI. 'Coupled with data center innovation like blade technology, virtualization, storage and backup, the Internet created a powerful deployment platform enabling organizations to benefit from lower costs through consolidation while empowering remote offices with control and performance.'Consolidation can achieve cost savings on space rental, hardware, licenses, personnel and training ' but only if done right from a technical, personnel and project management standpoint.'People look at data center consolidation as a technical thing,' said Fred Duball, director of the Service Management Organization at the Virginia Information Technologies Agency (VITA). 'Technology is the easy part. The hard part is the change management, getting people to understand why it is better and getting them motivated to make the consolidation succeed.'Federal data center consolidation has been going on for over a decade. In 1996, the Office of Management and Budget issued OMB Bulletin No. 96-02, 'Consolidation of Agency Data Centers,' directing the heads of all executive agencies to consolidate their data centers during the next two years. According to the bulletin, 'Industry experience suggest[s] operational savings of between 30 percent and 50 percent from consolidation compared to unconsolidated operations.'Many agencies have experienced the benefits from consolidation. But OMB's estimate of when agencies would see results ' 24 months ' could be a bit optimistic, especially if they must build a new data center.'A data center can be built in nine to 12 months, or 18 months for the bigger ones,' said Bob Sullivan, senior consultant at the Uptime Institute. 'But if you haven't done your two years of planning ahead of time, it won't turn out right.'Over the years, agencies have continued to consolidate data centers. The Defense Contract Management Agency went from 18 data centers to two. The Interior Department's National Business Center offers mainframe systems, Web site hosting, Unix, Linux and Windows servers, and disaster recovery services to agencies. At the state level, California, Maine, Oregon, Texas, Virginia and many others have their own consolidation projects under way.'People tend to forget that when you set up a little server center somewhere, you have to bring in generators, bring in power and air conditioning, in addition to the direct IT costs,' said Elbert Shaw, project manager at Science Applications International, who oversaw the consolidation of Army data centers and server rooms in Europe. 'There is also a lot of investment in people, training and licenses.'Shaw said consolidation also brings a big boost to redundancy and reliability. Under the Army's previous setup, each unit depended on its own Exchange server. With the consolidation, each data center had multiple, clustered Exchange servers. If one server in a cluster went down, the others would pick up the load. Then, if an entire cluster went down, it was mirrored to another cluster at that same data center. Finally, if the whole data center went down, the service was transferred to another mirrored site so there was no service interruption. That service availability wins over those who were reluctant to have their servers moved off-site.'Before, when a server in a remote site went down, it might be days before it came back up,' Shaw said. 'Once they realize their e-mail is always available, then they start liking it.'Shaw's project entailed consolidating IT services from 30 to 40 locations into two main data centers and two regional ones. The two main data centers were needed for failover. But SAIC found that a regional data center was needed in Belgium because of the unique needs involved in supporting NATO operations in the Benelux region. Italy also needed its own regional data center because of operational requirements, in addition to the availability of bandwidth over the Alps. The regional data centers fail over to the main ones, and the main ones fail over to each other.Because of the size of government organizations, most federal consolidation efforts take place within a single department or agency. At the state level, however, these typically are done across agencies. In 2003, Virginia created VITA to provide consolidated IT services to all state agencies. The state has more than 100 data centers, server rooms and closets, each requiring infrastructure and ongoing support.In 2006, VITA entered into a partnership with Northrop Grumman to provide IT services. The plan called for constructing two new data centers at a cost of $60 million that would house IT services for state and local agencies and school districts. The primary data center is a 192,000-square-foot Tier 3 facility, called the Commonwealth Enterprise Solutions Center, with 600 VITA and Northrop Grumman employees in Chesterfield County, just outside Richmond. This is backed up by a 101,000-square-foot disaster recovery facility in Russell County at the western end of the state. VITA started moving into the main facility in early July. The backup facility is scheduled to be completed this fall. Services will be transitioned from state agencies during the next two years.'We have more than 80 executive agencies,' Duball said. 'A dozen have full-scale data centers, and those would be good candidates for dramatic consolidation.'Although there are clear advantages to consolidation in many circumstances, it is not always appropriate.'These decisions are complex and agency-specific,' Keough said. 'If the applications or environment are small, critical, efficient or specialized enough, consolidation may not make sense.'The approach both the Army and Virginia took was to build the infrastructure and allow people to migrate over as they see the benefits. Shaw said that his group took the 'eat your own dog food' approach ' using the data centers themselves first.'They migrated the Signal Corps folks and G-6 first, so they had to live with their own pain,' he said. 'That proved that we were not afraid and also allowed us to learn a lot of lessons we could use with the other data center customers.'It also took a defined-services approach: Instead of trying to support whatever specialized software a particular unit was using, it built common services, such as Exchange, SharePoint and SQL Server applications that others could then subscribe to. Once people saw that they could get services from the signal community, they switched.Virginia, similarly, is not necessarily moving everything into the main data center. An agency may also have a particular mission-critical application that they want to keep close. 'We are not consolidating for consolidation's sake,' Duball said. 'Some things will have to remain at an agency location, but we believe the vast majority of the environment will be consolidated.'VITA will still manage the assets that aren't moved by installing management agents onto remote servers, PCs and network hardware.'By having that information in a CMDB, we can run reports to see what components need attention,' Duball said. 'That way, we can prevent problems from occurring and improve productivity for end users.'