SSA goes big on VOIP

The Social Security Administration is about a third of the way into a program to replace phone systems at 1,526 field offices nationwide with a single, centrally managed voice-over-IP solution that runs on the agency's data network.

By late February, the VOIP system had handled more than 1.6 million calls. SSA officials expect to add as many as 16 offices a week to the new system during the next two years, said Roderick Hairston, deputy associate commissioner of SSA's Office of Telecommunications and Systems Operations. When completed, the system will be one of the largest VOIP deployments in the world.

"For SSA to go with VOIP to this extent, they had to have a lot of faith in it," said Benjamin Moses, vice president of solutions at Nortel Government Solutions, the project's lead contractor.

The agency needed technology that could help increase productivity during the labor crunch officials expect in the coming years. "They are losing employees, and they are getting ready to deal with the baby boom explosion of retirees," Moses said.

SSA's network of field offices has 63,000 employees and a budget of about $10 billion a year. The agency paid $650 billion in Social Security benefits to 55 million people in 2008, and for years, it has been preparing for a wave of baby boomer retirements. SSA will be hit twice because many of its employees will reach retirement age at the same time the agency's workload spikes.

That process has already started. SSA's field office staff decreased by 4.4 percent from 2005 to 2008, according to a recent study by the Government Accountability Office. Visits to SSA offices increased from 42 million in 2006 to 44.4 million in 2008. Forty-four percent of the agency's staff is expected to retire by 2016, while the agency expects to be processing 1 million additional claims a year from retiring boomers by 2017.

Meanwhile, the agency's telephone infrastructure was aging. It had never been a single system but rather a collection of separate systems with different carriers at each office.

"Our system was more than 12 years old," Hairston said. "Some of it was past its end of life, and other parts were approaching it. We have been running out of parts," and the tasks of adding, dropping or moving lines were lengthy and labor-intensive. "We really had to do something to replace them. This was a perfect opportunity to look at what kind of system we needed."

What officials needed was a single system that could be centrally managed and offer a common look and feel for employees and customers. They wanted to be able to balance traffic among offices and re-route calls in case of an outage, and they wanted to be able to consolidate trunk lines to save money. The result of that assessment was the 10-year, $300 million Telephone Systems Replacement Project (TSRP).

Because it was a wholesale replacement rather than an upgrade, "it was pretty much a no-brainer" to go with VOIP, Hairston said. "We made the decision in 2003 to do it, but only after we did a pilot. It was a fairly new technology when we made the decision."

The pilot project was conducted from 2004 to 2006 at 43 offices, with separate systems operated by AT&T and Nortel as primary vendors. The results were satisfactory, and Nortel Government Solutions was awarded the TSRP contract in 2007. Implementation began in March 2008.

The system uses Nortel's Contact Center Manager Server, Communication Server 1000 for switching, Media Processing Server 500 for interactive voice response, Unified Messaging 2000 for voice mail and IP Phone 1100 Series handsets. Other members of the contractor team include General Dynamics, Black Box Network Services, Shared Technologies, York Telecom, High Wire Networks, NetIQ, Netcom Technologies, AttivaSoft and Pal-Tech.

There are separate parts to an IP phone system, Moses said. The IP telephony functions — the core switching and applications such as interactive voice response and call records — reside on the network and are independent of the means used to transport calls.

"VOIP is really transport," he said. "All we're doing is pointing to a [Session Initiation Protocol] trunk rather than to a [public switched telephone network] trunk. The applications are available either way." But VOIP allows trunk consolidation and better bandwidth use for improved pricing. "VOIP is a cost issue — it's not about application availability."

Combining the applications with the transport is the trickiest part of the implementation.

"If it was just providing a dial tone, it wouldn't be much more than putting a few servers on the network," Moses said. "This customer has an extremely interesting application environment."

One of the most unusual features of SSA's systems is the ability to partition call and performance data for each level of management. Managers log on through a Web front end, using SSA's authentication system, for access to the data that they are authorized to see. An office manager has access to data and security controls only for that office. District, regional and headquarters managers have access to progressively broader swaths of data.

Applications and dial tone are provided from four redundant service delivery points at Richmond, Calif.; Kansas City, Mo.; Durham, N.C.;and the SSA computer center in Baltimore. The service delivery points provide automatic failover in case of an outage.

"We have experienced an outage," Hairston said. "It works very well."

Each office has a separate gateway to the public switched telephone network for additional failover. In case of a local-area or wide-area network outage, the phone can automatically re-register to the PSTN. Emergency 911 service is directed through the PSTN gateway on a local trunk, so that the public safety answering point receives accurate information about the number and address from which the call is being made.

The handsets also do their part to ensure quality of service. Each acts as a voice quality agent, and when quality moves outside acceptable parameters, the system automatically alerts the service delivery point. Calls can be re-routed, problems identified and solutions launched without interrupting the call.

Ultimately, the quality of voice service depends on the quality of the network transporting it. Nortel works with SSA's Division of Network Engineering to analyze network health and troubleshoot problems before an office implements VOIP.

"In most cases, they are in pretty good shape," Moses said. "Our solution is somewhat tolerant" of network conditions.

"We learned from the pilot how to tune the network and get voice traffic where it needed to be," Hairston said.

SSA was fortunate that it had done a major network upgrade shortly before TSRP, during which it implemented Multiprotocol Label Switching. MPLS supports multiple services and simplifies traffic management for applications such as voice and video that are sensitive to latency. A few years ago, it was an expensive, leading-edge technology used primarily by large service providers to provision multiple services, but affordable high-performance MPLS routers now are available in a variety of sizes, making the technology practical for other organizations.

SSA still has a lot of work to do to transition field offices to VOIP, said Andre Pinto-Lobo, program manager at Nortel Government Solutions. "We are coming to the end of Year One," he said. The rate of implementation will increase during the second and third years and then slow again as the implementation concludes.

But "we're seeing the advantages already," Hairston said. System management has been simplified, there is better redundancy, and calls can be better routed to the proper person or automated service, which should please both employees and retiring baby boomers.