Measures of success

With a renewed emphasis on results, the government is pushing agencies to focus on the ends. Performance measures are the means.'Performance measures are both a vehicle for managing and a vehicle for communicating,' said Thomas Wegener, director of management information systems and research for the New York State Workers' Compensation Board. 'What you measure communicates the values of your organization.'As manager of the board's Performance Measures Project, Wegener helped the agency retool its business practices to optimize performance and results.A cornerstone of the initiative was the creation of an Electronic Case Folder to replace an archaic paper system. ECF streamlined the processing of hundreds of thousands of workers' claims.'It positioned us to distill the data from ECF and develop and implement a balanced set of performance measures for all of the business processes that were impacted by the re-engineering project,' Wegener said.For government at all levels, good performance measures are vital to maximizing results. You can't manage effectively without them.Office of Management and Budget officials say that because performance measurement keeps the focus on results, it is a central aspect of both the Government Performance and Results Act and OMB's Program Assessment Rating Tool (PART), which evaluates agency programs in terms of outcomes and results.'Performance measurement indicates what a program is accomplishing and whether results are being achieved,' OMB noted in its PART guide. Performance measurement 'helps managers by providing them information on how resources and efforts should be allocated to ensure effectiveness. It keeps program partners focused on the key goals of the program. And it supports development and justification of budget proposals by indicating how taxpayers and others benefit.' To read the PART guide, go to and enter 185 in the GCN.com/search box. Performance or outcomes measures reflect the ends that an agency is seeking to accomplish, said Thomas Darling, director of government and technology at the University of Baltimore's Schaefer Center for Public Policy.They put the spotlight on effectiveness'whether the agency is doing the right things to meet its mission goals, he said.'Inside each outcomes statement, there is a core concept,' Darling said. 'The trick is to try to measure that core concept.'At the federal level, program or mission performance and the performance of systems that support program objectives intersect with Exhibit 300 business cases, which agencies must supply to OMB to win funding for their projects.How do managers use business cases to align performance of systems with performance of the programs that those systems support?'Business cases ask for what contribution IT systems are going to make to program performance, and we ensure that those are consistent with the [program] goals that are assessed through the PART process,' said Robert Shea, OMB's counsel to the deputy director for management.'The PART assesses the quality of the measure,' he said. 'The business case requires an agency to state what improvements in program performance are going to occur as a result of the IT system, and those two have to be consistent. So they're mutually reinforcing.'Thus, program managers and IT managers have to be on the same page when it comes to measuring performance. In the final analysis, measures are all about mission outcomes, not about turf.Commitment to developing outcomes and measures also has to come from the top if the effort is going to succeed.'If you haven't got the big boss on board, it doesn't happen,' Darling said.Wegener agreed: 'You have to get strong leadership involved. So whose job is that? It's not the director of IT's job. It's the executive tier of an organization. It's absolutely essential that the top box in the organizational chart be involved with and supportive of a performance-measures project.'But observers say it's also critical to include stakeholders from all parts and levels of an organization.Staff members from an agency's lower bureaus or subdivisions tend to think in terms of their own outcomes, not those of the agency as a whole.'They are closer to output measures,' he said. 'They usually don't address the big, broad outcome you think about from an agency point of view,' Darling said.So frontline employees should play a role in developing the agency's overall outcomes and measures so they can see the bigger mission picture, he said.Though measures initiatives are fundamentally collaborative efforts, which part of the organization should oversee such projects?Wegener said measures projects don't necessarily have to be owned by a business unit. The IT side might be able to handle the job just as deftly.'One of the skill sets you find typically in an IT shop is the ability to implement a rigorous project management methodology, and that's really required for a performance measures project to be successful,' Wegener said.'So you might find that an IT shop would bring to the table the business analysis tools not too dissimilar from the business analysis tools you need to do good software development,' he added. 'You might find the ability to conduct a data assessment and to validate performance measures against the available data.'Developing outcomes and concomitant performance measures might be best accomplished with a more systematic approach, observers said.For example, the Indiana Revenue Department used Accenture LLP's public-sector value model to measure its performance between 1998 and 2002 as the department rolled out its Revenue Processing System, an online tax filing system.The PSV model 'is a tool we can use to see where we've been, where we are and what direction we want to go in the future'and a lot of that is driven by new technology,' said Ken Miller, Indiana Revenue commissioner.The PSV model mirrors the push toward focusing on results. It is based on the notion that government performance should be measured according to outcomes'the impact on mission goals of delivering a product or service'rather than outputs'the products or services themselves. Looking at outputs has been a traditional yardstick for government performance.For state revenue agencies, the model posits a small number of outcomes and measures an agency's performance by using compliance metrics that indicate its ability to deliver a specific outcome.For example, the PSV score for maximizing compliance rates is measured by such metrics as taxpayer error rates and the percentage of tax returns received on time.The analysis found that the Indiana department's performance improved steadily as core components of the electronic filing system were rolled out.The department expects its performance to increase as new technologies are incorporated into the system and the agency continues to align its IT projects with business outcomes, Miller said.Coming up with performance measures isn't always easy. And you're not always going to come up with perfect measures.'An automatic byproduct of successfully implementing performance measures is that you create a continuously improving organization,' Wegener said.