GAO: Common Criteria is not common enough
Connecting state and local government leaders
Auditors say the process takes too long and its effectiveness is not well-understood.
Many vendors understand the importance of getting products certified under a set of security standards called the Common Criteria Evaluation and Validation Scheme, but the organization that oversees the program has not done enough to educate agencies or vendors about it, according to a Government Accountability Office report released last week.
GAO also criticized the National Information Assurance Partnership (NIAP) for not providing metrics or evidence that the Common Criteria actually improves product security. In addition, the Common Criteria process takes so long to complete that agencies often find that the products they need are not on the list of certified offerings or that only older versions have been accredited, GAO's report states.
Products undergoing certification and accreditation can be obsolete by the time they are approved, said Daniel Kent, director of systems engineering for U.S. federal sales at Cisco Systems.
Ideally, the certification and accreditation process should take no more than six months, Kent said. However, in reality, 10 to 18 months is common, he said.
The government should establish centers of excellence for testing so agencies wouldn't have to duplicate their efforts and vendors wouldn't waste time and resources, he said.
It is possible to complete the testing process in as little as two to four weeks, said Helmut Kurth, chief scientist and lab director at atsec, an information technology security consulting firm that performs Common Criteria testing. That is fast enough to ensure that state-of-the-art technology can get out in the field.
'It's possible to do evaluation in parallel with development,' and labs and vendors must be prepared to do that, he said.
NIAP certification often is too slow for defense and intelligence agencies, said John Pescatore, vice president of Internet security research at Gartner. Only government labs can test at Common Criteria Evaluation Assurance Levels 5 through 7 ' the highest levels of scrutiny. NIAP now has fewer experienced testing employees and is not replacing them, which will further lengthen the process, he added.
To help remedy existing problems, NIAP program managers should create metrics that measure the program's effectiveness and collect data on the findings, flaws and fixes that resulted from NIAP testing, according to GAO's report.
Priscilla Guthrie, the Defense Department's deputy chief information officer, said in a written response to GAO's report that NIAP has been collecting such metrics since 2004 and is developing a template for an end-of-evaluation report that will review all changes to products and vendor procedures throughout the evaluation process.
The GAO report adds that Defense Secretary Donald Rumsfeld should order the National Security Agency and the National Institute of Standards and Technology, NIAP's sponsors, to develop workshops for agencies and vendors participating in the NIAP program.
Guthrie agreed that improving awareness and training is important. However, she added that NIST and DOD have cut support for NIAP to fund other priorities, making it impossible to allot extra money to such efforts.
DOD should instead direct partner vendors, evaluation laboratories and industry associations to create workshops using existing resources, Guthrie said. They should also get help from outside organizations, she added.
The problems the GAO report describes are not problems with NIAP itself, said Salvatore La Pietra, president and co-founder of atsec. 'It's easy for agencies to criticize NIAP, but they probably don't use the processes correctly in the first place' because they're not educated about them, he said. 'They have to do their homework.'
Pescatore said GAO's call for increased education and awareness of NIAP's function is overblown. Large vendors already know the process well and can afford millions of dollars for tailor-made product evaluations, he said.
Any education efforts should target smaller vendors ' with $10 million to $50 million a year in annual revenue ' that don't know about the NIAP process, don't know how expensive it is and have trouble affording it, Pescatore said. NIAP must do more than educate, he added. It must provide subsidies or reduce prices so smaller vendors can participate, he said.