Mary Ann Davidson | In defense of common criteria

THE COMMON CRITERIA Evaluation and Validation Scheme has been heavily criticized lately (GCN.com/850). Devised as an independent evaluation of security products against a set of standard criteria, Common Criteria has been faulted for being expensive and not providing a foolproof measure to increase security.Not everyone shares these views.Mary Ann Davidson, chief security officer at Oracle, for one, feels Common Criteria has a number of strengths. The value of assurance is the extent to which a vendor embraces it across its development processes. That said, since every vendor of [information technology] products claims, 'Our product is secure: trust us!' having a third party validate the product against the Common Criteria is tremendously valuable to customers, who otherwise would have to rely on unproven security claims. Also, many vendors, including Oracle, view the Common Criteria as the starting point for assurance, not the ending point. The Common Criteria allows vendors to start their evaluations with a lower Evaluation Assurance Level and improve their processes to meet a higher assurance level over time. The higher in assurance levels you go, the more aspects of your development process the evaluators validate, and thus you need more process to meet the requirements. This avoids an all-or-nothing benchmark that few vendors could meet and allows them to improve their assurance over time. Automated vulnerability assessment tools do not come into play in Common Criteria until you reach those Evaluation Assurance Levels that are higher than those mutually recognized under the Common Criteria Recognition Arrangement. The national schemes that use such tools do not release them to vendors, which means they are of no use in helping improve product security.The main value of automated vulnerability assessment tools is finding and fixing problems during development, before products ship. Also, automated vulnerability assessment tools are just one component of a robust, comprehensive assurance program. Oracle uses multiple tools as part of its Software Security Assurance program. There is no tool validation program. Anybody can create a tool and ' even if wildly inaccurate ' claim they found a problem, and the burden is on the product vendor to prove that there isn't a problem instead of on the tool vendor to prove its tool is accurate.Automated tools can find, at best, half the common security defects in software, and they miss many design defects. Also, in a product with millions of lines of code, if the tool has a 90 percent false-positive rate, the vendor could spend thousands of hours in nonrecoverable time chasing false alarms instead of actually improving security. Finally, these tools do not validate whether a product has any useful security functionality. The National Information Assurance Partnership is merely one of the national schemes under which vendors can evaluate their products. Under the Common Criteria Recognition Arrangement, vendors can do an evaluation up to Evaluation Assurance Level four that is accepted in other venues. In Oracle's experience, we can evaluate a large, complex product like our database in about six months. The length of the evaluation cycle has not been an impediment to customer adoption of the product. A vendor going through an evaluation for the first time or who does not have well-developed development processes may take longer to go through an evaluation. Labs do not have the final say on whether a product completes an evaluation successfully; the national schemes that certify the labs do. A lab doing substandard work would face scrutiny by the national scheme. Generally, vendors shop for labs based on expertise and cost. We do our evaluations primarily in the United Kingdom and in Germany because we found these labs to have a higher level of expertise in Oracle software and an acceptable cost versus labs in other countries. Oracle continues to invest significant resources in building market-leading new functionality and products that we evaluate under the Common Criteria so that our evaluation-aware customers will feel comfortable using the products. We also continue to improve development processes; for example, we have formal processes addressing security vulnerabilities that we have included in our evaluations under flaw remediation. Common Criteria has a number of strengths. Common Criteria considers both threats and the technical remedies needed to counter those threats; a product must have actual security functionality and assurance proof points for it. Common Criteria evaluation assurance levels are graduated, so that vendors can improve their assurance level over time. Also, secure development processes need to be demonstrable and repeatable.Common Criteria is flexible. Vendors can assert a security claim, such as the use of automated vulnerability tools, through the target of evaluation they choose. And because of the Common Criteria Recognition Arrangement, evaluations are cost-effective. A vendor can do one evaluation that is recognized and accepted in multiple venues. Many vendors who complain about Common Criteria did not experience the pre-Common Criteria days when vendors evaluated the exact same product in multiple countries.As far as challenges, Common Criteria is a committee-led organization representing 24 countries and is thus often slow to change. There is a lack of transparency in its structure, its decision-making and its technical review processes that the Common Criteria Vendors' Forum has raised. The organization is slowly starting to address these issues. Any critical piece of software should be designed in consideration of what kinds of threats the product is likely to face and the appropriate technical remedy for those threats. As threats evolve, so should Common Criteria.That said, the National Cyber Security Partnership recommended the use of automated vulnerability testing tools at lower assurance levels, which could be a useful change, provided that all the criteria are met. The tools themselves must be evaluated and validated for what they find and how well they find it. The vendor can only make claims about use of automated tools as part of an evaluation based only on the tools they actually use in development. The vendor must assure that the Common Criteria Recognition Arrangement still applies: The use of these tools is mutually accepted in multiple venues. And, finally, the vendor must continue to maintain control of its source code. That is, the company is protected by contractual relationships it has with its labs but is not required to give source to any other party. Since the National Information Assurance Partnership is just one of the national schemes under which vendors can do Common Criteria evaluations, vendors who want the Common Criteria improved should work within the Common Criteria Vendors' Forum. Oracle also believes it is duplicative and wasteful for individual schemes to want country-specific variants that would require vendors to evaluate their products only under those schemes.