Challenges grow up

A cheaper and more collaborative way of getting access to innovative technology could be as simple  as posing an intriguing challenge and offering a prize to whoever solves it. That’s become increasingly apparent in the four years since the American COMPETES Reauthorization Act of 2010 gave agencies authority to conduct competitions to spur innovation, solve tough problems and advance their core missions, according to the most recent annual report on the effectiveness of challenges.

Meaningful technology and ideas can be generated from these competitions. In 2015, a challenge created an architecture that improved the International Space Station’s email system so that it could handle large file attachments. There were 24 entries for the 18 challenges that were part of the email initiative. The overall cost of the challenge was $81,000; NASA estimates it would have spent $193,000 to develop this technology independently.

Overall, there were 116 prizes offered by 41 federal agencies in fiscal 2015, according to the report. Of these challenges, 83 percent had multiple goals and 47 percent were looking for multiple solutions.

There are headline-grabbing challenges such as the Defense Advanced Research Projects Agency’s Cyber Grand Challenge where competitors fought it out for a seven-figure first-place check. In the most recent iteration of this event, bug-hunting machines competed to find vulnerabilities in a simulated environment. The winners walked away with $2 million.

DARPA CGC Program Manager Mike Walker said events like this are an important part of finding answers to important questions. “I believe crowdsourcing was the right answer to that question and that getting global innovation in on the problem helped us get a much better result today,” he said.

But some of the challenges fly under the radar, offering nothing but bragging rights and drawing only modest participation. The Federal Trade Commission’s DetectaRobo challenge in June 2015 asked contestants to create algorithms to predict robocalls. There were 19 teams that signed up for the event even though no prize money was offered. Winning robosleuths  Ved Deshpande and M. Henry Linder walked away with nothing but a great item for their resumes.

As agencies have conducted more successful challenges, the process has evolved.

Partnerships have become part of recent challenges, with the number of cooperative challenges jumping from 56 percent to 68 percent between fiscal 2014 and 2015, the report noted. A partnership among the National Football League, GE, Under Armour and the National Institute for Standards and Technology for Head Health Challenge III, for example, sought to find innovative materials that absorb and dissipate energy to help protect against traumatic brain injury.

Beyond the administration’s role in laying the policy and legal groundwork to allow agencies to take maximum advantage of challenges and actively promoting their use, the General Services Administration and NASA support the challenges community through the Challenge.gov  program and NASA’s Center of Excellence for Collaborative Innovation (CoECI), respectively.

GSA’s Challenge.gov platform is available at no cost for federal agencies to host crowdsourcing competitions. Its experience over five years of running challenges -- case studies and step-by-step process guidance -- will be incorporated into a prizes and challenges toolkit slated to be released later this year.

The CoECI works across the entire federal government to help other federal agencies understand and conduct challenges. It has contracts with a global community of companies that provide crowdsourcing platforms so it can provide services on all aspects of implementing challenge-based initiatives, which allows other agencies to rapidly experiment with these new methods without standing up their own capabilities.

Other agencies have developed their own policies and guidelines for competitions and challenges. The Department of Health and Human Services led the way with the creation of a prize authority in 2011 and the establishment of the department’s goals and strategies for implementation. But NASA, the Environmental Protection Agency, Department of Homeland Security and others also have their own policies on challenges. Some of these include ways to streamline vendor contracting.

Third-parties are often used to design and implement challenges, and agencies have taken steps to simplify this process. The Office of Career, Technical, and Adult Education in the Department of Education awarded a five-year IDIQ contract to a develop and conduct its challenges in which a task order is issued for each challenge. The EPA expects to sign a blanket purchase agreement later this year to work better with GSA prize contractors.

Agencies that find challenges and competitions work well for them have even developed ways to stay connected with “solver communities.” Both the EPA and NASA have websites where people can find information on upcoming challenges and events. NASA’s website, NASA Solve, boasts 30,000 unique visitors every week. For interagency challenges, the Reclamation Water Prize Competition Center manages challenges related to infrastructure sustainability, ecosystem restoration and water availability for a number of agencies, including the U.S. Geological Survey, the National Oceanic and Atmospheric Administration, the U.S. Fish and Wildlife Service, NASA, EPA, the Department of Agriculture and NIST.

Additionally, designating staff to work on designing competitions, as well as providing training and mentoring programs to those employees, have helped institutionalize the challenge culture.

But the report says that even though challenges have proved successful, agencies need a  better understanding of how the innovation option can help them solve problems. Some agencies still might not think of holding a competition; others might be skeptical, and they could be right. A 2016 report by HHS, cited by the White House report, said that the “potential benefits and drawbacks” of challenges and prizes are not well understood.

“Even with demonstrated value in previous years, there exists a need to continue to communicate the benefits and lessons learned from federal prize competitions,” the White House report concludes.