What agile software development can do for government

One of the few areas of agreement in Washington is concern about the state of IT project management.

As the White House Forum on Modernizing Government recently concluded :

“. . . [M]ost Federal Government IT projects are too large and not sufficiently integrated into business unit operations. Multi-year Federal IT efforts are typically driven by technology managers — who often turn over during the life of the project — rather than agency business leaders. Agency business leaders are not held accountable for project success, and in turn do not adequately invest in IT project management. As a result, in comparison to industry best practices, Federal IT projects are too often marked by milestones spaced too far apart and deliverables that fail to deliver tangible end-user value. Further, Federal IT change efforts are typically managed in isolation from business operations, so those working on long-term solutions are too often not concerned with, or even aware of, the evolution of day-to-day business considerations. “

Having led major government IT programs, I can add that many of these challenges stem from an over-reliance on traditional software development models. These legacy approaches lock users into fixed and lengthy development processes that don’t readily accommodate change. This is despite the fact that requirements often evolve.

Fortunately, a framework – agile software development – exists for addressing many of these concerns. And agile has already been shown to deliver significant benefits for major government programs in terms of time-to-value, overall quality and their responsiveness to changing requirements.

Tradition doesn’t accommodate change

By definition, the traditional “waterfall” model for software development used throughout government is sequential and highly regimented. By locking down requirements at the onset of development and limiting subsequent stakeholder involvement, it often fails to:

• Identify and correct problems early in the process when it is less costly and disruptive to do so.
• Respond effectively to subsequent changing requirements that always exist.
• Incorporate lessons learned and other improvements into the final product.

To put this into perspective, waterfall builds upon traditional manufacturing models that rely on distinct segments – requirements analysis, design, development, quality assurance and so on. However, even manufacturers are now moving away from a strict reliance on these traditional models. Instead, they are embracing lean manufacturing, six sigma, just-in-time delivery and other emerging concepts to produce higher quality products, greater operational efficiency and responsiveness, and faster time-to-value.

Empirical, modular, collaborative

A similar revolution is impacting software development. For example, software giant SAP recently announced plans to use agile to reduce development times, extend stakeholder collaboration and improve its responsiveness to market change.

Related coverage:

DISA restructuring to provide more agile services

The good, the bad and the ugly of agile programming

Readers expound on the pros and cons of agile development

In terms of specifics, agile is an umbrella methodology encompassing a number of complementary disciplines. What they have in common is a commitment to delivering faster value, better collaboration and greater responsiveness to changing requirements.

One of agile’s principal differences from waterfall is that it is an empirical approach, relying on regular measurements and assessments to continually reevaluate and improve the solution. This empirical approach not only yields continuous improvement, but also the flexibility needed to respond to unforeseen demands.

Agile is also a more modular approach in which big deliverables are broken into smaller, self-contained components that can be developed and implemented more quickly. This is consistent with the shift to service-oriented architecture (SOA), software-as-a-service (SaaS) and cloud computing, where new features and functionality are added incrementally to deliver faster return-on-investment.

Finally, agile is more collaborative. Integrated project teams encompassing design, development and testing work in short duration sprints – typically two to four weeks – to deliver functioning software components. As a result, problems can be identified and remediated as they are created. Teams also meet regularly with stakeholders at the end of each sprint to assess performance and reevaluate priorities. This provides users with the opportunity to evaluate this functionality firsthand – is this what I envisioned? – and adjust their requirements as needed.

From a stakeholder perspective, agile delivers a number of strategic benefits:

Faster time-to-value. As functionality is delivered in a modular fashion, users capitalize on new features sooner.

Higher quality. Defects are caught and remediated earlier in the process as testing is fully integrated.

Risk mitigation. Program managers leverage real-time performance data to continually assess progress.

Better solutions. Continuous user feedback produces more effective solutions.

Government-ready

Agile has been used by a number of federal agencies and their contractors, including Agilex, to support their most high-profile projects.

For example, the Federal Health Architecture program, operating under the Office of the National Coordinator for Health IT, used agile to develop the award-winning Connect solution in less than a year. This was despite the challenges inherent in creating from scratch a nationwide health information exchange network, as the project also served as an important research & development exercise for ONC. For Connect, we delivered productivity that was five times the industry average while defect rates were just a quarter of similar projects. This has enabled eight consecutive quarters of on-time, on-budget product deliveries with new functionality added quarterly.

The U.S. Navy’s SPAWAR Command also relied on agile to develop the Post 9/11 GI Education Benefits Long-term Solution for the Veterans Affairs Department. As VA CIO Roger Baker explained to the media, “we developed this completely under an agile methodology. What’s important about that is that [education services] subject matter experts and my IT folks worked side-by-side, day-in and day-out to develop the user interface and the rules and the workflow in this system to be optimal for the processing of these types of claims by the folks that are the claims examiner.”

Emerging mandates

In issuing new guidance for managing federal IT projects, the Office of Management & Budget argued:

“. . . [T]his guidance initiates a re-examination of these expensive and lengthy investments in financial modernization solutions in favor of shorter-term, lower-cost, and easier-to-manage solutions. By dividing projects into smaller segments that deliver the most critical functionality more quickly, Federal agencies will achieve greater functionality sooner, better align projects to their organizations capacity to manage change, and reduce risk and cost.”

Agile can help achieve these results. As an empirical approach, it relies upon regular assessments to deliver greater transparency and accountability. It also mandates the continuing involvement of stakeholders in the decision-making process, which ensures that projects remain aligned with the requirements of the mission. In addition, it utilizes short-duration sprints and regular deliver timeframes to keep projects focused and on-track.

When to use agile

Although agile offers great potential, it isn’t a cure-all. In many cases, its greatest value is as a complement to existing processes.

Among the projects that benefit the most from agile are:

Fixed time and/or budget constraints. Agile works extremely well for projects with firm delivery dates and/or budgets, such as the VA’s Chapter 33 Long-term solution, which faces congressionally-mandated delivery deadlines. As noted earlier, traditional approaches lock-in requirements, budget and timeline early in the development process, often before the project’s full scope is clear. As a result, projects experience frequent delays and cost overruns as additional requirements are uncovered and incorporated into the plan. In contrast, agile is a dynamic process where priorities are continually reassessed against a fixed delivery deadline. Rather than delaying delivery while extraneous features are added, agile deliver core functionality as quickly as possible.

Research-driven initiatives. Within emerging fields, the project objective isn’t simply to deliver the solution, but also to identify new approaches to solving common challenges. The FHA’s Connect solution, which pioneered a new health information exchange model, is a good example of this scenario. Working with new ideas and concepts, project teams benefited from agile’s empirical approach as it provides a framework to continually test and refine their solution hypothesis.

Open-ended development. Due to their significant scope, some projects have fairly long-term and ongoing delivery timelines. These projects include major enterprise systems, such as ERP or clinical information systems, as well as those requiring frequent updates. SAP’s adoption of agile is one example. Agile provides the ability to break projects up into smaller components that can be delivered within a defined time period, typically 90 to 180 days. This makes the project more manageable while delivering faster time-to-value

Ultimately, the challenges that we face are real, but some of the answers are apparent as well. In this case, agile is clearly emerging as a solution to the cost, performance and other pressures facing major government IT programs.