Relief for integration headaches

Government IT managers are forced to strike a balance between a rapidly evolving technology landscape and the imperative to wring every last drop of value from previous investments. Often that means integrating new technology into legacy systems.

Making otherwise incompatible technologies play nice isn’t easy. Even a successful solution may have all the stability of a game of Jenga. The slightest change, such as the introduction of a new application or the migration to a new operating system, could upset everything. And what if the only person who knows how everything works goes on vacation – or heaven forbid, leaves? The entire mission can be compromised.

Here’s what IT managers can do to solve or mitigate two basic kinds of challenges: component integration and architectural integration.

Component integration challenges

Imagine having to rewrite desktop software because of a Windows upgrade. Or imagine having the office’s version of SAS unable to work with the latest version of Excel. These are examples of component integration challenges – making multiple software components and/or commercial-off-the-shelf products work together.

I won’t pretend to have all the answers since component integration challenges are so diverse, but here are suggestions:

●     Prefer browser applications to desktop applications. The once-superior feature set of desktop solutions was worth the configuration management and installation nightmares, but no longer. Note that browser applications aren’t necessarily Web applications. For example, developers can write apps in Google Dart that function as desktop apps living in Chrome. For those who insist on traditional desktop apps, choose operating-system agnostic technologies like JavaFX or Java Swing.

●     Consider virtualization and/or the cloud. If components can be successfully integrated in a virtual machine, changes to the standard office configuration and things like security scans become irrelevant. Just work in the VM. You can even run your VM in the cloud against data stored securely with the government flavor of Amazon Web Services. Or migrate desktop applications to Web applications deployed to Cloudbees or Heroku.

●     Opt for open source. Vendors have every right to maximize profits, but this often comes at the expense of easy integration as they seek to lock customers in to their stack. Open-source tools not only reduce cost, but they are usually built with integration in mind.  

●     Use Kanban. Agile development is no monolith. Although I prefer Scrum for application development, I prefer an agile approach called Kanban for component integration, where tasks are more like help-desk tasks flowing in continuously with varied priority. Kanban can be overlaid on an existing process to accelerate integration. Results are apparent in days or weeks rather than months.

●     Share knowledge. Rather than concentrate knowledge in one person, demand that several people on the team know how everything works. Whether through brown bags at lunch or mandatory training sessions, make this a priority.

●     Automate pipelines leveraging common formats. Pipelines exporting data from one component to the next via intermediate formats can improve efficiency significantly. This demands using data formats most tools understand like CSV, JSON, or XML. Write scripts to automate these pipelines.

These solutions demand investment, but so do current integration efforts. Consider these suggestions to maximize the return on your investment.

Architectural integration challenges

There is no more famous example of an architectural integration challenge than HealthCare.gov. It is its own beast but needs to communicate with a diverse array of applications in both public and private sectors – at the IRS, credit agencies, insurance companies, etc. Though they may not get as much press, there are many such large-scale architectures of similar complexity in government. 

Again, I won’t pretend to have all the answers, but I will offer one broad suggestion for your consideration: Use enterprise integration patterns.

The canonical text on the subject of the same name is meant primarily for developers, but the basis is straightforward: Ensure applications in the architecture know nothing about each other.

One pattern for achieving this integration nirvana is the message bus, a software component all applications communicate with. Applications send data in a canonical data format only to the bus. The bus can then modify messages as needed – transformed, split, aggregated, etc. –before delivering them to other applications. Producers of messages are completely decoupled from consumers of messages, so applications can be added or removed without disturbing the rest of the architecture. It takes skill to do this right, but it is quite feasible.

The need to integrate disparate components and applications will continue to be a necessity in government IT. Taking thoughtful steps will ensure investments yield the greatest value possible.