A mash made in heaven: Agencies mix data with social media

Government agencies would seem to be a natural fit for enterprise mashups — Web sites or applications that combine content from multiple sources into an integrated experience.

Federal and state agencies have volumes of constantly changing data. Moreover, the data is located across various departments and owned by disparate entities with different security models.

Making all of that information accessible to citizens, government employees and business partners – increasingly a goal of government – can seem like an insurmountable problem. But enterprise mashup software makers are offering improved user interfaces, standardized components and robust security as the platforms mature, making it easier than in the past to pull service-enabling components from multiple resources, industry and agency experts say.

For instance, the Defense Intelligence Agency uses JackBe’s Presto Mashup server software for its Project OverWatch, a virtual operating center application that integrates multiple intelligence sources into a desktop-like intelligence asset dashboard for real-time data analysis.

JackBe has standardized adapters that connect into Microsoft Excel spreadsheets, along with databases, news feeds and Web services. The company has a broad set of application programming interfaces that link to Presto.

“We wanted to get it down to the point where IT organizations can point, click and publish their data sources into Presto,” said John Crupi, JackBe's chief technology officer.

Presto doesn’t hold any data and is not a data warehouse, Crupi said. “We just point to where the data is, and [developers within an organization] put all of the security and governance in place in order to make that data accessible.”

More recently, JackBe has worked with DIA to help intelligence analysts filter data better. Presto “sits in the front of their search engines and other data sources and provides mechanisms to mix and match data coming from these sources, so users can have finely tuned filters,” he said.

Presto incorporates an Extensible Markup Language called Enterprise Mashup Markup Language to configure mashups. The software includes a visual tool called Wires that also creates EMML. Presto’s visual tool creates microapplications called mashlets, which can be viewed in Presto’s dashboard or within HTML pages or Web portals.

BIA Goes Composite

The Bureau of Indian Affairs, which launched a new Web site in October, uses a combination of technologies for mashups, or what officials in the Office of Information Development call composite applications.

The list includes Apache, Tomcat, DocuSign, Oracle 11g, Oracle Fusion Middleware stack – comprising WebCenter, Site Studio, Universal Content Management – and Oracle Business Process Execution Language (BPEL).

“We’ve noticed that there is interest in mashups and Web 2.0 services like Twitter, YouTube and Flickr, and everyone wants to integrate them with current or new applications they are developing,” said Daniel Joplin, OID’s information technology architect.

BIA’s portal is a composite application, he said. The agency purchased Oracle software to rapidly mix and match portal, security and Web services. Oracle WebCenter is used for creating portal functions, while Universal Content Management is used for the intranet because BIA does not need portal-like functionality now but might in the future, Joplin said.

This month, BIA is launching an Internet-facing application that will let American Indians complete an online survey about labor statistics within their tribes. Two years ago, when the labor force survey was last conducted, tribal members filled out the information on paper and faxed it into the bureau.

To digitize that process, BIA is using Tomcat and Apache on the Internet side, and the agency is working with a company called DocuSign, which provides a Web service for digital signing, said Brian Hardy, program manager of OID's information portal programs. The application connects to an Oracle database at the back end to generate reports.

Tribal members will be able to log in, authenticate themselves, complete the survey and digitally sign it. By using DocuSign, BIA can deploy the infrastructure needed to enable digital signatures for a low cost, Hardy said.

Moreover, OID wanted to gain experience with a smaller application, such as the labor force survey, and apply those lessons to its work on the portal and any work done with Web service vendors, Joplin said.

BIA is using Oracle BPEL to mix and match applications to add functionality to the portal. It also allows developers to create new applications in weeks instead of months, OID officials said.

BIA wants to increasingly add geographic information system maps to its mashups, said Chaeny Emanavin, branch chief of design services at OID.

The agency uses mashups via Google Maps for conferences. However, OID wants to work more with the agency’s GIS division, based in Albuquerque, N.M., to integrate detailed maps developed by GIS software maker ESRI into OID's applications and portal.

The Indian Trust and tribal maps are important because they help tribes determine boundaries if they want to build roads or housing projects and improve waterworks or other infrastructures. Maps also come in handy for pinpointing fires, outlining areas affected by natural disasters and cataloging mineral resources.

In one example of how the maps could be used as mashups, Emanavin said, BIA and the Homeland Security Department could overlay maps with their information on dams and flood zones so officials could see detailed information about low-lying areas in the event of floods.

Because Oracle has purchased and integrated best-of-breed technology into its middleware, BIA’s information technology team finds it easier to infuse applications with new services, such as content and identity management.

“Identity management can be a real pain to build into each application,” Joplin said. Because BIA chose an Oracle application that is service-enabled, IT employees don’t need to worry about user stores and rewriting log-ins. Application management also is easier because the IT operations group needs to manage only one system, he said.

A robust security infrastructure needs to be a major part of any mashup technology platform, said Peter Doolan, chief technology officer at Oracle’s public-sector division. A security infrastructure must allow for free movement of information without breaking any agency or legislative rules.

Problems with information sharing in the government are often the result of organizational and governance issues rather than technology, JackBe’s Crupi said. If more than one organization wants to combine data, how do you protect users’ data and the systems that provide the data? One agency might be using Microsoft SharePoint and Active Directory for storing user identities, another might use some other Lightweight Directory Access Protocol directory, and another might use a public-key infrastructure. Yet, the agencies want to create a seamless experience for users while moving across multiple systems and security models. That’s where a security infrastructure comes into play.

Where are mashups heading in 2010?

Social media and mashups will grow in importance, Oracle’s Doolan said. “Not just the customer-facing, commercial social-media technologies such as the Twitters and Facebooks,” he said. “We will start to see enterprises deploy similar types of systems internally within the firewall.”

Recent product announcements at Oracle Open World that revolved around skills identification are indicative of future trends, he said. For instance, Oracle unveiled Beehive 2.0, which analyzes information that flows through applications such as e-mail, wikis or chat messages. It builds a profile of people’s skill sets, so if two or more parties are involved in an online conversation and the system deems that another person with certain skills would be interested in joining, it invites that person into the conversation. It exposes that information through the mashup, Doolan said, and that person can decline or jump into the conversation and become an active participant.