CJIS system uses enterprise architecture, sign-on security to link law enforcement

The threat of terrorism has made the FBI more aware than most agencies of the need to share information across jurisdictions, but terrorism isn't the only reason the bureau shares data. To confront crime on many fronts, the bureau has developed the National Data Exchange (N-Dex), part of a four-year effort to gather the crime and incident data from the nation’s 18,000-plus law enforcement agencies into a single, Web-accessible investigative tool.

“The development of N-Dex marks the first time in U.S. history that local, state, tribal and federal criminal data has been openly shared,” said Andre Haynes, program manager at Raytheon Information Solutions, the prime contractor for N-Dex.

The project presented major challenges in data standardization, systems integration and security, and required a finely tuned search mechanism that was both fast and analytical.

The team behind the FBI’s N-Dex system includes, from left, Development Manager Jim Preaskorn, Assistant Director Daniel Roberts, Program Manager Jon Kevin Reid and Deputy Assistant Director Jerome Pender.

N-Dex  resulted from work by more than 140 employees at the FBI’s Criminal Justice Information Services (CJIS) division in West Virginia; 60 people from Raytheon, which was hired in February 2007; and contractors including Hewlett-Packard, IBM and Oracle.

To gain insight into user needs, the development team held meetings with local law enforcement officers who serve on CJIS’s Advisory Policy Board, conducted extensive technology and product research, and built several prototypes. “The challenge came in finding solid products that met the needs of the program, with the ability to scale sufficiently to accommodate the projected growth in data holdings, concurrent users and performance requirements,” Haynes said.

With so many agencies contributing data, integration would undoubtedly be a problem. “Standardization within the law enforcement community, with respect to electronic recordkeeping, was in its infancy when N-Dex was first conceptualized,” Haynes said. So the team helped local agencies map their record management systems to data-sharing specifications of the National Information Exchange Model (NIEM).

Because contributing agencies work under strict privacy and security rules, N-Dex would need a way to determine who could view what information. According to N-Dex development manager Jim Preaskorn, the team wrote an extension in a Security Assertion Markup Language-like language to leverage the security in the FBI’s Law Enforcement Online (LEO) portal. Roles are handled with two-factor authentication in the FBI’s Next Generation Identification (NGI) biometrics system, and users must go through LEO to reach N-Dex.

A three-tiered, color-coded system labels information according to its eligibility to leave the local jurisdiction. For example, yellow provides a basic pointer to information that fits the search criteria, but for details, the user must contact someone from the state agency that owns the data, Preaskorn said. Red means only people in that state can view the entire record.

N-Dex Increment One went live in March 2008, and by 2009 the system had 51 million records — five years' worth from 14 contributing agencies, Haynes said. This summer, Increment Two added subscription and notification features that alert users when new information matches their search queries. “It will let me know when someone searches on 3 Main Street,” Preaskorn said. “You [also] get notified when someone searches your data.” Arrest, booking and incarceration data were added, and user capacity doubled to 100,000. N-Dex also has chat room-like features that let investigators collaborate online, Preaskorn said.

Enterprise foundation

Much of N-Dex’s success is due to its reliance on CJIS’s enterprise architecture, which includes LEO for single sign-on and storage capacity for more than a billion records, according to Raytheon officials. Haynes said the off-the-shelf and custom software in N-Dex is built out of clearly defined components arranged in a Java Enterprise Edition framework, with separate layers for user interfaces, application processing and data. “Processes that are common across multiple functions have been identified and designed to be reusable by multiple higher-layer components,” he said.

N-Dex is one of CJIS’s first large systems to eschew proprietary mainframes for cheaper blade servers, in this case Hewlett-Packard hardware running Red Hat Enterprise Linux. The fault-tolerant cluster can support 3,250 concurrent users, provides 99.8 percent availability, and is scalable enough to support each increment’s doubling of capacity simply by adding servers, Haynes said. Computer Associates provides the security software, IBM provides security and data “ingest” appliances, and F5 appliances handle load balancing. Oracle Real Application Cluster (RAC) database servers provide six-second responses and can take in a million new records a day. Preaskorn said the system is centralized, with agencies submitting information to N-Dex’s data center, which then processes user queries.

Initiate Entity Resolution software from Initiate Systems currently serves as N-Dex’s search engine, but Preaskorn said Autonomy will be swapped in for the search component next year. Still, few will argue that the Initiate software’s ability to correlate seemingly unrelated information plays the key role in N-Dex.

It takes graphics to show relationships among data, and for that, Raytheon turned to ESRI’s ArcGIS Desktop for geospatial visualization, and i2’s Analyst Notebook. Haynes said an investigator might use ArcGIS to map locations of incidents involving a certain vehicle, or to confine searches on names to specific areas. Analyst Notebook highlights links within data to make them easier to identify, such as a suspect’s ties to others who were involved in similar crimes or who used a similar vehicle. “Using the i2 product, the investigator can pull back additional information on the vehicle or other people to help develop other leads in the case,” Haynes said. “The ability to display graphically the non-obvious ties within the data is a powerful tool.”

Next steps

Increment Three, slated for 2010, will again double the number of users while adding probation and parole data. Haynes said improvements to the correlation and visualization tools will help users spot crime networks, patterns and trends.

Investigators have been reaping the benefits that N-Dex was designed to deliver. In one success story, the system helped West Virginia investigators link a false name given by the driver in a routine traffic stop to his long rap sheet. In another, police in one large city used utility records to connect the aliases of people arrested in a drug house to known associates in another state.

Haynes and Preaskorn are fond of saying N-Dex helps law enforcement “connect the dots.” Eventually, they hope to offer it to the entire justice system, including courts, parole boards and prisons. It's an ambitious dream, but not a surprising one for a project rolled out in giant steps called “increments,” and comprising millions of simple facts, any one of which, when linked to others, might turn out to be the clue that stops the next terrorist attack.