NSA searches for novel intel answers in the Glass Box
Connecting state and local government leaders
Like Alice after she had eaten cake from a glass box in Wonderland, federal databases have grown to an unmanageable size.
Like Alice after she had eaten cake from a glass box in Wonderland, federal databases have grown to an unmanageable size. The National Security Agency is hoping for a different outcome from its own Glass Box'a solution to the problem of sifting intelligence from those databases.
Through the Advanced Research and Development Activity, analysts are using supercharged technologies to tease intelligence information out of massive federal databases.
The program, known as Glass Box, uses unclassified, open-source information to produce data used by the Novel Intelligence from Massive Data (NIMD) program, according to contractor Battelle Memorial Institute of Columbus, Ohio.
NIMD is an Advanced Research and Development Activity project to improve spy technology.
As intelligence agencies gather data at the rate of many quadrillions of bytes monthly, they are seeking to automate ex- traction of useful intelligence from the sea of zeros and ones.
Massive databases
The program deals with databases that are 'massive' in several senses, according to ARDA. For example, the sheer quantity of similar items in a database can be 'typically a petabyte or more,' the agency said. A petabyte is one quadrillion bytes.
'Some intelligence data sources grow at a rate of four petabytes per month now, and the rate of growth is increasing,' according to ARDA.
Other types of intelligence data can be termed massive because they consist of many different kinds of information, ranging from audio and video files to equations and chemical formulas. Still others can be massive because a single document contains links among many interdependent and nested data objects.
NSA awarded Battelle a $1.5 million contract for Glass Box work from last January through June next year.
According to contract documents, Glass Box is designed partly to evaluate intelligence algorithms developed by other, presumably classified, NIMD activities.
The program also is developing the Glass Box Analytic Environment, a hub task of the NIMD project. The contract documents describe NIMD as a research and development project intended to last for several years to discover the 'novel intelligence' that lurks undetected in the intelligence community's databases.
Novel intelligence falls in the category of information that causes intelligence analysts to gain new understanding of previously unappreciated or misunderstood threats, according to contract documents.
'This information may emerge from discoveries in the content and patterning of new data, or discoveries resulting from 'out-of-the-box' analytical reinterpretations of existing data,' according to the government.'
Decision methods
Glass Box analysis is intended to capture drafts of intelligence agents' reports, e-mails, memos and other intermediate products so the spy agencies can create a window into analysts' decision methods.
'We believe that the most powerful analytic environment will be one in which information about analysts is not only induced from their activities, as captured through Glass Box Analytic, but in which they also [build and use] models of the analytic targets that describe their knowledge, interests, hypotheses and biases,' according to ARDA.
A Battelle spokesperson stated in re-sponse to an e-mail inquiry that analysts working on the Glass Box experiment have processed more than 70 intelligence analysis tasks and produced more than 80 reports.
'From examining the Glass Box data, researchers have created visualizations of the analytic process, identified inefficiencies in the current analytic work environment, improved search techniques, created user models that help tools understand and predict analyst behaviors, and more,' the company said.
The agency had sought sources for Glass Box work in September 2004, according to Federal Sources Inc., a consulting firm that tracks IT contracts.
According to FSI, the government did not receive any responses to its sources-sought notice and decided last November to negotiate with previously identified potential contractors. Eventually, only Battelle submitted a proposal. The vendor and the government then negotiated an agreement.