Entity resolution's growing role in security efforts

 

Connecting state and local government leaders

Entity resolution enables the integration of records from many different sources and resolves these records into complete "entities" with a high degree of confidence, so that the associations within and among the entities can be identified and acted on, writes Scott Schumacher.

Identifying and tracking down persons of interest is one of our greatest national challenges.

Editor's Note

This is the first of a two-part commentary. Part two will be published Oct. 7.

As was demonstrated by both the Oklahoma City bombing and Sept. 11, 2001, terrorist attacks, national security threats can come from anywhere. The challenge and complexity of the problem has gotten even deeper as at least one naturalized U.S. citizen has been directly linked to terrorist activity overseas.

How can U.S. agencies successfully guard against current and emerging threats?

The Homeland Security, Defense and Justice departments, law enforcement agencies and the intelligence community are under increasing pressure to share information in order to more effectively identify and unravel threats before they happen. Yet, how can agencies reconcile the massive amount of related data that exists in bits and pieces across hundreds or thousands of disparate databases? How can agencies “connect the dots” between U.S. citizens and international terrorists with sufficient confidence to identify a potential threat?

Data-mining technologies have been used in the past to try and solve this challenge. However, these technologies require data to be aggregated in order to be mined. They also require that data be complete and correct, which is generally not the case with threat-related information that is often deliberately falsified or incomplete in order to avoid detection.

As a result, data mining alone is not sufficient to satisfy mission requirements.

The intelligence and law enforcement communities have recently begun using a type of technology designed to help connect the dots across data sets – relating to persons of interest, places of residence, vehicles, weapons and so forth. Called entity resolution, the technology enables the integration of records from many different sources and resolves these records into complete “entities” with a high degree of confidence. Once entities have been resolved, the associations within and among the entities can be identified and acted on.

The end result: Technology can be heavily leveraged to attack the signal-to-noise ratio problem faced by analysts and investigators and help them more easily connect the dots. Resolving entities and associated relationships with a high degree of confidence enables analysts and investigators to focus their efforts on the persons of interest and relationships that truly matter.

To the fore

Research firm and consultancy Gartner has been tracking the entity-resolution market for several years. “Entity resolution and analysis was previously an obscure technology that has come to the forefront as a result of world events and market forces where it is used to identify the use of false identities and networks of individuals who are attempting to hide their relationships to each other,” stated Gartner in “Hype Cycle for Master Data Management,” a report released in June.

In other words, when persons of interest are leaving trails of misleading information, agencies must be able to recognize conflicting data and act accordingly. That’s precisely what entity resolution does.

Today, most data exists within discrete stovepipes, associated with a particular application, system or architecture. For example, the details within the local arrest record of a U.S. citizen, including information associated with that person and the circumstances surrounding the arrest, are generally viewed and analyzed only within the context of the local law enforcement records management system. Similarly, details associated with suspects on a national watch list are generally viewed within the context of that application, run by a different agency. The various levels of law enforcement – from local and county level agencies to state sheriffs and federal intelligence agencies – have not traditionally shared data because of both political and technology roadblocks.

Entity resolution helps agencies transition from a system-centric to an entity-centric enterprise by decoupling entity data from contributing source systems. Once decoupled, relationships within and among the entities can then be identified and managed.

The resolved data set can then be made available to all users with appropriate access rights, through the standard applications already in place. Critically important for security and privacy, the data from the source systems can remain in place and under the control of the contributing application and/or organization.

When coupled with entity extraction technologies, entity data from unstructured data sets can also be integrated and can help connect other pieces of the puzzle as well.

By identifying and managing relationships between persons of interest and other individuals or objects, entity resolution delivers a more comprehensive view of people, places or things and their activity. By significantly mitigating the signal-to-noise challenge faced by analysts, they can then be much more proactive in identifying either the hot spots or patterns that would serve to thwart an attack – or contribute to solving or preventing a crime.

Marks of success

Implementations of entity-resolution technology have been widely successful. One of the largest implementations to date is the FBI’s National Data Exchange (N-DEx) program, a criminal justice information sharing system that will provide nationwide connectivity to disparate local, state, tribal and federal systems for the exchange of information.

N-DEx will provide law enforcement agencies with a powerful new investigative tool to search, link, analyze and share information (for example, incident and case reports) on a national basis to a degree never before possible. N-DEx will primarily benefit local law enforcement agencies in their role as the first line of defense against crime and terrorism.

The FBI system leverages entity resolution to associate many different types of data across the hundreds of millions of records contributed by the participating agencies and resolves this disparate data into entities. Once resolved, the relationships within and among the entities are then identified and acted on. This is initially done within the context of incident reports and investigations but also enables identification of potential relationships across these.

This type of information sharing is available to participating law enforcement agencies of all levels, from local to federal.

It is important to note that entity-resolution technology works with the information already in place – information already gathered by local, state and federal authorities. There are nearly 18,000 state and local jurisdictions in the United States. Each of these jurisdictions has evolved its information systems according to its own needs and concerns; data models and formats vary widely.

Entity resolution provides a means for participating agencies to retain the management and governance of their data while contributing to the benefit of all as a collective information sharing exchange.

Entity resolution enables contributing agencies to connect the dots between relationships in information, between systems and between evidence within investigations. With this information, agencies can then identify terrorist and criminal activity that was previously undetectable and act decisively to prevent or investigate incidents.

Local, state and federal authorities already maintain a wealth of valuable information – particularly regarding national security and threats against it. The goal is to enable agencies to share information while protecting the privacy and security of that data; to help agencies “connect the dots” so they can effectively and efficiently identify and track down persons of interest.

The ultimate goal is to continue to ensure the safety of our citizens and country. Entity resolution is a fundamental piece of securing our safety.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.