Shining light on the darknet
Connecting state and local government leaders
DarkSum crawls the darknet, indexing its content and making it viewable.
There’s much more to cyberspace than what can be found using Google, Bing or other popular search engines. In fact, a vast amount of internet content that doesn’t get indexed by traditional search engines resides in what is known as the deep web. The deep web is chock full of forums, databases and other content that can only be reached by typing in a specific URL.
A subset of the deep web, known as the darknet, is even more private. Not only is the darknet not indexed by popular search engines, it requires special software -- most often the Tor browser -- and, often, explicit authorization to access content.
Key to the privacy of the darknet are routers that move messages with multiple layers of encryption. Each router can peel off only a single layer of encryption, which reveals the data’s next destination. Unlike with standard internet messaging, darknet transmissions do not contain headers that reveal the sender or routing of the message.
Thanks to its high level of privacy, the darknet is popular with criminals, who use it to traffic in child pornography, drugs, weapons, stolen goods and hacking tools.
Yet while the darknet has been a favorite haunt of cyber criminals, there are also some good reasons that legitimate businesses may want to search and access darknet content. Some organizations may want to run their own private darknet for security reasons. At the same time, any government agency or business with sensitive information may want to scan the darknet to find proprietary data that might have been leaked or stolen.
DarkSum, a San Francisco-based company launched in 2015, offers a program of the same name that crawls the darknet, indexing its contents and making it viewable.
“Our clients care exclusively as to what content is posted, which is what we spider and monitor for,” said Darksum CEO Eric Michaud. “One example is if there are leaked/breached datasets available online.” Michaud said. DarkSum can also monitor postings on darknet sites, looking for threats to staff or facilities.
The spiders that DarkSum sends out to scan the darknet are, however, only the first step. Once the content is indexed, DarkSum applies machine-learning filters to categorize and prioritize content for threats. Finally, a human analyst reviews the potential threats and, when called for, issues a report to clients.
According to Michaud, DarkSum used to provide a Google-like interface so that clients could search the darknet for themselves, but the company quickly found that clients didn’t want that level of access. “We found clients do not want to search for themselves by creating crafted queries,” Michaud said. “Instead, we monitor a curated list of information -- locations of physical premises, project names, staff, assets, etc., that we work up with our customers during a risk assessment.”
Clients interface with the DarkSum system through a portal that allows the client’s security staff to review processed and prioritized threats.
Michaud added, however, that the company -- which was recently acquired by Intelliagg, a cybersecurity firm based in London and Stockholm -- may reintroduce the Google-like interface as part of Intelliagg’s broad platform of security tools.
The primary focus of DarkSum, Michaud said, is on corporate clients, “but by the nature of our systems, we do run across illegal activities, such as child exploitation.” When that happens, he said, “we pass off that information to law enforcement with hopes that something can be done.”