Semantic search: Still more luck than technology
Connecting state and local government leaders
While there has been significant improvement in search since Google launched in 1998, semantic search still has significant room for growth and innovation.
While watching the movie “Detachment” on Netflix, I grew curious about the scene where no parents show up for parents’ night at the high school. I wondered how prevalent poorly attended parents’ nights were across high schools in the United States. So I Googled it, typing in the following search: “Parents no show on high school parent’s night.” I received plenty of hits, but none on the problem of parents not attending a parents’ night at high school.
And there lies the problem: I know what I want to learn – in this case, statistics or stories indicating how common this phenomena is across America, but I am not translating that query into the right keywords, or more precisely the “exact” keywords, that will provide relevant results. In the jargon of information retrieval, this would be considered a low precision search because out of many hits there were no relevant ones.
Thinking further about the query, I came up with a new Google search: “parental involvement in urban schools.” Boom! I hit the mother lode of relevant results. What happened here?
The key difference was that instead of describing the symptoms of the phenomena I was interested in, I had to extract the meaning myself and interpret a possible cause of those symptoms. In essence, my search became a test of my hypothesis on the cause of those symptoms. Shaking my head, it was clear to me that search has a long way to go if it depends on the searcher coming up with a “magic phrase” that matches the most common description of the relevant results.
Fortunately, improved search is on everyone’s radar.
In an interview with Bloomberg TV, Yahoo’s Marissa Mayer said search can be improved through personalization and context. Her key point on personalization is that a search engine should be able to extract context from your search history, location, social data, etc. to deliver more relevant content. That resulted in her oft-quoted phrase, “In the future, you become the query.” Facebook is also experimenting with search via its Graph Search.
Google is also zeroing in on the weakness of current search results and is working hard to deliver semantic search. It has added the Knowledge Graph and the new Hummingbird algorithm. Finally, Business Insider predicts a new war over semantic search with Apple and startups gunning for Google.
What does this mean for government information managers? Basically, really important information discovery can’t rely on keywords and traditional search engines. Careful metadata curation, good categorization and understanding your users will provide more relevant results.
As for Google, I just hope they can soon get rid of the “I’m Feeling Lucky” button. Until they do, I don’t think they will have succeeded in convincing anyone that semantic search is more technology than luck.
Michael C. Daconta (mdaconta@incadencecorp.com) is the Vice President of Advanced Technology at InCadence Strategic Solutions and the former Metadata Program Manager for the Homeland Security Department. His new book is entitled, The Great Cloud Migration: Your Roadmap to Cloud Computing, Big Data and Linked Data.