DC transforms social media site with citizen-service analytics

The Department of Homeland Security and the FBI have been monitoring social media for several years to search for potential terrorists and criminals. Meanwhile companies have been scanning social media to gather intelligence about consumer perceptions of their brands. And the Centers for Disease Control is gathering data from Google Flu Trends and MappyHealth to keep tabs on health issues. 

Increasingly, however, government agencies are now turning to social media as a way to interact with citizens – to inform and to be informed. 

The most notable effort to date is Grade.DC.gov, a program launched by the District of Columbia in 2012. Grade.DC.gov gathers data from social media, as well as from comments contributed to its website. Nearly two years after launching the program, the surprise has been transformation of Grade.DC.gov from a public relations effort to something more. 

“When we started this we thought of it more as a transparency in communications exercise," said Matt Desjardines, D.C. communications officer. As the district expanded the program, however, they have come to see it as a service platform for interacting with residents. "The one thing that we quickly learned is that this has more benefit when used appropriately on the service side of things," said Desjardines. 

Desjardines cites the example of the district's Department of Aging learning through social media of complaints about the food at certain facilities. "They were serving some food that smelled very unpleasant and didn't taste good, and we were able to make that small subtle shift using GradeDC.gov social media to get the meal plan changed," said Desjardines. "People's lives were a little bit better off because of that."

How it works

The district contracted with newBrandAnalytics to collect and analyze data from the Grade.DC.gov website and from social media, including Facebook, Twitter, and Instagram. 

newBrandAnalytics doesn't just gather messages and pass them along, it runs them through the company's proprietary text-analytics engine to extract significance. "We identify certain keywords and then assign various levels of sentiment to them," said 

Zach Boisi, director of client services. "We grade on a five point scale." 

Recognizing that sentiment-analysis algorithms can miss sarcasm and slang – both of which are often directed at government agencies – the analytics team backs up the text-analytics engine with a quality-control network of humans who regularly sample the data flow. According to Boisi, a dozen newBrandAnalytics analysts are dedicated to the Grade.DC.gov project. "They probably know more about D.C. government that any D.C. resident," said Boisi. "If someone says 'this service is LOL,' or 'the service was the bomb,' they are able to discern it."

The results of the analysis are distributed in two ways. First, monthly grades for each of the 15 agencies currently covered are offered on the Grade.DC.gov website. Secondly, daily reports that cover the interactions from the day before are delivered to agency managers via e-mail.

Judging from the grades received over the past two years, the program has been effective. "Going back to the start of this program, the sentiment was not positive," said Boisi. "I think of the five agencies we launched with back in April 2012, four of the agencies had C- and one was C+." 

In fact, the grades for the 15 agencies involved in December 2013 included nine As. There were only two Cs and one D (which was based on only three reviews). "Over time, these grades have all trended upward," said Boisi. "And we have not changed our grading mechanism whatsoever." 

It is, of course, difficult to run a cost-benefit analysis on such a program, but Desjardine says the district government is very happy with its investment. While the contract with newBrandAnalytics was $170,000 for the first year, along with two $250,000 options years, he said, "I think it has paid off. It has really allowed us to tap into this growing communication stream from the public. It has shortened the feedback loop. And combining the data streams of different social media is really valuable from a managerial standpoint. GradeDC.gov allows managers to see everything in one place."

Best practices

Desjardins does offer some advice to government agencies looking to start similar projects. 

First, he says, it's important to assess each agency's level of expertise and comfort with social media. "Some agencies have really knocked it out of the park, like our District Department of Transportation. They have a great social media presence. They've always had a very well-followed Twitter feed that interacts with residents," he said. "But we also have agencies that don't quite have that level of social media expertise."

As a result, it's important that the project team reach out to agencies proactively. 

"For us it is a bit of a marketing problem for these agencies, Desjardines said. "We need to make sure that frontline employees at all of these agencies are asking for feedback to come through Grade.DC.gov. They've been asked to put up signage and train frontline workers to request more feedback. We want the good and the bad."

Finally, Desjardins recommends moving slowly. "We were the first in the country to do this," he said. "The one thing that really benefited us with was growing it slowly. We started with only five agencies and eventually grew to 15."