UX testing that works

A group of civic-minded Chicago technology developers are bringing to the Second City a set of business practices dedicated to improving the performance of citizen-facing software and apps.

A guiding principle of the group, the Smart Chicago Collaborative, is that the more that’s known about the preferences and behaviors of end users, the better the software and apps that can be designed for them.

Since 2013, the group has amassed a group of city residents to participate in user experience (UX) tests of civic software projects, ranging from apps for navigating the city’s public transit system to a project to build a new web site for Cook County.

The UX tests follow a set of guidelines for usability testing refined over time by the Civic User Testing Group (CUTGroup), the Chicago residents recruited to test the apps by Smart Chicago.

The CUTGroup meets both to look for errors in the performance of an app and  to check “classic human interface design in term of the presentation of information and workflows,” said Daniel X. O’Neil, a founder of Smart Chicago and author of a book about the CUTGroup project.

The CUTGroup program has become part of a trend in digital systems toward user-centered design and development, according to O’Neil, now director of product development and business strategy at Ad Hoc, LLC, a software development firm with a focus on government apps.

“The move toward user-centered design, user-centered systems and the increase in the number of user researchers in government contracts is profound,” said O’Neil, adding that the “CUTGroup has provided concrete ways for designers to listen to real people for whom they are creating.”

The trend also fits the focus of the General Services Administration’s 18F and U.S Web Design Standards initiatives, which gives developers plug-and-play design and code that GSA claims set “a new bar for simplicity and consistency across government services.”

To support demand for the data and testing necessary to support a user-oriented approach to programming, developers are turning to methods like the CUTGroup.  The group aims to find people whose characteristics make them target users for a particular project. For example, testers can be sorted by the devices they use to access the internet, their familiarity with 311 services, whether they’re veterans, of whether they're about to lose health insurance benefits.

“That kind of specificity is actually very difficult to get, and the CUTGroup allows for a really good method for recruiting regular people so they can be contacted and segmented on a moment’s notice,” O’Neil said.

Most of the tests are conducted in public computer centers, a way to emphasize human networks and improve the digital skills of participants. Another key policy: testers are paid. For filling out a CUTGroup profile, they receive a $5 gift card; if completing an app test, the prize is $20.

As the CUTGroup has been maturing, the group has moved into testing more government technology. “That wasn’t something we necessarily set out to do, ,  CUTGroup’s Project Coordinator Sonja Marziano said. “Instead it was something that came naturally from doing tests at the civic-tech level and then being asked to do it at the government level.”

The group worked with Cook County, Ill., to perform wireframe testing of a new web site that launched over the summer. It has also worked with the City of Chicago Department of Innovation and Technology on OpenGrid, an open source interface to help people visualize data, as well as a project to help residents navigate points of interest across the Cook County Forest Preserve.

For its tests, the collaborative relies a number of tools, including MailChimp to manage email campaigns and Wufoo, a tool to build forms for online surveys and event registrations. The group also taps the public library system for Wi-Fi and meeting space, from which it can operate as a community anchor.

The CUTGroup is currently exploring how to make the process of accessing open data more user friendly. In doing so, it is testing the City of Chicago Open Data Portal, Marziano said, an effort that will help users find datasets and applications that were created using open data.

One of the most important lessons learned when working with city agencies, she said, “is to make them partners and include them in the process of the test. It creates more buy in to make the changes happen when they have been part of the process since day one.”

Marziano said local citizens are often surprised by the volume of data generated by a test and how that data might be used in the future.  “For me it’s interesting to see how doing testing on these products influences how people will seek out data,” she said.

That in turn will lead to conversations throughout agencies about the future of the tests. “That’s a big goal here,” said Marziano, who aims to conduct usability tests throughout the development process.

“It would be ideal if as people are scoping out work, they are including more usability testing in the process -- before, during and after the development, knowing what that looks like and what resources it would take,” she said.

“That is a very important piece of what we do.”