A city where big data, analytics drive mobility choices

Planning and transportation officials in San Jose, California, are using a cloud-based decision support system (DSS) to help inform their projects as part of a citywide access and mobility plan.

The City Council voted Aug. 19 to adopt Move San Jose and a “Transit First Policy,” which will prioritize transit operations in plans and decision-making. A crucial part of these initiatives is DSS, a software tool that uses big data and analytics to help city officials determine the next best move.

“Move San Jose in general points us in the right direction. The decision support system actually consumes a bunch of big data sources and a lot of information about land use, current street designs, current transit accessibility, current bikeways – all that kind of stuff and says, ‘Hey, these are the places and these are the things that will help the most,’” said Ramses Madou, division manager of planning, policy and sustainability at the city’s Transportation Department.

Datasets that inform DSS, built by data analytics startup UrbanLogiq, include internal data about streets, including how many the city has, how many lanes each has, how they’re laid out and whether they have sidewalks. 

For example, UrbanLogiq geocoded 10 years of incident data and five years of traffic count data from the city, all of which came in multiple file formats, primarily PDF and Microsoft Excel documents. After cleaning and validating the data, it was uploaded and integrated into DSS. From there, users can query and see patterns and trends in the data and download raw historical data offline at any time, from anywhere.

Other data about e-scooters, land use, census blocks and equity information from the American Community Survey also factor into DSS. The city also procures data such as transportation information from StreetLight Data and Replica and open source data conforming with the General Transit Feed Specification that transit operators provide. DSS centralizes all of these into one platform.

“The way that everything comes together is in key performance indicators,” Madou said. The city and Arup, a global sustainable development consultancy, worked together to come up with 50 KPIs that support Move San Jose’s nine goals around environmental impact, equity and safety. “We normalized all of [the datasets] so that the key performance indicators are measured on a 10-point scale, where 10 is attainment or better, and then 1 is basically failing.”

Simply put, here’s how DSS works: If an aspect of a roadway needs a change, users make the change in the system, hit the run button and watch how the KPIs would move based on that adjustment. 

“The way the platform is set up, it includes different kinds of user experience areas. One is a simple data viewer,” said Joerg Tonndorf, an associate principal at Arup. “That’s where you go in and you can just basically look at one dataset at a time. It’s very easy to get there; anybody could really do that. It’s like a very simple geospatial mapping tool that you will find online.”

Another capability is post-project evaluation. “We chose to do Project Y because it moved the needle in this and that way. Two years on, did it do that, and should we be adjusting our hypotheses?” Madou said. “The real idea here is to … be able to make the right decisions as to what projects help us meet those goals the best and understand whether the actions we took in the past are lining up to what we said we’re going to do.”

The evaluations will require updated data. “Wherever available, we link datasets to APIs to almost provide an automatic update,” Tonndorf said. “Obviously, some of the other information that’s provided by vendors will have to be updated manually in a couple of years’ time, but the functionality is there to make that as seamless as possible.”

Additionally, when new datasets of value to DSS become available, the system can scale to ingest them, Tonndorf said. “The platform certainly allows for scaling and certainly allows for adding additional user features,” he said.

He sees public-sector interest in these types of systems growing. “San Jose has been forward-looking with the understanding that the world is changing much faster than it used to,” Tonndorf said. “There’s more uncertainty, such as COVID and its impacts or the current economic situation. So, cities need to have a tool that is much more reactive and quicker to basically do their planning work,” he said. “San Jose and other cities will need that much more to respond to immediate needs and to have an alternative to a more long-term planning approach.”

About three months ago, San Jose won a grant from the California Transportation Department to help fund the second phase of the program, which will involve more automation of data cleanup and processing, defining more projects to run through DSS and building a public-facing interface, Madou said. He expects work to start on that next March and last for nine to 12 months.

Historically, the city has run models over the course of a day or two to determine changes’ potential impacts on transportation. But models are mathematical representations of behavior and work on assumptions, Madou said. Because DSS uses observational data, it eliminates assumptions and reduces complexity. Plus, DSS can run in one hour, compared to 24 to 48.

“Anytime you say, ‘This is what we’re going to do,’ you just did research behind you and as time ticks on, assumptions start dwindling,” he said. “What we’ve given ourselves is the ability to make the next best decision over time, with changing circumstances. That’s the major difference here. Next year, the system will be updated because all the major road systems will be updated in the system, and the new projects that we’re looking at will be in the system. Of course, the assumption here is that our goals will probably be the same, but those could change too, and we could probably change the system to accommodate that, too.”

Stephanie Kanowitz is a freelance writer based in northern Virginia.