AI could aid public transit track inspections

NEW YORK CITY — Looking for track defects and other maintenance issues on public transportation is traditionally a dangerous, taxing job for the inspectors who have done it for decades.

Often operating with just a flashlight, they walk miles of track looking for loose bolts, warped rails and anything else that could create issues for trains as they navigate the network. Making things even trickier, trains might go past at all hours of the day and night, especially in 24-hour systems like the New York City subway.

But technology and artificial intelligence appear to hold promise for making those inspections easier and more efficient, based on recent results unveiled by New York’s Metropolitan Transportation Authority and Google Public Sector.

A prototype that retrofitted standard Google Pixel smartphones to subway cars along one line to capture vibrations and sound patterns through built-in sensors and microphones identified 92% of track defect locations that had been identified by inspectors. The prototype, known as TrackInspect, will now be expanded into a pilot program to look for what transit officials refer to as track non-conformities, MTA and Google Public Sector announced last week.

In a statement, New York City Transit President Demetrius Crichlow said AI helps “make the ride smoother for customers but also make track inspector’s jobs safer by equipping them with more advanced tools.” 

MTA Chief Technology Officer Raf Portnoy called it a “game-changer” for the agency.

In the initial pilot along the “A” line in New York City, the smartphones collected 335 million sensor readings, 1 million GPS locations and 1,200 hours of audio. That data was then combined with the city’s database and ingested by a Google Cloud-based machine learning tool and analyzed.

MTA could then send its inspectors to inspect where the data suggested there were non-conformities, take photos of the site to verify them, then work out what to do next by using the agency’s maintenance manual, which was uploaded to the Google Gemini generative AI tool.

It is a marked difference from having the MTA’s geometry cars currently out on the rails to find any defects, which travel around the network less frequently and so are less able to spot issues.

“The takeaway here is that the Pixel phones allowed us to come up with a fast, scrappy and an innovative way that enabled NYCT to use AI in a way that they hadn't been able to before,” Tricia Jamison, a cloud solutions architect at Google Cloud Public Sector, said on stage during the Google Public Sector, GenAI Live & Labs event last week in New York City. “There is still work here to do, but ultimately it’s going to help track inspectors find and resolve non-conformities faster.”

The first-of-its-kind pilot represents a new frontier for AI, which is already being embraced by transit agencies in various ways. A recent report by the Urban Institute found AI is already being used to detect obstacles on tracks, manage crowds, monitor air quality, provide real-time arrival information and enhance dispatching technology. The Urban Institute found it also is helpful for predictive maintenance.

Those kinds of emerging technologies can also help bus traffic flow, said Leslie Richards, a professor at the University of Pennsylvania and chair of the Transportation Research Board’s executive committee. Philadelphia set up a pilot where cameras would automatically detect bus lane violations, especially people parking in them. Within 10 weeks, the pilot had detected 36,000 violations on 10 bus routes in the city, so it is set for a broader rollout in April.

“We know that we'll be able to increase reliability,” Richards said during a panel discussion at the Google event. “We'll be able to get the buses moving faster.”

Similarly, Danette Carll, the lead project manager at the Port Authority of New York and New Jersey’s Innovation Hub, talked up the agency’s efforts to introduce autonomous vehicles to Newark Liberty International Airport, having already tested an autonomous shuttle at John F. Kennedy International Airport. The Port Authority issued what Carll described as a “request for innovation” to see what solutions are out there and what can be done.

It can be tricky to get people on board, she said, so you need to reach out early to colleagues who may be impacted by such innovations.

“The internal collaboration is so hugely important, getting people invested in our effort of innovation,” Carll said on stage. “It's timely. It costs people time, costs them a little bit of risk. You really want to make sure everyone's on board with what your vision is.”

As for the MTA, this pilot program could revolutionize how they find track defects, especially as the technology could have new pieces added to it as the need arises and can be retrofitted to existing train cars.

“Maybe we need additional thermal cameras, maybe we need additional phones for the front-facing video to see if there's any encroachments on the track,” said Rob Sarno, MTA’s assistant chief track officer. “Maybe we need phones off the side to see if there's anything on the platform, garbage-wise, that could be pushed on all this stuff. We've seen through the years that the influx of money or resources that would be needed to do that to a geometry car or a regular revenue car without this type of technology is so great. This is unbelievable for us to be able to do it with something so simple as off the shelf products.”