Census workers raise concerns about data quality, with bureau leaning on records rather than in-person counts

By David Rodriguez and Byard Duncan, Reveal

This story was originally published by Reveal from The Center for Investigative Reporting, a nonprofit news organization based in the San Francisco Bay Area. Learn more at revealnews.org and subscribe to the Reveal podcast, produced with PRX, at revealnews.org/podcast.

First, it was a series of problems with government-issued technology in the field. Then it was a wave of complaints about duplicative work, arbitrary terminations and haphazard management practices.

Now, Census Bureau workers from across the country claim that efforts to speed up and streamline the count generated major confusion – and, in some areas, may have reduced data quality.

The long-term ramifications of faulty data could be profound: Inaccurate numbers from the decennial census could affect funding for cities, counties and states – and determine how many seats each state gets in the House of Representatives. Historically, undercounts among communities of color, renters and other groups have meant these communities don’t receive their fair share of resources for programs such as Head Start, food stamps and Medicaid and face a loss of political representation. The census means money and power.

Census workers have told Reveal from The Center for Investigative Reporting that the bureau closed cases abruptly and without clear explanation, swapping in data from existing government records for the gold standards self-response or in-person enumeration. They also said operational standards fell by the wayside as the count’s completion deadline jerked back and forth to deliver the apportionment numbers on President Donald Trump’s desired timeline.

Since on-the-ground enumeration efforts concluded in mid-October, the bureau has repeatedly pushed back the proposed release date for its first set of results. According to reporting from NPR, the deadline has drifted from Dec. 31 to March 6 – and may be delayed further yet. Along the way, the bureau has released vague statements about “anomalies” in its data, but has provided few details about the scope of the problems or what caused them.

There are, however, abundant clues in the stories of workers on the ground, in more than 150 responses to Reveal’s ongoing survey seeking census workers’ experiences. 

‘Suddenly, the rug was pulled out from under us’

The Census Bureau’s 2019 Detailed Operational Plan, released in July 2019, outlined several measures aimed at reducing staff workload during 2020’s nonresponse followup stage – the period when census workers, known as enumerators, knock on doors of households that have not yet responded to the paper, phone or online questionnaire. Among these measures was a new procedure called administrative records modeling, which employed a combination of IRS, Medicare and other government data to count housing units and clear cases from enumerators’ workloads after they’d conducted a single visit – not the standard six attempts. According to Census Bureau policies, self-response produces the best data and in-person enumeration produces information whose quality far surpasses data gleaned from administrative records.

The bureau anticipated that tapping administrative records would make workers’ jobs easier, cutting the nonresponse followup case workload by an estimated 12.9%. But on the ground, it perplexed and frustrated workers, who saw hundreds of cases closed before their eyes in large batches – even as they believed in-person visits were still possible. 

In a whistleblower complaint filed with the U.S. Department of Commerce’s Office of Inspector General in November, Baltimore field supervisor Amanda Colianni alleged that thousands of cases in her area were “miscategorized as ‘completed’ … when in reality only one attempt had been made to visit those addresses and no information was garnered,” according to a synopsis from the Office of Inspector General.

On her view of the database that supervisors can access, Colianni claimed that she could see when a given household was attempted once by an enumerator and could see the enumerator’s notes. Then at 11 p.m., the system would end nightly processing and she would see the case resolved due to “Max Attempts w/o pop count.” Maximum attempts would typically mean six.

The Census Bureau’s employee relations branch dismissed Colianni’s claims, maintaining that cases being removed after one failed enumeration attempt was “part of the program” and attributed her concerns to “a misunderstanding of why these cases were removed.” 

Yet Colianni insists that the bureau leaned on administrative records too heavily – and too soon in the process. She witnessed cases resolved using administrative records that she and her team of enumerators could – and should – have continued visiting in person, she said.

“We definitely felt like we had done a lot of legwork on some of these cases to get better answers,” Colianni told Reveal. “And then suddenly, the rug was pulled out from under us when we noticed that they were mysteriously marked completed. We’d wasted time and didn’t understand why.” 

In response to queries from Reveal regarding how many cases were closed after a single unsuccessful visit, the bureau said it would release those details in its operational assessment reports scheduled for late 2021 and throughout 2022.

The use of administrative records to close cases isn’t new. In fact, this year’s modeling process was the culmination of more than a decade of research, according to Amy O’Hara, executive director of the Georgetown Federal Statistical Research Data Center. She previously founded the Census Bureau’s administrative data curation and research unit. She said these records, though powerful in providing the bureau with data characteristics, also have weak points.

“I’m concerned about the data lacking characteristics and census having time to try to append those characteristics thoughtfully,” O’Hara said, speaking of information on age, gender, race and ethnicity. “I know that the online data collection worked, but I’m curious if it was full and complete census records.”

In Oakland, California’s census office, field supervisors Gondica Nguyen and Peggy Lee Scott witnessed a phenomenon similar to what Colianni described.

In October, a former enumerator on Nguyen’s team asked her to follow up with a household the enumerator was previously assigned and wasn’t able to complete. The enumerator had met the residents two weeks earlier but wasn’t able to count them because a memorial service was taking place at the address. When Nguyen checked the status of that household later, the case was already closed – supposedly by the former enumerator.

That enumerator, confused, approached Nguyen. “She asked, ‘How is that possible?’ ” Nguyen said. “I was, like, I don’t know. But (the program) says that you left a population count because it said ‘max attempt with pop.’ ” 

After another supervisor raised similar questions, Nguyen said, she logged into the census database and filtered for all Oakland cases marked as closed due to “max attempt” after only one attempt. She claims her search produced more than a hundred pages of results, for a total of 21,866 cases. She highlighted this to the manager as a potential issue in early October, then again a week or so later.  

Scott, a colleague of Nguyen’s, said she first noticed cases marked as completed with “max attempt” when she started looking up addresses in a public housing project in Richmond, just north of Berkeley. She also reported to her area managers that cases were being closed by reaching “max attempts,” even though the database showed there had been only one attempt to visit.

When Scott and Nguyen brought up their questions and concerns around cases reaching the maximum number of attempts and showing one or no attempt made, they said, they got unsatisfactory explanations.

Like Colianni, Nguyen said she and her team could have easily acquired more robust information through in-person attempts; they just never got the chance. 

“I totally believed that they could have been enumerated,” she said. “In fact, I had information for a few of the cases that were closed, and I was told by upper management that there was no known way to update those cases.” 

The Census Bureau affirmed that it did not instruct supervisors to lower the number of attempts required to close cases based on “max attempts.” Yet in an internal slide deck obtained by Reveal, the bureau outlined options for relaxing some standards to increase enumerator productivity during the 2020 count. The proposed “adjustments” included decreasing enumerators’ required visits. Another suggested adjustment, to eliminate random re-interviews, was flagged for “potential reduction in quality of enumerator work.” 

The Census Bureau declined to comment on Reveal’s questions about these proposed policy changes.

“There is an old line, ‘If you can’t dazzle them with brilliance, baffle them with bullshit,’ ” Scott said. “I felt like we were being baffled by bullshit the whole time. There was no big picture (or if there was it was kept very private). There was a constant barrage of, ‘Get it done, do it yesterday,’ with no overall gameplan. There was NO leadership.”

‘I would call it sabotage’

As workers like Colianni, Nguyen and Scott struggled to understand why cases they believed could be completed in person were being closed before their eyes, others experienced significant pressure to cut corners and close cases – no matter what.

Jeff Black, an enumerator in Carmichael, California, near Sacramento, described the end of field operations in his community as “a mad scramble to count heads.” He and his wife, Lynne, who also knocked on doors for the bureau, said they both endured pressure from managers to close cases using incomplete information gleaned from lease agreements provided by apartment managers. If a resident’s spouse or children weren’t listed on the documents, the Blacks claim, they were instructed to state that only one person lived at the address.

“I objected strenuously,” Lynne Black said. “I said, ‘I ain’t doing it. I’ll quit.’ I said, ‘When I go do an interview, I do an interview. If that’s not what you want, you hired the wrong person.’”

They said these addresses were primarily located in low-income areas that house a high proportion of people of color. The Blacks said they believe that they could have collected better information had the Trump administration not truncated the count by two and a half weeks and attempted to end it a month early.

“I would call it sabotage more than anything else,” Jeff Black said. 

The Census Bureau has an operational plan with specific procedures developed to conduct the nonresponse followup, and enumerators are allowed to obtain a population count from a knowledgeable proxy during the final stage of closeout, according to former Census Bureau Director John Thompson, who served in that role for four years until 2017 and spent more than two decades at the bureau.

“We were closing cases without visiting,” a regional technician manager who worked in the Los Angeles office told Reveal. “I hate to say that it’s as simple as that. But in conversations that I was involved in (with local managers), it was, ‘You have to finish, you have to reach 100% and that’s a directive from our office.’ ”

This 100% goal represents the number of housing units enumerated through all data collection operations, including self-response and nonresponse followup. In the days following the end of data collection, the Census Bureau announced that 99.98% of all households were accounted for. But despite a call from the American Statistical Association to do so, the bureau has not made public its assessment of the data’s accuracy.

Meanwhile, news reports and lawsuits have buttressed census workers’ claims. Taken together, they illustrate the chaotic trickle-down effect of the timeline for a multibillion-dollar government operation that was shortened with little explanation or forethought. 

Now, after months of insisting its revised schedule wouldn’t cause data processing problems, the bureau has admitted that a series of “anomalies,” if left unresolved, could warp the count by millions. And it’s unclear what options President-elect Joe Biden will have to patch up potentially massive errors.

Meanwhile, government watchdogs have been dialing up scrutiny of the bureau. On Oct. 23, the Commerce Department’s Office of Inspector General demanded Census Director Steven Dillingham’s secretarial briefings. Since then, the department has initiated an evaluation of the count’s data quality and called for an audit of the bureau’s incident response process, which would assess how well the bureau responded to cybersecurity incidents. On Dec. 10, the House Oversight and Reform Committee issued a subpoena to Commerce Secretary Wilbur Ross, demanding that he hand over documents “relating to grave data problems with the 2020 Census.”

This story was edited by Sumi Aggarwal and Esther Kaplan and copy edited by Nikki Frick.

David Rodriguez can be reached at drodriguez@revealnews.org, and Byard Duncan can be reached at bduncan@revealnews.org. Follow them on Twitter: @DaveeJonesLock and @ByardDuncan.