Why the FBI's 2023 Estimates Were Likely Better Than Recent Years

Digging deep into the monthly crime data.

The FBI's revisions to the 2022 estimates have already gotten far more attention than they deserve given there is no impact on our understanding of national crime trenss. But recent demands from Congress ensure even more effort will be expended to understand a quirk of national crime data. Today, therefore, I'd like to dig deeper into a major reason for the national crime estimates for 2020 and 2022 changing (remember, we are ignoring 2021) and examine the evidence of improvement in the 2023 data.

The most likely culprit to the enormous revisions to 2021 and smaller revisions to 2020 and 2022 data likely share the same major culprit — less full reporting than in previous years. Here you can see what changed in the 2022 data:

Overall participation was up a lot in 2022 compared to 2021 with 93.5 percent of the US population covered by an agency that reported data to the FBI. That’s normal by historical standards though slightly lower than the 2010s when reporting was the most complete it has ever been.

But the share of the population that reported full 12 months of data was a bit lower than recent years in 2022 which isn’t obvious unless you dig deep into the data. Between 2017 and 2020, for example, 87.2 percent of the US population was covered with 12 months of crime data reporting when the FBI initially reported data for those years.

That share fell to under 60 percent in 2021 and was just over 80 percent in 2022. In 2023, however, nearly 90 percent of the country was covered by an agency reporting 12 months of crime data to the FBI.

The FBI didn’t just revise the 2021 or 2022 data, there has also been a sizable revision what was initially reported in 2020 as well. The number of violent crimes in 2020 has been revised down by a few thousand but nearly 1,000 more murders have been estimated now compared to what was initially published.

The most likely culprit for these revisions is a slightly lower share of the population being covered by full coverage than average in 2020 and 2022 as well as the FBI incorporating new data as those agencies catch up. The FBI's initial estimate that there were nearly 50,000 violent crimes in Pennsylvania in 2020 was derived from only 56 percent of the state’s metro areas reporting data to the FBI (including just 4 months from Philadelphia). In reality, that number was closer to 37,000, so it’s not surprising that violent crime was revised downward.

Luckily, the need for estimation does not appear to be as severe in the 2023 data as with recent years. Over 91 percent of the country was covered by an agency reporting at least 10 months of data in 2023, up from 84 percent in 2022 and 58 percent in 2021. Overall, there were 9 states in 2022 where the FBI estimated more than 10 percent of their violent crimes compared to just 5 states in 2023.

Fuller coverage means less estimation which means fewer revisions when data eventually gets submitted.

Much of the revision that the FBI made to the 2022 data reflected agencies that had not reported 12 months when the data was initially published which is clear when looking at changes at an agency level. Those agencies that were most responsible for the 2022 revisions largely reported full data in 2023.

Only 81 of the 122 agencies that had their 2022 murder count revised upwards by 2 or more reported 12 months of data to the FBI, but 113 of those 122 agencies reported 12 months of data for 2023. Only 237 of the 414 agencies (57 percent) that had their violent crime count revised upwards by 10 or more initially reported 12 months of data, but 374 of those 414 agencies (90 percent) reported 12 months in 2023.

Of course, just because an agency publishes 12 months doesn’t mean they won’t revise that count at some point, but revisions due to incorrectness were reasonably concentrated amongst a small handful of agencies. Four cities (Dallas, Baltimore, Louisville, and Nashville) accounted for an upward revision of 132 murders between what they initially submitted in 2022 and what their revised totals were. Fortunately the publicly reported figures for each of those four cities largely matches what was reported to the FBI suggesting those four won’t be a big problem in 2023’s data. Overall, just 23 agencies out of 19,000 accounted for around 40 percent of the upward murder revisions (there were also 103 agencies that revised down including Greenwood, SC).

Revisions to 2023 data may also not yield and increase, just look at 2020's violent crime count being revised down. The ‘human error’ in Oakland that caused an extra 10,000 violent crimes to be reported should be fixed in the upcoming revisions to 2023 data.

It’s hard to say what other agencies will have had issues in 2023 that get corrected and revised though. Going through this exercise creates optimism that the FBI’s revisions next year will be smaller as reporting is improving. But there clearly will be revisions — because there always are — which highlights how imprecise our national crime estimates are.

Ultimately, it's important to remember that revisions to past data occur every year, don't always lead to increases, the revisions to the 2022 data and upcoming revisions to 2023 do not change our understanding of the national crime trends which are based on multiple sources and allow for the uncertainty inherent in using flawed data.

FBI could (and should) communicate these changes more effectively, but confusing changes communicated poorly are certainly not a new feature of the Uniform Crime Report program — as anybody who has ever tried to use the FBI’s datasets can attest! Fortunately, the evidence suggests that reporting is becoming even better as the NIBRS transition heads towards completion.