Be Careful With Preliminary Crime Data

Monthly crime data from the FBI is coming, here's why you'll need to be careful.

This post marks 100 posts here at Jeff-alytics, arguably the most fun newsletter about crime data in the entire state of Louisiana. As a thanks for subscribing and reading his son's work, my dad agreed to buy a Super Bowl ticket for every subscriber if the Saints make this year’s Super Bowl in New Orleans!

The FBI released quarterly data covering the first quarter of 2024 in June showing massive, across-the-board declines in all types of crime nationally. I’ve argued previously that these findings are almost certainly overstating the degree of declining. In this post I’d like to use one Texas city to show why this happens and why it is much less of a big deal as the year goes on.

Wichita Falls is a city of just over 100,000 people northwest of Dallas along the Texas border with Oklahoma (Hook ‘em, 34-3). Wichita Falls submitted data through March to the Texas Department of Public Safety for inclusion in the quarterly report. DPS puts together the data from every Texas agency that submits and sends it up to the FBI for inclusion in the quarterly report.

Wichita Falls showed a huge 66 percent decline in violent crime in the first quarter and a smaller (but still large) decline in property crime of 15 percent in the FBI quarterly data.

Yet a review of Wichita Falls data shows violent crime actually declined less substantially in the first quarter than what was initially reported by FBI (which was reporting what was sent to DPS by the local police department) and property crime rose.

We can see how Wichita Falls’ reporting has changed over time thanks to the Real-Time Crime Index, because we captured the agency’s submissions to the Texas DPS for June, July, August, and September. What happened with Wichita Falls is that the agency underreported the first quarter and then stopped reporting data after the March submission, with nothing submitted to the Texas DPS in April through August.

Because they hadn’t submitted data for the second quarter, the FBI’s second quarter report did not include Wichita Falls. But when we checked Wichita Falls in September we saw that the agency had fully caught up with data now available for January through September 2024.

Problem solved!

With the September update came somewhat large increases in the agency’s crime counts for the first quarter of 2024. The below table shows the number of offenses by crime type originally submitted to DPS and the updated number that was recently submitted.

A deeper dive into property crimes by month highlights what happened. Burglary increased from 30 in February originally to 35 in the updated data and from 14 to 23 in March. Theft in March skyrocketed from 103 originally to 151 when updated, and motor vehicle theft went from 10 to 22 in March.

Violent crime in Wichita Falls was still down, quite a lot, in the first quarter of 2024. Digging into aggravated assaults and rapes shows the 2024 first quarter figures for each offense type aren’t huge historical anomalies for Wichita Falls suggesting those crimes were legitimately down rather than reflecting a reporting error. But violent crime wasn't down quite as much as shown in the first quarter data.

Property crime, by contrast, was probably up slightly in the first quarter of 2024 in Wichita Falls despite the FBI quarterly data reporting a 15 percent decline.

And therein lies the main challenge of the quarterly data as well as the upcoming monthly data. The FBI is reporting what is sent to it by the states, the states are recording what is sent to them by the agencies, the agencies are sending what they have as of the reporting deadlines. But it is prone to errors if you’re not careful because agencies still have a lot of time to catch up.

Agencies don’t have to submit final crime data for 2024 to the FBI until the spring of 2025, so the figures reported through March 2024 aren’t all completed while the figures through March 2023 are completed.

Usually the updates are small. A few thefts or a robbery added to the total from a couple of months ago. Sometimes, like in the case of Wichita Falls, the changes can be quite big. We run into this problem with the RTCI and do our best to audit out agencies that are clearly underreporting.

All of which is why the quarterly and monthly data needs to be approached with some skepticism early in the year. Because the FBI is simply reporting what is being sent up the chain there is a lot of room for added offenses which will only impact the current year (sometimes offense counts fall but that’s less usual).

These problems will be fixed later in the year, as is already the case in Wichita Falls this year. When the FBI’s Q3 numbers come out in December you’ll see a far closer representation to what the year end estimates might be though the underreporting problem will still persist. The improvement in accuracy in the quarterly data is clearly visible in the below table showing the percent change in crime types last year based on the Q3 data, the Q4 data, and the recently released final estimates.

The FBI’s third quarter data clearly predicted the country’s crime trends — declining violent and property crime with a massive drop in murder and a massive increase in motor vehicle theft — but the percent changes weren’t even close to reliable until the fourth quarter.

You can already see the same movement happening in the 2024 quarterly data, especially compared to where the Real-Time Crime Index thinks things stand through midyear 2024.

Being cautious when using the unaudited crime data will be especially important for when the FBI starts reporting monthly data in a few weeks. The FBI acknowledged this impending addition to CNN in a recent article (about the RTCI!). Per the article:

Further, the FBI told CNN on October 22 it plans to soon release new crime stats every month through its Crime Data Explorer (CDE) site.

“Rather than data being annually refreshed, it will be continuously updated, ensuring transparency in the reporting process,” the bureau said. “This will allow contributing agencies and states to view the data that they have submitted to the FBI and make corrections if needed.”

Some of this data will be really good while other cities will be quite suspect, and all of it is subject to change. This new monthly data will require patience and caution given the preliminary nature of what is being reported.

Just because the data may not be exact doesn't mean preliminary reporting is useless though! Knowing murder is down a lot this year while having only an educated guess how much is better than learning next Fall that it was down an exact estimated percent. Just be careful in the Wild West of preliminary data.