How The Crime Data Sausage Gets Made
A look under the hood at how a crime becomes a crime stat.
Lisa The Vegetarian is regularly considered one of the best Simpsons episodes ever made and — more importantly — one of my favorite episodes. There are many reasons why this is such a terrific episode, but none stand out as much as the scene where Lisa’s class is shown a short film on how meat gets prepared.
You may be rightly wondering where this intro is going, and I’m sure that my analyst trainers from my older Federal government days are groaning at the lack of an effective BLUF to this piece. Where this is going is that I thought it might be interesting to track how a crime goes from occurring to being reported by the FBI.
*Disclaimer* I am not a law enforcement agency so if I’ve missed or overly simplified a step in the process a bit too much please let me know!
Step 1: The Crime Happens
There were 8 major crime categories that agencies reported offenses for under the Summary Reporting System. There are 71 offenses that fall into 28 Group A offense categories under NIBRS.
If a criminal offense occurs that fits into one of the NIBRS Group A categories then it is the start of a long process that often — but not always — results in the crime being being reported by in the FBI’s annual count.
Of course, not every crime that occurs is successfully reported to the police. Let’s say that your car is stolen outside a bar in Springfield, Missouri. The National Crime Victimization Survey (NCVS) from 2021 suggests that roughly 3/4ths of auto thefts are generally reported to police, by far the highest share of successfully reported offenses per NCVS. Reporting for other types of crimes are much lower: around 60 percent of robberies, 40 percent of burglaries, and just over 20 percent of rape/sexual assaults.
So if your car is stolen in Springfield then you probably will call 911, and if you call 911 then it will probably be successfully reported to the police. If you decide not to call the police, or you leave the scene before they arrive, then that’s probably the end of the line for it being included in FBI statistics. If you call 911 to report your stolen car then your crime has moved on to the next step.
Step 2: A Call for Service Is Generated
Law enforcement agencies use Calls for Service to provide the basic metadata of what happened in an incident. If your car is stolen and you call 911 then an officer will be dispatched to meet you and take a report.
Sometimes an officer will witness a crime and an incident will be generated that way, but most of the major crimes that are reported are citizen generated incidents. The Calls for Service look like this and are a metadata record of hundreds of thousand or even million of incidents that occur in an agency’s jurisdiction each year.
Step 3: The Crime Is Recorded In A Records Management System
Most crime reports involve an officer arriving at a scene, interviewing witnesses, examining evidence, and eventually writing a report. Sometimes a crime is reported online or over the phone and a report can be made without physically talking to a victim.
The officer will input the information on what crime occurred into an agency’s Records Management System (RMS). The RMS is a repository where information can be accumulated and reported further up the chain. The RMS report will contain information about the victim, the crime that occurred — such as your stolen car, any offender or suspect information, and anything else that is pertinent to an investigation.
Step 4: The Agency Collects And Submits Monthly Data
There is no Federal requirement for agencies to submit crime data to the FBI. Some states require it by law, but for the most part agencies submit data voluntarily when they submit. Agencies that submit data typically do so monthly, but very few of them send data straight to the FBI. The vast majority sent to their state Uniform Crime Report program on a monthly basis.
Under NIBRS, there are Group A and Group B crimes. Agencies will send information on all Group A offenses that took place the previous month, to include “Administrative, Offense, Property, Victim, Offender, and Arrestee.” (See the NIBRS User Manual for more riveting details) Group B offenses are more minor offenses and only the number of arrests in those categories.
If your car is stolen and it is successfully reported then the agency will add that stolen car to all the other stolen cars that month. Information on all these incidents will be sent up the chain until it eventually reaches the FBI.
Step 5: The State UCR Program Submits Data To The FBI
Once the law enforcement agency collects its data it is sent to a state UCR program for aggregating. The state UCR programs help to manage local law enforcement agencies by providing training and guidance to ensure data is properly submitted. Many state UCR programs — though not all — have websites that post data above and beyond what the FBI asks for.
The North Carolina State Bureau of Investigation sums up the role of state UCR programs, writing:
“An obvious concern in the collection of crime statistics is the validity and uniformity of the data received. With the voluntary submission of crime reports from over 500 jurisdictions, the problem of attaining uniformity is readily apparent. Crime data submitted is reviewed for compliance with reporting standards to ensure accuracy and completeness. Data reliability is a high priority and noted deviations or arithmetical adjustments are brought to the attention of the submitting agency by contact through a UCR analyst.
DCI has also developed an audit program to assess and document the accuracy and integrity of the crime data reported to the UCR program. Although DCI makes every effort through audits, editing procedures, training practices and correspondence to assure the validity of the data it receives, the statistics' accuracy depends primarily on the adherence of each contributor to the established standards of reporting.”
So your state UCR program takes each agency’s aggregated data and, after making sure it isn’t horribly flawed, sends it up to the FBI.
Step 6: The FBI Collects And Audits The Data
There are 18,000 law enforcement agencies nationwide that collect crime data, and state UCR programs will collect and send to the FBI data from the nearly 70 percent that are NIBRS compliant this year and other non-NIBRS agencies that are still reporting SRS data.
There the data undergoes another rigorous audit to ensure that only accurate data has been reported. As the FBI summarizes it:
“In order to fulfill its responsibilities in connection with the UCR Program, the FBI edits and reviews individual agency reports for both completeness and quality. Members of the national program’s staff contact the state UCR Program in connection with crime-reporting matters and, as necessary, when approved by the state, individual contributors. Upon request, staff members conduct training programs within the state on law enforcement record-keeping and crime-reporting procedures. Following audit standards established by the federal government, the FBI conducts an audit of each state’s UCR data collection procedures once every 3 years. Should circumstances develop whereby the state program does not comply with the aforementioned requirements, the national program may institute a direct collection of data from law enforcement agencies within the state.”
Step 7: The FBI And Friends Create Crime Estimates
The FBI produces and reports crime data each year, but the FBI is not a statistical agency. Former FBI Director Robert Mueller is quoted as once saying about the FBI’s role in the UCR program "We collect, we announce, we pass on; we do not analyze."
The job of producing estimates, therefore, falls on the Bureau of Justice Statistics (BJS) and research institute RTI International. People much smarter than me in these two agencies have created detailed procedures for developing NIBRS estimates that were as good as humanly possible.
Step 8: The Report Is Published
FINALLY, crime data will see the light of day. The FBI does publish quarterly estimates on a 3 month delay, but there have not been any national quarterly estimates since the end of 2021 and the data that is released is unaudited — and frequently wrong — quarterly crime counts from about 300 agencies with over 100,000 people.
The Uniform Crime Report for any given year is finally published in September or October after the previous year has concluded. The primary report, known as the Crime in the US used to be published on the FBI’s website with all tables and data relatively easily accessible and downloadable.
In 2020, however, the FBI began exclusively publishing data on the Crime Data Explorer, a website created to house crime data and test the sanity of users. There is actually substantially more data on the CDE than used to be available in annual reports, but simple data can be maddeningly difficult to find.
If you want to know how many murders were cleared in 2019 you would simply go to the 2019 Crime in the US page, click on clearances, and select table 25 which gives us that data in an easy to navigate format on its own dedicated website. No downloading, and if I want to share that source with an interested party or cite it in a piece I can just use that web address.
The process is a bit more complex if you want to know that same fact for 2020. To find the murder clearance rate for 2020 on CDE you would click on Documents and Downloads then scroll down to the 4th set of reports titled Crime in the United States. Next you click on the year and change in to 2020 then select ‘Collections’ and scroll to the 4th to last which is Offenses Known to Law Enforcement. Select the download button which will download a zip file. From there you open the zip, find the Tables_25-28_Clearances folder, and open up Table 25. It’s the same table, but lord help you if you want to share or cite the data or otherwise explain how you got that figure.
Conclusion
And that long, elaborate process is how every major crime that occurs in the US gets reported by the FBI. Well, not every major crime, but those crimes that are successfully reported to a police department which appropriately reports its data to a state UCR program that correctly reports its data to the FBI.
Hi Jeff: My opinion:
"Crime Data Explorer, a website created to house crime data and test the sanity of users."
Reading BJS reports requires skill and the ability to notice subtle differences as to what's reported which is why so many get BJS data wrong or the fact that, for many, it's too intimidating for practitioners to use.
The FBI has now raised a similar bar for practitioners and crime writers. As you state, what used to be a simple process via their old system now requires immense effort and it's easy to misinterpret the data.
Federal crime data is now a nightmare on many levels. There was a national effort to create federal documents in plain language and it's noteworthy that the National Institute of Justice and the COPS office seems (in my opinion) to have embraced this.
BJS data can be a nightmare. They used to provide, from time to time, context from other research but no more.
But it's the FBI that has now transcended any effort to make their material easy to access or understandable. Your focus on city data may be the only way to understand urban crime.
Plain language and easy access to crime data seems to be a thing of the past.
Best, Len.
I note that you're eliding how something goes from an "incident" to a "clearance". Can you talk a little more about that process, and how it affects annual data? What happens if, say, there's a conviction that gets overturned on appeal? How, if at all, does that affect an agency's clearance rate?