I created a crime data chat bot a while ago and I told you all about it two-ish months ago. Here’s the post I wrote back then in case you missed it or want to re-read:
The assistant is at NIBRSAI.com and it is designed to answer your crime data questions while highlighting the limitations of the available data and avoiding analytic AI slop. The goal remains, when ready, to add it to our newly released Crime Index website over at CrimeIndex.org. It wasn’t perfect before, and it isn’t perfect now. There are undoubtedly bugs and responses it won’t generate perfectly, but I think it’s getting better and want to figure out by how much!
This project is quite ambitious in my humble opinion.
There are more than a billion rows of NIBRS data, millions of rows of current and historical crime, clearance and staffing data, hundreds of thousands of homicide reports, and tens of thousands of rows of hate crime data (oh, I added hate crimes data). Historical SRS data captures crime back to the 1930s and clearances and staffing back to the late 1950s. Supplementary Homicide Report and current crime data are completely different sources covering different aspects of crime.
I want to include all of it and have the bot produce accurate answers in just a few seconds.
There have been challenges.
I built the assistant using my knowledge and expertise, but I wasn’t initially very good at querying the assistant from the perspective of someone who doesn’t have a strong working knowledge of the data sources and subject matter. Getting hundreds of real-world queries was an enormous boon that showed what it did well (a lot!) and what it didn’t do well (a lot!). In the end, all I want is the bot to query a database and return an answer, but the beauty of the chat bot is also its main challenge — there are a lot of ways to ask questions.
The main problem I found was twofold: first, the assistant wasn’t always very good at understanding what a user was asking and translating that into a usable SQL query to find the correct data. And second, users weren’t always very good at picking sources and phrasing questions in a way that maximized the assistant’s potential for generating an appropriate SQL query.
Basically, both the users and model had too much freedom and it was leading to fewer correct answers.
The problem started on the original main page:
Users were asked to choose a data source before asking a question. This was a great system for me because I usually know if I want to query NIBRS data or SHR data or current crime data. These are all distinct data sources in my mind with clearly distinct outputs.
This was not a great system for users who do not know the difference between NIBRS and SHR, which is going to be many if not most users. It’s unfairly limiting to expect users to know the data schema as a prerequisite to using the assistant.
Now there is a new home page which grounds your query in the question you’re asking rather than the source you’re asking it from. There are model queries below that hopefully help guide a user into both using appropriate language and highlighting the different kinds of questions that can be asked thereby subtly introducing the available data sources. The purpose of the home page is to help users craft a better query, not make them crime data sourcing experts.
Another big change is that the assistant is now far more deliberate in determining exactly what you’re asking for before asking the model to weigh in. Some crimes and geographies are ambiguous or not covered in the available data, so the site now works with the user to solve those ambiguities on the front end to craft a better query that avoids confusing the model. As a result, the AI model guesses far less often and uses more concrete rules to match the user’s question to the available data.
Let’s say you want to know how many carjackings there have been each month in Memphis since 2021. The assistant confirms the correct query is being run by figuring out the data source (carjackings are determined through NIBRS), the place (Memphis, TN which has just the police department), the metric being requested (carjacking count), the timeframe (since 2021), and the unit (count of incidents).
This process takes all the guesswork out of crafting the proper query for both the user and model. The result is this lovely chart.
There are a ton of other changes to make responses faster and honest to what’s available in the FBI’s (still flawed) datasets. For example, the FBI does not track shootings, but if you ask for shooting data it’ll help you pick something we do have data on.
The assistant is also much better at picking out geographic ambiguity and forcing the user to decide before it gets to building the query.
And it still gives you the option to choose data sources when multiple sources could be used to answer the same question:
The assistant still runs on Claude’s Haiku though I’ve used both Claude and Codex (OpenAI’s answer to Claude Code) as a sort of tag team engineering team. I asked Codex to summarize the changes we’ve made over the last two months and here’s how it summarized the improvements:
“In practical terms, the site is better at three things: answering common questions quickly, refusing questions it cannot answer honestly, and making its remaining uncertainty visible instead of hiding it behind a polished but unsupported result.
The main limitation is unchanged: the underlying FBI data still has uneven agency participation and coverage. What has improved is that the site is much less likely to confuse those data limitations with actual crime trends.”
In other words, the site should work much better now than before and you should give it another whirl if you’re so inclined. It should be better constrained from giving bad answers and it should be better at helping users to give it questions it can properly answer.
The assistant is getting closer to being a truly helpful crime data assistant. Please feel free to use it and provide whatever feedback you’d like. Hopefully the next update post about this project will be me telling you it is nearly perfect and ready for adding to the Crime Index!
New on the Jeff-alytics Podcast
This week’s guest is Insha Rahman, the president of the Vera Institute of Justice, the oldest and largest justice reform organization in the country. Insha has spent the past ten years building out everything from bail reform research to a 501c4 focused squarely on the politics of public safety and brings a fascinating perspective to the conversation around making perceptions of crime and safety more responsive to what the data says is actually happening.
We talk about what criminal justice reform looks like today in 2026 and how it might look into the future. We get into why the data behind reforms like bail reform rarely matches the political story told about them, what cities like Camden, NJ got right during their first homicide-free summer in fifty years, and why Insha thinks the gap between crime perception and crime reality might finally be starting to close.
You can also catch it on the Jeff-alytics YouTube page where I’ll be posting episodes and video clips, so be sure to like and subscribe there if you’re so inclined!
And while you’re here, be sure to check out these other recent great episodes:
Director, NORC Center on Public Safety & Justice John Roman
Criminologist Andy Wheeler
Filmmaker Lynn Novick
Georgetown Professor Christy Lopez
I've mostly been using it to query NIBRS data (for which the FBI's data tools are just inexplicably cumbersome and terrible). Just for that, it's a God-send.
I agree that the chat bot has gotten better, although i still find that it helps a *lot* to have at least a basic handle on the different data sources. For example, i use it a lot to pull offender information. I understand that when i tell it "show me violent crimes by year" the response will be a count of incidents, and when it tell it "show me violent crimes by year by offender age" the response will be a count of offenders - so in both queries i'm just asking about "violent crimes" but the unit of analysis changes based on adding a criterion. If i didn't understand that, i'd have a hard time reconciling those queries.
Maybe it's reasonable to expect the chat bot to be able to give a user highly contextual, query-specific guidance like that, but it feels like a tall order - in which case, providing (or just linking to) some more in-depth explainers for each data source might help?
As someone who works in juvenile justice, it's extremely valuable to have easy access to data sources that span adults & juveniles, so you can compare trends among juveniles to trends among older age cohorts. What NIBRS seems to be telling us, now that you've given us a good way to query it, is that - for all the media focus on violent crime involving youth over the past few years - the age group among whom violent crime has really gone up is adults aged 35 and older (it's down considerably among younger adults, and while the numbers for the under 18 population fluctuated a lot in the COVID era, they're lower now than they were in 2019). That obviously doesn't mean that violent crimes committed by young people aren't a problem, but it does undermine a lot of common misconceptions about how much young people contribute to violent crime in our society. Thank you for making this tool available!
This is pretty slick. Better and easier to use than the actual data exploration tool on the FBI website. While I can't make the exact graph I want to show my students, the table feature is nice. I can just make it myself with the numbers. Thanks!