Wonderful resource, Jeff--this is a hugely valuable effort! One thing I am curious about is how you will handle situations where an agency that is currently reporting data today stops reporting data in the future, or you want to add a new agency. For people who stop reporting, if you completely take an agency out, then that has the effect of making past values from today's RTCI different from previous editions of the RTCI, which doesn't seem ideal. If you don't take it out, then you can get "increases" or "decreases" in crime that simply reflect compositional changes, which is definitely bad. And for newly reporting agencies, are you going to require that an agency produces numbers all the way back to the earliest point in the index before it can be included? That avoids to compositional issues but may limit your ability to bring new agencies online, as many jurisdictions who launch new real-time reporting systems might not include data far back into the past.
I think this is a situation where you might consider not actually including all available agencies but instead doing a weighted average of some subset of agencies (one obvious approach would be to choose the weights to match the trajectory of actual realized national numbers in earlier years), like they do with the Dow Jones Industrial Average or S&P 500.
As a data visualization nut, I’m obsessed!!! How do I volunteer? Absolutely great work!
No need to volunteer, but please spread the good word and if you have contact with an agency that we don't have data for please send them our way!
Wonderful resource, Jeff--this is a hugely valuable effort! One thing I am curious about is how you will handle situations where an agency that is currently reporting data today stops reporting data in the future, or you want to add a new agency. For people who stop reporting, if you completely take an agency out, then that has the effect of making past values from today's RTCI different from previous editions of the RTCI, which doesn't seem ideal. If you don't take it out, then you can get "increases" or "decreases" in crime that simply reflect compositional changes, which is definitely bad. And for newly reporting agencies, are you going to require that an agency produces numbers all the way back to the earliest point in the index before it can be included? That avoids to compositional issues but may limit your ability to bring new agencies online, as many jurisdictions who launch new real-time reporting systems might not include data far back into the past.
I think this is a situation where you might consider not actually including all available agencies but instead doing a weighted average of some subset of agencies (one obvious approach would be to choose the weights to match the trajectory of actual realized national numbers in earlier years), like they do with the Dow Jones Industrial Average or S&P 500.
Agreed on that