Data Validity Issues


This continues our series of student reflections and analysis authored by our research team.


The Prosecution Project is a multi-year research initiative that is run by Dr. Michael Loadenthal, of Miami University’s department of sociology, and a cohort of students. Our team has built a “code book” to help us turn the prose of court documents and news articles into consistent data values. While the code book seeks to mitigate subjective interpretations of case details, some variables — like ideological affiliation — are less black and white and require an amalgamation of different context clues from multiple sources to definitively define. As the project’s scope and team have expanded over the last two years, different variables and values have been interpreted in increasingly varying ways. That is okay, as the existing understanding of a given variable may not always be the best. Furthermore, discussions of our coding process have engaged my critical thinking skills, taught me how to articulate my research process to other teammates, and given me a clearer grasp of the Prosecution Project’s potential impact on the public and law enforcement community. However, discordance in variable interpretations — while beneficial to my overall personal growth — has made it much harder to build samples in the analysis stage of our project.

For the last month and a half, I have been analyzing our dataset in order to understand domestic terrorism motivated by anti-government belief systems. I narrowed down the 1194 cases of political violence coded to a sample of 162 cases that were ideologically aligned with anti-government extremism.

Of course, “anti-government,” to those whose research focus lays elsewhere, can be interpreted in many ways. Anti-government could be used to describe an anarchist who does not believe in the existence of a government that exercises authority without justification. Or, anti-government might be used to label a Jihadist who seeks to terrorize the American government and people by attacking a federal institution. These examples show the many ways in which “anti-government” ideological affiliation can be incorrectly but understandably assigned to defendants who clearly do not possess a rightist anti-government belief system. This is significant because any defendants incorrectly included in my sample would have skewed my data for key variables like jail sentence, ethnicity, tactic, etc.

I individually assessed each of the 26 cases in the database ideologically coded as “rightist unspecified” and “unclear” (Loadenthal et al. 2018). For each case, I read the narratives briefly describing the case and assessed the following variables: group affiliation, foreign affiliation, motivation for choosing their target (labeled in our database as “target: why”). I included defendants affiliated with right-wing, anti-government groups (e.g. Oklahoma Constitutional Militia) even if their ideological affiliation for a specific attack was unclear. I excluded defendants who possessed unspecified rightist ideologies but exclusively chose their targets based on the religion, race, or foreign nationality of the person or property targeted. Ultimately, this case by case filtering process enabled me to verify that each case included in my sample possesses the fundamental characteristics of homegrown anti-government extremism.

This blog post demonstrates one of the many ways in which our research cohort is democratically creating a communal research process, making mistakes, positively reacting to our mistakes, and reiterating and improving our database and processes. The Prosecution Project is a labor of love that is being built through a culture of collaboration, complementary skills, and continuous learning.

– Nikki Gundimeda

Leave a Reply

Your email address will not be published. Required fields are marked *