class: center, middle, inverse, title-slide .title[ # Multi-Method Designs Centering Case Studies ] .subtitle[ ## Multi-Method Research II ] .author[ ### <large>Jaye Seawright</large> ] .institute[ ### <small>IQMR 2025</small> ] .date[ ### June 27, 2025 ] --- class: center, middle <style type="text/css"> pre { max-height: 400px; overflow-y: auto; } pre[class] { max-height: 200px; } </style> --- --- ### Pure Qualitative Research Is Fine! --- ### Integrative Research and Qualitative Methods - Quantitative researchers sometimes, but not always, identify assumptions in the way that quantitative researchers do - Other times, they talk about boundary conditions, limitations, challenges, needs in their research, etc. --- ### Some Ideas 1. Enhance testing-based process tracing by reinforcing surprising steps in the process-tracing argument 2. Enhance discovery-based process tracing by broadening the range of subject matters explored 3. Machine learning to help position texts within large collections 4. Quantitative components to help move between levels of analysis --- ### Enhance Surprising Process-Tracing Steps - The best process-tracing evidence strongly supports one hypothesis but is very surprising under other hypotheses. - This motivates people who accept those hypotheses to reinterpret. - Adding additional evidence with different epistemological properties makes such reinterpretation harder, and therefore speeds up necessary theoretical change. --- ### Party-System Collapse  ---  ---  ---  ---  ---  ---  ---  ---  --- ### Lacombe (2018) The NRA deploys ideas to construct a politicized group social identity among gun owners, who are then easy to mobilize into political participation. --- ### Lacombe (2018) 1. Archival research 2. Structural topic models 3. Qualitative and quantitative content analysis 4. In-depth reading 5. Time-series analysis 6. Process tracing ---  ---  --- The perceived opponents of gun rights consist of several distinct groups, the three most prominent of which are politicians, the media, and lawyers. --- Politicians are described as: bureaucrat(ic), reformer(s), big city, urban, elitist, special interests, tyrannical, and "F" troop (politicians who have received "F" ratings from the NRA). --- The media is described as: liar(s), coward(ly), elitist, phony, cynical, devious, shameless, and propaganda/propagandists. --- Lawyers as: greedy, fat-cat, opportunist(s), big city, urban, elitist, phony, cynical, and liar(s). --- A set of more general characteristics is used to portray gun regulation proponents as un-American, including: fanatic(s), extreme/extremists, radical(s), hysterical, anti-liberty, Communist(s), tyrannical, globalist, and internationalist. Finally, gun control supporters are described as "anti-gunners" and "the gun ban crowd." ---  ---  --- Case studies showing that dissemination and textual influence include broad arguments and not just word-use patterns. ---  --- Case studies of letters written to Presidents Johnson and Bush in response to NRA calls to action. --- ### Enhance Discovery-Based Process-Tracing - It is much easier to add a new topic/theoretical theme to qualitative research before it begins than after data collection is complete. --- ### Enhance Discovery-Based Process-Tracing 1. Collect some kind of data set related to the question of interest, that is as inclusive as possible. 2. Use a machine-learning method to identify predictors related to the topics of interest. 3. Treat those predictors as clues about themes to explore in qualitative research. --- ### Example: Rise of the U.S. Alt-Right - Alex Jones, InfoWars, and the Alt-Right - It is likely that whatever is responsible for the growth of the alt-right has left traces in the discussions on InfoWars broadcasts. --- ### Lasso Regression OLS regression minimizes: `\(\sum(Y - X \hat{\beta})^2\)` --- ### Lasso Regression Lasso regression minimizes: `\(\sum(Y - X \hat{\beta})^2 + \lambda |\hat{\beta}|\)` ---  ---  ---  ---  --- { ---  ---  ---  ---  --- ###Alex Jones and the Proud Boys | Outcome | Word | Coefficient | | :--------: |:--------------------:|:-----------:| | Alex Jones | Pappert | 6.70 | | Alex Jones | Sanctimoniously | 6.63 | | Alex Jones | 87778925398777892539 | 4.34 | | Alex Jones | Bolsonaro | 4.22 | | Proud Boys | Zionist | 0.11 | | Proud Boys | Loudmouth | 0.08 | | Proud Boys | Collusion | 0.01 | Words and Coefficients Associated With Alex Jones and the Proud Boys --- ### Positioning Texts in a Collection - Qualitative researchers often discuss texts drawn from large collections, as a way of characterizing both the individual text and the broader collection. --- ### Positioning Texts in a Collection 1. Run a simple text-as-data model on the text collection as a whole. 2. Summarize the topics selected from the model using qualitative readings of selected texts. 3. Use statistics related to the model to show how individual texts selected for close reading relate to the collection as a whole, and also to describe the overall population. --- ### Example: January 6th Legal Documents - The main DOJ Jan. 6th website lists about 2,700 documents related to cases against people involved with Jan. 6th. ---  ---  ---  ---  ---  ---  --- - Topics 3 and 7 are various kinds of judicial procedural documents, shared across various kinds of cases. - Topic 2 is mostly plea bargains, while topic 5 is mostly charge sheets. --- - For Topic 1, the most representative text is a stipulation of facts in the case of Jonathan Davis Laurens. ---  ---  ---  --- - The most-representative document for topic 4 is an FBI affidavit about the activities of Brandon Nelson and Abram Markofski. ---  ---  --- - Topic 6 is most typically represented by a statement of offense for Daniel Ray Caldwell. ---  ---  ---  ---  --- - With respect to topic 8, the most representative document is a statement of offense for Matthew Capsel. ---  ---  ---  ---  --- - With this framework, we can position specific documents we want to read qualitatively within the broader collection. - We can also get side benefits, like answering questions about the relative prevalence of different degrees of violence. --- ###Degrees of Violence | Topic | Average Membership | | :-----------------------: |:--------------------:| | 4 (Least Violence) | 28% | | 1 (Near Violence) | 10% | | 6 (Violence) | 3% | | 8 (Violence, Recruitment) | 9% | Texts' Average Degree of Topic Membership --- - Returning to our main agenda, suppose we want to closely read one case. To pick an actually randomly selected example consider the statement of facts connected with Joseph Howe. The model tells us that Howe belongs in topic 8 (96% membership). ---  ---  ---  ---  --- ### Moving Between Levels of Analysis - In any kind of research, it can be tough to move between levels of analysis. - Embedding a quantitative design component can sometimes save time, effort, and money in covering another level of analysis. --- ### Example: South American Party Trajectories after Collapse - Focus groups to establish a nationally shared set of meanings regarding parties. - Surveys to test whether party organizational resources were evenly shared across geography. --- ### Two More Challenging Designs 1. Combining Game Theory and Qualitative Research 2. Statistically Testing Generalizability of Qualitative Research --- ### Hands-On On [this website](https://jnseawright.github.io/practice-of-multimethod/Chapter-3.html#Process-tracing_elite_messaging_on_climate_change), work on the exercise *Process-tracing elite messages on climate change.*