Reproducible QCA studies with fs/QCA

The ability to reproduce published empirical studies is central to progress in science. Reproducibility requires the proper storage and sharing of data, which cannot be taken for granted, and a detailed, step-by-step description of the empirical analysis. Statistical research has no problems with the latter, in principle, because a script can be compiled (do-file etc.) that can be easily shared.

The invention of the R package for QCA allows QCA users to do the same which is one major advantage of this package. There are no numbers on the relative use of the package versus the established fs/QCA software offered by Ragin, but my hunch is that fs/QCA is still the dominant software.

Fs/QCA does not allow one to write a script because it is based on a graphical user interface (GUI), but it nevertheless allows users to ensure reproducibility, a feature that is largely ignored (based on my knowledge of the applied QCA literature). Everything you do with your data is documented in the results window, the window which pops up first when you open the software, and can be exported via File -> Save -> Output. The file type is .out, but out-files can be edited with ordinary text editors.

This is basically what you need to document what you did. The only challenge is to produce a file that exactly captures the analysis. Most probably, the more cumbersome method is to try to run the analysis from beginning to end in exactly the same way as is outlined in the manuscript. Once you forget a step, take one that is unnecessary or implement it incorrectly, you have to rerun the analysis from scratch. In the second variant, you simply make sure that all steps required for reproduction are contained in the out-file and then utilize any text editor to remove unnecessary entries. Here, you run the risk of producing an edited file that is missing key information for reproduction.

In either case, the fact that fs/QCA is GUI-based should not stop one from properly documenting the empirical analysis. For the sake of living up to the standards of the social sciences, we hopefully will see more such documentations of QCA research in the future.

About Ingo Rohlfing

I am a political scientist. My teaching and research covers social science methods with an emphasis on case studies, multi-method research, causation, and causal inference. I also became interested in matters of research transparency and credibility. ORCID: 0000-0001-8715-4771
This entry was posted in QCA, Qualitative Comparative Analysis, replication, reproducability, set relation, set theory and tagged , , , . Bookmark the permalink.

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.