Call for Papers “Advancing the Reproducibility of Psychological Assessment Across Borders and Populations”

The role of psychological assessment across a variety of educational, clinical, and organisational settings is expanding. This rapid uptake is closely related to the growing demand for evidence-based practice and policymaking, along with the evidence of the economic benefits of effective measurement. This practice-oriented development of the discipline demands close attention to be paid to ensure the psychometric quality and translatability of methods and tools across borders, domains, and populations. Indeed, researchers have widely commented on such issues, including the acknowledgement that the majority of the findings and tools in the behavioural science are derived from WEIRD (Western, educated, industrialized, rich, and democratic) populations (Henrich, Heine, & Norenzayan, 2010).

While this is a challenge for researchers, it provides fruitful grounds to advance both science and its role in society through better methodological standards. In particular, it demands addressing obstacles such as the selection of appropriate methods for assessment conducted across borders, populations, and languages. Doing so effectively improves the efficiency of data collection, achieving cross-cultural validation, and improving accessibility of assessments. Investment in such questions further creates potential to train a new generation of psychometricians able to work in dynamic contexts and to deliver complex concepts to various audiences outside the immediate assessment field. For such reasons, this Call for Papers is specifically aimed to highlight work conducted by junior researchers, particularly for studies either adapting to a new setting or language, or in attempting methods that combine multiple settings and languages as a feature of their research.

Along with a growing need to adapt assessments and their delivery, there is a concern that in a scientific climate where novel and positive results are considered more publishable than replication studies and negative findings, researchers may have little incentive to conduct important validation work and publish failed replications. Such issues are gaining considerable attention, particularly in light of the replication crisis reported in psychological science (Open Science Collaboration, 2015). Registered Reports (RR) are a form of manuscript submission that has been suggested as a potential tool to improve the reproducibility and combat the misaligned incentive structures of the field (Munafò et al., 2017). In the field of assessment research, RRs should outline assessment details and analysis plans and are submitted for review before data collection begins. This format is favoured as it holds the potential to help eliminate questionable research practices and to ensure that the peer review process is not biased by the reported results.

Given the growing use of psychological assessment outside the lab and its potential societal consequences, this Call for Papers aims to encourage the use of highest standards of scrutiny such as RRs in the assessment field. For this edition, we take a broad view as to what qualifies as assessment. We are primarily interested in the methodological underpinnings of conducting psychological research with subjective measurement components in multiple settings, substantively new settings, and/or in languages where the measure has not previously been validated.

Using the RR format, researchers can define what would be considered acceptable cut-offs and successful adaptations a priori. Manuscripts can therefore be judged on their methodological merit, regardless of outcome. For this special issue, RRs of empirical studies aiming to either develop novel tools or to adapt and test existing measures in novel settings will be mandatory for all submissions. We see numerous benefits in this: researchers can receive review and feedback early in the process when it is likely to be most valuable; the field will benefit from improved transparency and research quality. We hope that such practices will advance the assessment field and its utility in a variety of contexts and provide a platform for junior researchers to make contributions while receiving direct and early input from senior experts prior to carrying out their studies.

Submission Process and Timeline

All manuscripts should be submitted electronically at http://editorialmanager.com/ejpa.

Submitted papers should adhere to the author guidelines of EJPA (https://www.hogrefe.com/j/ejpa). Also, all submitted papers should be concise and take the word limits outlined in EJPA’s author guidelines (5,000 words for single-study empirical articles and 7,500 words for multi-study reports) as a general orientation for manuscript length; however, for explicitly stated reasons, papers that combine theoretical ideas with empirical illustrations can be longer. Exceptions regarding the word limits are possible upon request and should be justified by the authors in the cover letter.

We invite authors to submit their papers by

October 31, 2019

Questions concerning the special issue can be sent by email to one of the following editors:

Kai Ruggeri (kai.ruggeri@columbia.edu)

Matthias Ziegler (zieglema@hu-berlin.de)