Conference: Le défi de l’explosion des travaux sur les vaccins durant l’épidémie de COVID-19 : un rapport pour favoriser la cumulativité et le dialogue interdisciplinaire – Congrès Association Française de Science Politique

Thematic session on Conducting a literature review in political science: methodological and software innovations, professional and epistemological issues

Date: 2-4 July 2024

Location: Grenoble

Scientific directors: Vincent Caby (Sciences Po Grenoble) vincent.caby@iepg.fr
Deborah Galimberti (Sciences Po Saint-Germain-en-Laye) deborah.galimberti@sciencespo-saintgermainenlaye.fr

Scientific presentation:

Conducting a literature review is a “mandatory step » in any research project (Muller [2018] 1990); Surel 2015; Coman et al. 2022). Conducting a review consists in identifying, analyzing and synthesizing prior scientific publications on a research topic, and determining what is known, unknown and debated (Knopf, 2006). This exercise enables political scientists to verify the novelty and originality of their research, to reflect on their unconscious biases. However, handbooks of political science research methods say little about how to conduct a literature review. There is no mention of the different review methods, or of the growing number of systematic reviews in English-speaking political science (Arksey & O’ Malley 2007; Gough et al. 2013). These handbooks remain silent on the software and tools that can help political scientists in their work.

Outside France, a strong interest in literature reviews is emerging. Firstly, central journals in the discipline are publishing an increasing number of articles consisting of literature reviews. These articles follow a variety of review methods which include meta-analysis of quantitative research (Godefroidt 2023) but also increasingly qualitative research, as for meta-ethnographies (Boswell & Smedley 2022). This variety is fueled by interdisciplinarity and software innovation. Some reviews involve importing protocols from other disciplines (the PRISMA method, see: Sabbe et al. 2020). Others apply textual quantitative methods to data extracted from bibliographic databases, such as SCOPUS (lexicometric analysis of abstracts – see: Caby & Frehen 2021). Some are experimenting AI models, such as SCITE. AI, to report for a given article which other published research confirm or refute its conclusions.

This interest is also reflected on debates about systematic reviews. Systematic reviews have the distinctive feature of following a pre-established, explicit and justified method (Gough et al. 2012). A review is systematic when its author describes how he or she searched for, identified, analyzed and synthesized prior scientific publications on his or her research topic, and why. These systematic reviews take stock of a large volume of publications, thanks to online bibliographic databases. According to advocates of systematic reviews, applying the same standards that prevail for empirical surveys, could contribute to increase transparency and minimising biases (Dacombe 2018).

However, it remains an open question whether these review methods fulfill their promises. Their added value should be tested empirically. Under what conditions do they allow to reduce biases such as the theoretical approach of review’s authors, or the language of publications? Which other biases are they exposed to? These include the biases inherent to bibliometric databases and indicators. Systematic reviews are often associated to a positivist approach that pays little attention to the construction and circulation of concepts. However, these reviews could also serve postcolonial and feminist perspectives analyzing epistemic inequalities (Fricker, 2007) in the making of scientific knowledge (Bacevic, 2021).

The development of meta-analyses, systematic reviews and the diffusion of new AI tools call for a wider reflection on the ‘routine’ practice of conducting a literature review. The novel contribution of this methodological ST is to consider the literature review as a method in its own right. A reflection on the different review methods, epistemological approaches, tools and software, and their biases, could bring together all political scientists – insofar there is a consensus on the fact that all research projects entail a state of the art. Taking seriously literature reviews and opening up a reflection on review methods can benefit the development of the discipline, by synthetizing the results of numerous existing studies, evaluating their quality, identifying gaps and biases and therefore research agendas.

Proposals may illustrate and discuss a review method, or a tool facilitating all or part of this exercise, with a reflexive stance. They might present: the context in which it is used, the epistemological approach, its aims (to explore an emerging subject, to address a well-researched question, to analyze the structure of a field – the transnational circulation of a concept, the evolution of a concept over time, unequal relations between geographical areas), its constraints (timing, technical expertise) and its benefits (in terms of minimizing biases). Proposals comparing the advantages and disadvantages of two methods or tools are welcome.

References:

Arksey H., O’ Malley L. (2005) Scoping studies: towards a methodological framework, Int. Journal of Social Research Methodology, 8(1), 19-32.

Bacevic, J. (2021). Epistemic injustice and epistemic positioning: towards an intersectional political economy. Current Sociology, 71(6), 1122-1140.

Boswell, J., & Smedley, S. (2022). The potential of meta-ethnography in the study of public administration: a worked example on social security encounters in advanced liberal democracies. Journal of Public Administration Research and Theory, 33(4), 593–605.

Caby, V., & Frehen, L. (2021). How to produce and measure throughput legitimacy? Lessons from a systematic literature review. Politics and Governance, 9(1), 226-236.

Coman, R. et al. (2022). Méthodes de la science politique : De la question de départ à l’analyse des données. LLN : De Boeck Supérieur.

Dacombe, R. (2018). Systematic reviews in political science: what can the approach contribute to political research?. Political Studies Review, 16(2), 148-157.

Déloye, Y., & Mayer, N. (2019). Les trois défis de la science politique française : Retour sur la période 1968-2018. Idées économiques et sociales, (1), 28-39.

Fricker, M. (2007). Epistemic injustice: power and the ethics of knowing. Oxford: OUP

Godefroidt, A. (2023). How terrorism does (and does not) affect citizens’ political attitudes: a meta‐analysis. American Journal of Political Science, 67(1), 22-38.

Gough, D. et al. (2013). An introduction to systematic reviews. London: Sage

Goyal, N., & Howlett, M. (2018). Lessons learned and not learned: Bibliometric analysis of policy learning. In: C. Dunlop, C. Radaelli, & P. Trein (Eds.), Learning in public policy (27-49). Cham: Palgrave Macmillan.

Greffet, F. (2023). A view from France. French Politics, 21, 139-146.

Hong, Q. (2023) Les revues de littérature systématiques mixtes. LIEPP.

Knopf, J. W. (2006). Doing a literature review. PS: Political Science & Politics, 39(1), 127-132.

Mour, C. (2022). La pratique des revues de littérature exploratoires interdisciplinaires au LIEPP. LIEPP

Muller, P. (2018). Les politiques publiques. Paris : PUF

Sabbe, M., Moyson, S., & Schiffino, N. (2021). Citizen‐agency versus state‐agency at the frontline in prisons and probation services: A systematic literature review. Social Policy & Administration, 55(1), 206-225.

Smith, A. (2020). A glass half full: the growing strength of French political science. European Political Science, 19, 253-271.

Surel, Y. (2015). La science politique et ses méthodes. Paris : Armand Colin

Program:

Présidence & discussion :
François Bonnet (CNRS), Vincent Caby (Sciences Po Grenoble), Deborah Galimberti (Sciences Po Saint-Germain-en-Laye)

Chloé Bérut (Ca’ Foscari University), Coder des articles ? Le cas d’une revue de littérature systématique sur les politiques européennes du digital

François Bonnet (CNRS), Vincent Caby (Sciences Po Grenoble), Deborah Galimberti (Sciences Po Saint Germain en Laye) , Les revues de la littérature en science politique : innovations méthodologiques et logicielles, enjeux professionnels et épistémologiques

Nihit Goyal (Delft University of Technology), Innovative enough? A bibliometric review and computational text analysis of research on policy innovation

Stéphane Moyson (UCLouvain), Mathias Sabbe (ULiège), Inès Lequeux (ULiège), Anthony Ricotta (UCLouvain), Nathalie Schiffino (UCLouvain), Leçons de cinq applications de l’approche PRISMA : forces, challenges et mise en perspective

Jeremy K. Ward (INSERM, CERMES3), Le défi de l’explosion des travaux sur les vaccins durant l’épidémie de COVID-19 : un rapport pour favoriser la cumulativité et le dialogue interdisciplinaire

Participants:

BERUT Chloé chloe.berut@unive.it, BONNET François francois.bonnet@umrpacte.fr, CABY Vincent vincent.caby@iepg.fr ,GALIMBERTI Deborah deborah.galimberti@sciencespo-saintgermainenlaye.fr, GOYAL Nihit Nihit.Goyal@tudelft.nl, LEQUEUX Inès ines.lequeux@uliege.be , MOYSON Stéphane stephane.moyson@uclouvain.be, RICOTTA Anthony anthony.ricotta@uclouvain.be, SABBE Mathias mathias.sabbe@uliege.be, SCHIFFINO Nathalie nathalie.schiffino@uclouvain.be, WARD Jeremy