After the Paper | Scoping studies: Advancing the Methodology
Published in Protocols & Methods
Here, we reflect on the conceptualization, writing and publication of Advancing the Methodology, and describe three lessons drawn from the experience that have influenced our career trajectories.
We wrote Advancing the Methodology as trainees (DL and HC were PhD students, and KKO was a postdoctoral fellow) in the School of Rehabilitation Science at McMaster University. Each of us had recently used Arksey & O'Malley's (2005; fondly referred to hereafter as A&O) pioneering methodological paper to undertake an (at the time!) novel method of knowledge synthesis: the scoping review. Their paper offered an excellent road map, but also illuminated new challenges. A&O deftly recognized the work to be done, stating:
“One of the purposes of the present paper is to stimulate discussion about the merits of scoping studies, and help develop appropriate methods for conducting such reviews……We look forward to seeing how the debate progresses.”(p31)
As trainees, we felt ourselves to be less than ideal candidates to answer this call, but a fortuitous occasion of sage mentorship from DL’s doctoral supervisor, Dr. Cheryl Missiuna, changed our minds. Dr. Missiuna suggested that our collective experiences (published by DL in Research in Developmental Disabilities, HC in the Canandian Journal of Occupational Therapy , and KKO in Aids and Behavior) had value, and that the three of us partner to take on this challenge. Her encouragement gave us the confidence to undertake the work. She taught us Lesson #1: Look for ways to foster confidence in your trainees. She embodied this lesson by promoting our independence. Once the seed was planted to write a paper about our collective experiences, we were left to organize ourselves and do the work. Which we did! In addition to encouragement, Dr. Missiuna provided the funds to ensure the paper was open access. Her belief in our abilities, combined with the post-publication evidence of interest in the paper, nurtured our confidence as early career researchers, helping to establish us early on as methodologists in a rapidly developing field.
It helped that we were in the right place at the right time. Figure 1 illustrates the growth in scoping review (or scoping study) publications between 1997 and 2022, showing how the rate of scoping review publication was on the cusp of rapid acceleration following the 2005 A&O paper. However, we combined good luck with adequate preparation: we had each read widely, illuminating the gap that needed to be filled. In retrospect, we see this as Lesson #2: Question the status quo. We embody this lesson in nudging our students (and our colleagues!) to consider how advancing existing methodologies could be a component of their work.
For example, HC was an external examiner on a student’s comprehensive exam that occurred just a week before publication of the PRISMA scoping review reporting guidance. She asked the student to reflect on how the new guidance might influence her (already written in preparation for publication!) review. That question, along with sage encouragement from HC, spurred the student to write an editorial based on their experiences of altering their paper to reflect the new reporting guidance. We encourage our trainees to follow existing methodical guidance, but we also prompt them to continually reflect on how a methodology could be adapted to better address the research question. Over the years, lesson #2 has motivated us to question the other aspects of our field that could benefit from disrupting the status quo.
Figure 1: Number of scoping reviews by year
Note: Figure was created by conducting an electronic search for ‘scoping study’ or ‘scoping review’ from 1997 to March 2022 in Medline, EMBASE, CINAHL, and PsycINFO.
Importantly, we also benefited from mentors who believed in the value of collaboration. Combining our experiences into one paper made it stronger than if any one of us had written it alone, leading us to Lesson #3: Consider the process of doing the work to be equally important as its outcome. This can be difficult to remember in academia, where outcomes are key to advancement, specifically those that can be tallied in an annual progress or impact report. Yet the impactful, collaborative process of co-ideating and co-writing gave us an early model to replicate and expand in our subsequent careers.
Indeed, Advancing the Methodology spurred new collaborations with researchers doing similar knowledge synthesis methodological work, formalized in a Canadian Institutes of Health Research Knowledge Dissemination grant in 2015 led by KKO. The grant brought together an international working group in Toronto to discuss scoping review methodological improvements. At this meeting, Dr. O’Malley herself presented the historical perspectives and origins of the scoping study (NB: A&O used the term scoping study). Figure 2 shows one of our brainstorming boards from this meeting – as you can see, a rich idea generation experience!
We joined forces with other groups working in this same area to continue collaborative efforts, including Drs S. Strauss and A. Tricco's group at the University of Toronto, who had expertise in knowledge syntheses and developing reporting guidelines (including a scoping review published in 2016 in BMC Research Methodology. This collaboration led to a 2014 Editorial in the Journal of Clinical Epidemiology and the publication of the PRISMA guidelines for Scoping Reviews. Further collaborations expanded to include another team working on reporting guidance for scoping reviews: Mikah Peters from the Joanna Briggs Institute (JBI).
Figure 2: Graphic Facilitation from the Advancing the Field of Scoping Study Methodology International Meeting held in 2015 (funded by the Canadian Institutes of Health Research) brainstorming board.
Amazingly, as of March 2022, Advancing the Methodology has been cited 6792 times (Google Scholar) and downloaded over 170,000 times. We never anticipated the attention the paper has received, nor the wide-ranging impacts of its publication on our careers. These impacts include the considerable time and energy spent on mentoring, advising, and presenting on scoping reviews, exposing us to a plethora of new research interests, perspectives, and worldviews.
Scoping reviews: Looking to the future
Many lively discussions have been held with the goal of debating the identity, conceptualization, definitions, and terminology related to scoping reviews. The most interesting were debates related to the value of quality assessment of included studies in a scoping review, the debate on terminology – is it a study, or is it a review? - and the pros and cons, from a scientific perspective, of the inherently iterative nature of the scoping process, as emphasized in the A&O paper. Interestingly, these issues remain relevant to this day. We also note emerging areas of debate such as how best to present scoping results for a wide range of stakeholders, how best to conduct a qualitative synthesis of data, and when to conduct the (optional) consultation exercise. We look forward to ongoing discussions!
We end with an expression of our collective gratitude for this experience. Publishing this paper had both measurable (the citations!) and immeasurable impacts on our career trajectories, influencing our dedication to the essential scientific pursuits of collaboration and mentorship. We would like to amplify A&O’s original call by encouraging others - at any stage of their careers - to join in methodological advancement efforts to benefit evidence-informed practice, research and policy.
Acknowledgements
The authors acknowledge the contributions of Sarah Zarshenas, Research Associate at University of Toronto, in gathering the data for and creating Figure 1. Kelly K. O’Brien (KKO) is supported by a Canada Research Chair in Episodic Disability and Rehabilitation from the Canada Research Chairs Program.
References
- Levac, D., Colquhoun, H. & O’Brien, K. K. Scoping studies: Advancing the methodology. Implement. Sci. 5, (2010).
- Arksey, H. & O’Malley, L. Scoping studies: towards a methodological framework. Int. J. Soc. Res. Methodol. 8, 19–32 (2005).
- Levac, D., Rivard, L. & Missiuna, C. Defining the active ingredients of interactive computer play interventions for children with neuromotor impairments: A scoping review. Res. Dev. Disabil. 33, (2012).
- Colquhoun, H. L., Letts, L. J., Law, M. C., MacDermid, J. C. & Missiuna, C. A. A scoping review of the use of theory in studies of knowledge translation. Can. J. Occup. Ther. 77, 270–279 (2010).
- O’Brien, K., Wilkins, A., Zack, E. & Solomon, P. Scoping the field: identifying key research priorities in HIV and rehabilitation. AIDS Behav. 14, 448–458 (2010).
- Tricco, A. C. et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann. Intern. Med. 169, 467–473 (2018).
- Miller, E. & Colquhoun, H. The importance and value of reporting guidance for scoping reviews: A rehabilitation science example. Aust. J. Adv. Nurs. 37, (2020).
- O’Brien, K. K. et al. Advancing scoping study methodology: A web-based survey and consultation of perceptions on terminology, definition and methodological steps. BMC Health Serv. Res. 16, (2016).
- Tricco, A. C. et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med. Res. Methodol. 16, (2016).
- Colquhoun, H. L. et al. Scoping reviews: Time for clarity in definition, methods, and reporting. J. Clin. Epidemiol. 67, (2014).
11. Peters, M. D. J. et al. Scoping reviews: reinforcing and advancing the methodology and application. Syst. Rev. 10, 263 (2021).
Follow the Topic
-
Implementation Science
This journal publishes research relevant to the scientific study of methods to promote the uptake of research findings into routine healthcare in clinical, organizational, or policy contexts.
Related Collections
With Collections, you can get published faster and increase your visibility.
Breaking Frameworks: Revisiting, Extending, Integrating, and Theorizing Implementation Frameworks
The field of implementation science has amassed a large number of frameworks3,4,5. These are sometimes also called models, but because the term “model” is used in many other contexts in research, we will use the term “framework.” While many of these frameworks express a goal of supporting research in the implementation of evidence-based practices and programs, researchers and especially new entrants to the field continue to express confusion and uncertainty about how to use existing frameworks and which to use for what purposes. New frameworks are often developed without clarity about how they fit within the existing corpus of frameworks.
Despite the large number of frameworks 4, their use often reflects a lack of deep understanding of the content of the frameworks. Implementation researchers often describe frustration with existing frameworks while continuing to use them. A major issue is that once published in a peer-reviewed venue, there is no clear path to suggest changes or updates to the frameworks. A few, such as the Consolidated Framework for Implementation Research (CFIR)6, the Exploration, Preparation, Implementation, Sustainment (EPIS) framework7,8, and the RE-AIM framework9, have been updated through processes determined by a relatively small group of researchers10; others remain essentially fixed as they were published, or updated once but not again11,12. This can lead to reification of the frameworks in their original form. These issues may constitute a major “sticking point” for advancing the science of implementation, as well as contributing to complexity for implementation practitioners who use frameworks as tools developed through the science. Emerging global health priorities, including health equity, structural racism, coloniality, climate and planetary health, digital transformation, and policy implementation, raise questions about whether existing frameworks adequately capture power, history, resource constraints, political economy, community agency, and cross-setting adaptation13. The increased geographic scope of published studies adds to concerns about whether theories and frameworks current in the literature support the broader scope.
We also note the importance of understanding the function of existing frameworks, most clearly addressed in the seminal 2015 paper by Nilsen describing an initial taxonomy of theories, models, and frameworks in implementation science5. We note that this paper is now over a decade old. Proposing additional taxonomic categories of frameworks, models, and theories is an important step yet to be taken.
This background informs this collection proposal. We are calling for manuscripts to address the issues, which may include methods (what methods can be used to update or extend existing frameworks), perspective or commentary manuscripts (why is this important), and empirical papers offering new insights, updates, and extensions of existing frameworks. We would also welcome papers that explicitly focus on theorizing based on existing frameworks, focusing on prediction and explanation rather than description14. However, manuscripts proposing new frameworks will be considered only if they clearly demonstrate how the proposed contribution builds on, revises, synthesizes, tests, or challenges existing frameworks, and why a new framework or a substantial extension is necessary. The existing body of frameworks and models within implementation science and practice constitutes an important catalog of knowledge. Our goal is to build on that existing knowledge.
Examples of topics include, but are not limited to:
- Innovative papers that develop new substantive theories or significant theoretical extensions to existing theories
- Methods for classifying and categorizing existing frameworks
- Proposing new domains and constructs for existing determinant frameworks
- Synthesizing across existing process frameworks to describe common elements and areas of departure
- Practical guidance on how to use existing tools such as the “Assess the Dissemination and Implementation Models Webtool” or useful new tools and approaches to help people select and use existing frameworks (these are likely to be assessed for Implementation Science Communications rather than Implementation Science)
This Collection welcomes submissions of a range of article types. Should you wish to submit to this Collection, please read the submission guidelines of the journal you are submitting to, i.e., Implementation Science or Implementation Science Communications, to confirm that the type is accepted by the journal you are submitting to.
Articles for this Collection should be submitted via our submission systems in Implementation Science or Implementation Science Communications. During the submission process, you will be asked whether you are submitting to a Collection. Please select "Breaking Frameworks: Revisiting, Extending, Integrating, and Theorizing Implementation Frameworks" from the dropdown menu.
Articles will undergo the standard peer review process of the journal in which they are considered, Implementation Science or Implementation Science Communications, and are subject to all of the journal’s standard policies. Articles will be added to the Collection as they are published.
The Editors have no competing interests with the submissions that they handle through the peer-review process. The peer review of any submissions for which the Editors have competing interests is handled by another Editorial Board Member who has no competing interests.
Publishing Model: Open Access
Deadline: Mar 09, 2027
Learning From the Past and Shaping the Future
Implementation Science has published 2,197 papers since its inception in 2006 (as of May 27, 2026). Implementation Science Communications has published 938 papers since it began in 2019 (also as of May 27, 2026). In addition to papers published in these two journals, a larger number of papers focusing on implementation science in health care have been published in other journals over the last 20 years, conservatively around 22,300. This is based on a PubMed search using a specific search string[1], conducted on May 27, 2026. It is important to note that landmark papers in the field were published before 2006. Our purpose in marking this anniversary is to reflect on the field as a whole.
While much of the growth in the literature has come from high-income countries, there has been an increase in the number and scope of papers from lower and middle-income countries, fueling the overall growth.
Arguably, the growth in the literature and underlying research studies shows that the science and practice of implementation have moved from the periphery to mainstream health research. We are interested in papers that document and analyze the change over the last couple of decades—although the history of the field prior to 2006 is also of interest—and propose how this shapes the future of the field. This may be based on bibliographic or citation analysis, surveys among researchers, or other sources. Papers that only review the past, without analysis and future direction, will not be seen as responding to this call.
Examples of topics include, but are not limited to:
- Content analysis or systematic reviews of empirical publications from Implementation Science and/or Implementation Science Communications
- Content analysis of editorials and research agenda-setting articles from both journals, including papers focusing on implementation science published in other journals
- Bibliographic/citation analysis of publications over the 20 years of IS, including other papers published in other journals
- Analyses of geographic, disciplinary, authorship, funding, or institutional patterns in implementation science
Submissions should include critical interpretive analysis of existing literature and provide new insights, ideas, and thoughts from reflection on the existing literature.
This Collection welcomes submissions of a range of article types. Should you wish to submit to this Collection, please read the submission guidelines of the journal you are submitting to, i.e., Implementation Science or Implementation Science Communications, to confirm that the type is accepted by the journal you are submitting to.
Articles for this Collection should be submitted via our submission systems in Implementation Science or Implementation Science Communications. During the submission process, you will be asked whether you are submitting to a Collection. Please select "Learning From the Past and Shaping the Future" from the dropdown menu.
Articles will undergo the standard peer-review process of the journal in which they are considered, Implementation Science or Implementation Science Communications, and are subject to all of the journal’s standard policies. Articles will be added to the Collection as they are published.
The Editors have no competing interests with the submissions that they handle through the peer-review process. The peer review of any submissions for which the Editors have competing interests is handled by another Editorial Board Member who has no competing interests.
[1] ("Implementation Science"[Mesh] OR "implementation science"[tiab] OR "implementation research"[tiab] OR "dissemination and implementation"[tiab] OR "translation science"[tiab] OR "knowledge translation"[tiab]) AND 2006:2026[dp]
Publishing Model: Open Access
Deadline: Mar 09, 2027
Please sign in or register for FREE
If you are a registered user on Research Communities by Springer Nature, please sign in