A literature review is an essential component of almost any research project. It serves as the foundation for advancing knowledge, facilitates theory development, closes mature research areas, and uncovers novel research areas Webster and Watson Because literature reviews are so prevalent, there exist already several comprehensive resources that guide authors through the steps necessary to conduct a literature review e.
Surprisingly, there is a still considerable variance in the understanding of what a literature review is and, consequently, in the quality of systematic literature reviews. Often, researchers seem unfamiliar with the process, structure, and presentation of systematic literature reviews and produce merely descriptive, annotated bibliographies of loosely connected research, which makes it unnecessary complex and difficult for the readers to follow the literature review.
The literature review therefore does not achieve its main goal of summarizing and categorizing knowledge. There is also the misconception that literature reviews are less rigorous or easier to write than empirical articles. However, conducting a literature review of high quality requires an in-depth understanding of the necessary processes and skills and is by no means a trivial endeavor.
It also requires some experience in the respective field, as the interpretation of the results of the studies included in the literature review is subjective and by no means trivial. Motivate the topic and state the research question The abstract and introduction are crucial elements of any research article. Additionally, a literature review needs a crisp and concise motivation. It is important to not only motivate why a topic warrants investigation but also why the authors choose to approach the topic in the form of a systematic literature review.
Perhaps the most important element of an introduction is the research question that guides the remainder of the literature review. Therefore, we encourage authors to carefully develop and clearly state their research question s in the introductory section.
Identify the relevant literature in a systematic way A distinguishing feature of a systematic literature review is that the review process should be transparent and reproducible. The authors need to clearly outline their search strategy for identifying relevant literature in a systematic way to establish as much transparency as possible.
This involves a description of the databases where the literature search was conducted, a definition of the search terms and keywords used to identify literature, and a careful description of the practical e. Notice that the application of screening criteria e.
Choose the right balance between breadth and depth When conducting a systematic literature review, authors often face the dilemma of choosing between breadth and depth when identifying and describing prior studies.
In general, a good systematic literature review is characterized by the right balance between breadth and depth by including all relevant studies but only describing important studies in more detail in a structured way. This dilemma is often difficult to solve, as a literature review should be coherent and cover a research field as a whole, but it should not be an endless, overly descriptive summary of all studies that the authors identified.
To solve this dilemma to some extent, authors should make use of tables and figures to convey the most important concepts and information in an efficient fashion. For example, figures can be used to illustrate the development of the number of studies over time and can also illustrate which topics have attracted the most research. Of course, tables and figures should be used in a sensible fashion and should never present the main focus of the literature review.
The breadth and depth of a literature review also depend on the maturity of the research field. A literature review on a mature topic requires that the authors analyze and synthesize a large body of literature, in comparison to a review on a more novel field where only few studies exist.
The authors of this editorial were once challenged to summarize the literature on the intersection of entrepreneurship and innovation. At first, this seemed to be an impossible task. However, after having developed a more concise research question see suggestion 1 , developing clear inclusion and exclusion criteria see suggestion 2 , focusing on breadth instead of depth see suggestion 3 and on concepts rather than studies see suggestion 4 , the task became doable.
The whole process took many iterations and was very work-intensive, but we managed to deliver a literature review at the intersection of innovation and entrepreneurship or vice versa see Block et al. Focus on concepts, not studies Authors need to decide how to summarize and categorize the literature identified. While it is possible to summarize literature in a chronological or even alphabetical order, we believe that literature reviews should be concept-centric.
This involves a careful identification and evaluation of the underlying concepts used in the review, which then guide the analysis conducted. Focusing on concepts instead of studies helps authors to identify the research debates they aim to contribute to and helps to ensure a better structure throughout the manuscript. Hence, a systematic literature review needs to be based on sound logical and conceptual reasoning.
This can but need not lead to a new conceptual framework with propositions. In this sense, writing a systematic literature review very much resembles the writing of a conceptual theory paper. Derive meaningful conclusions Closely connected to the previous point, we want to reiterate that a systematic literature review must go beyond a mere descriptive summary of prior literature.
While it is important to provide a descriptive overview on the topics and studies included, it is essential to go one step further and to synthesize and interpret this knowledge. The literature review should derive meaningful conclusions and needs to answer the question: What do we learn from this summary?
This includes carefully evaluating and deriving implications, pointing out gaps in the literature, and outlining avenues for future research. The librarian's roles in the systematic review process: a case study. Journal of the Medical Library Association. Egger JB. Use of recommended search strategies in systematic reviews and the impact of librarian involvement: a cross-sectional survey of recent authors.
PLoS One. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol. PubMed Article Google Scholar. McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews.
Weller AC. Mounting evidence that librarians are essential for comprehensive literature searches for meta-analyses and Cochrane reports. Two physiotherapists, one librarian and a systematic literature review: collaboration in action. Health Info Libr J. Foster M. An overview of the role of librarians in systematic reviews: from expert search to project manager.
Database selection in systematic reviews: an insight through clinical neurology. A competency framework for librarians involved in systematic reviews. New activities and changing roles of health sciences librarians: a systematic review, Emerging roles for biomedical librarians: a survey of current practice, challenges, and changes. The systematic review team: contributions of the health sciences librarian. Med Ref Serv Q. Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects.
Reproducibility of literature search reporting in medical education reviews. Academic medicine : journal of the Association of American Medical Colleges. Impact of librarians on reporting of the literature searching component of pediatric systematic reviews. Scoping reviews: establishing the role of the librarian. Reproducibility of search strategies is poor in systematic reviews published in high-impact pediatrics, cardiology and surgery journals: a cross-sectional study. Fehrmann P, Thomas J.
Comprehensive computer searches and reporting in systematic reviews. Research Synthesis Methods. Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Systematic Reviews. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health technology assessment Winchester, England. Few systematic reviews exist documenting the extent of bias: a systematic review.
How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Literature searching for social science systematic reviews: consideration of a range of search techniques. Petticrew M. Time to rethink the systematic review catechism? Effectiveness of different databases in identifying studies for systematic reviews: experience from the WHO systematic review of maternal morbidity and mortality.
Felson DT. Bias in meta-analytic research. Publication bias in the social sciences: unlocking the file drawer. Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews.
Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. Language bias in randomised controlled trials published in English and German. Lancet London, England. The inclusion of reports of randomised trials published in languages other than English in systematic reviews.
Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary. The effects of excluding treatments from network meta-analyses: survey. The contribution of databases to the results of systematic reviews: a cross-sectional study. Searching for unpublished trials in Cochrane reviews may not be worth the effort.
Evidence mapping for decision making: feasibility versus accuracy - when to abandon high sensitivity in electronic searches. German medical science : GMS e-journal. The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Gough D. Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Res Pap Educ. The challenges of searching for and retrieving qualitative studies.
West J Nurs Res. Asking more of qualitative synthesis: a response to Sally Thorne. Qual Health Res. Booth A, Carroll C. Systematic searching for theory to inform systematic reviews: is it feasible? Is it desirable? An assessment of the efficacy of searching in biomedical databases beyond MEDLINE in identifying studies for a systematic review on ward closures as an infection control intervention to control outbreaks.
Syst Rev. Assessing the validity of abbreviated literature searches for rapid reviews: protocol of a non-inferiority and meta-epidemiologic study. Trading certainty for speed - how much uncertainty are decisionmakers and guideline developers willing to accept when using rapid reviews: an international survey.
Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health. Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches.
Effectiveness of search strategies for qualitative research about barriers and facilitators of program delivery. Eval Health Prof. Abbreviated literature searches were viable alternatives to comprehensive searches: a meta-epidemiological study. Article PubMed Google Scholar. Res Synth Methods. Text mining for search term development in systematic reviewing: A discussion of some methods and challenges.
Search terms and a validated brief search filter to retrieve publications on health-related values in Medline: a word frequency analysis study. PubMed Google Scholar. Andrew B. Clear and present questions: formulating questions for evidence based practice. Library Hi Tech. Clinical effectiveness and cost-effectiveness of tests for the diagnosis and investigation of urinary tract infection in children: a systematic review and economic model.
A population search filter for hard-to-reach populations increased search efficiency for a systematic review. Routine development of objectively derived search strategies. Prospective comparison of search strategies for systematic reviews: an objective approach yielded higher sensitivity than a conceptual one. Craven J, Levay P. Recording database searches for systematic reviews - what is the value of adding a narrative to peer-review checklists?
A case study of nice interventional procedures guidance. Evid Based Libr Inf Pract. A mapping review of the literature on UK-focused health and social care databases. Younger P, Boddy K. When is a search not a search? Increasing number of databases searched in systematic reviews and meta-analyses between and Bethel A, editor Search summary tables for systematic reviews: results and findings. HLC Conference a. Adams CE, Frederick K. Psychol Med.
So many databases, such little clarity: searching the literature for the topic aboriginal. Canadian family physician Medecin de famille canadien. Lawrence DW. What is lost when searching only one literature database for articles relevant to injury prevention and safety promotion?
Injury Prevention. Searching one or two databases was insufficient for meta-analysis of observational studies. Should meta-analysts search Embase in addition to Medline? Stevinson C, Lawlor DA. Searching multiple databases for systematic reviews: added value or diminishing returns?
Complementary Therapies in Medicine. Control Clin Trials. Systematically retrieving research: a case study evaluating seven databases. Res Soc Work Pract. Beyer FR, Wright K. Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Supplementary search methods were more effective and offered better value than bibliographic database searching: a case study from public health and environmental enhancement in Press.
Cooper C, Booth, A. A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: A methodological review. In Press. BMC Systematic Reviews. Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ Clinical research ed. Hinde S, Spackman E.
Bidirectional citation searching to completion: an exploration of literature searching methods. Identifying evidence for public health guidance: a comparison of citation searching with web of science and Google scholar. Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews. Efficiency and contribution of strategies for finding randomized controlled trials: a case study from a systematic review on therapeutic interventions of chronic depression.
Journal of public health research. Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care. Health Libr Rev. Bethel A. De-duplication of database search results for systematic reviews in EndNote.
Reviewing retrieved references for inclusion in systematic reviews using EndNote. Gall C, Brahmi FA. Medical reference services quarterly. Zotero: a bibliographic assistant to researcher. J Pharmacol Pharmacother. Zotero: harnessing the power of a personal bibliographic manager. Nurse Educ. PLoS Med. No consensus exists on search reporting methods for systematic reviews. Toews LC. Compliance of systematic reviews in veterinary journals with preferred reporting items for systematic reviews and meta-analysis PRISMA literature search reporting guidelines.
Quality of search strategies reported in systematic reviews published in stereotactic radiosurgery. Br J Radiol. Reporting quality of search methods in systematic reviews of HIV behavioral interventions — : are the searches clearly explained, systematic and reproducible?
Analysis of the reporting of search strategies in Cochrane systematic reviews. A comparison of quality of abstracts of systematic reviews including meta-analysis of randomized controlled trials in high-impact general medicine journals before and after the publication of PRISMA extension for abstracts: a systematic review and meta-analysis. Compliance of systematic reviews articles in brain arteriovenous malformation with PRISMA statement guidelines: review of literature. Journal of clinical neuroscience : official journal of the Neurosurgical Society of Australasia.
Goodwin N. Poor methodological quality and reporting standards of systematic reviews in burn care management. International wound journal: Faggion CM; Endorsement of PRISMA statement and quality of systematic reviews and meta-analyses published in nursing journals: a cross-sectional study. BMJ Open. Methods for documenting systematic review searches: a discussion of common issues.
Reporting standards for literature searches and report inclusion criteria: making research syntheses more transparent and easy to replicate. PRESS peer review of electronic search strategies: guideline statement. An evidence-based practice guideline for the peer review of electronic search strategies. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both.
ROBIS: a new tool to assess risk of bias in systematic reviews was developed. Relevo R, Balshem H. Finding evidence for comparing medical interventions: AHRQ and the effective health care program. Medicine Io. Download references. The open access fee for this publication was paid for by Exeter Medical School. You can also search for this author in PubMed Google Scholar. CC conceived the idea for this study and wrote the first draft of the manuscript.
All authors revised the manuscript prior to submission. All authors read and approved the final manuscript. Correspondence to Chris Cooper. Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Appendix tables and PubMed search strategy. Key studies used for pearl growing per key stage, working data extraction tables and the PubMed search strategy. DOCX 30 kb. Reprints and Permissions.
Cooper, C. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. Download citation. Received : 20 September Accepted : 06 August Published : 14 August Skip to main content. Search all BMC articles Search. Download PDF. Abstract Background Systematic literature searching is recognised as a critical component of the systematic review process.
Method A literature review. Results Eight key stages were determined relating specifically to literature searching in systematic reviews. Conclusions Eight key stages to the process of literature searching in systematic reviews were identified. Background Systematic literature searching is recognised as a critical component of the systematic review process. Table 1 Guidance documents audited for this literature review Full size table.
The key stages of literature search guidance as identified from nine key texts. Full size image. Results and discussion Our findings We were able to identify consensus across the guidance on literature searching for systematic reviews suggesting a shared implicit model within the information retrieval community.
Unique guidance Only one document provided guidance on the topic of when to stop searching [ 2 ]. Agreement between guidance: Defining the key stages of literature searching Where there was agreement on the process, we determined that this constituted a key stage in the process of literature searching to inform systematic reviews. Table 2 The order of literature search methods as presented in the guidance documents Full size table.
Limitations Other handbooks exist A potential limitation of this literature review is the focus on guidance produced in Europe the UK specifically and Australia. The handbooks are potentially linked to one another What is not clear is the extent to which the guidance documents inter-relate or provide guidance uniquely.
This is a literature review A further limitation of this review was that the review of published studies is not a systematic review of the evidence for each key stage. Conclusions This literature review would appear to demonstrate the existence of a shared model of the literature searching process in systematic reviews.
References 1. Article Google Scholar 2. Book Google Scholar 3. Google Scholar 6. Google Scholar
A short while ago, David posted this comment on How I use Scrivener for academic writing :. I promised I would write a post about it. So here it is. But before we get down to the mechanics of Excel…. The methodology for my Professional Doctorate is autoethnography. Initially, a systematic review of the literature appealed to me. I like structure. At the same time as conducting my systematic review, I was reading a minimum of one article on autoethnography per day.
I was following bibliographies. Each article I read led me to another, then another, then another. And I made my choices based on the knowledge that I wanted to accumulate. For example, I wanted to specifically look at autoethnographic articles which were located in the educational world. Articles about disability and discrimination were very powerful, but they were not within the scope of my review.
Rather than follow a linear process, an abductive approach allows for the type of intuition I was using when finding the next article. I suspect that I am drawn to an abductive approach because its focus is on interpretation and understanding new perspectives. These are phrases that are strongly associated with autoethnography. It makes sense to me that an autoethnographer would approach the literature in this way.
One step at a time. One article or book at a time. I wanted to create a table that I could manage effectively. Where I could move the data with ease. My spreadsheet is basic. But its simplicity works for me. Originally, I used headings from lit reviews that I had seen elsewhere. There were some columns I never filled in.
And I found it hard to remember what some headings were for. So, over time, I made my own, to suit the purpose of my review and the topic I am reviewing in this case autoethnography. The photo above only shows the columns I can see when I open the spreadsheet. In unhappy thoughts, I might note if the text is fairly hard going and perhaps needs a few reads, or any contradictions I might have picked up.
I wanted to capture if the author of the work raised any ethical issues, and if I had any. This is furthest away as it is not my primary concern at the moment. I regularly move my columns around. This is incredibly useful if, like me, you add rows in all over the place. I know I can insert a new row anywhere I like and with the touch of a button Excel will put it in alphabetical order. Same goes for any other column.
Definitely have to try using Excel. Thanks for sharing your experience! This is really helpful! Also, like this method of managing literature review because I can use filters in Excel. Like Like. This is really helpful — thanks for sharing your process. I was wondering if you could comment on the benefits of using spreadsheets for this over a reference manager like Zotero, which enables notes and annotations etc. I use my lit review spreadsheet and a reference manager I use EndNote.
When I read a new article, I capture the information on my spreadsheet first. Then I go to EndNote and add the article there too. I use Minor Spreadsheets in two different ways. These are articles that could be related to other work I want to do. I add to these sheets on an ongoing basis to save me time later. The second way I use Minor Spreadsheets is when I start a new paper.
I pull articles from my Major Spreadsheet and throw them in a new one. Now that I have an existing foundation for the literature, I can go to Google Scholar to build on what I already have instead of starting from scratch. These Minor Spreadsheets are typically much more focused than my Major Spreadsheet. For example, in the Major Spreadsheet I use the x to identify articles under one overarching disability theme and in the Minor Spreadsheet I take all these and look more closely at type of disability, level of education, and accommodations.
Now, lots of folks would use the spreadsheet approach and then store their articles with annotations in another program. I then have a folder for my Major Spreadsheet and all its articles. For my Minor Spreadsheets, I typically start by copying and pasting articles from the Major Spreadsheet and the folder of articles.
This is why you end up with folders for Minor Spreadsheets where the numbers are all over the place, which is okay. A final note: I number dissertations differently than other pieces. I started numbering those at and have gone up from there.
I wanted to differentiate dissertations in some way so that I could easily find them in my folders usually because I look at dissertations to see how others have done certain things. You could also differentiate other pieces in your folder like books, by starting at , for example. Again, you can customize all of this to what works for you. Do you think the spreadsheet approach would work for you?
What other methods do you use to organize your literature review work? Expand comments Hide comments. View the discussion thread. We have retired comments and introduced Letters to the Editor. Share your thoughts ». Advertise About Contact Subscribe. Coronavirus Live Updates - July 16, Strategies for using nested spreadsheets to organize your lit review. By Kathleen Clarke. October 24, Here are two pointers for your Major Spreadsheet: First, start early and add often. The Number System Now, lots of folks would use the spreadsheet approach and then store their articles with annotations in another program.
Read more by Kathleen Clarke. Inside Higher Ed Careers Hiring?