What steps are involved when conducting a systematic literature review? After explaining what they mean by a systematic literature review, our Evidence Synthesis Team here outline the 7 stages of developing an HTA-ready, gold-standard systematic literature review.
What is a systematic literature review?
A systematic literature review (SLR) (often shortened to systematic review [SR]) is essentially a thorough analysis of all relevant research on a specific topic. True systematic literature reviews must be comprehensive, transparent, and reproducible.
A systematic literature review identifies, appraises, and summarises literature to answer a prespecified question through a clearly defined protocol. Reviewers systematically screen and analyse published evidence from multiple sources to identify relevant studies and extract data.
How are systematic reviews used in healthcare?
The health and life sciences industry use systematic reviews to evaluate the efficacy and safety of new medicines, diagnostics, and medical devices. Systematic reviews provide a comprehensive synthesis of existing research, helping stakeholders to make informed decisions about market access and regulatory approval.
Systematic reviews offer reliable evidence-based information to support informed choices and practices. By aggregating data from multiple studies, systematic reviews offer a robust foundation for assessing the effectiveness and cost effectiveness of treatments.
With systematic literature reviews seen as the gold standard for evidence-based medicine, decision-makers, from regulatory agencies to payers and health insurers, expect to see a systematic review in your evidence dossier.
Systematic reviews are often published in peer-reviewed academic journals. These publications serve as authoritative references for payers, policy-makers, and other stakeholders involved in healthcare decision-making. Publishing evidence is often a key step in the market access journey.
The 7 steps of a systematic literature review
There are several published guidelines, such as those issued by the Cochrane Collaboration®[1] and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA statements) [2], which provide guidance on conducting and reporting a robust systematic literature review.
The process of conducing a gold-standard systematic literature review typically involves 7 stages:
- Protocol and registration
- Electronic database search
- Title and abstract screening
- Full publication review
- Hand searching
- Data extraction and quality assessment
- Report
Step 1: Protocol and registration
The first step in a systematic literature review is to write the protocol. The protocol must clearly define the study question and describe the methodology to be used.
The reviewer should begin the protocol by describing the background, aim, and purpose of the systematic literature review. They should also clarify the study question.
The protocol should then define the eligibility criteria used to assess each publication. This is a list of inclusion and exclusion criteria that indicates whether a publication is eligible for inclusion in the systematic literature review. The eligibility criteria specify the Population, Intervention, Comparator, Outcomes, and Study design (PICOS). These criteria can also specify certain factors, such as the publication date, language, and location of the included studies.
The protocol should also state the methodology intended for each stage of the review, as this ensures that the systematic literature review is robust and reproducible, and conforms to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) checklist [3].
After writing the protocol, reviewers should register the systematic literature review with the International Prospective Register of Systematic Reviews (PROSPERO) to prevent duplicate reviews and facilitate potential future publication.
Step 2: Electronic database search
The reviewer conducts an electronic database search to identify all potentially relevant publications that should be included in the review. Each search should be specifically designed to reflect the PICOS. The search parameters are specifically designed to capture all relevant literature, with a balance of sensitivity and recall, so that large numbers of irrelevant records are not retrieved.
The search should interrogate all relevant databases, such as PubMed®, Embase®, MEDLINE®, and the Cochrane Library.
All search sources, methods, and strategies should be included in the report to ensure that the search is transparent and replicable.
Step 3: Title and abstract screening
Step 2 should provide the reviewer with a list of published research that may be relevant to their study question. As this can be quite a long list, the first step is to screen the results using the title and abstract.
Reviewers screen the title and abstract of identified citations to determine if they meet the eligibility criteria outlined in the protocol. Publications may be excluded based on various factors, including the following:
- Disease is not relevant to the study question
- Intervention is not relevant to the study question
- Duplicate publication to another in the search
- Study design is not relevant in this case
- Animal/in vitro study, if study question is on human health (or vice versa)
- The publication reports the protocol only, with no results
- Linked to another publication in the list (such as a conference abstract that has been superseded by a full journal article)
At this stage, citations should not be excluded solely because they do not report a relevant outcome in the abstract, as the full publication may contain additional outcomes not mentioned in the abstract. If it is at all unclear as to whether a publication meets any of these criteria, then it is typically included. For example, it may not be easy to identify all relevant outcomes or subgroups from an abstract, so the publication will be included at this stage.
To prevent reviewers from overlooking any relevant citations, two separate reviewers should independently screen the title and abstract.
Step 4: Full publication review
In Stage 4, reviewers proceed to assess any studies identified as potentially relevant during the title and abstract screening through a full publication review. This involves a complete review of each publication to determine if it meets each of the eligibility criterion, with a particular focus applied to the Methods and Results of each publication. At this stage, reviewers can exclude studies that do not have relevant outcomes. Similarly to title and abstract screening, this should be conducted by two independent reviewers.
During this stage, it maybe be helpful for reviewers to capture certain study outputs, such as study design, number of participants, trial location, and reported outcomes. Mapping these outputs helps to identify publications of the same study and prioritise publications for data extraction. It is particularly useful for systematic literature reviews that are focused on real-world evidence (RWE), as this can identify a large number of relevant publications.
Step 5: Hand searching (and reference checking)
In addition to the database search, reviewers should conduct ad hoc searching of supplementary sources. To do this, the reviewer should search conference abstracts from relevant conference proceedings from the past 3 years using key terms.
It is also important to search through references of the eligible studies already identified. References may contain other relevant studies that should be included in the new systematic literature review.
When using the systematic literature review to support a reimbursement submission, it's beneficial to explore various global health technology assessment (HTA) websites for relevant submissions. This can yield valuable insights and potentially unique outcomes data.
Step 6: Data extraction and quality assessment
The reviewer then extracts data from the relevant identified publications into a data extraction table (DET). The DET captures details of each clinical trial, most commonly:
- The study design of each clinical trial, such as the number of participants enrolled, the eligibility criteria of the study, and information about treatment
- Baseline characteristics of enrolled participants, such as age, sex, disease duration, and disease-specific characteristics
- Outcomes of interest
- Risk of bias
If the DET will be used in a subsequent meta-analysis, it is crucial to design it in a way that supports this.
In some systematic literature reviews, it may be appropriate to extract data from the publication directly into the final report, rather than into a DET. This may be appropriate in a smaller systematic literature review, or for ones with limited outcomes of interest.
Simultaneously, reviewers should conduct a quality assessment of each study to evaluate potential sources of bias. Various validated tools and checklists are available for appraising and assessing the risk of bias in studies. The most appropriate assessment will depend on your study design.
Step 7: Report
The final stage of a systematic literature review is writing the report. The aim of the report is to answer the study question defined in the protocol. A systematic literature review report will typically include the following sections:
- Executive Summary
- Introduction and background
- Aims and objectives
- Methodology
- Result tables and summary of findings
- Discussion
- Conclusion
The report should follow the guidance provided by PRISMA, including the PRISMA diagram. This diagram details the number of citations identified during the database searches and handsearching, and the number of citations included and excluded at each stage of the review, as well as the reasons for exclusion.
The methodology outlined in the report should align with the methodology described in the protocol.
The discussion should describe how the results have answered the study question. It should also address the robustness of the evidence base, outline any data gaps, and recommend areas for future research.
The final report will then be ready for inclusion in your evidence dossier. It can be used in global value dossiers and reimbursement dossiers for HTA submissions. Most reports are also submitted to reputable journals for publication.
Conducting a systematic literature review
Do you have a question about the systematic literature review process? Are you looking for an expert to develop an systematic literature review to support your market access and reimbursement strategy? To speak to our Evidence Synthesis Team, email info@mtechaccess.co.uk.
