Are you new to the world of evidence synthesis for market access? Are you unsure about some of the terminology used by specialists who are preparing your evidence dossier? Here, our experts answer some of the most frequently asked questions (FAQs) about systematic literature reviews and network meta-analyses.
4 FAQs about systematic literature reviews for Pharma and Medtech market access
What is the difference between a systematic literature review and a meta-analysis?
A systematic literature review seeks to gather and synthesise all relevant data, whether clinical or non-clinical, pertaining to a research question. This can be achieved through either narrative synthesis or a meta-analysis. A meta-analysis is a statistical method used to combine clinical outcome data from various sources, such as clinical trials, into a single analysis to compare interventions. Researchers often conduct meta-analyses after a systematic review has identified all relevant clinical data.
What is the difference between a targeted literature review and a systematic literature review?
A systematic literature review follows robust pre-specified methodology. It is good practice to pre-register protocols for a systematic review on a public database. These protocols will clearly outline the research question(s) to address, along with systematic literature review methods, search strategy, and any pre-planned analyses. This helps ensure transparency of the systematic literature review processes. Researchers should register protocols before extracting any data.
A targeted literature review focuses its scope and employs restricted methods compared with systematic literature reviews. A targeted review may only look at a single database and may have a restricted search strategy to reduce the number of results for a more focused analysis. In addition, single reviewers may be employed. As such, a targeted review will not be as robust, as there is an increased likelihood of missing studies of relevance or introducing bias.
Are systematic literature reviews qualitative or quantitative?
Systematic literature reviews can be qualitative, quantitative, or a bit of both. This will depend on your research question(s) and the evidence/data available. The best approach depends on what the reviewers discover while designing the review protocol and scoping the literature initially. Reviewers should clearly document the planned evidence synthesis in the review protocol and note any changes during the review.
How long should a systematic literature review take?
This depends on the research question(s) posed and the outputs required. As an example, take a systematic literature review that has 3,000 search results, with 200 results that requiring full-text screening. Of these, 35 are included for data extraction. We would expect a systematic literature review of this size to take between 12–18 weeks to complete from protocol development through to reporting.
11 FAQs about meta-analyses for Pharma and Medtech reimbursement
What is meta-analysis in healthcare?
A meta-analysis is a statistical method of ‘pooling’ data from multiple studies that have addressed a similar research question to provide a combined effect estimate for a clinical outcome(s) (e.g. complete response) from multiple data sources. Hence providing more power and confidence in the observed effect. Meta-analyses in healthcare combine clinical evidence from studies to ensure comprehensive assessment of treatment effectiveness across multiple interventions, most notably for health technology assessment (HTA).
Meta-analyses can include data from various types of studies (e.g. randomised controlled trials [RCTs], observational studies, single-arm studies). The study design determines the type of analysis required.
What is the difference between conventional meta-analysis and network meta-analysis?
A conventional meta-analysis compares two trial arms (e.g. drug versus placebo) from multiple studies in a pairwise manner. It aims to determine which treatment is most effective. In contrast, a network meta-analysis can compare multiple trial arms, even if they have not been directly compared in a head-to-head trial. Its goal is to determine the most effective treatment option.
When would you use a meta-analysis?
Researchers utilise meta-analysis when they plan to include multiple studies addressing the same research question. These studies typically involve similar populations, interventions, and outcomes. Homogeneity or similarity of the trial characteristics should be proven prior to including within a meta-analysis. Meta-analysis helps to synthesise and analyse data from these studies to draw robust conclusions.
Can you conduct a systematic literature review without a meta-analysis?
Yes, a meta-analysis is not always necessary. After conducting a systematic literature review, researchers synthesise all included data, whether that be statistically or narratively. The data can be used to support evidence generation requirements in various ways, such as incorporating these into HTA submissions.
If good head-to-head clinical trial data are available, it may not be necessary to conduct a network meta-analysis for the purpose of HTA. However, if other studies are available that consider interventions of interest, a network meta-analysis would be favourable to ensure all relevant data on the interventions of interest are captured. An economic model submitted to an HTA agency can then explore scenarios including all available data or just the data from the key clinical trial.
In some cases, data may not be homogenous meaning a robust meta-analysis, or network meta-analysis, is not possible.
What is the major advantage of a meta-analysis?
Meta-analyses provide greater statistical power by combining several studies into what is effectively one larger study. As a result, we can be more confident in the data. Some meta-analyses, such as indirect treatment comparisons (ITCs), enable comparisons between interventions where no direct comparison exists, for example, via an RCT.
What do you write/include in a pharmaceutical meta-analysis?
When conducting a meta-analysis of clinical trials, the aim is to identify how effective one treatment is in comparison with another. Statistical analyses are conducted on the data to estimate treatment effect. In general, the results you would expect to see include:
- Effect estimate: overall pooled estimate based on multiple studies included in the meta-analysis showing whether studies generally favour an intervention or comparator. Can be derived in many forms including, but not limited to, odds ratio, risk ratio, rate ratio mean difference, standardised mean difference
- Confidence intervals (e.g. 95%): highlights the variation associated with the effect estimate derived from multiple studies to see how much confidence should be applied to the observed effect
- p-value for effect estimate: indicates the statistical significance (i.e. p<0.05=reject null hypothesis; p≥0.05=failure to reject the null hypothesis) associated with the pooled effect estimate
- Heterogeneity (e.g. I2): provides insight on whether studies included in the pooled analyses differ in terms of the direction of effect observed (i.e. one study significantly favours the intervention, whereas another significantly favours the comparator)
What are the assumptions for meta-analyses and network meta-analyses?
For meta-analyses, homogeneity is a key assumption in being able to draw robust conclusions from pairwise comparisons. In the context of network meta-analysis, homogeneity is a key assumption to check there is no imbalance in the distribution of effect modifiers across the different types of direct treatment comparisons. Another important assumption to take into account for a network meta-analysis is coherence.
How many studies are needed for a meta-analysis?
Performing a meta-analysis requires a minimum of two studies. Yet, if the studies contain small sample sizes with wide confidence intervals, experts will warn against performing a meta-analysis with just two studies. This is mainly because of the imprecision observed in the effect estimate, which makes the analysis redundant for making clinical decisions about the statistically compared drugs. Many will often report two studies qualitatively and only perform a meta-analysis if they have at least three studies included.
Can a meta-analysis be qualitative?
Yes, albeit these are rare and often used in the discipline of psychology to assess key themes developing from qualitative data. These are often referred to as meta-syntheses.
What is an ITC?
An indirect treatment comparison involves conducting a meta-analysis to compare two treatments that have not been directly compared in head-to-head trials (e.g. through an RCT). Indirect treatment comparisons are conducted when a common comparator exists. Two trials may have different treatments but the same comparators, allowing that common comparator to act as an anchor. For example, where direct evidence is available for treatment A versus treatment B and for treatment A versus treatment C, the ITC allows a comparison of treatment B versus treatment C.
What are the assumptions for indirect treatment comparisons?
Homogeneity (i.e. similarity) across included trials is a key assumption for indirect treatment comparisons. In addition to homogeneity, transitivity (i.e. the ‘anchor’ comparator is similar across trials) and consistency (i.e. direct and indirect evidence are in agreement) are also important assumptions when performing indirect treatment comparisons. If these assumptions are not met, the results of the indirect treatment comparison are likely to be misleading. In the case of transitivity, if the ‘anchor’ comparator is not similar between trials, then you are introducing indirectness into the direct comparison, which is used as the platform to calculate your indirect comparison.
Do you have a different question for our experts? Send your question to info@mtechaccess.co.uk.
Our experts are here to support all your evidence synthesis, generation, and communication requirements, including all aspects of systematic literature review and network meta-analysis. We also offer education and training on key principles of evidence synthesis. To learn more, email info@mtechaccess.co.uk.
