4 FAQs about systematic literature reviews for Pharma and Medtech market access

What is the difference between a systematic literature review and a meta-analysis?

A systematic literature review seeks to gather and synthesise all relevant data, whether clinical or non-clinical, pertaining to a research question. This can be achieved through either narrative synthesis or a meta-analysis. A meta-analysis is a statistical method used to combine clinical outcome data from various sources, such as clinical trials, into a single analysis to compare interventions. Researchers often conduct meta-analyses after a systematic review has identified all relevant clinical data.

What is the difference between a targeted literature review and a systematic literature review?

A systematic literature review follows robust pre-specified methodology. It is good practice to pre-register protocols for a systematic review on a public database. These protocols will clearly outline the research question(s) to address, along with systematic literature review methods, search strategy, and any pre-planned analyses. This helps ensure transparency of the systematic literature review processes. Researchers should register protocols before extracting any data.

A targeted literature review focuses its scope and employs restricted methods compared with systematic literature reviews. A targeted review may only look at a single database and may have a restricted search strategy to reduce the number of results for a more focused analysis. In addition, single reviewers may be employed. As such, a targeted review will not be as robust, as there is an increased likelihood of missing studies of relevance or introducing bias.

Are systematic literature reviews qualitative or quantitative?

Systematic literature reviews can be qualitative, quantitative, or a bit of both. This will depend on your research question(s) and the evidence/data available. The best approach depends on what the reviewers discover while designing the review protocol and scoping the literature initially. Reviewers should clearly document the planned evidence synthesis in the review protocol and note any changes during the review.

How long should a systematic literature review take?

This depends on the research question(s) posed and the outputs required. As an example, take a systematic literature review that has 3,000 search results, with 200 results that requiring full-text screening. Of these, 35 are included for data extraction. We would expect a systematic literature review of this size to take between 12–18 weeks to complete from protocol development through to reporting.

11 FAQs about meta-analyses for Pharma and Medtech reimbursement

What is meta-analysis in healthcare?

A meta-analysis is a statistical method of ‘pooling’ data from multiple studies that have addressed a similar research question to provide a combined effect estimate for a clinical outcome(s) (e.g. complete response) from multiple data sources. Hence providing more power and confidence in the observed effect. Meta-analyses in healthcare combine clinical evidence from studies to ensure comprehensive assessment of treatment effectiveness across multiple interventions, most notably for health technology assessment (HTA).

Meta-analyses can include data from various types of studies (e.g. randomised controlled trials [RCTs], observational studies, single-arm studies). The study design determines the type of analysis required.

What is the difference between conventional meta-analysis and network meta-analysis?

A conventional meta-analysis compares two trial arms (e.g. drug versus placebo) from multiple studies in a pairwise manner. It aims to determine which treatment is most effective. In contrast, a network meta-analysis can compare multiple trial arms, even if they have not been directly compared in a head-to-head trial. Its goal is to determine the most effective treatment option.

When would you use a meta-analysis?

Researchers utilise meta-analysis when they plan to include multiple studies addressing the same research question. These studies typically involve similar populations, interventions, and outcomes. Homogeneity or similarity of the trial characteristics should be proven prior to including within a meta-analysis. Meta-analysis helps to synthesise and analyse data from these studies to draw robust conclusions.

Can you conduct a systematic literature review without a meta-analysis?

Yes, a meta-analysis is not always necessary. After conducting a systematic literature review, researchers synthesise all included data, whether that be statistically or narratively. The data can be used to support evidence generation requirements in various ways, such as incorporating these into HTA submissions.

If good head-to-head clinical trial data are available, it may not be necessary to conduct a network meta-analysis for the purpose of HTA. However, if other studies are available that consider interventions of interest, a network meta-analysis would be favourable to ensure all relevant data on the interventions of interest are captured. An economic model submitted to an HTA agency can then explore scenarios including all available data or just the data from the key clinical trial.

In some cases, data may not be homogenous meaning a robust meta-analysis, or network meta-analysis, is not possible.

What is the major advantage of a meta-analysis?

Meta-analyses provide greater statistical power by combining several studies into what is effectively one larger study. As a result, we can be more confident in the data. Some meta-analyses, such as indirect treatment comparisons (ITCs), enable comparisons between interventions where no direct comparison exists, for example, via an RCT.

What do you write/include in a pharmaceutical meta-analysis?

When conducting a meta-analysis of clinical trials, the aim is to identify how effective one treatment is in comparison with another. Statistical analyses are conducted on the data to estimate treatment effect. In general, the results you would expect to see include:

• Effect estimate: overall pooled estimate based on multiple studies included in the meta-analysis showing whether studies generally favour an intervention or comparator. Can be derived in many forms including, but not limited to, odds ratio, risk ratio, rate ratio mean difference, standardised mean difference
• Confidence intervals (e.g. 95%): highlights the variation associated with the effect estimate derived from multiple studies to see how much confidence should be applied to the observed effect
• p-value for effect estimate: indicates the statistical significance (i.e. p<0.05=reject null hypothesis; p≥0.05=failure to reject the null hypothesis) associated with the pooled effect estimate
• Heterogeneity (e.g. I2): provides insight on whether studies included in the pooled analyses differ in terms of the direction of effect observed (i.e. one study significantly favours the intervention, whereas another significantly favours the comparator)

What are the assumptions for meta-analyses and network meta-analyses?

For meta-analyses, homogeneity is a key assumption in being able to draw robust conclusions from pairwise comparisons. In the context of network meta-analysis, homogeneity is a key assumption to check there is no imbalance in the distribution of effect modifiers across the different types of direct treatment comparisons. Another important assumption to take into account for a network meta-analysis is coherence.

How many studies are needed for a meta-analysis?

Performing a meta-analysis requires a minimum of two studies. Yet, if the studies contain small sample sizes with wide confidence intervals, experts will warn against performing a meta-analysis with just two studies. This is mainly because of the imprecision observed in the effect estimate, which makes the analysis redundant for making clinical decisions about the statistically compared drugs. Many will often report two studies qualitatively and only perform a meta-analysis if they have at least three studies included.

Can a meta-analysis be qualitative?

Yes, albeit these are rare and often used in the discipline of psychology to assess key themes developing from qualitative data. These are often referred to as meta-syntheses.

What is an ITC?

An indirect treatment comparison involves conducting a meta-analysis to compare two treatments that have not been directly compared in head-to-head trials (e.g. through an RCT). Indirect treatment comparisons are conducted when a common comparator exists. Two trials may have different treatments but the same comparators, allowing that common comparator to act as an anchor. For example, where direct evidence is available for treatment A versus treatment B and for treatment A versus treatment C, the ITC allows a comparison of treatment B versus treatment C.

What are the assumptions for indirect treatment comparisons?

Homogeneity (i.e. similarity) across included trials is a key assumption for indirect treatment comparisons. In addition to homogeneity, transitivity (i.e. the ‘anchor’ comparator is similar across trials) and consistency (i.e. direct and indirect evidence are in agreement) are also important assumptions when performing indirect treatment comparisons. If these assumptions are not met, the results of the indirect treatment comparison are likely to be misleading. In the case of transitivity, if the ‘anchor’ comparator is not similar between trials, then you are introducing indirectness into the direct comparison, which is used as the platform to calculate your indirect comparison.

Do you have a different question for our experts? Send your question to info@mtechaccess.co.uk.

Our experts are here to support all your evidence synthesis, generation, and communication requirements, including all aspects of systematic literature review and network meta-analysis. We also offer education and training on key principles of evidence synthesis. To learn more, email info@mtechaccess.co.uk.

The post FAQs about systematic literature reviews and network meta-analyses in healthcare appeared first on Mtech Access.

The post 6 reasons to partner with a specialist market access agency appeared first on Mtech Access.

The post Making evidence more accessible: reflections from Virtually Cochrane appeared first on Mtech Access.

The post appeared first on Mtech Access.

‘Time to event’ or ‘survival’ is an important endpoint used in efficacy assessments of medical interventions, particularly in disease areas such as oncology. The Cox proportional hazards (PH) model is the most commonly applied statistical model for the analysis of survival data in randomised controlled trials (RCTs). Hazard ratios (HRs) are reported from the Cox PH model as a measure of relative treatment effect. Meta-analyses of survival data are usually performed using this individual summary statistic and this approach assumes that the PH assumption holds; in other words, the relative hazards of an event are constant over time.

A major challenge encountered when conducting meta-analyses with survival data is the failure of RCT publications to report the methodology and validity of the statistical modelling approach (1). In a recent review of RCTs from five major oncology journals it was found that the use of the PH assumption to model survival outcomes is likely to have been inappropriate in many instances (1). In such cases, the estimates of relative treatment effect from the trials, expressed as HRs, and the subsequent meta-analyses which usually utilise these HRs, will be biased. This is especially problematic in network meta-analyses (NMA) where the PH assumption is imposed across multiple studies. The use of robust approaches to evidence-based medicine are vital to clinical decision-making and alternative approaches for the analysis of survival data are needed.

The non-PH acceleration failure time (AFT) model represents an alternative option to the Cox PH model when the effect of treatment is to accelerate or delay the event of interest (2). We suggest that a viable option is to independently analyse pseudo-individual patient-level data from all trials that contribute to a survival endpoint network, using both the Cox PH and AFT models. The assumptions of both modelling approaches can be assessed formally, and the most appropriate method can be used to obtain relative treatment effect estimates for use in meta-analysis.

Our evidence generation experts at Mtech have published research using this approach in what we believe is the first NMA based on trial-level acceleration factors for first-line treatments in non-small cell lung cancer (3). This approach represents a robust alternative when there is strong evidence to suggest the PH assumption does not hold in trials of a connected evidence network suitable for meta-analysis.

For further details on any aspects of evidence generation contact us at info@mtechaccess.co.uk.

References

1. Batson S, Greenall G, Hudson P. Review of the Reporting of Survival Analyses within Randomised Controlled Trials and the Implications for Meta-Analysis. PloS one. 2016;11(5):e0154870.
2. Patel K, Kay R, Rowell L. Comparing proportional hazards and accelerated failure time models: an application in influenza. Pharmaceutical statistics. 2006;5(3):213-24.
3. Batson S, Mitchell SA, Windisch R, Damonte E, Munk VC, Reguart N. Tyrosine kinase inhibitor combination therapy in first-line treatment of non-small-cell lung cancer: systematic review and network meta-analysis. Onco Targets Ther. 2017;10:2473-82.

The post Novel approach to the network meta-analysis of survival data appeared first on Mtech Access.

There are two main statistical approaches to modelling ordered categorical data in network meta-analysis (NMA):

1. Analyse as binary data using an independent binomial logit model to estimate the treatment effects for each category separately
2. Analyse as categorical data using a single multinomial probit model with a shared treatment effect across all categories

The binary approach requires fewer assumptions and the random effects model is easier to implement with this model (particularly important if significant heterogeneity is anticipated in the evidence network). However, this approach does ignore the correlation and natural ordering of the outcomes and therefore is not considered ‘efficient’ in terms of using all the information available. The categorical approach uses all available information, accounting for competing risks and the natural ordering of the outcome response categories. A key assumption of the multinomial probit model is the shared treatment effect; that is, the relative treatment effect is the same regardless of the response cut-off. It is essential that this assumption is deemed reasonable to pursue the second approach.

The two approaches both have their advantages and disadvantages. Therefore, it is important to understand the assumptions required for each model and their associated limitations before determining the most appropriate and robust analysis method for your data. Particularly when conducting NMA for health technology assessment, the rationale as to the choice of model and validation of the underlying assumptions of that model are essential.

For further details on the analysis of categorical outcomes in NMA or on any aspects of evidence generation contact us at info@mtechaccess.co.uk.

The post Modelling Categorical Data in Network Meta-Analysis appeared first on Mtech Access.

A recently published meta-epidemiological study evaluated the current state of handling of effect modification in NMA (1). In summary, the study identified 77 NMA articles published in 2013 and of these, approximately 25% failed to explore potential effect modifiers. Of those that did, 21% did not apply any methods to adjust for them, while the following approaches were reported:

• 52% performed sensitivity analyses
• 38% performed subgroup analyses
• 35% performed meta-regression analyses

The reporting of methods and results addressing effect modification were also incomplete across approximately 50% of the NMAs. Overall, the study highlights deficiencies in current reporting practices for addressing effect modification. It also raises concerns over the potential impact that the failure of the reporting of steps to address effect modification may have on the interpretation and validity of NMA results.

At Mtech Access we understand the importance of addressing effect modification in NMA, fully reporting the methods employed and providing a clear interpretation of the results. We have extensive experience of dealing with effect modification in NMA across a range of indications, using approaches such as sensitivity analyses, subgroup analyses and network meta-regression.

For further information on any aspects of systematic review and NMA please contact our in-house experts at Mtech Access.

Reference

1. Kovic B, Zoratti MJ, Michalopoulos S, Silvestre C, Thorlund K, Thabane L. Deficiencies in addressing effect modification in network meta-analyses: a meta-epidemiological survey. Journal of clinical epidemiology. 2017;88:47-56.

The post Addressing effect modification in network meta-analysis appeared first on Mtech Access.