Network Meta-Analysis Methodology

Comprehensive methodological guidance for conducting rigorous network meta-analysis following NICE DSU and PRISMA-NMA guidelines.

When to Use This Skill

• •Planning a network meta-analysis
• •Assessing transitivity and consistency
• •Interpreting treatment rankings
• •Choosing between frequentist and Bayesian NMA
• •Designing NMA sensitivity analyses
• •Reviewing NMA code or results

Fundamental Assumptions

1. Transitivity Assumption

Definition: If we can estimate A vs B directly and B vs C directly, we can estimate A vs C indirectly, provided the studies are sufficiently similar.

Requirements:

• •Studies comparing different treatments should be similar enough to have been included in the same RCT
• •Effect modifiers should be balanced across comparisons
• •No important differences in study-level characteristics

Assessment:

For each comparison in network, check:
├── Population similarity
│   - Age, sex, disease severity
│   - Biomarker status, prior treatments
├── Outcome definitions
│   - Same definition of response/event
│   - Same time point of assessment
├── Treatment definitions
│   - Dose, duration, route
│   - Concomitant medications
└── Study design
    - Randomization, blinding
    - Follow-up duration

Presenting Transitivity Assessment:

• •Create table of study characteristics by comparison
• •Highlight any systematic differences
• •Use forest plots stratified by comparison

2. Consistency Assumption

Definition: Direct and indirect evidence for the same comparison should agree (within random variability).

Relation to Transitivity:

• •Transitivity is untestable (conceptual)
• •Consistency is testable (statistical)
• •Consistency violations suggest transitivity violations

Consistency Assessment

Global Consistency Tests

Design-by-Treatment Interaction

# netmeta
decomp.design(nma_result)
# Tests overall consistency across network
# Q statistic partitioned into within-design and between-design

Q Statistic Decomposition

• •Q_total = Q_heterogeneity + Q_inconsistency
• •Test Q_inconsistency against chi-square distribution

Local Consistency: Node-Splitting

# netmeta
netsplit(nma_result)

# gemtc
nodesplit_model <- mtc.nodesplit(network)

Interpretation:

Caution: Multiple testing - expect some false positives.

Net Heat Plot

netheat(nma_result)
# Visual display of inconsistency
# Red: high inconsistency contribution
# Blue: low inconsistency

What to Do with Inconsistency

1. •Check data - errors in data entry
2. •Investigate sources - which comparisons differ
3. •Explore heterogeneity - meta-regression on potential modifiers
4. •Consider splitting network - if clinical rationale exists
5. •Report transparently - don't hide inconsistency
6. •Use inconsistency model - as sensitivity analysis

Treatment Rankings

Frequentist (netmeta)

P-scores

netrank(nma_result, small.values = "bad")
# P-score: probability of being better than average treatment
# Ranges 0-1
# NOT probability of being best

Bayesian (gemtc)

SUCRA (Surface Under Cumulative Ranking Curve)

sucra(mtc_result)
# Similar interpretation to P-score
# Based on cumulative ranking probabilities

Probability of Being Best

rank.probability(mtc_result)
# Full ranking probability matrix
# Prob_best = P(rank = 1)

Interpretation Cautions

Critical: Rankings are uncertain - always present with uncertainty measures.

Problems with rankings:
├── Small differences → different rankings
├── Wide credible intervals often ignored
├── Multiple treatments may be effectively tied
├── Rankings don't consider clinical relevance
└── "Best" might have limited evidence

Best Practice:

• •Report ranking probabilities, not just point ranks
• •Show cumulative ranking plots
• •Consider clustering treatments by effect
• •Discuss clinical significance alongside statistical

Model Selection

Fixed vs Random Effects

Bayesian Prior Selection

Treatment Effects

prior_trt = prior_normal(0, sd)
# sd should be large enough to be weakly informative
# Consider scale of effect measure (log OR ~2-3 is large)

Heterogeneity (τ)

prior_het = prior_half_normal(scale)
# Scale depends on expected heterogeneity
# Turner et al. informative priors available

Model Comparison

DIC (Deviance Information Criterion)

# Lower is better
# Difference of ~3-5 is meaningful
dic(model1)
dic(model2)

Residual Deviance

• •Compare to number of data points
• •Should be close if model fits well

Network Geometry

Key Considerations

Network Structure Assessment:
├── Connectivity
│   - All treatments connected (directly or indirectly)?
│   - Star network? (single common comparator)
│   - Well-connected?
├── Evidence Distribution
│   - Some comparisons well-informed, others sparse?
│   - Imbalanced networks problematic
├── Multi-arm Trials
│   - Must account for correlations
│   - Contribution to network
└── Placebo/Active Control
    - Consider clinical relevance of network anchor

Contribution Matrix

# netmeta
netcontrib(nma_result)
# Shows % contribution of each direct comparison to each estimate

Network Graph

netgraph(nma_result,
         plastic = FALSE,
         thickness = "number.of.studies",
         multiarm = TRUE,
         points = TRUE)

Reporting Checklist (PRISMA-NMA)

Methods

• • Network geometry description
• • Transitivity assessment approach
• • Effect measure and rationale
• • Model choice (fixed/random, frequentist/Bayesian)
• • Prior specifications (if Bayesian)
• • Consistency assessment methods
• • Ranking methods and interpretation
• • Sensitivity analyses planned

Results

• • Network diagram
• • Study characteristics table by comparison
• • Pairwise MA results (for direct evidence)
• • NMA results for all comparisons
• • League table
• • Consistency assessment results
• • Treatment rankings with uncertainty
• • Sensitivity analysis results

Common Pitfalls

1. Ignoring Transitivity

• •Must assess before running NMA
• •Not just a formality - fundamental requirement

2. Over-interpreting Rankings

• •"Treatment A ranked #1" without uncertainty
• •Small differences may give different rankings
• •Clinical relevance matters more than rank

3. Selective Consistency Reporting

• •Report all node-split results
• •Don't dismiss inconsistency findings

4. Multi-arm Trial Handling

• •Must account for correlations
• •Software handles this, but check it's done correctly

5. Sparse Networks

• •Very uncertain indirect comparisons
• •Consider if NMA is appropriate

Quick Reference Code

Frequentist (netmeta)

library(netmeta)

# Fit NMA
nma <- netmeta(TE, seTE, treat1, treat2, studlab,
               data = pairwise_data,
               sm = "OR",
               reference.group = "Placebo",
               random = TRUE)

# Network graph
netgraph(nma, plastic = FALSE, multiarm = TRUE)

# Forest vs reference
forest(nma, reference.group = "Placebo")

# League table
netleague(nma)

# Consistency
netsplit(nma)
netheat(nma)

# Rankings
netrank(nma, small.values = "bad")

Bayesian (gemtc)

library(gemtc)
library(rjags)

# Create network
network <- mtc.network(data.ab = arm_data)

# Fit model
model <- mtc.model(network,
                   likelihood = "binom",
                   link = "logit",
                   linearModel = "random")
result <- mtc.run(model, n.adapt = 5000, n.iter = 50000)

# Check convergence
gelman.diag(result)

# Summary
summary(result)

# Rankings
rank.probability(result)
sucra(result)

# Node-splitting
nodesplit <- mtc.nodesplit(network)
ns_result <- mtc.run(nodesplit)
summary(ns_result)

Resources

• •NICE DSU TSD 2: https://www.sheffield.ac.uk/nice-dsu/tsds
• •PRISMA-NMA: Hutton et al. 2015
• •Dias et al. (2018): Network Meta-Analysis for Decision Making
• •Salanti (2012): Ann Intern Med - Intro to NMA