Agreement between mega-trials and smaller trials: a meta-research study

Importance: Mega-trials can provide large-scale evidence on important questions. Objective: To explore how the results of mega-trails compare to the meta-analysis results of trials with smaller sample sizes. Data Sources: Clinicaltrials.gov was searched for mega-trials until 10.01.2023. PubMed was searched until June 2023 for meta-analyses incorporating the results of the eligible mega-trials. Study Selection: Mega-trials were eligible if they were non-cluster non-vaccine randomized control trials (RCTs); had a sample size over 10,000; and had a peer-reviewed meta-analysis publication presenting results for the primary outcome of the mega-trials and/or all-cause mortality. Data Extraction and Synthesis: For each selected meta-analysis, we extracted results of smaller trials and mega-trials included in the summary effect estimate, and combined them separately using random effects. These estimates were used to calculate the ratio of odds ratios (ROR) between mega-trials and smaller trials in each meta-analysis. Next, the ROR were combined using random-effects. Risk of bias was extracted for each trial included in our analyses (or when not available, assessed only for mega-trials). Main Outcomes and Measures: The main outcomes were the summary ROR for the primary outcome and all-cause mortality between mega-trials and smaller trials. Sensitivity analyses were performed with respect to the time of publishing, masking, weight, type of intervention, and specialty. Results Of 120 mega-trials identified, 39 (33%) had significant benefits for the primary outcome and 18 (15%) had significant benefits for all-cause mortality for the intervention. In 35 comparisons of primary outcomes (including 85 point estimates from 69 unique mega-trials and 272 point estimatesfrom smaller trials) and 26 comparisons of all-cause mortality (including 70 point estimates from 65 unique mega-trials and 267 point estimates from smaller trials), ROR was 1.00 (95% CI, 0.97-1.04) and 1.00 (95% CI, 0.97-1.04), respectively. For the primary outcomes, smaller trials published before the mega-trials had more favorable results than the mega-trials (ROR 1.05, 95% CI, 1.01- 1.10), and than the subsequent smaller trials (ROR 0.91, 95% CI, 0.85-0.96). Conclusions and Relevance: Meta-analyses of smaller studies show in general comparable results with mega-trials, but smaller trials published before the mega-trials give more favorable results than the mega-trials.