MATES: A tool for evaluating the quality of reporting of meta-analyses

Meta-analyses, embedded in systematic reviews, are pivotal in today's scientific landscape for reconciling conflicting findings, increasing statistical power, and charting new research directions. However, poor reporting practices that conceal technical details and potential limitations often need to be revised to maintain their reliability. Despite existing reporting guidelines, a comprehensive tool has yet to be tailored to appraise the reporting quality of the quantitative aspects of meta-analysis in environmental sciences. To bridge this gap, we introduce the Meta-analysis Appraisal Tool for Environmental Sciences (MATES), a checklist of items to assess the reporting quality of meta-analyses. To develop MATES, we used an adapted Delphi process involving workshops (11-16 participants), a survey (193 participants), and validation (30 participants). This process resulted in a 14-item checklist, encompassing the environmental science communities view of important reporting elements. The validation, across 50 meta-analyses, indicated that the tool is repeatable (an average intra-class correlation of 88.97%) and time-efficient (17.00 ± 11.77 min) to implement. To enhance the accessibility and usability of MATES, we created an interactive web-based app that features training and implementation modules https://kylemorrisonisshiny99.shinyapps.io/MATES_shiny/. We also discuss how to interpret the MATES results, potential use cases of MATES and evaluate the development methodology. Overall, MATES provides authors, readers, reviewers, and editors with a reliable and user-friendly tool to assess the reporting quality of meta-analyses in the environmental sciences.