1 Overview

This report presents the four main meta-analytic specifications used to support the manuscript. The primary analysis is the direct replication dataset, which is followed by three complementary analyses: computational reproduction based on the published supplementary dataset, a risk-of-bias–adjusted sensitivity analysis, and a recycling-adjusted sensitivity analysis.

For each specification, the report provides a forest plot, pooled estimate and heterogeneity statistics, regression tests of funnel plot asymmetry, two funnel plot views, and a leave-one-out influence analysis.

2 Main analyses

2.1 Direct replication

This is the primary analysis for the manuscript. It is based on the independently reconstructed dataset and reflects the direct replication of the original review workflow using the eligibility decisions adopted in the present study.

2.1.1 Forest plot

Forest plot for the direct replication.

2.1.2 Pooled estimate and heterogeneity

Analysis Estimate Lower Upper SE t df p tau2 I2
Direct replication -1.20773 -1.506364 -0.909095 0.149 -8.1 56.9 < 0.001 0.985 76.7%

2.1.3 Regression tests of funnel plot asymmetry

Precision Intercept SE (Intercept) t df Intercept p Slope SE (Slope) Slope p n_used
SE -5.130 0.554 -9.26 106 < 0.001 1.733 0.293 < 0.001 108
1/sqrt(n) -9.143 3.826 -2.39 106 0.019 0.803 0.897 0.372 108

2.1.4 Funnel plot using standard error as the precision metric

2.1.5 Funnel plot using 1/√n as the precision metric

2.1.6 Influence analysis (leave-one-out)

Dropped k_remaining Pooled SMD [95% CI] Delta % change
Saravi_2016a_Group1 107 -1.16 [-1.45, -0.87] +0.047 +3.9%
Saravi_2016b_Group5 106 -1.16 [-1.45, -0.88] +0.044 +3.6%
Saravi_2016b_Group1 106 -1.16 [-1.46, -0.87] +0.043 +3.6%
Saravi_2016a_Group5 107 -1.17 [-1.46, -0.88] +0.040 +3.3%
Saravi_2016b_Group2 106 -1.17 [-1.46, -0.88] +0.040 +3.3%
Saravi_2016a_Group2 107 -1.17 [-1.46, -0.88] +0.039 +3.3%
Singh_2016_Group1 106 -1.17 [-1.46, -0.88] +0.038 +3.1%
Deak_2005 107 -1.24 [-1.54, -0.94] -0.028 -2.3%
Tong_2017_Group5 105 -1.24 [-1.53, -0.94] -0.028 -2.3%
Souza_2017_Group2 106 -1.18 [-1.48, -0.88] +0.027 +2.2%
Norden_2015_Group1 107 -1.23 [-1.53, -0.93] -0.026 -2.2%
Zheng_2015_Group1 106 -1.23 [-1.53, -0.93] -0.026 -2.1%
Mahmoud_2017_Group1 106 -1.23 [-1.53, -0.93] -0.025 -2.1%
O’Connor_2009_Group1 106 -1.23 [-1.53, -0.93] -0.025 -2.1%
Tong_2017_Group3 105 -1.23 [-1.53, -0.93] -0.025 -2.1%
Vogt_2016 106 -1.23 [-1.53, -0.93] -0.025 -2.1%
Xu_2017_Group1 106 -1.23 [-1.53, -0.93] -0.025 -2.1%
Tong_2017_Group1 105 -1.23 [-1.53, -0.93] -0.023 -1.9%
Arakawa_2012_Group2 106 -1.23 [-1.53, -0.93] -0.022 -1.8%
Chijiwa_2015_Group2 107 -1.23 [-1.53, -0.93] -0.022 -1.8%
Saravi_2016a_Group3 107 -1.19 [-1.48, -0.89] +0.022 +1.8%
Xie_2014_Group1 106 -1.23 [-1.53, -0.93] -0.022 -1.8%
Saravi_2016b_Group3 106 -1.19 [-1.49, -0.89] +0.021 +1.7%
Saravi_2016b_Group4 106 -1.19 [-1.49, -0.89] +0.021 +1.8%
Souza_2017_Group1 106 -1.23 [-1.53, -0.93] -0.021 -1.7%
Saravi_2016a_Group4 107 -1.19 [-1.49, -0.89] +0.020 +1.7%
McKim_2016 107 -1.23 [-1.53, -0.92] -0.019 -1.6%
Burke_2014_Group4 107 -1.23 [-1.53, -0.92] -0.018 -1.5%
Liu_2015 105 -1.22 [-1.53, -0.92] -0.016 -1.4%
Majidi_2016_Group1 105 -1.22 [-1.53, -0.92] -0.016 -1.3%
Rinwa_2013_Group2 106 -1.19 [-1.49, -0.89] +0.016 +1.4%
Burke_2014_Group1 107 -1.22 [-1.53, -0.92] -0.015 -1.2%
Burke_2014_Group2 107 -1.22 [-1.53, -0.92] -0.015 -1.2%
Chijiwa_2015_Group1 107 -1.19 [-1.49, -0.89] +0.015 +1.3%
Singh_2016_Group2 106 -1.19 [-1.49, -0.89] +0.015 +1.3%
Xie_2014_Group2 106 -1.19 [-1.49, -0.89] +0.015 +1.3%
Rinwa_2013_Group3 106 -1.19 [-1.50, -0.89] +0.014 +1.2%
Singh_2017_Group1 107 -1.19 [-1.50, -0.89] +0.014 +1.1%
Nagpal_2013 106 -1.22 [-1.53, -0.92] -0.013 -1.1%
Singh_2017_Group2 107 -1.22 [-1.52, -0.92] -0.013 -1.0%
Zheng_2014 106 -1.20 [-1.50, -0.89] +0.013 +1.0%
Tong_2017_Group6 105 -1.20 [-1.50, -0.89] +0.011 +0.9%
da Silva Dias_2016 107 -1.22 [-1.52, -0.91] -0.010 -0.9%
Molina-Hernández_2018a 107 -1.22 [-1.52, -0.91] -0.008 -0.7%
Molina-Hernández_2018b_Group2 107 -1.22 [-1.52, -0.91] -0.008 -0.7%
Wang_2017_Group1 106 -1.22 [-1.52, -0.91] -0.008 -0.6%
Arakawa_2012_Group1 106 -1.21 [-1.52, -0.91] -0.007 -0.6%
Xu_2017_Group2 106 -1.21 [-1.52, -0.91] -0.007 -0.6%
Burke_2014_Group3 107 -1.21 [-1.52, -0.91] -0.006 -0.5%
O’Connor_2009_Group2 106 -1.20 [-1.51, -0.90] +0.006 +0.5%
Molina-Hernández_2018b_Group1 107 -1.21 [-1.52, -0.91] -0.005 -0.4%
Tong_2017_Group2 105 -1.20 [-1.51, -0.90] +0.005 +0.4%
Zheng_2015_Group2 106 -1.21 [-1.52, -0.91] -0.005 -0.4%
Amorim_2017 106 -1.20 [-1.51, -0.90] +0.004 +0.4%
Mahmoud_2017_Group2 106 -1.21 [-1.52, -0.91] -0.004 -0.4%
Majidi_2016_Group2 105 -1.21 [-1.52, -0.91] -0.004 -0.3%
Norden_2015_Group2 107 -1.21 [-1.52, -0.91] -0.004 -0.3%
Wang_2017_Group2 106 -1.20 [-1.51, -0.90] +0.004 +0.4%
Rinwa_2013_Group1 106 -1.21 [-1.52, -0.90] -0.002 -0.1%
Tong_2017_Group4 105 -1.21 [-1.51, -0.90] -0.000 -0.0%

2.2 Computational reproduction

This analysis uses the published supplementary effect-size dataset to test whether a rebuilt analytical pipeline can recover the original quantitative signal. It therefore evaluates reproducibility of the published analysis more directly than the independently reconstructed dataset does.

2.2.1 Forest plot

Forest plot based on the original supplementary dataset, used to verify that the rebuilt pipeline reproduces the published signal.

2.2.2 Pooled estimate and heterogeneity

Analysis Estimate Lower Upper SE t df p tau2 I2
Computational reproduction -1.074619 -1.405311 -0.743927 0.163 -6.58 37.1 < 0.001 0.791 75.7%

2.2.3 Regression tests of funnel plot asymmetry

Precision Intercept SE (Intercept) t df Intercept p Slope SE (Slope) Slope p n_used
SE -5.478 0.925 -5.92 69 < 0.001 1.779 0.458 < 0.001 71
1/sqrt(n) -1.527 4.900 -0.31 69 0.756 -0.834 1.066 0.437 71

2.2.4 Funnel plot using standard error as the precision metric

2.2.5 Funnel plot using 1/√n as the precision metric

2.2.6 Influence analysis (leave-one-out)

Dropped k_remaining Pooled SMD [95% CI] Delta % change
21 70 -1.00 [-1.31, -0.70] +0.071 +6.6%
20 69 -1.02 [-1.33, -0.70] +0.057 +5.3%
33 70 -1.12 [-1.44, -0.80] -0.046 -4.3%
9 69 -1.12 [-1.44, -0.80] -0.045 -4.2%
23 69 -1.03 [-1.36, -0.71] +0.041 +3.8%
29 68 -1.11 [-1.44, -0.78] -0.039 -3.6%
6 70 -1.11 [-1.44, -0.78] -0.038 -3.5%
35 69 -1.11 [-1.44, -0.78] -0.034 -3.2%
27 68 -1.11 [-1.44, -0.77] -0.033 -3.1%
37 69 -1.11 [-1.44, -0.77] -0.033 -3.1%
4 70 -1.10 [-1.44, -0.77] -0.029 -2.7%
18 69 -1.05 [-1.38, -0.71] +0.029 +2.7%
25 68 -1.10 [-1.44, -0.77] -0.028 -2.6%
19 69 -1.05 [-1.38, -0.71] +0.027 +2.5%
24 69 -1.05 [-1.38, -0.71] +0.027 +2.5%
5 70 -1.05 [-1.38, -0.72] +0.026 +2.4%
22 70 -1.05 [-1.38, -0.72] +0.024 +2.2%
30 68 -1.05 [-1.39, -0.71] +0.024 +2.2%
13 70 -1.10 [-1.43, -0.76] -0.021 -2.0%
8 70 -1.06 [-1.39, -0.72] +0.018 +1.7%
1 69 -1.06 [-1.39, -0.72] +0.017 +1.6%
11 68 -1.09 [-1.43, -0.75] -0.017 -1.6%
26 68 -1.06 [-1.40, -0.72] +0.016 +1.5%
7 70 -1.09 [-1.43, -0.75] -0.014 -1.3%
32 69 -1.06 [-1.40, -0.72] +0.014 +1.3%
2 70 -1.09 [-1.43, -0.75] -0.013 -1.2%
34 70 -1.09 [-1.43, -0.74] -0.011 -1.0%
10 69 -1.07 [-1.40, -0.73] +0.008 +0.7%
28 68 -1.07 [-1.41, -0.73] +0.007 +0.7%
16 69 -1.08 [-1.42, -0.74] -0.006 -0.6%
17 69 -1.07 [-1.41, -0.73] +0.006 +0.6%
3 70 -1.07 [-1.41, -0.73] +0.005 +0.4%
14 70 -1.08 [-1.42, -0.74] -0.005 -0.4%
15 70 -1.08 [-1.42, -0.74] -0.005 -0.5%
31 69 -1.08 [-1.42, -0.74] -0.005 -0.5%
39 70 -1.08 [-1.42, -0.74] -0.004 -0.4%
12 68 -1.07 [-1.41, -0.73] +0.002 +0.2%
36 69 -1.08 [-1.42, -0.73] -0.001 -0.0%
38 69 -1.07 [-1.41, -0.73] +0.001 +0.1%

2.3 RoB-adjusted

This sensitivity analysis restricts the dataset to studies not classified as high overall risk of bias. Its purpose is to assess whether the primary result remains stable after excluding studies considered more methodologically vulnerable.

2.3.1 Forest plot

Forest plot for the lower-risk subset.

2.3.2 Pooled estimate and heterogeneity

Analysis Estimate Lower Upper SE t df p tau2 I2
RoB-adjusted -1.057365 -1.424675 -0.6900558 0.181 -5.85 34.2 < 0.001 0.809 72.9%

2.3.3 Regression tests of funnel plot asymmetry

Precision Intercept SE (Intercept) t df Intercept p Slope SE (Slope) Slope p n_used
SE -5.165 0.863 -5.99 63 < 0.001 1.864 0.450 < 0.001 65
1/sqrt(n) -4.815 3.721 -1.29 63 0.200 0.125 0.884 0.888 65

2.3.4 Funnel plot using standard error as the precision metric

2.3.5 Funnel plot using 1/√n as the precision metric

2.3.6 Influence analysis (leave-one-out)

Dropped k_remaining Pooled SMD [95% CI] Delta % change
Saravi_2016a_Group1 64 -0.98 [-1.32, -0.64] +0.081 +7.7%
Saravi_2016a_Group2 64 -0.99 [-1.34, -0.64] +0.064 +6.0%
Saravi_2016a_Group5 64 -0.99 [-1.34, -0.64] +0.064 +6.1%
Souza_2017_Group2 63 -1.01 [-1.37, -0.65] +0.047 +4.4%
Deak_2005 64 -1.10 [-1.47, -0.73] -0.042 -3.9%
Tong_2017_Group5 62 -1.10 [-1.47, -0.73] -0.042 -4.0%
Saravi_2016a_Group3 64 -1.02 [-1.38, -0.65] +0.040 +3.7%
Mahmoud_2017_Group1 63 -1.10 [-1.47, -0.72] -0.038 -3.6%
O’Connor_2009_Group1 63 -1.09 [-1.47, -0.72] -0.037 -3.5%
Saravi_2016a_Group4 64 -1.02 [-1.39, -0.65] +0.037 +3.5%
Tong_2017_Group3 62 -1.09 [-1.47, -0.72] -0.037 -3.5%
Tong_2017_Group1 62 -1.09 [-1.47, -0.71] -0.033 -3.1%
Xie_2014_Group1 63 -1.09 [-1.46, -0.72] -0.032 -3.0%
Arakawa_2012_Group2 63 -1.09 [-1.46, -0.71] -0.031 -3.0%
Chijiwa_2015_Group2 64 -1.09 [-1.46, -0.71] -0.031 -3.0%
Souza_2017_Group1 63 -1.09 [-1.46, -0.71] -0.030 -2.8%
Chijiwa_2015_Group1 64 -1.03 [-1.40, -0.66] +0.029 +2.7%
Xie_2014_Group2 63 -1.03 [-1.40, -0.66] +0.029 +2.7%
McKim_2016 64 -1.08 [-1.46, -0.71] -0.027 -2.5%
Burke_2014_Group4 64 -1.08 [-1.46, -0.71] -0.025 -2.4%
Tong_2017_Group6 62 -1.03 [-1.41, -0.66] +0.025 +2.4%
Zheng_2014 63 -1.03 [-1.40, -0.66] +0.024 +2.3%
Majidi_2016_Group1 62 -1.08 [-1.46, -0.70] -0.022 -2.0%
Burke_2014_Group1 64 -1.08 [-1.46, -0.70] -0.020 -1.9%
Burke_2014_Group2 64 -1.08 [-1.46, -0.70] -0.019 -1.8%
O’Connor_2009_Group2 63 -1.04 [-1.42, -0.66] +0.016 +1.5%
Tong_2017_Group2 62 -1.04 [-1.42, -0.66] +0.015 +1.4%
Wang_2017_Group2 63 -1.04 [-1.42, -0.67] +0.013 +1.2%
Molina-Hernández_2018b_Group2 64 -1.07 [-1.45, -0.68] -0.008 -0.8%
Wang_2017_Group1 63 -1.06 [-1.44, -0.68] -0.008 -0.7%
Arakawa_2012_Group1 63 -1.06 [-1.44, -0.68] -0.006 -0.6%
Tong_2017_Group4 62 -1.05 [-1.43, -0.67] +0.005 +0.5%
Burke_2014_Group3 64 -1.06 [-1.44, -0.68] -0.004 -0.4%
Mahmoud_2017_Group2 63 -1.06 [-1.44, -0.68] -0.003 -0.2%
Molina-Hernández_2018b_Group1 64 -1.06 [-1.44, -0.68] -0.003 -0.3%
Majidi_2016_Group2 62 -1.06 [-1.44, -0.68] -0.000 -0.0%

2.4 Recycling-adjusted

This sensitivity analysis evaluates whether the primary result remains stable after excluding comparisons considered more likely to reflect repeated use of the same animals across multiple experiments or outcomes.

2.4.1 Forest plot

Forest plot for the recycling-adjusted dataset.

2.4.2 Pooled estimate and heterogeneity

Analysis Estimate Lower Upper SE t df p tau2 I2
Recycling-adjusted -0.9530282 -1.212945 -0.6931112 0.129 -7.36 50 < 0.001 0.663 71.0%

2.4.3 Regression tests of funnel plot asymmetry

Precision Intercept SE (Intercept) t df Intercept p Slope SE (Slope) Slope p n_used
SE -4.474 0.773 -5.79 94 < 0.001 1.422 0.392 < 0.001 96
1/sqrt(n) -0.535 2.893 -0.19 94 0.854 -0.914 0.664 0.172 96

2.4.4 Funnel plot using standard error as the precision metric

2.4.5 Funnel plot using 1/√n as the precision metric

2.4.6 Influence analysis (leave-one-out)

Dropped k_remaining Pooled SMD [95% CI] Delta % change
Saravi_2016a_Group1 95 -0.90 [-1.15, -0.66] +0.051 +5.3%
Saravi_2016b_Group1 94 -0.91 [-1.16, -0.66] +0.046 +4.8%
Singh_2016_Group1 94 -0.92 [-1.17, -0.66] +0.037 +3.9%
Souza_2017_Group2 94 -0.92 [-1.18, -0.67] +0.030 +3.2%
Tong_2017_Group5 93 -0.98 [-1.24, -0.72] -0.027 -2.8%
Deak_2005 95 -0.98 [-1.24, -0.72] -0.026 -2.7%
Rinwa_2013_Group2 94 -0.93 [-1.19, -0.67] +0.024 +2.5%
Mahmoud_2017_Group1 94 -0.98 [-1.24, -0.71] -0.023 -2.4%
Norden_2015_Group1 95 -0.98 [-1.24, -0.72] -0.023 -2.5%
Xu_2017_Group1 94 -0.98 [-1.24, -0.71] -0.023 -2.4%
Zheng_2015_Group1 94 -0.98 [-1.24, -0.71] -0.023 -2.4%
O’Connor_2009_Group1 94 -0.98 [-1.24, -0.71] -0.022 -2.3%
Rinwa_2013_Group3 94 -0.93 [-1.19, -0.67] +0.022 +2.3%
Singh_2016_Group2 94 -0.93 [-1.19, -0.67] +0.022 +2.3%
Tong_2017_Group3 93 -0.98 [-1.24, -0.71] -0.022 -2.3%
Vogt_2016 94 -0.98 [-1.24, -0.71] -0.022 -2.3%
Chijiwa_2015_Group1 95 -0.93 [-1.19, -0.67] +0.020 +2.1%
Xie_2014_Group2 94 -0.93 [-1.19, -0.67] +0.020 +2.1%
Singh_2017_Group1 95 -0.93 [-1.19, -0.67] +0.019 +2.0%
Tong_2017_Group1 93 -0.97 [-1.24, -0.71] -0.019 -2.0%
Tong_2017_Group6 93 -0.93 [-1.20, -0.67] +0.019 +2.0%
Xie_2014_Group1 94 -0.97 [-1.23, -0.71] -0.019 -2.0%
Arakawa_2012_Group2 94 -0.97 [-1.23, -0.71] -0.018 -1.9%
Chijiwa_2015_Group2 95 -0.97 [-1.23, -0.71] -0.018 -1.9%
Souza_2017_Group1 94 -0.97 [-1.23, -0.71] -0.017 -1.8%
Zheng_2014 94 -0.94 [-1.20, -0.67] +0.017 +1.8%
McKim_2016 95 -0.97 [-1.23, -0.70] -0.015 -1.6%
Burke_2014_Group4 95 -0.97 [-1.23, -0.70] -0.014 -1.5%
Liu_2015 93 -0.97 [-1.23, -0.70] -0.013 -1.3%
O’Connor_2009_Group2 94 -0.94 [-1.20, -0.68] +0.013 +1.4%
Tong_2017_Group2 93 -0.94 [-1.20, -0.68] +0.013 +1.3%
Amorim_2017 94 -0.94 [-1.21, -0.68] +0.011 +1.1%
Majidi_2016_Group1 93 -0.96 [-1.23, -0.70] -0.011 -1.2%
Wang_2017_Group2 94 -0.94 [-1.21, -0.68] +0.011 +1.2%
Burke_2014_Group1 95 -0.96 [-1.23, -0.70] -0.010 -1.1%
Burke_2014_Group2 95 -0.96 [-1.23, -0.70] -0.010 -1.0%
Nagpal_2013 94 -0.96 [-1.23, -0.70] -0.008 -0.9%
Singh_2017_Group2 95 -0.96 [-1.23, -0.70] -0.008 -0.8%
Rinwa_2013_Group1 94 -0.95 [-1.21, -0.68] +0.007 +0.7%
Tong_2017_Group4 93 -0.95 [-1.21, -0.68] +0.007 +0.7%
da Silva Dias_2016 95 -0.96 [-1.22, -0.69] -0.005 -0.5%
Majidi_2016_Group2 93 -0.95 [-1.22, -0.68] +0.003 +0.3%
Molina-Hernández_2018b_Group1 95 -0.95 [-1.22, -0.69] +0.002 +0.2%
Molina-Hernández_2018b_Group2 95 -0.96 [-1.22, -0.69] -0.002 -0.2%
Norden_2015_Group2 95 -0.95 [-1.22, -0.69] +0.002 +0.2%
Wang_2017_Group1 94 -0.96 [-1.22, -0.69] -0.002 -0.2%
Arakawa_2012_Group1 94 -0.95 [-1.22, -0.69] -0.001 -0.1%
Mahmoud_2017_Group2 94 -0.95 [-1.22, -0.69] +0.001 +0.1%
Molina-Hernández_2018a 95 -0.95 [-1.22, -0.69] -0.001 -0.1%
Zheng_2015_Group2 94 -0.95 [-1.22, -0.69] +0.001 +0.2%
Burke_2014_Group3 95 -0.95 [-1.22, -0.69] +0.000 +0.0%
Xu_2017_Group2 94 -0.95 [-1.22, -0.69] +0.000 +0.0%

3 Comparative summary plot

This section compares the main analytical datasets used in the project. The direct replication serves as the primary reference point for interpretation, while the computational reproduction and the sensitivity analyses are presented to show how the pooled estimate changes under different defensible analytical decisions. For visual reference, the plot also includes the pooled estimate reported in the original study.

[1] “report_tables/pooled_summary_all_models.csv”

4 Summary table across models

This section compares the main analytical datasets used in the study. The direct replication is the primary analysis, and the remaining analyses are shown to indicate how the pooled estimate changes under different analytical assumptions and sensitivity restrictions.

Analysis Estimate Lower Upper SE t df p tau2 I2
Direct replication -1.2077297 -1.506364 -0.9090950 0.149 -8.10 56.9 < 0.001 0.985 76.7%
Computational reproduction -1.0746192 -1.405311 -0.7439270 0.163 -6.58 37.1 < 0.001 0.791 75.7%
RoB-adjusted -1.0573653 -1.424675 -0.6900558 0.181 -5.85 34.2 < 0.001 0.809 72.9%
Recycling-adjusted -0.9530282 -1.212945 -0.6931112 0.129 -7.36 50.0 < 0.001 0.663 71.0%