Supplementary File 1
Statistical Methods
Two-level meta-analysis model:
The two-level model is set up as follows:
(1)
(2)
,
,
Model (1), which is the “intercept only” model, is equivalent to the random-effects model for meta-analysis described by Hedges and Olkin [1], is ES of study j, is the ES estimation of study j, the intercept is the mean ES estimation across all studies, is the random effect of study j, is the level-1 sampling error for study j and is the sample size of study j. It is assumed that and have a normal distribution with known variances and , respectively.
Heterogeneity test:
The heterogeneity test is an important step of meta-analysis. Because the variance of ESs across studies, , indicates how much these ESs vary across studies, heterogeneity test is equivalent to testing the null-hypothesis that variance of the residual error , indicated by , is equal to zero. If the test of is statistically significant, the study ESs are considered heterogeneous, then ESs are to be pooled using random-effects model. Otherwise, ESs are to be pooled using fixed-effects model [2].
Exploratory multilevel meta-analysis model:
Some explanatory variables are added to model (1) to form the exploratory multilevel meta-analysis model (model (2)), where is study- or ecological-level variables, and is the regression coefficient.
Because exploring the possible sources of heterogeneity across studies is a key methodology feature of meta-analysis of observational studies [3,4], the multilevel meta-analysis with covariates approach in model (2) was applied to identify variables that explain differences between GSI scores of cross-sectional surveys [2], only when there was obvious evidence of statistical heterogeneity. Each study- or ecological-level variable was separately included in the model and its corresponding regression coefficient was tested by using Wald Chi-square test [5] in univariate analysis. Considering the number of included surveys was small, the results of multivariate analysis might possibly be unstable for lack of power and the method for independent variable selection in multivariate multi-level meta-regression has not been well-established [6,7], multivariate analysis was not performed after univariate analysis. Instead the percentage of level-2 variance explained by an individual variable was calculated in order to measure the strength of relationship between covariate and SCL-90-R GSI score.
ES calculation:
For ES synthesizing of cross-sectional surveys, is one of SCL-90-R subscale or GSI scores of CMWs and its corresponding standard error is calculated through SD divided by square root of sample size ().
For ES synthesizing of case-control studies, is value [8], defined as , and its corresponding standard error is calculated by .
The pooling of SCL-90-R scores and Cohen’s d values and sub-group analysis by study quality were performed in model (1). Sub-group analyses were performed to evaluate the impact of study quality on pooled SCL-90-R scores. Exploratory multilevel meta-analysis with covariates were performed in model (2). Other statistical details refer to the web pages of UCLA ATS [9].
References:
1. Hedges L, Olkin I (1985) Statistical methods for meta-analysis. Academic Press, Orlando
2. Hox J (2010) Multilevel Analysis: Techniques and Applications, Second Edition. LAWRENCE ERLBAUM ASSOCIATES, London
3. Sutton AJ, Abrams KR., Jones DR, Sheldon TA, Song F (2000) Methods for Meta-Analysis in Medical Research. John Wiley, Baffins Lane, Chichester
4. Stroup D, Berlin J, Morton S, Olkin I, Williamson G, Rennie D, Moher D, Becker B, Sipe T, Thacker S (2000) Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 283 (15):2008-2012
5. Yang M, Li X (2007) Multilevel Model in Medicine and Public Health. Peking University Medical Press, Beijing
6. Morton S, Adams J, Suttorp M, Shekelle P Meta-regression Approaches: What, Why, When, and How? Technical Review 8(Prepared by Southern California–RAND Evidence-based Practice Center, under Contract No 290-97-0001). AHRQ Publication No 04-0033 Rockville, MD: Agency for Healthcare Research and Quality March 2004
7. Cooper H, Hedges LV, Valentine JC (2009) The handbook of research synthesis and meta-analysis. Russell Sage Foundation, New York
8. Cohen J (1988) Statistical power analysis for the behavioral sciences. Second Edition edn. Lawrence Erlbaum Associates, Inc., Hillsdale, New Jersey
9. Services UAT MLwiN Textbook Examples: Multilevel Analysis Techniques and Applications by Joop Hox, Chapter 8: The Multilevel Approach to Meta-Analysis. http://www.ats.ucla.edu/stat/mlwin/examples/ma_hox/chapter8.htm. (accessed Dec 13, 2011)