Appendices

Methods

Table. Generalizability (G) Study ofTeamwork, NP Autonomy and Independent

Practice, and NP-Administration Relations Scales

Source / Teamwork / Autonomy
and Independent Practice / NP-Administration Relations
Estimate of Variance Component
NP / 0.206 / 0.189 / 0.365
Practice / 0.012 / 0.000 / 0.055
Practice(NP) / 0.000 / 0.000 / 0.000
Item / 0.022 / 0.017 / 0.071
Item*NP / 0.160 / 0.186 / 0.214
Practice(Item*NP) / 0.000 / 0.000 / 0.000
Residual / 0.000 / 0.000 / 0.035
Universe Score Variance Component / 0.206 / 0.189 / 0.365
Relative Error Variance Component / 0.032 / 0.037 / 0.024
Absolute Error Variance Component / 0.036 / 0.041 / 0.032
Generalizability Coefficients
G Coefficient (Ep2) / 0.865 / 0.835 / 0.939
Phi Coefficient(Φ) / 0.850 / 0.822 / 0.919
Note. This table presents the estimates of variance component from different sources and the generalizability coefficients. The relative generalizability coefficient was calculated as follows: universe score variance component/(universe score variance component+relative error variance component). The absolute generalizability coefficient was calculated as following: universe score variance component/(universe score variance component+absolute error variance component). (1, 2)

Multilevel Modeling

A procedure for fitting multilevel linear models in SAS 9.3, PROC MIXED (3), was conducted to build the models. Covariates included NPdemographics (age, sex, race, education) and work characteristics, including years of experience, average work hours, main practice site (physician’s office, community health center, hospital-based clinic, or other), number of NPs in the practice site, location of the practice site (urban or non-urban), and NPs’ patient panel information. Those with p-values < 0.20(4) were entered into the multilevel linear regression model to account for the hierarchical design of the data where 314 NPs (Level-1) were nested in 163 organizations (Level-2). The main predictors in the models were the organizational-level AIP and NP-AR scores, which were included as Level-2 variables. Covariate variables measuring NP demographicsand work characteristics were Level-1 measures, and organizational-level characteristics were Level-2 measures.Since Level-2 contextual effects were of substantive interest, centering at grand mean was applied to both Level-1 and Level-2 continuous variables in the multilevel linear regression model (5). Plots for examining the pattern of interaction effect between Level-1 and Level-2 predictors were drawn to help determine if interaction effect should be included in the final model. We used scatterplots (Figures 1A-1F) to check interaction effect between each Level-1 covariate and Level-2 predictor, and results demonstrate there is no strong evidence to include any of the interaction terms in the final model. The decision to include or not to include a random component of a Level-1 covariate in the final model was made based on the differences in Akaike's Information Criterion (AIC): smaller AIC values indicate better fit. Note that when these values are negative, lower numbers in absolute values are preferred (3).

Figures

Figure 1. Scatterplot of Level-2 Predictors versus Outcome by Level-1 Covariate

1A.

1B.

1C.

1D.

1E.

1F.

References

1.Cronbach LJ, Gleser GC, Nanda H, Rajaratnam N. The Dependability of Behavioral Measurements. New York, NY: Wiley; 1972.

2.Burns KJ. Beyond classical reliability: Using generalizability theory to assess dependability. Res Nurs Health. 1998;21(1):83-90.

3. Singer J. Using SAS PROC MIXED to Fit Multilevel Models, Hierarchical Models, and

Individual Growth Models. J Educ Behav Stat. 1998;23(4):323-55.

4.Hosmer DW, Lemeshow S. Applied logistic regression. 2nd ed. New York: John Wiley & Sons, Inc.; 2000.

5.Enders CK, Tofighi D. Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychol Methods. 2007;12(2):121-38.