STAT 460Lecture 15: Review10/27/2004
1) Goal: Draw valid conclusions (with a degree of uncertainty) about the relationship between explanatory variable(s) and an outcome variable in the face of the limited experimental resources and the randomness characteristic of the world.
2) One detailed example: Experiment to study the effects of a “teamwork”vs. a “technical” intervention on productivity improvement in small manufacturing firms.
3) Principles of Statistics
a) A statistical view of the world
The Population A Sample
b) Hypothesis testing
i) ANOVA: Let μi for i in 1 to k be the population mean for the outcome for group i. H0: μ1=…=μk vs. H1: at least one pop. mean differs
ii) Regression: Let β0 be the population value of the intercept and β1 be the population value of the slope of a line representing the mean outcome for a range of values of explanatory variable 1. H0: β0=0 vs. H1: β0≠0 H0: β1=0 vs. H1: β1≠0 etc.
iii) ANOVA decision rule: reject H0 if Fexperimental>Fcritical equiv. to p<α
iv) Regression decision rules: reject H0 if |t|>tcritical equiv. to p<α
v) If we choose α=0.05, then when the null hypothesis is true we will falsely reject the null hypothesis (make a type 1 error) only 5% of the time. (This only guarantees that we will correctly reject the null at least 5% of the time when the null is false.)
c) Characteristics of problems we can deal with so far:
i) Quantitative outcome variables and a categorical explanatory variable (ANOVA)
ii) Quantitative outcome variable (regression) or one of each (ANCOVA form of regression)
iii) We require the following assumptions to make a specific model that we can analyze:
(1) the outcome is normally distributed with the same variance at each set of explanatory variable values;
(2) the subjects (actually, errors) are independent;
(3) the explanatory variables can be measured with reasonable accuracy and precision.
(4) For regression, the relationship between quantitative explanatory variables and the outcome is linear on some scale.
(5) For one-way ANOVA, the k population means can have any values, i.e. there is no set pattern of relationship between the outcome and the explanatory variables.
d) Examples of data we have not learned to deal with:
e) Experiments vs. observational studies
i) In experiments, the treatment assignments are controlled by the experimenter. Randomization balances confounding in experiments.
ii) In randomized experiments, association can be interpreted as causation.
iii) In observations studies, causation can be in either direction or due to a third variable.
f) The assumptions (fixed x, independent errors, normality with the same variance (σ2) for any set of explanatory variables, plus linearity for regression models) are needed to calculate a “null sampling distribution” for any statistic.
i) This tells the frequency distribution of that statistic over repeated samples when the null hypothesis is true, thus allowing calculation of the p-value.
ii) The alternate sampling distributions require more information and more difficult calculations.
iii) Note that we do not require normality or equal variance for explanatory variables.
g) The p-value is the probability of getting a sampling statistic (e.g. ANOVA F or regression t) that is as extreme or more extreme that the one we got for this experiment if the null distribution were true.
h) The one-way ANOVA F value has a particularly nice interpretation. MSbetween in the numerator estimates σ2 from the model if the null hypothesis is true, and something bigger otherwise. MSwithin in the denominator estimates the σ2 regardless of whether the null hypothesis is true or not. So F is around 1 when the null hypothesis is true and bigger otherwise.
i) Degrees of freedom count the amount of information in a statistic by subtracting the number of “constraints” from the number of component numbers in the statistic. Use df to check that an analysis was set up correctly or to obtain certain information about an analysis from statistical output. E.g. the df in MSbetween is k-1 (one less than the number of groups being compared).
4) Experimental Design concepts
a) Generalizability
i) Assure that the population from which samples of subjects could be drawn is not too restricted. Assure that treatment and environmental conditions are not too restrictive.
ii) Be more and more careful in stating conclusion as they apply to populations less and less restrictive than where the sample came from.
iii) Balance against increased control for increased power.
b) Power is the chance that the null hypothesis will be rejected for some specific alternate hypothesis. In one-way ANOVA, e.g., power is increased by increasing the F statistic, .
i) Don’t study small effects
ii) Decrease σ2 by decreasing subject, treatment, environmental and measurement variation
iii) Increase sample size
c) Interpretability: don’t make alternate explanations easy to defend
i) Use a control
ii) Use blinding of subject and/or experimenter
iii) Use randomization to prevent confounding
5) EDA is used to find mistakes, become familiar with data and coding, anticipate results, and roughly check for appropriate models.
a) All models: check descriptive statistics for quantitative variables and frequency tables for categorical variables.
b) ANOVA: side-by-side boxplots of outcome by the categorical explanatory variable.
c) Regression: scatterplot with outcome on the y-axis.
d) ANCOVA: scatterplot with separate symbols for each group.
6) Analysis details
a) ANOVA
Response: Productivity
Treatment: Control (0), Teamwork (1), Technical (2)
The MEANS Procedure
Analysis Variable : Productivity
N
Treatment Obs Mean Std Dev N Minimum Maximum
ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ
0 30 0.5533333 0.2823709 30 -0.0500000 0.9900000
1 35 0.6237143 0.3365402 35 -0.1100000 1.2200000
2 25 1.0324000 0.2769067 25 0.4400000 1.4800000
ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ
The ANOVA Procedure
Class Level Information
Class Levels Values
Treatment 3 0 1 2
Number of observations 90
The ANOVA Procedure
Dependent Variable: Productivity
Sum of
Source DF Squares Mean Square F Value Pr > F
Model 2 3.59417575 1.79708787 19.54 <.0001
Error 87 8.00333981 0.09199241
Corrected Total 89 11.59751556
R-Square Coeff Var Root MSE prtivity Mean
0.309909 42.49257 0.303303 0.713778
Source DF Anova SS Mean Square F Value Pr > F
Treatment 2 3.59417575 1.79708787 19.54 <.0001
ANOVA
Productivity
b) Regression and ANCOVA
The REG Procedure
Model: MODEL1
Dependent Variable: Productivity
Analysis of Variance
Sum of Mean
Source DF Squares Square F Value Pr > F
Model 5 4.66103 0.93221 11.29 <.0001
Error 84 6.93649 0.08258
Corrected Total 89 11.59752
Root MSE 0.28736 R-Square 0.4019
Dependent Mean 0.71378 Adj R-Sq 0.3663
Coeff Var 40.25939
Parameter Estimates
Parameter Standard Standardized
Variable Label DF Estimate Error t Value Pr > |t| Estimate
Intercept Intercept 1 0.79282 0.09866 8.04 <.0001 0
EmployeeCount 1 -0.00578 0.00202 -2.87 0.0052 0.58066
Teamwork Teamwork 1 0.01054 0.14145 0.07 0.9408 0.01431
TechSkill TechSkill 1 0.17951 0.13708 1.31 0.1939 0.22398
TeamEmployee 1 0.00300 0.00244 1.23 0.2220 0.32180
TechEmployee 1 0.00686 0.00243 2.82 0.0060 0.63719
Parameter Estimates
Variable Label DF 95% Confidence Limits
Intercept Intercept 1 0.59662 0.98902
EmployeeCount 1 -0.00978 -0.00177
Teamwork Teamwork 1 -0.27075 0.29183
TechSkill TechSkill 1 -0.09309 0.45211
TeamEmployee 1 -0.00185 0.00785
TechEmployee 1 0.00202 0.01169
7) Residual analysis
8) Transformation and Interpretation
Coefficients(a)
Model / Unstandardized Coefficients / Standardized Coefficients / t / Sig.B / Std. Error / Beta
1 / (Constant) / 1.417 / .284 / 4.989 / .000
Teamwork / -.035 / .409 / -.047 / -.085 / .932
Technical Skills / -.618 / .388 / -.772 / -1.593 / .115
Teamwork/Log10(Employee Count) interaction / .126 / .248 / .303 / .509 / .612
tech/Log10(Employee Count) interaction / .710 / .243 / 1.471 / 2.919 / .005
Log10(Employee Count) / -.564 / .182 / -.495 / -3.091 / .003
a Dependent Variable: Productivity
1