ECD 560 Lecture Notes Class Four

  1. Internal Validity
  2. Internal Validity – degree to which the researcher has ensured that the conclusions stated follow from the variables studied and no other research factors
  3. These factors are called threats to Internal Validity
  1. Maximize the effects of the independent variable on dependent variable
  2. Experimental & quasi experimental where researcher is able to manipulate independent variable
  3. Independent Groups = comparable Groups
  1. Minimize error factors
  2. Error = any event, characteristic, or situation that is unsystematic and that fluctuates randomly
  3. subject error – due to unpredictable changes in subjects physical or mental states during research participation
  4. fatigue, bored, anxious, loss of attention
  5. maturation – directional changes in subjects’ physical or mental state during research participation
  6. Procedural error – random, unpredictable fluctuations occurring during the administration of Ind. Variable. This becomes critical in longitudinal or repeated treatment designs.
  7. Measurement error – inherent in instruments used to obtain subjects’ scores.
  8. Potential sources – inconsistent rater of behavior, interviewer who is inconsistent in testing protocol, scorer fatigue, unpredictable scorer leniency, BAD TEST.
  9. Recommendations for reducing error.
  10. Pilot Research (dry run, dress rehearsal) – running research procedures to ensure that the variables are administered and recorded consistently.
  11. Develops standardized procedural sequence
  12. Experts have opportunity to observe and give feedback of the researcher implementing the independent variable.
  13. Able to obtain feedback from subjects.
  1. Control extraneous variables
  2. Types of extraneous variables
  3. selection bias – when comparing different groups, it is possible for the groups to differ on a variable known to influence the dependent variable. (snowballing)
  4. Subject bias – participants’ attitudes that systematically slant investigative results in a particular direction.
  5. Hawthorne effect – wanting to look good
  6. Researcher Bias – subtle feedback researcher gives subjects about behavior
  7. History – unexpected event that affects subjects during their research participation
  8. Maturation – systematic change during course of study that has nothing to do with variables under study.
  9. Testing Effects – by being exposed to testing materials initially it affects the participants’ subsequent measurements
  10. Mortality – subjects do not complete study
  11. Instrumentation – systematic lack of standardization or procedures that are unbalanced
  12. Regression – regression toward the mean – outliers tend to become means over time
  1. Factors influencing variables – allows to predict when extraneous variables would be more pronounced and what steps to take to control for extraneous variables.
  2. Research Type – may help to reduce extraneous variables or help to pronounce effects
  3. Correlational
  4. Quasi-experimental
  5. Experimental
  6. Research Method (procedures).
  1. Controlling or assessing extraneous variable
  2. matching – match subjects in groups on some variable known to be related to the dependent measure.
  3. Prescreen to obtain data
  4. Using subject status on variable as randomizer
  5. blocking – purposefully adds a dependent variable so that its effect can be assessed.
  6. Statistical control – statistical analysis that takes into account variance and error.
  7. Placebo Control – increases emphasis that treatment is effective
  8. Unobtrusive measurement – allows for genuine responses.
  9. subject blind method – subjects do not know what they are being observed on
  10. researcher blind – data collectors are unaware of research hypothesis
  11. Postinvestigative inquiry - qualitative assessment
  12. standardized conditions
  13. counterbalance measures – different assessments used to get same info.
  1. External Validity
  1. Relationship between Internal and External Validity
  2. Internal validity – confidence in the integrity of results of the study
  3. External validity – generalizability
  4. 2 aspects
  5. the appropriateness of the generalization from the study to the target population, setting, and operations
  6. the identification of other populations, settings, and operations to which these results can be generalized.
  1. Population External Validity – population of subjects applicable to the study
  2. Accessible population – one that is readily available who are willing to participate in the study and are at the right place at the right time
  3. Volunteers
  4. Location
  5. Target population – the population to which one wants to generalize
  1. Ecological External Validity – the settings in which the study’s results would occur
  2. Describe the setting of the research precisely while making inferences as to other settings in which the researcher would expect to obtain the same results.
  3. Researcher effect
  4. Novelty & disruption
  5. Interaction of setting and treatment effects
  1. External Validity of Operations – whether another researcher could obtain the same results using similar operational definitions of the independent and dependent variables.
  2. Measurement of dependent variable
  3. Pretest sensitization
  4. Posttest sensitization
  5. Measurement of the independent variable
  6. Posttest administration
  7. Multiple treatment Interference
  1. Methods for increasing external validity – best way repeat the design with different subjects
  2. Researcher can:
  3. Randomize subjects
  4. Purposeful sampling
  5. Choose appropriate research design
  6. researcher expectancy design – expanded treatment comparison design
  7. single organism design
  8. factorial design
  9. multiple dependent variables
  10. consumer can:
  11. replication
  12. meta-analysis
  13. knowledge = careful consumer
  1. 5 types of descriptive research (Qualitative) – relates, portrays, and recounts
    variables the way they are. A snapshot of the variables.
  1. case study – in-depth, intensive study of a particular unit, such as an in-depth description of history, symptoms, or treatments. When used with larger units this research is called ethnographic observation or inquiry.
  2. Advantages
  3. development of preliminary hypotheses for further explanation
  4. Potential to capitalize on natural contexts
  5. Depth of description
  6. Focus on idiographic inquiry
  1. Disadvantages
  2. Due to idiographic nature of research cannot generalize
  3. Strong possibility of researcher bias.
  4. Recommendations
  5. multiple observers
  6. reliable measurement – interrater reliability
  7. replicate if at all possible
  1. survey research – describes some variable by indicating its frequency in a particular population. Also called epidemiological, prevalence, and incidence research.
  2. Advantages
  3. Large amount of information that can be collected from many participants at the same time.
  4. potential for sampling from large, arbitrary populations
  5. can be of great practical utility
  6. Disadvantages
  7. potential bias in responses since not a standard measure
  8. return rates – what does it say about those who returned the survey and those who didn’t?
  9. Recommendations
  10. develop clear description of the arbitrary population and sampling procedures
  11. use preexisting survey instruments
  12. include distracter items not relevant to study
  13. alternating positive and negative statements
  14. pilot study
  1. correlational research – examines the way variables co-vary or co-relate.
  2. Advantages
  3. allows investigator to analyze simultaneously how many variables co-relate
  4. useful with vast array of research questions
  5. Disadvantages
  6. Correlation does not imply causation
  7. the problem is correlation does not explain all exceptions
  8. lack of control
  9. important to distinguish b/w strength of the correlation and statistical significance.
  10. Recommendations
  11. use high quality measuring tools if at all possible
  12. avoid truncation – limiting the range of scores
  13. random sampling of unrestricted population
  1. developmental research – typically include time as a factor of research as independent variable. Looking for changes or effects of change over time.
  2. 3 types
  3. longitudinal
  4. cross-sectional
  5. cross-sequential
  6. Advantages
  7. sampling behavior at different times
  8. longitudinal – comparability between the groups across time should be perfect (same groups)
  9. cross-sectional – allows to sample comparable groups at different points
  10. cross-sequential – compromise between two disadvantages and advantages both reduced from strengths of other two.
  11. Disadvantages
  12. longitudinal – time consuming in collecting data, longer the time frame – the more susceptible to threats to internal validity arise (esp. attrition & history)
  13. cross- sectional – comparability of groups must be controlled for and even then results must be viewed cautiously
  14. Recommendations
  15. make decisions on length of observations based upon understanding of developmental theory
  16. need to explicitly indicate how threats occur and how much of a concern it represents
  17. must ensure comparability of groups to best of ability
  1. Making Sense of Results: Analyzing Research Data
  1. General Research Issues and Definitions
  2. Descriptive Statistics – used to describe the sample and to communicate the nature of the sample to the reader
  3. Measures of central tendency – mode, median, and mean
  4. Measures of variability (how different the scores are from the average) – range, interquartile range, standard deviation
  5. Measures of Skewness – normal distribution, negatively skewed distribution, positively skewed distribution
  6. Measures of relationships – Correlation
  1. Inferential Statistics – deal with the process whereby statements are made about the population of interest
  2. Predictive in nature and require statistical analysis with established confidence intervals.
  1. ex post facto research – study of variables that have not been manipulated by researcher. Variables still seen as independent variables though only manipulation is choice of inclusion in study.
  2. Advantages
  3. can examine multiple variables at the same time
  4. useful as precursor to experimental study
  5. Disadvantages
  6. independent variable is still an attribute and observations are simply associations and no comments on causation can be made
  7. represents comparison of nonequivalent groups which is a confound
  8. Recommendations
  9. guard against conclusions that go beyond the data
  10. base design on existing research and use this as guide to formulate clear research questions
  11. need to operationally define levels between variables
  12. use random sampling from a population
  13. identify potential confounds and attempt to control for them