Syllabus
Outline
Department, Number, Title and Credit Hours:
CESP 811: Principles of Measurement and Program Evaluation (3 credit hours)
Semester: Fall 2008 [Thursday from 4:30-6:55]
Location: Lindquist Hall 429B
Faculty Member: Dr. Randy Ellsworth, Professor of Educational Psychology
Office Hours: By appointment
Telephone: 978-6943 (office); 733-5811 (home)
Note: Weather Cancellations -- Call 316-978-6633 (select 2) to obtain information on weather related class cancellations.
FAX: 316-978-3102
E-mail Address:
Catalog Description: This course provides coverage of the trans-disciplinary field of program evaluation including history and current trends, alternative program evaluation models, program evaluation standards, program evaluation procedures,data collection instrument development and interpretation, data analysis, and reporting of evaluation results.
Prerequisites: CESP 701 and CESP 704 or equivalent
Textbook(s) and Related Material:
Lyman, H. B. (1998). Test scores and what they mean. Boston: Allyn and Bacon.
Stufflebeam, D. L, & Shrinkfield, A. J. (2007). Evaluation theory, models, & applications. San Francisco: Jossey-Bass.
Major Topics:
Major Topics / Guiding Principles from the Unit Conceptual FrameworkProfessionalism and Reflection (PR) / Human Development and Diversity (HDD) / Connection of Teaching Experiences and Assessment (CTA) / Technology (T) / Content Knowledge, Pedagogical Content Knowledge, and Alignment with Standards (CKS) / Collaboration (C)
History/current trends in evaluation / X / X / X
Program evaluation standards / X / X / X
Alternative evaluation models / X / X / X
Program evaluation procedures / X / X / X
Assessment technical issues / X / X / X / X
Analysis of data / X / X / X / X
Evaluation report writing / X / X / X / X
Technology Expectations:
Students will be expected to have a basic working knowledge of the Statistical Package for the Social Sciences (SPSS) software. Laptop computers with SPSS software will be used periodically during class. Laptop computers will be made available from the College’s Technology Center. Students will use SPSS to analyze evaluation data during in-class learning times and to complete questions included on examinations.
Program Goals Addressed:
Educational Psychology: The graduate will:
1. select appropriate procedures for analyzing data.
2. write research reports in accordance with professional standards.
3. make decisions that are informed by research.
School Psychology: The graduate will:
1. use research and evaluation to address educational problems.
Course Assignments:
Reading assignments, in- and out-of-class exercises, and examination study guides are designed to provide you with the necessary evaluation knowledge and experiences that will be assessed in this course.
Grading:
Multiple choice and applied item mid-term (tentatively scheduled for October 4) and final exams (scheduled for December 13, 5:40-7:30) will be administered to assess your understanding of the basic learning outcomes identified in this syllabus. In addition, class assignment and projects will be assigned during the course with specific point values awarded for completion and quality of work. Grades will be reported in terms of percent correct and letter grade.
Ultimately your course grade will be a composite of your grades on the mid-term exam (35%), a comprehensive final exam (35%), and completion of class assignments/projects (30%).
Academic Honesty:
A standard of honesty, fairly applied to all students, is essential to a learning environment. Students abridging a standard of honesty must accept the consequences; penalties are assessed by appropriate classroom instructors or other designated people. Serious cases may result in discipline at the College or University level and may result in suspension or dismissal. Dismissal from a college for academic dishonesty constitutes dismissal from the University (see WSU Student Code of Conduct contained in the WSU Graduate Catalog).
Special Needs:
If you have a physical, perceptual, psychiatric/emotional, medical, or learning disability that may impact your ability to carry out assigned course work, contact the Office of Disability Services (ODS), Grace Wilkie Annex, room 173. (Voice/TDD 978-3309). ODS will review your concerns, confirm your disability, and determine, with you, what accommodations are necessary. All information and documentation of your disability is confidential and will not be released by DS without your written permission.
Major Topics:
v Assessment technical issues
o Types of assessments
§ Norm referenced
§ Criterion referenced: (Standards based, Objectives/Outcome based)
§ Objective tests
· Multiple choice
· True-false
· Completion
· Matching
§ Rubrics
§ Interviews
§ Focus groups
§ Observation
§ Surveys
§ Documents and records review
o Reliability
§ Test-retest
§ Alternate form
§ Split-half
§ Internal consistency
§ Error variance
§ Standard error of measurement
§ Factors that influence reliability
o Validity
§ Face
§ Content
§ Empirical/Predictive/Concurrent (or criterion related)
§ Construct
§ Usability
§ Factors that influence Empirical validity
o Derived scores
§ Age/grade equivalent
§ Percentile rank and percentile bands
§ Stanines
§ Standard scores (z, T, and variations)
§ Deviation IQs
§ NCE
§ Test result profiles
o Sources of assessment instruments
§ Publisher test catalogs
§ Mental Measurements Yearbook
§ Web sites
§ Criteria for screening tests
o Testing ethics
§ Who is entitled to test information?
§ How are tests to be administered?
§ Is “coaching” for a test permissible?
v History of evaluation
v Program Evaluation Standards
v Definition of “evaluation”
v Current trends in evaluation
v Evaluation models
v Impact of NCLB on program evaluation
o Experimental design issues
o Best practices/evidence based practices
o What Works Clearinghouse
o Budgeting for evaluation
v Quantitative/qualitative, multi(mixed)-method approaches
v Action based research/evaluation process
v Formative/summative evaluation
v Different conceptions of evaluation
o Standards based
o Objectives oriented
o Culturally Responsive
o Inclusive
o Participatory
o Democratic
o Utilization focused
o Appreciative inquiry
v Program evaluation procedures
o Diagnosis—Identification of:
§ stakeholders
§ program goals
§ key evaluation questions
§ appropriate data sources
§ timelines
§ appropriate reporting procedures
§ appropriate audiences
§ budgetary needs
o Implementation
§ Design of instruments and procedures
· Reliability and validity
· Qualitative/quantitative
§ Data base design
· SPSS, Excel, FileMaker Pro, etc.
§ Data analysis/interpretation
· Coding data
· Selecting appropriate data analysis procedures
· Organizing results for ease of interpretation
§ Reporting Results
· Development of preliminary reports
· Gathering of stakeholder reactions
· Incorporating reactions
· Delivering a final report
Schedule of Classes:
August21 (first class)
28
September
4
11
18
25
October
2 (tentative mid-term exam)
9
16 (Fall Break—No Class)
23
30
November
6
13
20
27 (Thanksgiving Break—No Class)
December
4
11
18 (Final Exam)
1