Annual Assessment Workshop #4 Measurement and Assessment Plans

February 25th and 26th, 2013

  1. Background

A.  Relationship between Annual Assessment and Program Review

1.  Annual assessment as the foundation for program review.

2.  Program review and assessment are required by WASC.

B.  Levels (course, program, institution) of PSLOs – What do you want students to know when they complete a course, major program of study, degree?

Example:

1.  Institutional Level: Graduates from our campus can apply quantitative reasoning to real world

2.  Program Level: Students who complete the Psychology program can use statistical tools to analyze and interpret data from psychological studies.

3.  Course Level: Students who complete this course can calculate and interpret a variety of descriptive and inferential statistics.

  1. Rubrics
  2. How to build your own
  3. Holistic rubric - one global, holistic rating for a product or behavior
  4. Analytic rubric - separate, holistic ratings of specified characteristics of a product or behavior
  5. Steps
  6. Identify what you are assessing (e.g., critical thinking).
  7. Identify the characteristics of what you are assessing (e.g., appropriate use of evidence, recognition of logical fallacies). These are the characteristics.
  8. Describe the best work you could expect using these characteristics.
  9. Describe the minimally acceptable product using these characteristics.
  10. Describe an unacceptable product.
  11. Develop descriptions of intermediate-level products and assign them to intermediate categories.
  12. Develop a scale that runs from 1 to 5 (unacceptable, marginal, acceptable, good, outstanding), 1 to 3 (novice, competent, exemplary), or any other set that is meaningful.
  13. Ask colleagues who were not involved in the rubric's development to apply it to some products or behaviors and revise as needed to eliminate ambiguities.
  14. Rubric Information and Generators
  15. DePaul University Webpage
  16. Rubi Star
  17. iRubric
  18. Grademark
  19. APA style paper submission graded with a rubric
  20. Students upload their papers to turn-it-in
  21. Faculty member enters rubric into grademark and attaches it to the assignment.
  22. Papers are graded and the grades are automatically put into the gradebook.
  23. Downloading the report: http://www.youtube.com/watch?v=jhKcGyysFoA
  1. Pretest/Posttest designs using Moodle (or Scantron)
  2. Develop MC quiz/exam
  3. Moodle – students can complete it on any computer, in class or out of class. Data file is automatically created.
  4. Scantron – feed into scantron machine and data file is automatically created.
  5. Item Analysis is an option
  6. To create a data file for comparing pretest and posttest scores:

·  IDS assists with downloading data files (will work with IDS to simplify this task).

·  Use Excel (or other software such as SPSS) to create scores and compare scores across time

  1. Indirect Measurement of PSLOs
  2. These measures are not direct assessments of student learning but can be useful when trying to understand student performance.
  3. Survey results available on the Institutional Planning and Analysis website: http://www.csusm.edu/ipa/surveys/index.html

·  National Survey of Student Engagement (NSSE) Freshmen Survey (Cooperative Institutional Research Program -CIRP)

·  CSUSM Alumni Survey

·  CSUSM Senior Survey (CSS)

·  CSUSM Graduation Survey

·  Survey of Student Opinion

·  CSUSM Senior Survey (CSS)

·  CSUSM Graduation Survey

·  Survey of Student Opinion

  1. Assessment Schedules
  2. Develop PSLOs for the program.
  3. Develop curriculum map to show which courses address which PSLOs.
  4. Identify which PSLO(s) will be addressed in each year of the cycle. (The first year of Program Review is reserved for the self-study so no annual assessment data is collected that year.)
  5. Which courses address the PSLO and what kinds of assignments do they use? You do NOT have to use a common assignment.
  6. Think about specific evidence that you will collect (e.g., through embedded assessment) that addresses the PSLO.
  7. How will you score it (e.g., rubric, m/c through moodle)?
  8. What will you do with the information?
  1. Templates for the assessment plans (undergraduate and graduate programs) are available on the assessment website http://www.csusm.edu/assessment/2012_2013/12_13_assessment_material.html
  1. 2012-2013 Assessment Report
  1. Maximum of 2 pages – please do not include data
  2. Reflect on what you know about your students’ performance on the PSLOs up to this point.
  3. Reflect on what your program has done with the information it has.
  4. Tell us what additional resources you need (e.g., workshops, books) to complete next year’s assessment.

1