Reports from work completed at Faculty Retreat August 18, 2005

Assessing DMS Report

18- and 33-Credit Checkpoints – Dispositions

Anne Roberts, Terri Swim, Steven Crites, Dorace Smith

1. What does the data tell us about our candidates’ performances?

It shows that the students’ beliefs are higher than their behaviors. While there is not a large difference, there is a consistent difference. The data here shows that students feel they are not getting enough diverse experiences.

Since the scores are rather high (usually on a self-report people rate themselves lower than would an outside reviewer), we suggest that we look at the questions to make sure we are asking the right questions and that there is enough variable within the questions to make sure students are really meeting the standards.

2. What does this data tell us about the assessment assignment and rubric?

We need something in addition to self-report such as

Behavior analysis

Video-taping

3. What, if any, implications are there for program or curriculum changes?

While experience is low, we have already added service learning for diversity. This may not be reflected in the data we were given to analyze.

4. What implications does this have for the UAS of the program(s) in which this assessment is used?

It seems that the questions are conceptual and we suggest instead that students show us through their behavior their dispositions. A more valid response would have students analyze their behavior through video-tape.

5. In not more than 3 sentences, write a summary statement about this report:

Students rate their beliefs slightly higher than their behaviors.

Since scores are almost all on the high side, we may need to consider re-visiting the tools (questions) we use to assess.

We are concerned that since the average score (expect for experience) is right around 4, is there room for improvement (little room for growth over time), are psychological factors at play here (names were on the survey), and are students really meeting the standards?

Assessing Narrative Report

18- and 33-Credit Checkpoints

Anne Roberts, Terri Swim, Steven Crites, Dorace Smith

1. What does the data tell us about our candidates’ performances?

It is limited but a good start to this anecdotal data.

It will give a different perspective than something like the Elem & Sec Dispositions.

It would be nice to add some numbers.

2. What does this data tell us about the assessment assignment and rubric?

We debated whether this is anecdotal or qualitative data.

3. What, if any, implications are there for program or curriculum changes?

This study will help us understand whether or not and how we need to make curricular changes. We will be able to do this once we have more data.

4. What implications does this have for the UAS of the program(s) in which this assessment is used?

How can we get more participation by providing possible incentives?

Interesting pilot data (good start)

5. In not more than 3 sentences, write a summary statement about this report:

Really hard to draw conclusions since there were only 3 participants at each checkpoint.

This is an interesting pilot.

Need stronger incentive to gain participation, such as building it in as a class assignment (but this is questionable).

Aside thoughts:

Let’s have shorter examples (of exemplars) in the Program Guide and UAS Booklet … not about quantity of writing, but quality.

It would help to change the requirements to reducing the number of reflections from 2 to 1 (see top of page 24, Program Guide and UAS Booklet) so that it combines the ideas of scholar and practitioner.