Conclusions from ACES Study

Conclusions from ACES Study

These conclusions are based on two ACES studies submitted during October 2005 based on placement data from Fall 2003 and Fall 2004. These findings were presented to the Mathematics Department Assessment Committee in January 2006.

Inability of SAT to predict success in Math classes:

In MAT 120, Fall 03, average SAT Math score of students who got A’s was identical to average of those who got F’s.

During Fall 03, students in High College courses (MAT 108, 124, and 225) who got C’s had an SAT Math score that was on average 9 points higher than those students who got B’s.

During Fall 04, students in MAT 120 that received a grade of C, D, or F had average SAT Math scores that were nearly identical (438, 437, and 434, respectively).

Similarly, students in HighCollege math courses who got C’s, D’s or F’s had average SAT Math scores that were nearly identical (523, 522, and 521, respectively).

In MAT 181/211, Fall 04, average SAT Math score of students who got B’s and C’s was lower than the average of those who got F’s.

There is a very weak statistical relationship between a student’s SAT Math score and their eventual grade in the course they are placed into.

Placement Criteria

Based on the submitted data, the ACES report calculates a “probability of success” based on SAT Math, and when available, Accuplacer and a “composite” score based on both. For example, according to the report, based on Fall 2003 data, a student with an SAT Math score of 408 or higher has an 50% chance of success in a HighCollege math class.

The study recommends making a placement policy of placing into a course only those students who have a 50% chance of success or higher in the course, based on the model they have calculated.

For example, for HighCollege math classes during Fall 2003, if we use the recommended cut-off for SAT Math scores (that is, place students into MAT 120 if their SAT Math score is 408 or above), then 77% of the students would have been correctly placed. However, looking at the Accuplacer data for these class, there is actually a negative correlation. This means that according to the data, students who scored higher on the Accuplacer would be predicted to do worse in these classes than students who scored lower.

All of this data suggests that the SAT Math score is a poor predictor of student success, and Accuplacer is only marginally better.

Options

Several options present themselves based on these findings.

• Require all students to take the Accuplacer test. One reason why the Accuplacer data may not have been a good predictor of student success is that not all students are required to take it.
• Design a new placement test local to Shippensburg that ideally would reflect the kinds of knowledge we expect students to have going into various levels of math classes.
• Look for a new source of placement testing. One option would be a math placement assessment designed by the Mathematical Association of America (MAA) and administered electronically using the Maple TA software (which we already own).