Questions from Jim Sauerberg presented to Senate, December 6, 2017

And

Responses from Test Optional Task Force, January 12, 2018 for Senate meeting January 17, 2018

Question 1

What is the upside? That is, what advantage do we expect from becoming test-optional? Are there students from under-represented backgrounds who are currently excluded from SMC because we are test-mandatory who will soon attend SMC? We are already at an 80% acceptance rate; won't this just increase that?

Response:

A common criticism of the SAT is that it is culturally biased, and that a focus on test scores deters otherwise qualified minority, low-income, first-generation, female and other students from applying.

De-emphasizing tests in the admissions review process will support a more SMC mission aligned practice. Additionally, it allows for a more holistic review process where other aspects of the applicant can be considered that may more directly contribute to academic success, social acclimation and contribution to the SMC community.

Colleges that have made the SAT optional report that their applicant pools are more diverse and that there has been no drop in academic quality. A majority have reported an increase in applications which led to an increase in selectivity.

Question 2a

I am a bit suspicious of the claim that we can be test-optional for admission but test-semi-mandatory for placement. If the purpose of going test-optional is to attract students who don't wish to take a standardized test, telling them later 'please take a standardized test' is cross-messaging and likely supplies us with information of dubious quality.

Response:

Information from other collegiate institutions that have implemented test optional admissions policies indicates that the majority of applicants will actually submit test scores at the time of application if this is made an option. Subsequent to enrollment, institutions have found that few applicants lack test scores and are willing to share them after an offer of admission has been made if they are determined to enroll. The incentive (impetus) to their forwarding them is to be freed from a requirement for a placement examination. The task force discussed that were ACT or SAT scores not required at the time of admission that this would present little additional work for the departments provided that students were informed they were needed for advisement purposes.

Question 2b

I can be convinced that _overall_ GPA is strongly correlated with _overall_ SMC success. Is it as clear that [level of HS Math completion and grades in those final courses] is strongly correlated with [success in SMC math courses]? If so, which HS courses, which SMC courses, and how strongly is the correlation?

Response:

Investigating the relationship between various predictors (including with existing data the relationship with SAT-Math) will be a central part of the validity studies conducted by the campus (Institutional Research in particular) over the next several years.

Question 2c

Reducing the emphasis on SAT/ACT Math scores in favor of HS classes & scores adds work to the mathematics placement process (one that several other departments depend on). Is it clear where the additional human-power will come from?

Response:

The expectation is that campus resources (Admissions; Institutional Research) will provide support in identifying and making available the needed measures here.

Question 3

re: the 2008-12 data

Question 3a

The 2008 & 2009, pre-Great Recession students classes were quite different than the 2011 & 2012 CalState Shrinkage classes. Is there concern with lumping them together?

Response:

A new analysis was run, dividing the data into the 2008-2009 cohorts and the 2010-2012 cohorts. The two new results strongly replicate the results from the larger dataset.

2008-2009
SAT quartile
1 / 2 / 3 / 4
GPA / 1 / 44% / 45% / 41% / 51% / 44%
quartile / 2 / 63% / 67% / 63% / 59% / 64%
3 / 66% / 66% / 85% / 61% / 70%
4 / 67% / 86% / 77% / 83% / 80%
56% / 59% / 65% / 68% / 62%
2010-2012
SAT quartile
1 / 2 / 3 / 4
GPA / 1 / 47% / 45% / 51% / 37% / 46%
quartile / 2 / 63% / 58% / 62% / 55% / 60%
3 / 59% / 70% / 65% / 70% / 67%
4 / 79% / 71% / 75% / 74% / 74%
Total / 57% / 60% / 64% / 65% / 62%

Question 3b

Gregg's Table 2 covers 3000+ students. How many students are in each of the 16 bins? I'd like to know if there are enough outliers ("Highest GPA but worst SAT" and "Lowest GPA but best SAT") to be really meaningful.

Response:

The extreme corners do have fewer students than the other bins, but there is a degree of representation. And there is significant representation in the bins adjoining the extreme corners. Here is the bin distribution:

SAT quartile
1 / 2 / 3 / 4
GPA / 1 / 413 / 286 / 186 / 84
quartile / 2 / 232 / 233 / 180 / 115
3 / 134 / 203 / 179 / 180
4 / 66 / 101 / 178 / 307

Question 3c

The conclusion seems to be, "Of the students who apply, are admitted, and attend SMC, GPA is a much stronger predictor of success than SAT. Therefore, of all students who might consider SMC, GPA will be a satisfactory predictor of success." Are we ok with treating this conditional probability as probability?

Response:

No, this is not the conclusion. The conclusion is that once HSGPA is taken into account, SAT adds very little, if anything, to the prediction of success. Therefore, the SAT should be omitted from admissions decisions and be replaced with other more valid predictors of success, e.g., achievement in critical high school courses, with the result being increased overall prediction of success.

Question 3d

Do we have such data for the STEM areas?

Response:

For STEM areas the more relevant analysis might be to examine the joint relationship of HSGPA and SAT-MATH score on student success (four year graduation). Initial results for all new 2008 and 2009 freshmen indicating an intended major in a STEM area appears to replicate the overall findings, that is, there is a consistent relationship of HSGPA with graduation within each SAT-Math quartile.

SAT-MATH QUARTILE
1 / 2 / 3 / 4
HSGPA
QUARTILE / 1 / 47% / 27% / 46% / 30%
2 / 54% / 73% / 33% / 78%
3 / 62% / 60% / 65% / 50%
4 / 86% / 90% / 82% / 80%