Assessment Plans
Student Affairs
Compiled by Lori E. Varlotta,
Vice President for Student Affairs
Fall 2008
Throughout the late 1990s and early 2000s the assessment program within Sacramento State’s Division of Student Affairs, for those departments that had initiated one, focused primarily on student satisfaction and program improvement. Realizing that the former foci were a bit askew from the student learning emphasis that was taking center stage in the aftermath of Learning Reconsidered, the Vice President for Student Affairs redirected the Division’s Assessment Program. The move—from a student satisfaction/program improvement program to a student learning outcome-based program—began in October 2005. Two years later, changes are afoot. Though no “final destination” has been reached, significant progress has been made.
Within the first year of implementation, each Student Affairs director was first charged with explicitly aligning his/her departmental mission with those of the Division and the University. Next, directors were asked to identify the two to three overarching planning goals that would broadly frame their work during the upcoming years. Finally, directors were expected to articulate at least one significant student learning outcome that they would like students who participate in their programs or utilize their services to achieve.
Directors were charged, during year-two of implementation, with developing instruments and collecting data to measure the learning that occurred. Working with the Vice President for Student Affairs and/or staff within the Office of Institutional Research, the directors designed or borrowed instrument(s) that presumably could measure the student learning outcomes associated with their respective programs or services. As expected, in phase-one of data collection, some instruments and assessment approaches proved to be more reliable than others. Despite the necessary revisions that needed to be made on several pre and post tests, observed competency exercises, and emerging rubrics, the leadership team and the directors celebrated the fact that within a two-year timeframe each department had laid the rudimentary foundation for evidence-based decision making and outcome-based assessment.
At Sacramento State’s Division of Student Affairs, we realize we have much student-learning- assessment-ground yet to cover. Still, we take great pride in the direction we have charted for ourselves, as committed Student Affairs professionals, and in the outcomes our students are likely to achieve.
The following pages detail the emerging assessment plan that each Department is formulating. If you have general comments or questions about the document, please send them to . If you have specific questions about the outcomes associated with a certain program area, please contact the Director identified as the point person for that particular department.
Academic Advising Center
2008-09 Assessment Plan
Mission Statement
Note: Departmental mission must be directly aligned with those of the University and the Division. This statement should include approximately 3-5 sentences that identify the name of the department, its primary functions, modes of delivery and target audience.
Mission: The Academic Advising Center offers new student orientation, mandatory freshman advising, and advising on General Education and graduation requirements. The Center engages students in a developmental process which helps clarify and implement individual educational plans consistent with their skills, interests, and values. Through individual appointments, group advising sessions and presentations, the professional staff, faculty advisors, and student interns help students understand the university’s academic requirements as well as its policies and procedures. As a result, students are better prepared to make informed decisions about their education and persist towards a timely graduation.
Planning Goals
Note: Planning Goals are broad statements that describe the overarching, long-range intentions of an administrative unit. Goals are used primarily for general planning, as the starting point for the development and refinement of program objectives or student learning outcomes. (UCF Administrative Handbook, University of Central Florida).
Goal 1: Help students make informed decisions about their education and persist toward a timely graduation.
Goal 2: Provide comprehensive first year advising through a three-phase program
Program Objectives or Student Learning Outcomes
Note: The Objectives or Outcomes can be one of two types: program objectives or student learning outcomes. The former are related to program improvement around issues like timeliness, efficiency and participant satisfaction. The latter addresses what a student learns or how a student changes by participating in the program or utilizing the service. Both program objectives and student learning outcomes are measurable statements that provide evidence as to how well you are reaching your goals.
Student Learning Outcome 1
Students will retain key information between Phase I and Phase II of the First Year Advising Program. At the end of Phase II, students will be expected to correctly answer at least 85% of targeted questions that they previously answered at the end of their freshman orientation experience (Phase I).
Rationale: For the past two academic years, more than 85% of new students participating in the initial two phases of the First Year Advising Program (Phase I Freshman Orientation and Phase II First Semester Advising), demonstrated increased understanding of the University’s General Education requirements, academic standards, and resources available to assist them (Appendix A). Based on two academic years of positive results, the Advising Center Director became interested in determining how well information acquired during Phase I was retained by freshmen through their first semester. To test student learning retention, the Orientation Leaders (at Orientation, Phase I) and First Year Advisors (fall semester, Phase II) administered post-tests approximately four months apart, to determine how well freshmen were able to demonstrate retention of the information they were provided.
This student learning outcome supports baccalaureate learning goals Number 4- Information Competence (Phase I); Number 2- Analysis and Problem Solving (Phase II); and Number 1- Competence in the Discipline (Phase III).
Measures
Note: Measures describe the methodology and timeframe for data collection. Measures also should identify the population being surveyed and/or tested. Provide materials such as survey instruments, check lists, focus group protocols, etc. in an appendix.
During Phase I, Orientation Leaders administer a pre-test to a random sampling of students at the beginning of each Orientation day. There are 10 multiple choice questions on the test.The percentage of correct responses on this pre-test helps the Orientation Coordinator determine what information to highlight during the day. Immediately after Orientation, all students are given a post-test to determine whether knowledge has increased during the day by analyzing pre- and post-test results.
For the past two academic years, new students have correctly answered at least 85% of questions on the post-test for the past two academic years (Student Affairs Assessment Plans for 2007/2008 and 2008/2009)
During October and November of the Fall 2008 term, the Academic Advising Center required new freshmen to attend an academic advising session prior to registration for the spring term (Phase II of the First Year Advising Program). At the end of the advising session, students completed a post-test which included five important questions that were previously asked in the Orientation post-test. The post-advising test contains 10 multiple choice questions. The advisors expected at least 85% of freshmen to be able to correctly answer the same questions on both post-tests.
Results
Note: Results include a brief narrative of findings, and/or essential tables or graphs. The results should indicate the extent to which the program objective or student learning outcome was met.
Staff focused their analysis on the five questions that were the same for both the Phase I and Phase II post-tests (see Appendix A for comparative results). Question numbers and the corresponding percentage for students who correctly answered the questions are indicated below:
Phase I Post-Test (Orientation) / % Students who Answered Correctly / Phase II (First Term Advising Session) / % Students who Answered CorrectlyQ2 / 98% / Q3 / 92%
Q3 / 88% / Q7 / 80%
Q5 / 96% / Q10 / 92%
Q7 / 94% / Q8 / 98%
Q9 / 77% / Q4 / 94%
Students correctly answered questions on four out of the five subject areas more than 90% of the time (see Appendix B for complete Phase II post-test results). For one subject area, however, students did not meet the 85% learning objective threshold after their Phase II advising session. The number of students who correctly answered this subject area (identifying the minimum GPA to remain in good academic standing) declined from 88% after Orientation to 80% after Phase II advising.
Conclusions
Note: The conclusion should summarize briefly the collection and analyses of data. It should also “close the loop” by identifying what decisions and/or program modifications were made on the basis of these analyses.
Based on these results, orientation and advising staff concluded that students who participated in both Phase I and Phase II activities did retain important academic and support services information. In response to the lower performance on Question Three, staff will review information regarding minimum GPA that is provided during the Phase II Advising Session. Both assessment methods and instruments will be re-evaluated for potential improvements in the coming year.
Student Learning Outcome 2
After participating in an intrusive advising process that helps students improve their learning skills and retain information, undeclared students who have been reinstated after academic disqualification will score at least 80% on a post-test that is completed 3 months following the workshop.
Rationale: Undeclared students who have been reinstated after academic disqualification are much less likely to persist and graduate than other students. The Academic Advising staff has reported that this group of students historically needs additional assistance to develop critical learning skills to help them succeed academically.
This student learning outcome supports baccalaureate learning goals Number 4- Information Competence; and Number 2- Analysis and Problem Solving
Measures
Advising staff sent an invitation to undeclared students who had been academically disqualified at the end of Fall 2008 to participate in an advising session. During this session, advisors discussed important academic requirements and service information with students. Using Student Voice technology, the Academic Advising Center staff administered a learning strategies web-based post-test for participants in this program. The post-test is comprised of 6 multiple-choice questions (Appendix C) and staff e-mailed the link for the post-test to the student after they participated in the advising session.
Results
Of the 40 students that participated in their advising appointment during the Fall 2008 term, 24 completed the post-test in Spring 2009. Even though advisors covered information related to the content of the post-test, in the 3 – 4 months that passed between orientation and their first advising session, students only answered one question (number 5) correctly more than 80% of the time (see Appendix C for assessment results).
Conclusions
This assessment data was first collected in Spring 2009. The assessment results will be evaluated based on student ongoing progress and retention (measured after Fall census September 28, 2009). As such, the assessment will continue for 2009 and 2010. Using the results of this assessment, Advising Center staff will re-evaluate their advising sessions to determine how key information is communicated to this population and develop a consistent approach, including additional tools to help both students and advisors.
On a related note, the Advising Center Director also plans to review potential strategies for disqualified students who are reinstated to help ensure that this population is advised for more than one semester, either by Academic Advising staff or their major department, until the student has attained good academic standing and is better positioned for success. Assessment of student learning for reinstated students will be the topic of a future student learning objective.
Program Objective 1
To develop and implement a new intrusive advising program for second year students who are placed on academic probation by Spring 2009.
Rationale: After extensive analysis during the Spring 2008 term, the University’s Retention Working Team determined that students on academic probation had much lower first year retention rates compared to peers who maintained good academic standing (49% vs 89% respectively). On average, 22% of first time freshmen were placed on academic probation by the end of their first year. Students on academic probation had much lower six year graduation rates compared to their peers who maintained good academic standing (13% vs. 48%). Based on this information, the Vice President for Student Affairs and the University Provost asked the staff in the Academic Advising Center to develop and implement a new advising program for these students to help increase their retention and graduation rates.
Measures
Since graduation rates will not be available until several years after the implementation of the program, the Advising Center Director decided to initially collect year-to-year retention rates for students who participate in the new advising program. The Director will compare retention rates for students who participate in the program during 2008/2009 with previous years to identify any changes that may be attributable, all or in part, to the second year advising program.
Results
The Academic Advising Office developed the new second year advising program for students on academic probation during late summer and early Fall 2009, and four new Academic Advising professionals were recruited, hired and trained. A one-page summary of this program is provided in Appendix D.
For 2007/2008 new entering freshmen, advising staff identified 520 students who were placed on academic probation after their first year. In early Spring 2009, advising staff invited these students to an advising appointment. If students did not come in for advising after two invitations, the advising staff placed holds on their academic records to prevent them from being able to register for Fall 2009. Of the 520 students invited to participate, 277 (53%) attended their advising session. Advising staff removed registration holds after each student’s session. Registration holds for the remaining 243 students will be removed only after these students come in for advising in the future.
Initial results for the program will be available after the Fall 2009 Census Date when retention rates are calculated.
Conclusions
In Spring 2010, after the first full-year of implementation, the Director will review the program with staff and students to determine if modifications will be needed. A student learning outcome assessment will be developed for this program in 2009/2010.
Program Objective 2
The first-year retention rate for Fall 2007 new freshmen who participate in the First Year Advising program will be at or above 80%
Rationale: This is the third year of the three phase First Year Advising Program: Orientation (Phase I), fall advising (Phase II), and spring advising (Phase III). Students who participated in the first two years of this program exhibit an 11% higher persistence rate than students who matriculated in previous years. Advisors will collect this data for at least 5 consecutive academic years to continue to evaluate the effectiveness of the program.
Measures
Retention rates for each cohort are measured after the fall census in early October. Cohort retention rates for students who completed all the phases of the first year advising program will be assessed at that time with previous years to identify any initial changes.
Results
88% of students in the Fall 2007 cohort who completed all three advising phases (orientation, fall semester advising, and spring semester advising) continued to be enrolled in the University after one year. This result is 11% higher than the 77% Fall 2007 freshmen Cohort retention rate.
Conclusions
Even though there are many other contributing factors to student retention, the increased retention rate for students who participate in all three phases of the First Year Advising Program demonstrates a strong correlation between their participation in the advising program and a corresponding increase in 1st year retention rates. Based on this conclusion, significant emphasis will be placed on increasing the number of freshmen who participate in all three phases of the First Year Advising Program.
Questions regarding the programs and outcomes delineated in this section should be addressed to Beth Merritt Miller, 916-278-6531.