College Reading and Learning Association Council for the Advancement of Standards in Higher Education
CAS Assessment:
Using Student Learning Outcomes to Transform Program Evaluation
Student learning and development outcomes are an essential part of learning center evaluation. Learning center professionals self-assess for a variety of reasons: to know if mission and goals are being achieved or to meet demands of campus assessment initiatives and regional institutional accrediting organizations. Regardless of the motivation, assessing student learning outcomes is a necessary and fascinating aspect of evaluation, and the CAS standards for learning assistance programs provide an outline of learning outcomes to consider.
Karen S. Agee, Ph.D.
Reading and Learning Coordinator, Emerita
University of Northern Iowa
CRLA’s Director on CAS Board of Directors
College Reading and Learning Association Conference
Revolutionalizing Learning to Enhance Student Success
The Boston Park Plaza Hotel
November 9, 2013
On a scale of 1 (clueless) to 10 (guru), how knowledgeable are you of student learning outcomes (SLO) assessment strategies?
1 2 3 4 5 6 7 8 9 10
Learning objectives for this workshop:
1. Understand the purpose of CAS LAP standards and student learning and development outcomes.
2. Identify student learning outcomes useful to participants’ programs.
3. Contextualize learning outcomes assessment.
4. Learn options for writing outcomes statements.
In a 2011 survey of CRLA members, 81% of respondents said it was very or somewhat important to them to learn more about assessment. CAS standards and guidelines for learning assistance programs provide the framework to become expert in self-assessment. This conference session focuses on one aspect of that self-assessment: student learning outcomes.
Although the CAS standards and learning outcomes are not in themselves revolutionary, adding student learning outcomes assessment to current program evaluation strategies can revolutionize program evaluation by focusing assessment on desired outcomes rather than services provided and resources expended. Instruction and services should be intentionally focused on student learning (Simpson, 2002).
College Reading and Learning Association (CRLA) is a member of the Council for the Advancement of Standards in Higher Education (CAS), a consortium of professional organizations in higher education advocating self-assessment. CAS standards (“must” statements) and guidelines (“should” and “may” statements) have been developed for more than 40 areas of higher education, including learning assistance programs. Standards and guidelines are developed and reviewed in a consensus process.
The CAS standards and guidelines for learning assistance programs (LAP) are made available to CRLA members on the Members Only page as a benefit of membership. (To review the CAS LAP standards and guidelines, enter the Members Only page at http://www.crla.net/membership/index.htm.) The CAS LAP standards and self-assessment guides are also available at https://store.cas.edu/catalog/index.cfm. As a member of CAS with long service on the CAS Board of Directors, CRLA encourages its own members to use CAS standards for self-assessment of their programs.
A critical part of self-assessment using CAS standards is assessment of student learning and development outcomes. From a variety of sources and several perspectives, including Bloom’s taxonomy (as revised in Anderson & Krathwohl, 2001), CAS created a list of six domains and 28 dimensions of student learning and developmental outcomes. (See http://www.cas.edu/index.php/learning-and-developmental-outcomes/ for citations and a description of the development of this list of outcomes.)
The CAS process of developing standards by consensus has been critically important in stimulating certification and professional development programs in learning assistance. In 1989, CRLA initiated tutor training program certification to ensure minimum standards for tutor training (CRLA, 2012). The National Association for Developmental Education (NADE) developed the NADE Self-Evaluation Guides (Clark-Thayer, 2009) in the 1990’s. In 1998, CRLA created a similar standards process for mentor training certification (CRLA, 2013). All three certification programs are endorsed by the Council of Learning Assistance and Developmental Education Associations (CLADEA), through which six professional organizations have developed processes for examining and approving certifications in the field. CRLA publications—including the CRLA Tutor Training Handbook (Deese-Roberts, 2003) and CRLA Handbook for Training Peer Tutors and Mentors (Agee & Hodges, 2012)—provide examples of best practices that meet its ITTPC and IMTPC certification standards, and the latter handbook incorporates learning outcomes into every training module.
Unfortunately, learning centers and developmental education programs have not universally adopted the revised (2008) CAS learning and development outcomes for program evaluation. Professional staff in tutoring centers may wonder how they can possibly assess the learning of students who drop in for assistance; in fact, CAS learning outcomes may be especially useful in assessment of student learning outside classroom settings. CAS standards were first developed for student affairs programs and seem especially appropriate for campus departments like academic advising and learning assistance centers. Learning center administrators and staff need to become knowledgeable about program evaluation and student outcomes assessment, including self-assessment using CAS standards and learning outcomes (Christ, Sheets, & Smith, 2000; Trammell, 2005).
1. Methods of assessing student learning outcomes (Maki, 2004)
a. Direct methods
• Students demonstrate learning via some form of standardized test focusing on aspects of student learning
· Examples: CAAP, CLA, ETS Proficiency Profile (formerly MAPP), GRE subject tests, PRAXIS exams
b. Indirect methods
• Students report perceptions of their learning and the educational environment that supports that learning
• Examples: CSEQ, NSSE, NSLLP
c. Performance-based methods
· Students represent learning in response to assignments or projects that are embedded in their educational experiences
• Examples: Successful solution of a math problem not previously understood, demonstrated writing skill, panic replaced by confidence
But learning centers can create more direct methods than standardized tests!
2. CAS Learning and Developmental Outcomes (2008)
· General Standard
Programs and services must assess relevant and desirable student learning and development outcomes and provide evidence of their impact on student learning and development.
Programs must articulate how they contribute to or support students’ learning and development in the domains not specifically assessed.
· 6 domains, 28 dimensions
3. Assessment plan outline
- Review/revise mission
- Identify program goals and learning goals
- Align major programs and services with goals
- Specify desired outcomes
§ Program and operation goals
§ Learning and development goals
- Map learning and development goals to CAS learning domains and dimensions
f. Determine assessment approach
4. Write student learning outcome statements
· Specify who, under what conditions, what, to what extent, how, why
· Consider immediate learning and longer-term outcomes
· Consider rubrics for assessing and informing students
Three Assessment Approaches for LAPs
a. Dorothy Williams’ Measures of Metacognitive Development
www.lsche.net/resources/lrnr_asses/assess_slo_metacgntv_rubric.htm
§ SLOs and instruments developed for metacognitive development, levels of critical thinking, locus of control, and use of learning styles
§ Rubrics developed for metacognitive motivation, acquisition, retention, and performance
§ SLOs and instruments used in general tutoring, writing center, math center, reading center, academic skills center
§ SLOs and instruments used for both appointment-based and drop-in services of these offices
b. UNI Reading & Learning Center’s Knowledge-Skills-Attitudes
§ Developed from UNI’s Qualities of an Educated Person project by Karen Agee and Peer Instructors
§ CAS learning and developmental outcomes categorized by K-S-A
§ K-S-A used for Ask-a-Tutor appointments and Coordinator consultations
§ K-S-A assessment made collaboratively with student whenever possible
c. May and Harris’ Scale: Where Are You Now?
§ Developed to assess problems and progress in tutoring sessions
§ Used by tutors and student clients to make progress visible
Metacognitive RubricPerformance
Ability to Demonstrate Understanding of Information and Processes
1 / 2 / 3 / 4 / 5
Does not try to predict what will be on the exam / Is unsuccessful in predicting possible exam questions / Attempts to predict and practice potential exam questions before the test / Is usually able to accurately predict and practice potential exam questions before the test and practice retrieval. / Is able to accurately predict and practice potential exam questions well before the text and practice retrieval.
Does not attempt to apply appropriate processes or information / Applies inappropriate processes or information / Sometimes knows when to apply appropriate processes or information (example, knows when to use the appropriate formula or methodology) / Usually knows when to apply appropriate processes or information (example, knows when to use the appropriate formula or methodology) / Always knows when to apply appropriate processes or information (example, knows when to use the appropriate formula or methodology)
Does not attempt to transfer and integrate knowledge from one source to another / Is unable to transfer and integrate knowledge from one source to another / Is sometimes able to transfer and integrate knowledge from one source to another / Is able to transfer and integrate knowledge from one source to another / Easily transfers and integrates knowledge from one source to another
Can not control stress during assessments / Has no techniques to control stress during assessments / Attempts to use techniques to control stress during assessments, but they are not always successful and appropriate / Uses successful and appropriate techniques to control stress during assessments / Regularly uses successful and appropriate techniques to control stress during assessments
References
Agee, K., & Hodges, R. (2012). CRLA handbook for training peer tutors and mentors. Mason, OH: Cengage Learning.
Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching and assessing: A revision of Bloom's taxonomy of educational objectives (Complete ed.). New York, NY: Longman.
Christ, F., Sheets, R., & Smith, K. (Eds.). (2000). Starting a learning assistance center: Conversations with CRLA members who have been there and done that. Clearwater, FL: H&H.
Clark-Thayer, S. (Ed.). (2009). NADE self-evaluation guides (2nd ed.). Clearwater, FL: H&H.
College Reading & Learning Association (CRLA). (2012). International Tutor Training Program Certification: About ITTPC. Retrieved from http://www.crla.net/ittpc/about_ittpc.htm
College Reading & Learning Association (CRLA). (2013). International Mentor Training Program Certification: About IMTPC. Retrieved from http://www.crla.net/imtpc/about.htm
Council for the Advancement of Standards in Higher Education. (2012). CAS professional standards for higher education (8th ed.). Washington, DC: Author.
Deese-Roberts, S. (Ed.). (2003). CRLA tutor training handbook (rev. ed.). Hastings, NE: College Reading & Learning Association.
Maki, P. L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus.
Simpson, M. L. (2002).Program evaluation studies: Strategic learning delivery model. Journal of Developmental Education, 26(2), 2-4, 6, 8, 10, 39.
Trammell, J. (2005). Learning about the learning center: Program evaluation for learning assistance programs. The Learning Assistance Review, 10(2), 31-40.
LEARNING ASSISTANCE PROGRAMS
CAS Standards and Guidelines
Part 1. MISSION
The primary mission of Learning Assistance Programs (LAP) must be to provide students with resources and opportunities to improve their ability to learn and to achieve academic success.
LAP must develop, disseminate, implement, and regularly review their missions. The mission must be consistent with the mission of the institution and with professional standards.The mission must be appropriate for the institution's student populations and community settings. Mission statements must reference student learning and development.
LAP must collaborate with faculty members, staff, and administrators in addressing the learning needs, academic performance, and retention of students.
Models of LAP vary, but must have the following goals:
· ensure that students are the central focus of the program
· assist students in achieving their personal potential for learning
· introduce students to the academic expectations of the institution, the faculty members, and the culture of higher education
· help students develop positive attitudes toward learning and confidence in their ability to learn
· foster students’ personal responsibility and accountability for their own learning
· provide a variety of instructional approaches appropriate to the skill levels and learning styles of students
· assist students in applying newly learned skills and strategies to their academic work
· support the academic standards and requirements of the institution
Models of LAP should also share the following common goals:
· provide instruction and services that address the cognitive, affective, and sociocultural dimensions of learning
· provide to faculty members, staff, and administrators, both services and resources that enhance and support student learning, instruction, and professional development
Part 2. PROGRAM
The formal education of students, consisting of the curriculum and the co-curriculum, must promote student learningand development outcomes that are purposeful, contribute to students' realization of their potential, and prepare students for satisfying and productive lives.
Learning Assistance Programs (LAP) must collaborate with colleagues and departments across the institution to promote student learning and development, persistence, and success.
Consistent with the institutional mission, LAP must identify relevant and desirable student learning and developmentoutcomes from among the six domains and related dimensions:
Domain: knowledge acquisition, integration, construction, and application
· Dimensions: understanding knowledge from a range of disciplines; connecting knowledge to other knowledge, ideas, and experiences; constructing knowledge; and relating knowledge to daily life
Domain: cognitive complexity
· Dimensions: critical thinking, reflective thinking, effective reasoning, and creativity
Domain: intrapersonal development
· Dimensions: realistic self-appraisal, self-understanding, and self-respect; identity development; commitment to ethics and integrity; and spiritual awareness
Domain: interpersonal competence
· Dimensions: meaningful relationships, interdependence, collaboration, and effective leadership.
Domain: humanitarianism and civic engagement
· Dimensions: understanding and appreciation of cultural and human differences, social responsibility, global perspective, and sense of civic responsibility
Domain: practical competence
· Dimensions: pursuing goals, communicating effectively, technical competence, managing personal affairs, managing career development, demonstrating professionalism, maintaining health and wellness, and living a purposeful and satisfying life
[LD Outcomes: See The Council for the Advancement of Standards Learning and Developmental Outcomes statement for examples of outcomes related to these domains and dimensions.]