Outcomes Assessment for Continuousupps No. 01.03.04

Outcomes Assessment for ContinuousUPPS No. 01.03.04

ImprovementIssue No. 2

Effective Date:07/14/2017

Next Review Date: 06/01/2020 (E3Y)

Sr. Reviewer: Director, University

Planning and Assessment

01.POLICY STATEMENTS

01.01As indicated in its mission statement and shared values, Texas State University values thoughtful reflection, collaboration, planning, and evaluation as essential for meeting the changing needs of those we serve. Outcomes assessment is an essential function forgathering critical information used to inform decision-making leading to continuous improvement.

01.02In accordance with Comprehensive Standard 3.3.1 of ThePrinciples of Accreditation: Foundations for Quality Enhancement of the Southern Association of Colleges and Schools Commission on Colleges (SACSCOC), Texas State’s accrediting body:

“The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas: (Institutional Effectiveness)

3.3.1.1educational programs, to include student learning outcomes

3.3.1.2administrative support services

3.3.1.3academic and student support services

3.3.1.4research within its mission, if appropriate

3.3.1.5community/public service within its mission, if appropriate”

01.03Texas State has established an outcomes assessment program that effectively identifies and assesses outcomes for student learning in educational programs, administrative and academic/student support services, research, and community and public service initiatives.

01.04This policy describes the process by which Texas State schedules, conducts,and approves assessment activities on campus to ensure useful data are collected.

01.05For Texas State’s purposes, outcomes assessment activities are expected to be informative and not punitive, and are conducted predominantly for the purpose of continuous improvement of programs and services. The focus should be placed on the area, rather than singling out individuals.

01.06Texas State encourages each area to employ methods that reveal weaknesses. Texas State acknowledges that this will involve submitting results with low success rates with respect to the target in question, and considers this a mark of effective assessment. The aim is not to show that the area is performing well. Rather, the aim is to determine how instruction or services might be improved.

01.07The annual assessment cycle followed at Texas State includes two phases (Phase I – planning and Phase II – results reporting). In early fall of each year, assessment plans are developed orrevised to include the mission statement, outcomes, methods and measures of assessment. In late spring for educational program outcomes and summer for all other outcomes, results, plans of action, and evidence of improvement are reported for each outcome.

01.08All outcome assessment plans are maintained in the electronic repository, the Texas State Outcomes System. Reports are accessible to all Texas State faculty and staff with a user ID. Refer to the Outcomes Assessment website for specific cycle due dates, tools, and other pertinent information.

02.DEFINITIONS

02.01Area – unit that provide assessments (i.e., office, department, unit, school, college, center, or institute), depending on the type of outcome.

02.02Assessment – the gathering of information in a prescribed manner and interpreting that information to make decisions about policy, curriculum, instruction, initiatives, or assessment practices.

02.03Outcomes – the desired results that an area anticipates finding, expressed in general terms.

02.04Methods – strategies, along with tools or instruments, used to gauge progress toward attaining outcomes. Method statements should clearly indicate how an outcome will be assessed, who will be assessed, whenand whereit will be assessed, and what specifically will be assessed.

1. Direct – assessment tools that evaluate actual performance (e.g., exam scores, portfolios, observations, records of error, time, cost, efficiency, productivity, etc.).

1. Indirect – assessment tools that evaluate perceived learning or performance based on opinions (e.g., student or employer perceptions, perceived satisfaction levels, perceived timeliness, perceived capability, etc.).

02.05Measures –operationalized variables or constructs used to assess the level of an attribute. Measures include intended performance targets, expressed in specific terms.

02.06Results – the findings amassed from the corresponding method.

02.07Action Plans – future steps to be taken toward attaining continuous improvement based on the reported results.

02.08Evidence of Improvement – current performance target level achieved as compared to previous findings, expressed in specific terms; intended to present result patterns from year to year, semester to semester, etc.

03.EDUCATIONAL PROGRAMS OUTCOMES PROCEDURES

03.01Educational program student learning outcomes identify the knowledge, skills, and attitudes that students are expected to demonstrate by the end of their course of study in a particular program. Chairs, directors, and program coordinators for each academic program will lead faculty discussions to gain collective input and consensus to create and assess appropriate, measurable student learning outcomes for each program.

03.02All academic programs identified by a distinctive four-digit CIP code are required to identify and measure student learning outcomes.

03.03Phase I in creating a system for assessment of student learning outcomes begins with a thorough review of the program mission. Mission statements should be:

1. brief and clearly defined;

1. distinctive to the program;

1. reflective of the core purpose, primary functions, and activities of the program; and
2. consistent with the mission of the university, college, and department or school.

03.04Once the mission is in place, at least five measurable student learning outcomes for undergraduate programs and three for graduate programs, as well as two support service outcomes (refer to Sections 04.05 through 04.09 for support service assessment process) specific to the educational program,are identified. These outcomes should align with the mission and overall goals of the program, as well as reflect the level of the program. Ideally, these outcome statements should demonstrate higher levels of cognitive, psychomotor, and affective learning domains.Outcomes:

1. describe the knowledge, abilities, and attitudes of a successful graduate;

1. focus on issues pertinent and important to the program;

1. reflect learning throughout the program curriculum;

1. focus on student learning rather than instructional or administrative processes; and

1. focus on key areas where outcomes assessment will enable the program to improve.

03.05Each identified outcome should have two methods of measurement that will be used to assess achievement of the corresponding outcome. Multiple methods of measurement are necessary to assure reliability and validity. Direct methods are preferred. Measures should:

1. effectively and accurately reflect learning addressed in the outcome;

1. apply to multiple sections of the same course, regardless of location or mode of delivery;

1. allow for the collection of accurate, valid, and reliable data for each method and measure in a reasonable and efficient manner within the academic year;

1. provide detail on who will be assessed, how the assessment will be conducted, when and where the assessment will take place, and what specific knowledge, skills, values, and attitudes will be assessed; and

1. clearly indicate the desirable level or standard of performance (target).

03.06Once the outcomes,methods and measures are in place, a system should be developed for implementation and assessment. This system should identify:

1. how and where the concepts, skills, and values are being taught;

1. who will be involved in the implementation and measurement process;

1. how individual student learning will be measured;

1. what instruments (i.e., exams, assignments) and scoring rubrics will be needed to quantify student learning;

1. when the learning will be measured; and

1. how the data will be evaluated.

03.07Phase II begins with collecting and analyzing the assessment data. Assessment results should be reportedin a brief, but detailed summary for eachmethod and measure. Discussions of findings should include:

1. a reflection upon and discussion of the findings amassed from the corresponding method and measure;

1. results based on reliable and valid data collection methods;

1. level of success in achieving the desired performance target;

1. indicators for further improvement that can be gleaned from the results;

1. evidence of learning related to the outcome aggregated for all sections of the course and disaggregated based on delivery, location, mode, and other pertinent factors;

1. student accomplishments and success;

1. improvement over previous assessment cycles; and

1. effectiveness of previous action plans.

03.08Based on the assessment results, plans of action should be developed. Action plans should lead to continuous improvementin student learning. Action plans should also be feasible considering the availability of time and resources.

03.09A synopsis of evidence of improvement gained from actions taken based on previous outcomes assessments should also be provided. This evidence of improvement should focus on continuous improvement in student learning and should be founded in the assessment results.

03.10Once completed, outcomes reports are reviewed and approved by the appropriate program chair or director and college dean.

03.11At the end of each phase of the assessment cycle, outcomes reports are audited by members of the Council of Associate Deans for Academic Assessment. Audit reports provide useful feedback for improving outcomes reports. These audit reports can be found on the Texas State Outcomes System.

04.SUPPORT SERVICE OUTCOME PROCEDURES

04.01 Support service outcomes identify core functions for providing administrative and academic/studentsupport services. Department directors and administrators for each identified area across every division will lead staff discussions to gain collective input to create and assess appropriate, measurable outcomes for each support service area.

04.02All areas that directly report to a vice president or special assistant to the President (units), as well as those that directly report to unit heads (departments), are required to identify and measure support service outcomes.

04.03Phase I in creating a system for assessment of support service outcomes begins with a thorough review of the program mission. Mission statements should be:

1. brief and clearly stated;

1. distinctive to the area;

1. reflective of the core purpose, primary functions, and activities of the support service area; and

1. consistent with the mission of the university, division, unit, and department or office.

04.04Once the mission is in place, three to five measurable outcomes specific to the support service area are identified. These outcomes should align with the mission and overall core functions of the support service area. Ideally, these outcome statements should reflect continuous improvement of services. Outcomes:

1. focus on issues pertinent and important to the support service area as specifically defined in the mission statement;

1. focus on key areas where outcomes assessment will enable the support service area to improve, with a focus on the end result and not the means to get there;

1. clearly articulate the intended result or action desired; and

1. align with the previous year’s identified action plans.

04.05Each identified outcome should include two methods of measurement that will be used to assess achievement of the corresponding outcome. Multiple methods of measurement are necessary to assure reliability and validity. Direct methods are preferred. Measures should:

1. align with the intent of the outcome;

1. allow for the collection of accurate, valid, and reliable data for each method and measure in a reasonable and efficient manner within the academic year;

1. provide detail on who will be assessed, how the assessment will be conducted, when and where the assessment will take place, and what will be assessed;

1. clearly indicate the acceptable level or standard of performance (target); and

1. align with the previous year’s action plans.

04.06Once the outcomes,methods and measures are in place, a system should be developed for implementation and assessment. This system should identify:

1. who will be involved in the implementation and measurement process;

1. what measures will be needed to quantify service levels;

1. what instruments and scoring rubrics will be needed;

1. when the services will be measured; and

1. how the data will be evaluated.

04.07Phase II begins with collecting and analyzing the assessment data. Assessment results should be reported in a brief, but detailed summary for eachmethod and measure. Discussions of findings should include:

1. a reflection upon and discussion of the findings amassed from the corresponding method and measure;

1. results based on reliable and valid data collection methods;

1. a level of achievement of performance targets;

1. discussion regarding implementation of prior year’s action plans and how they impacted performance; and

1. indicators for further improvement that can be gleaned from the results.

04.08Based on the assessment results, plans of action should be developed.Action plans should be feasible considering the availability of time and resources and should lead to continuous improvement of services.

04.09A synopsis of year-to-year evidence of improvement gained from actions taken based on previous outcomes assessments should also be provided for each outcome. This evidence of improvement should focus on continuous improvementin services and should be founded in the assessment results.

04.10Once completed, outcomes reports are reviewed and approved by the appropriate unit head and vice president.

04.11At the end of each phase of the assessment cycle, outcomes reports are audited by the director of University Planning and Assessment. Audit reports provide useful feedback for improving outcomes reports. These audit reports can be found on the Texas State Outcomes System.

05.RESEARCH OUTCOME PROCEDURES

05.01 Research outcomes identify the scholarly and creative activities expected of faculty, staff, and students in the research area. Academic deans and university research center and institute directors for each research area will lead discussions to gain collective input to create and assess appropriate, measurable outcomes for each research area.

05.02All research areas,including academic colleges with faculty and university research centers and institutes,are required to identify and measure research outcomes.

05.03Phase I in creating a system for assessment of research outcomes begins with a thorough review of the program mission. Mission statements should be:

1. brief and clearly defined;

1. distinctive to the research area;

1. reflective of the core purpose, primary functions and activities of the area; and

1. consistent with the mission of the university and college, as well as the university research strategic plan.

05.04Once the mission is in place, at least three measurable research outcomes specific to the area are identified. At least one of the research outcomes must specifically address student involvement in research. These outcomes should align with the mission and overall goals of the research area. Ideally, these outcome statements should contribute to the achievement of the university research strategic plan. Outcomes:

1. focus on issues pertinent and important to the area, as specifically defined in the mission statement;

1. focus on key areas where outcomes assessment will enable the area to improve; and

1. clearly articulate the intended result or action desired.

05.05Each identified outcome should have two methods of measurement that will be used to assess achievement of the corresponding outcome. Multiple methods of measurement are necessary to assure reliability and validity. Direct methods are preferred. Measures should:

1. align with the intent of the outcome;

1. allow for the collection of accurate, valid, and reliable data for each method and measure in a reasonable and efficient manner within the academic year;

1. provide detail on who will be assessed, how the assessment will be conducted, when and where the assessment will take place, and what will be assessed;

1. clearly indicate the acceptable level or standard of performance (target); and

1. address areas where the research areacan make a difference.

05.06Once the outcomes, methods and measures are in place, a system should be developed for implementation and assessment. This system should identify:

1. who will be involved in the implementation and measurement process;

1. what measures will be needed to quantify research initiatives;

1. what instruments and scoring rubrics will be needed;

1. when research initiatives will be measured; and

1. how the data will be evaluated.

05.07Phase II begins with collecting and analyzing the assessment data. Assessment results should be reported in a brief, but detailed summary for eachmethod and measure. Discussions of findings should include:

1. a reflection upon and discussion of the findings amassed from the corresponding method and measure;

1. results based on reliable and valid data collection methods;

1. a level of achievement of performance targets;

1. discussion regarding implementation of prior year’s action plans and how they impacted performance; and

1. indicators for further improvement that can be gleaned from the results.

05.08Based on the assessment results, plans of action should be developed. Action plans should be feasible considering the availability of time and resources and should lead to continuous improvement.

05.09A synopsis of year-to-year evidence of improvement gained from actions taken based on previous outcomes assessments should also be provided for each outcome. This evidence of improvement should focus on continuous improvement and should be founded in the assessment results.

05.10Once completed, outcomes reports are reviewed and approved by the appropriate academic dean or university research center or institute director and the associate vice president for Research and Sponsored Programs.

05.11At the end of each phase of the assessment cycle, outcomes reports are audited by the assistant vice president for Research and Sponsored Programs. Audit reports provide useful feedback for improving outcomes reports. These audit reports can be found on the Texas State Outcomes System.

06.COMMUNITY AND PUBLIC SERVICE OUTCOME PROCEDURES

06.01Community and public service outcomes describe outreach activities across the university. Designated administrators within the Academic Affairs, Student Affairs, and University Advancement divisions will lead staff discussions to gain collective input to create and assess appropriate, measurable outcomes for their community and public service initiatives within their divisions.