2013 Annual Program Review For

Kaua`i Community College

2013 Annual Program Review for

ASSESSMENT COMMITTEE

October 31, 2013

Program Description

The mission of the Kaua’i Community College Assessment Committee is to encourage and promote a culture of assessment that is embraced by the entire KCC community of faculty, staff, administration, and students. The primary purpose of outcomes assessment is to assure institutional fidelity and to foster ongoing improvements in student learning and development. To meet this mission, the Assessment Committee has the following goals, which serve as our Program Outcomes:

1) Foster the efficient collection and subsequent dissemination of information about student learning and service outcomes assessment to all campus units.

2) Facilitate appropriate interpretation, understanding, and subsequent use of assessment information in College and Unit decisions.

3) Ensure that course, program, and institutional student learning outcomes and service outcomes are measurable and appropriate.

Part I. Quantitative Indicators

Outcome #1) Foster the efficient collection and subsequent dissemination of information about student learning and service outcomes assessment to all campus units.

Responsibility #3) Seek input from the campus community about what assessment information will be most useful and the most efficient means of gathering the information. This may include developing and administering surveys and evaluations.

The committee has identified a need to communicate more effectively with non-instructional program areas to strengthen its efforts in this area.
Committee representatives are regularly asked to solicit feedback from their constituents about assessment practices and possible improvements.
The Assessment Coordinator is available to meet with any faculty member, staff member, division, or department to assist with assessment planning and implementation upon request.

Responsibility #6) Regularly communicate with the campus about assessment efforts, findings, and implications.

Now that an assessment database has been populated with initial data, the IR has been able to present a report on ISLO achievement across campus for the first time. This data was made available to the campus at Fall Convocation 2013. Subsequently, program-level data was distributed to program coordinators and division chairs in Accounting, ABRP, AMT, BTEC, CULN, ECE, ETRO, HOST, Liberal Arts, and the Registered Nurse programs. Responses to the 4th qualitative question on the CARDs for Spring 2013, which outline any APRU requests based upon assessment data, were disseminated to Division Chairs and program coordinators in October 2013 for inclusion in this year’s APRUs.

Responsibility #7) Provide accessible, up-to-date, and high-quality information about assessment practices and campus assessment policies and procedures to the campus via a committee website and/or Intranet documents.

All committee documents, agendas, and minutes are available at Campus Docs/Assessment. The Assessment folder also includes a small archive of assessment-related professional articles, templates for all current reporting forms for all courses, copies of current ISLO documentation, current non-instructional assessment plans and outcomes, current PSLO matrices for all instructional programs, a rubric bank, and a folder of support materials to aid faculty in developing outcomes and assessments. The committee’s Mission and Charter, and documents detailing the college’s existing assessment procedures, are available in the same CampusDocs folder, along with a regularly-updated file detailing all course and program outcomes formally approved by the committee.
In addition, the AC confidentially maintains folders of Annual Assessment Afternoon reports, individual course and program assessment reports, and faculty completion of assessment reports of data. These are available upon request to appropriate viewers.
For instructional programs, there has been a general increase in assessment report completion rates and quality of data collected (see chart below). A database system has been devised which allows the CARD data to be used for program-level and institutional-level assessments. It is currently being populated with CSLO/PSLO and CSLO/ISLO matrices based upon current Program Action Requests and catalog information, CSLOs for all courses with existing assessment data, and assessment data from Fall 2012. Further CARD data will be input as time and workload permits. Once the database is fully-populated, the Assessment Coordinator (AC) and Institutional Researcher (IR) plan to work with Computer Services to develop a new interface which will allow faculty to input data directly into the database via a secured website, eliminating the need for the Excel-based CARDs—the committee is hopeful that this will further encourage greater faculty participation in the reporting of assessment results.

Semester / Discrete Course Sections Offered / Discrete Course Sections Assessed / % Course Sections Assessed / % Discrete Courses Assessed
Fall 2007 / 209 / 0 / 0 / 0%
Spring 2008 / 206 / Plans created for 34 / 16% / 21%
Summer 2008 / 10 / 0 / 0 / 0
Fall 2008 / 212 / Plans created for 22 / 10% / 14%
Spring 2009 / 239 / 28 / 12% / 13%
Semester / Discrete Course Sections Offered / Discrete Course Sections Assessed / % Course Sections Assessed / % Discrete Courses Assessed
Summer 2009 / 16 / 0 / 0 / 0%
Fall 2009 / 252 / 83 / 33% / 38%
Spring 2010 / 264 / 74 / 28% / 31%
Summer 2010 / 11 / 1 / 9% / 9%
Fall 2010 / 268 / 97 / 36% / 45%
Spring 2011 / 256 / 117 / 46% / 45%
Fall 2011 / 255 / 184 / 65% / 62%
Spring 2012 / 244 / 178 / 57% / 61%
Fall 2012 / 276 / 196 / 71% / 75%
Spring 2013 / 263 / 235 / 89% / 90%
Summer 2013 / 13 / 4 / 31% / 31%

Responsibility #10) Assessment policies and procedures shall be reviewed annually for effectiveness.

In Spring 2013, the Committee reviewed its existing processes for gathering assessment data for individual courses. In response to faculty complaints about the current CARD system, the committee developed the Alternative Assessment Reporting Methods Policy.
Based upon input from the Curriculum Committee, Faculty Senate, the VCAA, and from individual faculty and divisions via their representatives on the committee, the Assessment Committee Mission and Charter is currently being revised to more effectively outline the criteria used by the committee to review and approve outcomes. The requirement for the committee’s approval of all outcomes before curriculum committee consideration will be emphasized.

Outcome #2) Facilitate appropriate interpretation, understanding, and subsequent use of assessment information in College and Unit decisions.

Responsibility #4) Plan and organize the Annual Assessment Afternoon, including facilitating necessary course-, program-, and institutional-level activities related to assessment.

The AC and IR have continued to produce reports that track each particular course’s assessment results across sections and semesters for the Annual Assessment Afternoon (AAA) held in March 2013. Faculty received concrete quantitative and graphical feedback on their courses (as reported on the CARD system), and were provided with guidance in interpreting the results and implementing improvements. In response to feedback from faculty who found the quantity of data to be overwhelming, the Assessment Coordinator manually annotated all course print-outs, indicating areas of significant improvement or decline, and suggesting topics for discussion. In addition, summaries of narrative/qualitative data from submitted assessments (CARDs) were distributed with the print-outs to facilitate discussion.

Responsibility #5) Work with the Professional Development Coordinator and individual divisions and units to plan and organize needed professional development activities related to assessment.

The committee and the College Council (Oct. 17, 2013 meeting) have identified a need to provide more training to faculty and staff about writing and assessing student learning outcomes and service outcomes. More training and information about assessment and service outcomes is a need for most academic support and administrative support units, and will be addressed as the year progresses. The AC will be seeking to collaborate with the Professional Development Coordinator to facilitate workshops.

Responsibility #8) Encourage and facilitate collaborative assessment efforts across campus.

This has not been a priority during the past year.

Responsibility #9) Promote active participation in assessment by all faculty, staff, and students in instructional and non-instructional areas.

Assessment efforts for non-instructional programs continue and have been identified as a priority item for the committee for the 2013-2014 academic year. At present, all units should be reporting on their Service Outcomes (SOs) in their APRUs, the primary point of assessment reporting for non-instructional units.
The Spring 2013 CARD completion rates were the highest they have ever been, thanks largely to a coordinated effort among the administration and divisions to emphasize assessment reporting.
The AC met with Student Services in May 2013 to assist the unit with identirying appropriate assessments for their identified SOs.
The AC met with the new VCAS (March 12, 2013) to identify units possibly needing assistance with identifying and assessing Service Outcomes.

Outcome #3) Ensure that course, program, and institutional student learning outcomes and service outcomes are measurable and appropriate.

Responsibility #1) As part of the regular curriculum approval process, review and approve or make recommendations on all Course Student Learning Outcomes (CSLOs) and Program Student Learning Outcomes (PSLOs) as new courses/programs are developed and as existing courses/programs are modified. For both types of outcomes, the committee will ensure that SLOs are, at minimum, measurable, appropriate, and sufficient.

At present, the committee is revising its charter to reflect a fully-transparent set of criteria to be used to evaluate outcomes submitted for approval. The major criteria are measurability, appropriateness, and sufficiency.
A Task Force jointly formed by the Curriculum Committee, the VCAA, and Faculty Senate has requested that the Assessment Committee be responsible for reviewing and approving on Course Outlines all CSLOs, course-level assessment tasks, and CSLOs’ alignment with PSLOs and with ISLOs. The committee voted to accept this responsibility, and will enact this for the Spring 2014 semester.
To date, the committee has reviewed and approved 144 courses’ CSLOs. Three courses (PHYS 151L, PHYS 170L, and MGT 120) have been returned to the proposers for revisions that are still pending.
All programs’ PSLOs have been “grandfathered” by the committee, and each program’s PSLOs will be thoroughly reviewed upon the next Program Action Form submitted. To date, 5 programs have had PSLOs formally reviewed and approved by the committee: Accounting, Sustainability, AS in Business, AAS in BTEC, and Culinary Arts.

Responsibility #2) Review and revise, as necessary, the college’s general education core—the Institutional Student Learning Outcomes (ISLOs)—when the college’s Mission is reviewed and/or revised, ensuring that the ISLOs are measurable and appropriate, and that all programs offered at the college are aligned with the core outcomes.

The committee reviewed and revised the ISLOs in Spring 2013.
The college’s first ISLO report, using CARD data from the Fall 2012 semester, has been produced. In addition, the IR has selected specific questions from the CCSSE Survey to supplement the ISLO data. Examining these two very different measures will allow us to see how students perceive their KCC experience, and what they are learning, and compare it to our faculty’s perceptions of student learning within the General Education realm.

ISLO # / ISLO Title / CARD Assessment (met benchmark)
1 / Written Communication / 85%
2 / Oral Communication / 85%
3 / Reading / 91%
4 / Symbolic Reasoning / 82%
5 / Integrative Thinking / 74%
6 / Information Literacy / 97%
7 / Technological Competency / 88%
8 / Teamwork / 96%
9 / Respect for Diversity / 73%
10 / Ethics / 77%
CCSSE Contributions to Assessing Institutional Learning Outcomes / Improvement / National Comparison
ISLO # / ISLO / 2010 / 2012 / Grade / Small Colleges / Kaua‘i CC / Grade
1 / Written Communication / 2.69 / 3.01 / 90% / 2.74 / 3.01 / 86%
2 / Oral Communication / 2.39 / 2.60 / 81% / 2.41 / 2.60 / 78%
3 / Reading / 2.51 / 2.66 / 73% / 2.51 / 2.66 / 73%
4 / Symbolic Reasoning / 2.69 / 2.80 / 67% / 2.70 / 2.80 / 66%
5 / Integrative Thinking / 2.86 / 2.98 / 67% / 2.81 / 2.98 / 74%
6 / Information Literacy / 2.69 / 2.88 / 78% / 2.73 / 2.88 / 73%
7 / Technological Competency / 2.63 / 2.80 / 74% / 2.74 / 2.80 / 59%
8 / Teamwork / 2.51 / 2.70 / 78% / 2.44 / 2.70 / 85%
9 / Respect for Diversity / 2.60 / 2.88 / 86% / 2.47 / 2.88 / 95%
10 / Ethics / 2.56 / 2.78 / 81% / 2.46 / 2.78 / 90%

*A detailed explanation of the specific CCSSE questions selected for this analysis and the means by which these scores were derived can be found in the 2013 Follow-Up Report submitted to ACCJC in October 2013.

Part II. Analysis of the Program

The Assessment Committee is responsible for ensuring that all operations of the college are being actively and accurately assessed, that student learning is being documented, and that improvements are made to services and instruction based on reliable data. Its operations are critically important to the college’s ongoing accreditation and efficiency. Since taking its current form, with representation from all instructional and non-instructional programs (with the exception of OCET), the committee has been able to address instructional, non-instructional, andadministrative areas.

Last year (2012/2013)

At its first meeting in August, 2011, the Assessment Committee identified seven priority goals for the 2012/2013 academic year and noted three action items related to these goals, as follows:

1. Create position and hire Full-time Assessment Coordinator (from 2011/2012 AY APRU request). Approved by College Council in 2011 but not pursued. New half-time assigned time position implemented in Spring 2013.

2. Continuance of a small operating budget for the Committee. 2011 APRU request for $300 was approved. A separate budget code was not assigned, so the committee’s expenditures on this budget are housed under Academic Support.

3. Assessment Database Subscription. This request was predicated upon the campus’ adopting the LiveText assessment database system on a campus-wide basis. The 2012/2013 Perkins-funded pilot was successful, and at the time of the request plans were being made to secure further grant monies to facilitate a larger cross-campus trial of the software. The grant funding fell through, however, so the project did not move ahead in the intended manner. A new initiative may be possible in the 2014/2015 AY, but price structures for student subscriptions to the database will have changed.

This year (2013/2014)

At its first meeting in August, 2013, the Assessment Committee identified one priority goal for the 2013/2014 academic year: improving non-instructional program assessment. This includes all units within Student Services, Administrative Support, and Administrative Services. The committee agreed that, while a workable process is now in place for instructional assessment, we need for turn our focus to providing non-instructional programs with a better understanding of assessment, developing their own Service Outcomes, gathering and documenting data to assess these outcomes, and facilitating an effective method for reporting and analyzing this information that leads to program improvement.

Part III. Action Plan

Goal Alignment

UH System Goals, KCC Goals, Strategic Goals / Program Goals
Goal 1: Educational Effectiveness and Student Success
KCC Goal 1: Access &
KCC Goal 2: Learning and Teaching
Strategic Goals: Student Recruitment, Retention and Success of All Students and Particularly
○Remedial/Developmental Students
○Non-traditional Students in Career and Technical Programs
○Increased Completion of Degrees, Certificates, and Licensure
○Increased Transfer Rate
○Increase opportunities for potential students to experience KCC
Relevant Curriculum Development
○Sustainability/Green Jobs
○Health
○STEM
○DOE-KCC English Alignment
○Distance Education
○Create/Strengthen k-12 - four-year degree pathways
Completion of
○Course Student Learning Outcomes (CLOs)
○Program Learning Outcomes (PLOs)
○Institutional Learning Outcomes (ILOs)
○Course Action Forms (CAFs)
Assessment Activities and Analysis / 1)Improving non-instructional program assessment.
2)Continue populating and refining assessment database, including improving user interface for possible direct faculty input.
UH Goal 2: A Learning, Research and Service Network
KCC Goal 3: Workforce Development & KCC Goal 5: Community Development
Strategic Goals: Increased Job Placement and/or Performance through
○Revised or New Curriculum
○Better Coordination with Business and Industry / 1) Improving non-instructional program assessment
UH Goal 3: A Model Local, Regional and Global University
KCC Goal 6 Diversity
Strategic Goals:
○Fostering Global Understanding and Intercultural Competence
○Increased Enrollment and Successof International Students / 1) Improving non-instructional program assessment
UH Goal 4: Investment in Faculty, Staff, Students and Their Environment
KCC Goal 4: Personal Development
Strategic Goals:
○Professional Development Directed to Any of the Above Goals
○Enriching Student Experience, Particularly Directed to Any of the Above Goals
○Increasing the Efficiency, Effectiveness and Sustainability of the KCC Environment / 1)Improving non-instructional program assessment
2)Continue populating and refining assessment database, including improving user interface for possible direct faculty input.
UH Goal 5: Resources and Stewardship
KCC Goal 5 Community Development
Strategic Goals:
○Reduce Deferred Maintenance
○Address Health and Safety Issues
○Promote Sustainability / 1) Improving non-instructional program assessment

ACTION PLAN

Program Goal / Action Item / Resources Needed / Person(s) Responsible / Timeline / Indicator of Improvement / PLO impacted / Status
1)Improving non-instructional program assessment / 1) Ensure SOs for all non-instructional units are appropriate, sufficient, and measurable.
2) Assist non-instructional units to identify data needs and develop/implement assessment plans. / Time / Assessment Committee; Assessment Coordinator;VCSS; VCAS; Appropriate Unit Supervisors; Professional Development Coordinator / Fall 2013 / Spring 2014 / SOs developed for all non-instructional units and clearly-articulated assessment plans in place. / All non-instructional unit Service Outcomes / Ongoing
2)Continue populating and refining assessment database, including improving user interface for possible direct faculty input. / 1) Continue aligning CSLOs, PSLOs, and ISLOs for required courses in all instructional programs.
2)Input all alignments for all instructionalprograms.
3) Prepare all existing CARDs for importing to the database.
4) Develop a user-friendly interface for faculty use. / Assigned Time for AC, IR, and appropriately-skilled IT specialist. / Assessment Coordinator; Institutional Researcher; Computer Services?
This may become subsumed under the LiveText project currently funded by Perkins grant. / Fall 2014 / Project completion. / Assessment of all PSLOs for all programs. / Ongoing

Part IV. Resource Implications