Part V: the IMPACT REPORT of the QUALITY ENHANCEMENT PLAN

Part V: THE IMPACT REPORT OF THE QUALITY ENHANCEMENT PLAN

Section 1. Title and brief description of the QEP as initially presented

Theme Title. Eastern Kentucky University will develop informed, critical and creative thinkers who communicate effectively.

Description. The primary focus of the QEP is the development of students who use higher-order thinking skills to explore, evaluate, expand, and express ideas, which is encapsulated as E4.

EKU is achieving its goal of developing informed, critical and creative thinkers through a wide variety of activities and projects. Originally, three levels of initiatives were identified: university-wide (broad based) initiatives, individual or unit-specific initiatives, and strategic planning initiatives.

Level 1: University-Wide Initiatives

A New First-Year Course: The Critical and Creative Thinking Foundation was created to develop an effective first-year experience course that will aid in the transition to college for all first-year students entering EKU. The goal is to provide foundation knowledge and skills in critical and creative thinking that address the QEP to all first-year students, and to integrate the development of problem-solving skills into both the academic and social lives of all first-year students.

Thinking and Communicating Across the Curriculum (TCAC): A Writing Intensive Program was created to achieve both the QEP and the General Education goals through a systematic program of developing skills in critical and creative thinking, written communication, and oral communication throughout a student’s baccalaureate experience.

The Studio for Academic Creativity: Developing Critical and Creative Thinking through Development of Writing, Speaking, and Research: An academic help center was created to help students become more articulate, learning to communicate clearly and mastering the art of critical and creative thinking.

The Service-Learning Project was created to promote faculty and staff engagement in projects with external partners in order to enhance student development as informed, critical and creative thinkers who communicate effectively, through service learning activities.

Critical and Creative Thinking in General Education: Revision of EKU’s General Education (GE) Program and the development of the QEP occurred in the same time period, and information gathered from faculty, staff, and students informed the development of both the QEP and GE Goals. The GE Program helps students to become informed, independent thinkers by developing competencies in communication, quantitative analysis, and critical thinking. All GE learning goals focus directly on critical thinking and written/oral communication.

Level 2: Unit-Specific Initiatives

The CACTUS Project (Citizens’ Assembly for Critical Thinking about the United States) is a course that was created by faculty in the Department of Government to provide an arena in which students could develop the skills of critical and creative thinking, specifically as it relates to their roles as engaged citizens in American democracy.

The Science in Society Project: Promotion of Critical Thinking with a Thematic Interdisciplinary Focus: This cross-disciplinary, team-taught course was created to promote student critical and creative thinking through the development of lesson plans, learning activities, and campus events that would encourage students to discover, interpret, and analyze scientific and technological information, as well as produce creative solutions to societal problems caused by scientific and technological progress.

Level 3: Strategic Plan Initiatives

When EKU adopted the QEP its mission statement was “EKU is a student-centered, comprehensive public university dedicated to high-quality instruction, scholarship, and service.” EKU’s strategic plan required academic departments with educational learning objectives to have a student learning outcome for either critical/creative thinking or communication.

General Education: EKU’s Strategic Plan requires every GE course to include learning outcomes for critical thinking, and many GE courses include learning outcomes for written/oral communication. Additionally, these outcomes must be assessed every two years, using standard rubrics for each GE Block; these rubrics include EKU’s written and oral communication rubrics, and customized critical/creative thinking (CCT) rubrics, developed from EKU’s generic CCT Rubric.

Other Major Initiatives

Faculty Development: A professional development plan was implemented to provide faculty and staff opportunities to hone their knowledge and skills, in order to help them establish an ideal environment for students to become informed critical and creative thinkers who communicate effectively.

QEP Management Plan: The Office of Quality Enhancement Programs was created and staffed to oversee and manage the QEP implementation, public relations, and marketing activities to inform and engage the campus community. The office included a full-time director, administrative assistant, professional development/service learning coordinator and assessment analyst. Additionally, multiple teams of QEP personnel, faculty, and staff provided guidance on implementation and assessment issues for all of the University-wide initiatives. A budget of close to $1 million annually has been sustained for the EKU QEP from 2007 to present, and the University is committed to this level of funding.

Section 2. initial goals and intended outcomes of the QEP

The over-arching goal of the EKU QEP is to improve students’ critical/creative thinking and communication skills. To capture comprehensive evidence of student learning, summative and formative assessment processes are being implemented, using direct and indirect assessment methods. Formative assessment is conducted during a program or course and provides immediate information for improving student learning at the course and individual levels. Summative assessment is used to check the level of student learning at the end of the program. Direct methods ask students to demonstrate their learning through tools such as objective tests, essays, presentations, classroom assignments, and cooperative supervisor/employer evaluations. Indirect methods ask them to reflect on their learning through tools such as surveys and interviews. The original outcomes of the initiative are described below:

Planning Objectives: Tracking Program Progress

A.Identification of QEP-related student learning objectives and assessment measures in every department

B.Provide funds for and help develop six initiatives to help students achieve the QEP goals:

1. First-year course (University-wide)
2. Communication-intensive courses (TCAC Program) (University-wide)
3. Studio for Academic Creativity (University-wide)
4. Service Learning Project (University-wide)
5. CACTUS Project (Unit specific)
6. Science in Society Project (Unit specific)
7. Establish The Center for Quality Enhancement to assure effective implementation, coordination, and marketing of the QEP
8. Create a professional development plan to help faculty/staff support the QEP
9. Follow the developed assessment plan to evaluate QEP activities

Intended Student Learning Outcomes

A.Summative Assessment of Critical/Creative Thinking and Communication

1.Freshman-Senior Task. A University-developed freshmen and senior written task evaluated with EKU’s CCT and WC Rubrics would show

(a)improvement in CCT and WC from freshman to senior year (longitudinal data)

(b) higher senior than freshman scores (cohort comparisons)

(c) improvement of seniors’ scores over time (cohort comparisons)

2.NSSE. Items related to the QEP from the NSSE would show

(a) more positive scores from seniors than freshmen, and more positive scores from seniors over time (cohort comparisons)

(b)more positive scores from seniors than freshmen, and an increase in these differences (change scores) over time (longitudinal data)

(c) more positive scores from seniors than from transfer students at the time of transfer to EKU, and an increase in these differences over time (cohort comparisons)

C.Formative Assessment of Critical/Creative Thinking and Communication

1.GE Courses. Student work in GE courses evaluated with EKU’s GE Rubrics would show at least 85% of students achieved the assessed learning objectives for that course.

2.TCAC (Communication-Intensive Courses). Student work evaluated with EKU’s CCT and WC/OC Rubrics would show at least 70% of students achieved at least a “competent” rating on CCT and WC/OC.

3.New First Year Course. Student work evaluated with EKU’s CCT Rubric would show at least 60% of students achieved at least a “developing” rating on critical thinking.

4.The Studio for Academic Creativity. Student work evaluated with assessment instruments customized for each project would show that students significantly improved critical thinking, written, and/or oral communication skills.

5.Service Learning Courses. Student work evaluated with EKU’s CCT and WC Rubrics would show at least 85% of students improved critical thinking and written communication skills.

6.Employer Survey. Items related to the QEP from a University-developed employer survey would show at least 70% of seniors rated at least “very good” in both critical thinking and communication by their employers.

Section 3. Changes Made to the QEP

A.Infrastructure. The plan allowed for The Center for Quality Enhancement to be established. That was changed to the “Office of Quality Enhancement Programs” and the goal to assure effective implementation, coordination, and marketing of the QEP remained the same. After 5 years QEP tasks were made a permanent part of the duties of Associate Dean of University Programs.

B.QEP Programs/Initiatives. Originally there were six main initiatives funded:

All of the University-wide initiatives are still operating and going strong, with a few modifications:

New First Year Course. The original plan proposed that a carefully designed and uniform course be developed to allow a specific place to introduce concepts of critical and creative thinking for all first year students. The EKU Council on Academic Affairs (CAA) did not approve it as mandatory for all first year students. From 2008 to 2011 360-574 students enrolled voluntarily in the course each year as one option for their University orientation requirement. Effective fall 2012, CAA approved the course for all first-year undeclared students, and 810 students enrolled in the course. However, students with a declared major do not routinely take this course, and additional initiatives were implemented to expose more first-year students, and all Teacher Education students, to critical thinking learning strategies:

• Communications (CMS) Program Faculty Development is a unit-specific program with broad impact because every student must take one CMS course as part of general education. Faculty were trained to integrate specific critical thinking and communication strategies in CMS courses. Students’ critical thinking and oral communication skills are evaluated via required assessments of general education courses; the intended outcome is that at least 85% of students achieve the learning objectives.
• English Department Faculty Development is a unit-specific program with broad impact because students complete two freshman composition courses as part of general education. English Department faculty are being trained to integrate specific critical thinking learning strategies in composition courses. Students’ written communication skills are evaluated in the second composition course via required assessment of general education courses; the intended outcome is that at least 85% of students achieve the learning objectives.
• College of Education (COE) Diversity Project is a unit-specific program with broad impact because every Teacher Education student is exposed. Critical and creative thinking was integrated into modules that focus on cultural-competence issues in education. Teacher Education candidates’ use of critical thinking skills as applied to cultural competence are evaluated with a customized rubric developed from EKU’s CCT Rubric. The intended outcome is that 100% of Teacher Education candidates demonstrate proficiency in critical thinking, as applied to cultural competence.

Thinking & Communicating Across the Curriculum (TCAC) Program. The proposed program included one required communication-intensive (W) course in general education (GE), and two required W courses in each major. It became clear that the proposal was too ambitious in that degree programs were not prepared to develop W courses for a variety of reasons, such as the lack of professional development to teach W courses, and the necessary cap on enrollment in W courses. Therefore, CAA did not approve the TCAC Program, but instead required each student to complete just one W course in general education. Even this requirement proved to be problematic, and it became clear that the number of W courses approved for GE was not sufficient to accommodate student need for the courses. At that time faculty were trained to develop W courses for their majors, and students were allowed to fulfill the need for a W course in GE or in their major. Changing the requirement from three courses to one course for each student effectively eliminated the “program.” Thus, assessment of student work is at the course level instead of a programmatic level; intended outcomes did not change.

Unit-Specific Programs. The two unit-specific programs were determined to be successful, but without broad impact, and so were not continued with QEP funds.

C.Assessment Plan.

Summative Assessment of Critical Thinking. Originally, the plan was to create a discipline-free assignment, appropriate for use in both freshman and senior courses that would prompt students to demonstrate their critical thinking and written communication skills. That student work product was to be evaluated by faculty using EKU’s CCT and WC Rubrics. The task of developing a discipline-free assignment became more difficult than expected and we discovered that Tennessee Tech University had developed an instrument that met our needs. Their NSF Grant funding allowed them to conduct validity tests well beyond what we would have been able to do in the short period of time. After carefully exploring this tool, the Critical Thinking Assessment Test (CAT) was adopted as our major assessment instrument, but the intended outcomes are the same:

National Survey of Student Engagement (NSSE). The original plan proposed comparing data, longitudinally, from freshmen to seniors. However, NSSE experts recommend against this approach, and recommend instead comparing freshmen and seniors to norms from benchmarks universities. The outcome focused on longitudinal data was deleted.

Additional summative assessment instrument for critical thinking and communication. As a result of its participation in the Voluntary System of Accountability EKU currently tests freshmen and seniors with the Collegiate Assessment of Academic Proficiency (CAAP) critical thinking and essay writing modules. Scores show the performance of each cohort, the expected performance based upon ACT scores, and a value-added score referring to the gains made by seniors as compared to freshmen (by ACT). The critical thinking portion of the CAAP measures students' skills in clarifying, analyzing, evaluating, and extending arguments. The essay writing portion of the CAAP measures students’ skills in formulating assertions, supporting assertions with appropriate evidence, organizing and connecting major ideas, and expressing ideas in clear, effective language. We will report CAAP scores as one of our assessment measures for as long as the University can support the costs. The intended outcomes are (a) seniors will score at or above the expected mean based upon entering ACT scores; (b) the gain in EKU CAAP scores from freshman to senior year will be at or above expected levels.

Graduates in the Workplace. Instead of creating a survey for employers of EKU’s alumni, instruments and processes that were already in use are being used to collect similar data. All students employed through EKU’s Co-op Program are evaluated by their employers every semester. Data are being used to assess employers’ perceptions of the critical thinking and communication skills of seniors in the workplace. The outcome remained the same.

SECTION 4. IMPACT ON STUDENT LEARNING / UNANTICIPATED OUTCOMES

Impact on Student Learning