Executive Assessment Summary, 1999-2000
Executive Summary of Assessment of Student Learning and Development
Number 3
1999-2000
We are what we repeatedly do. Excellence, then, is not an act, but a habit. --Aristotle
Introduction
What Is Assessment?
The word “assessment” often brings to mind “tests,” “grades,” “evaluations,” and so on. In higher education today, the term “assessment” has come to mean much more than that. Assessment goes beyond the testing and grading of students by professors. It is more than evaluation. Rossman & El-Khawas (1987), in their monograph for college presidents and chief academic officers, summarize the purposes of the assessment process this way: “…to ask important questions about student learning, to get some meaningful information on these questions, and to use the information for academic improvement” (p. 3). In the words of another researcher, assessment, as it is understood by most in higher education today, is “the measurement of the educational impact of an institution on its students” (Terenzini, 1989).
The goal of the assessment process is excellence. Excellence results from the habitual improvement of all institutional practices that affect student learning and development, from the classroom to the athletic field. This task begins with setting clear objectives for student learning and development that are consonant with the University’s mission statement. It continues with the habitual gathering of meaningful information to measure the accomplishment of institutional, departmental, and classroom objectives. The assessment “loop” is complete only when the information gained from this process informs planning and decision-making in each and every department, and results in constructive change.
Why Do Assessment?
The assessment movement in higher education began with public insistence on educational accountability. After two decades, this movement shows no signs of going away. Performance-based or outcomes-based budgeting is part of spending formulas for public colleges and universities in at least half the states (Lopez, 1999). Effective assessment programs are required by all six of the higher education accrediting associations. The North Central Association, which accredits Concordia University, St. Paul, requires institutions it accredits to provide evidence of the effectiveness of their educational programs (North Central Association of Colleges and Schools Commission on Institutions of Higher Education, 1996). Lopez, of the North Central Association, predicts that the “public demand for educational accountability will continue to require colleges, universities and systems of higher education to document and improve student learning” (1999, p. 6).
Accountability mandates aside, the essential goal of assessment is to improve the quality of student learning by improving the practices that impact learning. Improved institutional practices result in institutional excellence. And institutional excellence benefits all stakeholders: students and their families, faculty and staff, the church and the community (Wolff & Harris, 1994).
Relationship of Assessment Activities to the Evaluation of Institutional Effectiveness
The assessment of student learning and development, which is covered in this document, is only one aspect of the evaluation of institutional effectiveness (Lopez, March 1996). The evaluation of other aspects of institutional effectiveness is carried out by department chairs, deans, program directors, and vice-presidents as they complete program evaluations, annual reports, and other activities to evaluate the achievement of Concordia University’s strategic goals. The following sections summarize the ongoing departmental and institutional activities that assess student learning and development.
Summary
The reports from departments on their academic majors have made very evident that faculty are seriously engaged in reflecting on their practice and improving the curriculum and student performance. Faculty collaboration and their discussion of assessment activities and results were clearly documented by most departments. Many professors and departments took extra time and resources to seek feedback from students about their academic program, in addition to using the University’s standard course evaluations. Some courses involved students in assessment by having them actually create and administer surveys or interviews, which asked fellow students in the department about issues that were important to them. The delineation of clear goals and objectives for students to achieve in their majors continues to be a strength of the assessment process at Concordia University. The effective assessment of student achievement of those objectives is, in most cases, being thoughtfully deliberated and improved upon. The departmental reports suggest that departments with more ambitious assessment plans or limited personnel find it difficult to implement them due to limited resources and time. Reports of assessment activities from areas outside of academic departments were lacking.
Efforts by faculty to improve student learning were impeded by the wide range of abilities, motivation, and preparation that students bring to their academic experience at Concordia University. Faculty in both the traditional and non-traditional undergraduate programs expressed particular concern that poor writing skills on the part of some students hinder their academic achievement. The Bush Foundation grant proposal written in July 2000 summarized data showing that the range of Concordia student scores on the ACT for the mid-50% (second and third quartiles) was the widest of those institutions belonging to the Minnesota Private Colleges Consortium (Hackett, 2000). Research conducted in preparation for the grant proposal found that faculty are concerned about teaching students with “diverse experiences, preparations, and learning styles,” as well as under-performing students (Hackett, 2000, p. 2-3). Of those students who left Concordia University after their first year, 20% did not return because they had been academically disqualified.
In spite of these challenges, faculty reported that, in general, students are satisfactorily meeting the goals and objectives they have set for academic programs, although they often note a wide degree of variability in students’ performance. The challenge for the institution will be to provide sufficient time and resources to allow faculty and staff to develop and implement assessments that will target learning areas to be improved and the means to improve them.
Learning and Development in General Education
General Education Curriculum
During the 1999-2000 academic year, members of the general education committee focused on modifying the general education goals and requirements to more accurately reflect the academic course offerings. Last summer, the University assessment committee met with the general education committee to develop a plan assessing general education outcomes using the revised goal areas. The committees proposed a two-tiered plan. The first tier is at the course level. Faculty teaching courses within goal areas (e.g., history and political science) will develop outcomes for students taking any course in that goal area, and will determine assessments that will demonstrate student learning in that area. This task was begun by academic areas during the August 2000 faculty retreat.
The second level of assessment would involve assessing skills or outcomes that are common across general education offerings, for example, writing, critical thinking, or quantitative reasoning skills. The assessment and general education committees determined to tackle one of these broader areas at a time, with at least one year being devoted to studying outcomes in one area before attempting the next one. There was consensus that writing should be assessed first, since there is the sense, based on faculty experiences in reading and evaluating students’ written work (see below), that this is an area of immediate concern. This project will be undertaken during the 2000-2001 academic year.
The Bush Foundation Grant Proposal Findings
The Bush Foundation grant proposal written in July 2000 summarized data reflecting student academic preparation for the traditional programs. The proposal indicated that the range (17-27) of student scores on the ACT (a college entrance exam for students entering traditional programs) for the mid-50% (second and third quartiles) was the widest of those institutions belonging to the Minnesota Private Colleges Consortium (Hackett, 2000). Moreover, the lowest point in the range (17) was two points lower than the minimum required for admission.
Survey research conducted in preparation for the grant proposal found that faculty are concerned about teaching students with “diverse experiences, preparations, and learning styles,” as well as under-performing students (Hackett, 2000, p.2-3). In this survey, faculty also indicated that they do want to learn ways to help under-performing students, and it was clear from this study that Concordia University faculty want to help students to succeed. The Bush Foundation grant proposal addressed the need for faculty development activities that would enhance the ability of professors to promote quality learning in the context of the disparate needs of learners. The outcome of that grant proposal is pending.
Another fact that may reflect students’ preparation for college-level study is that, of those students who left Concordia University after their first year, 20% did not return because they had been academically disqualified.
Additional Faculty Input
In the non-traditional, undergraduate degree-completion programs, faculty reports have expressed concern that some students enter their programs unprepared for the level of writing required for a university program. Faculty in the traditional academic programs have also indicated concern that poor writing skills continue to hinder the achievement of some students. In addition, an informal needs assessment was conducted by the associate dean of academic support by polling faculty on their perception of where students show the greatest need for additional academic help. The area of writing ranked highest, with 96% of the faculty who responded (n=25) citing writing skills as those most in need of additional intervention. Reading skills and English for students whose primary language is not English ranked second and third, respectively.
Student Input
In student focus groups conducted in conjunction with preparation for the Bush Foundation grant proposal, students shared their own perspectives on their learning. They acknowledged that learning requires more than the memorization of facts, that it entails true comprehension. They indicated that collaboration with peers often contributed to their understanding of course material, and that, for students earning higher GPAs, competition was an important motivator for their performance. The students indicated that they are further motivated by courses of specific interest to them and those that are relevant to their chosen careers. They revealed that a professor’s passion and excitement about the subject area have a significant impact on their attitude toward a course. They stated a preference for interactive classroom atmospheres and small classes, and looked for opportunities to talk with professors. They saw the use of technology in courses as predominantly “Power Point” presentations and communication with faculty and peers via e-mail. Most students indicated that the use of technology is acceptable to them only if it does not supplant the professor, and is used only to enhance learning in conjunction with the more desirable live interactions (Cook Research and Consulting, Inc., 2000).
Learning and Development in the Academic Major
Academic departments documented their implementation of assessment and their use of assessment results for improvement by means of the annual assessment report for each major. These reports consist of four sections (see Appendix I). The first section outlines the expected learner outcomes, or goals and objectives, for that major. The second section indicates which assessment activities were conducted that year for students in that major. The third section reports the actual results of those assessment activities, that is, how students as a group actually performed when measured against the goals and objective for their major. Finally, the fourth section reports how those results were used by the department to improve or strengthen curriculum and instruction in the major.
Assessment reports for 1999-2000 were completed for 24 undergraduate academic majors and four graduate programs. This number represents a slight decline in the number of departments completing reports. These reports will be reviewed by the University assessment team, which will provide feedback to academic departments using North Central Association guidelines (Lopez, March 1996) and principles of good assessment practice (American Association for Higher Education, 1992). This process is intended to lead to improvement in the assessment of student learning and of program curricula and procedures at the department level. These reports will be retained in the office of assessment and used to document the progress of assessment activities.
Brief summaries of 1999-2000 assessment activities for majors, as reported in the annual assessment reports from academic departments, follow.
College of Arts and Sciences
Department of Art
No assessment report was submitted by this department.
Department of Biology
What was formerly the department of biology is now a part of the department of natural science and mathematics.
Department of Business
The department of business offers three majors: accounting, marketing, and finance. The previous year, the department had a preliminary accreditation visit by the Association of College Business Schools and Programs (ACBSP) and was granted candidacy status. No further information was provided about the status of this accreditation. In all three majors, course assessment activities were used to assess learning in the major, and a standardized exam in business (Major Field Test in Business) was administered. No results of the standardized exam were included in the reports, and its relationship to the learning outcomes for the majors was not elucidated.
The previous year’s assessment reports indicated weaknesses in statistics, spreadsheets, and database skills and a faculty response that added MAT102: Statistics to the required business curriculum and a computer lab component to the introductory accounting classes. No information was provided in the current report to indicate whether these strategies improved those weaknesses.
In the marketing major, assessment activities emphasized performance assessments, which were a part of on-going coursework. (Performance assessment requires students to demonstrate their ability to carry out the skill or technique, or apply the concept.) A project requiring secondary data analysis exposed students to the vast internet resources available. Faculty stated their intention to increase the use of the internet for subsequent library projects. Frequent use of course evaluations and classroom assessment techniques also provided student input on how course experiences and assignments affected their learning. Students indicated that activities in the marketing major, like case studies, class discussions, marketing plans, experimental methods, etc., helped them to “put theoretical knowledge into practice.” Faculty reported that students satisfactorily demonstrated achievement of the objectives for the major, and indicated that their use of performance assessments and frequent student input would be continued, with an increase in the use of the internet.
The assessment report for the accounting major indicated the use of weekly quizzes, homework problems, and four semester exams to assess student achievement of the learning outcomes for the major. Results provided in the report did not indicate in what areas student performance was weak or strong and did not indicate how the curriculum or instruction could be improved.
No assessment report was submitted for the finance major.
Department of Communication Studies
Assessment activities are in place throughout the course of study for the communication major, and results of these activities are discussed on a regular basis by faculty in department and “Basic Course” meetings. At the completion of their program, senior communication majors in the required capstone course submitted papers to the annual Undergraduate Communication Research Conference at St. Thomas University (St. Paul); 100% of them were accepted and presented in April 2000. Other activities used to assess student achievement of learner outcomes included internship journals, reflection papers, group projects, synthesis papers, portfolios, presentations, technology use, media analysis, and videotaped service learning projects. The reported frequency and detail of departmental discussion of curriculum and pedagogy in the process of assessment and the resulting integration of course objectives across the curriculum are noteworthy.
Based on assessment results from 1999-2000, faculty set clear goals regarding curriculum and instruction. These goals include: determining the best course in which to address persuasion and rhetorical criticism; developing effective strategies for increasing racial and gender sensitivity; revising research and video assignments; reviewing the means for teaching media theory; discussing applications for technology in the introductory course; arranging for the technology necessary for its inclusion; coordinating with other departments to allow for student productions on community access channels; and incorporating a communications ethics statement into each department course and student portfolio. The activities and tools that were determined by faculty and by student feedback to successfully teach and/or assess will be continued and, in some cases, emphasized.
In addition, the department accomplished many of the goals it set previously based on assessment results, including reformulating learner outcomes for the major, strengthening accountability in student internships using an on-line checklist, reviewing the sequence, content, and variety of courses offered, continuing to evaluate expansion of technology applications, and revision of the departmental mission statement
Department of English and Modern Languages
The department of English and modern languages offers two majors, English and secondary English teaching. Seniors in both majors completed three outcomes assessment activities in spring 2000: the senior capstone course, ENG499: Framing the Literary Tradition; a written comprehensive exam; and an oral comprehensive exam. Results were reportedly mixed regarding the successful student achievement of the outcomes delineated for these majors. In general, some students were able to adequately demonstrate the required skills, while the others struggled to demonstrate them at a level appropriate for an academic major. As in the previous year, faculty noted some improvements inthe area of genre, since coursework in the major had greater emphasis in that area. They also noted stronger responses in poetry explication than in prose explication. Areas found to need continued attention were preparation and analysis of readings by students and meaningful student-led discussions. Several students still had difficulty writing clearly, using thesis statements and supporting arguments, and applying theories of literary criticism and methods of analysis.