Shoreline Community College
ANNUAL OUTCOMES ASSESSMENT REPORT—2001-02
Assessment Liaison/s Phone Email
Jim James 206-546-6949
Pam Dusenberry 206-546-4626
Note: This report is available electronically at:
A. Highlights of Major Assessment Activities/Projects
For the fourth year, a major portion of Shoreline's Assessment allocation was dedicated to faculty assessment development efforts through an RFP process coordinated by the Institutional Effectiveness Committee. Remaining funds supported the Office of Institutional Effectiveness, Assessment and Research and the Assessment Liaison .
For 2001-02 funds were awarded to 9 faculty projects representing most of Shoreline's academic divisions:
· Global Awareness General Education Outcomes Assessment Team Project, Tim Payne, Adam Sowards Intra-American Studies & Social Sciences, ,
· Portfolio Assessment of Outcomes in the Visual Arts, Mike Larson, Humanities,
· Quality Guidelines for Online Teaching & Learning, Diana Knauf, Intra-American Studies & Social Sciences,
· Information Literacy Assessment, Gary Parks, Humanities, Joanna Tillson Library/ Media Services, ,
· Follow-up Performance of Students who have Completed Developmental English & ESL Courses, Aura Rios-Erickson. Humanities, Sally Rollman, Business Administration, ,
· Interdisciplinary General Education Outcomes Assessment Team Project, Pam Dusenberry, Humanities,
· Revision and development of outcomes and student assessment tools for individual VCT courses, Mary Bonar, Dick Davis, Humanities, ,
· Assessment and Coordination of the Interdisciplinary Studies Program, Neal Vasishth, Humanities,
· English Offerings General Education Integration and Assessment Project, Paul Cerda, Humanities
B. Examples of Educational Improvements Made/Actions Taken
· General Although the number of faculty projects dropped from 16 in 2000-01 to nine in 2001-02, projects tended to be somewhat broader in scope and to involve more faculty per project. Four of the nine projects addressed assessment aspects of Shoreline's newly revised General Education Requirements.
· Global Awareness General Education Outcomes Assessment Team Project
o The “Global Awareness General Education Outcomes Assessment Team Project”, undertaken in the 2001-02 academic year, involved faculty members in several academic disciplines in discussing the process of assessing student performance of the Global Awareness General Education Outcome. A team of instructors from History, Economics, Political Science and International Studies met five times during the year to collaboratively discuss how faculty members in the Social Sciences are currently assessing student performance of Global Awareness outcomes. It is our intention that the work we have done here will be useful to faculty members in many academic disciplines as they globalize their curricula.
Our initial goals for the project included the following:
1. build an interdisciplinary understanding of how student achievement of Global Awareness Outcomes can be described and assessed,
2. create evaluation criteria for the Global Awareness General Education Outcome that could be applied across disciplines,
3. compile samples of outcomes that might be used in constructing MCOs,
4. compile a notebook of interdisciplinary assessment tools usable in a variety of disciplines,
5. build an increased awareness among faculty about each other’s disciplines, and
6. evaluate the effectiveness of this project in assessing student achievement of Global Awareness General Education Outcomes.
A more thorough discussion of each of these points follows.
o Additional project detail is contained in Appendix A.
· Portfolio Assessment of Outcomes in Visual Arts
Foundation Art Courses: Drawing series ( Art 105, 106, 107), 2-D Design (Art 109), Ceramics Series (Art 201, 202, 203) and Painting series (Art 256, 257, and 258)
o Continuity with diversity established as a departmental value. Creation of a valuable resource of assessment material resulted from the direct collaboration with tenured and associate faculty responsible for delivery of content.
o The discussions have provided a framework for common portfolio assessment tools related to outcomes. Differences in perspective offered by each course sequence were discussed and documented.
o The review of outcomes and assessment activities in these sequences will provide a framework for updating MCOs for the affected courses.
o Course specific outcomes were identified and clarified in a more complete fashion than in the MCO where they are merely listed. Hopefully this process will result in a resource that complements and is more effective than the MCOs. Presently MCOs focus mainly on general education outcomes.
o The necessary requisites of the facility for group and individual critique in painting and drawing classes were documented. Critique is a fundamental tool of assessment in these classes.
o Optional spaces for the 2-dimensional design course have been identified and will not impact the content nor the budget.
o A departmental policy emphasizing in-hand return and limiting the time of storage for student work is now being considered
o Direct collaboration with tenured and associate faculty responsible for delivery of content, brought about the exchange and collection of assessment material supporting both continuity and diversity in instruction.
o Course specific outcomes were identified and clarified in a more complete fashion than in the MCO where they are merely listed.
o Assessment strategies for both individual and class assessment that support the dynamics of teaching and learning activities were discussed, but are not necessarily part of the grading process,.
o The review of outcomes and assessment activities in this sequences will provide a framework for updating MCOs for the respective courses.
o Additional project detail is contained in Appendix B.
· Quality Guidelines for Online Teaching & Learning
o PROJECT GOAL
The major goal of this project was for a group of faculty familiar with and committed to the online learning program to propose a set of guidelines for teaching online and hybrid courses at Shoreline Community College. At present, no clearly stated standards exist at the college for online offerings. It is important that guidelines be established to ensure the distance-learning program provides a uniformly high standard of online course delivery.
o PROJECT ACTIVITIES
WORKING RETREAT: EXPLORING GUIDELINES FOR QUALITY TEACHING AND LEARNING IN AN ONLINE CONTEXT, 10/18-19/2001
Team members Diana Knauf and Betsey Barnett participated in this working retreat sponsored in part by the State Board of Community and Technical Colleges and WAOL with funding from the U.S. Department of Education’s Learn Anywhere Anytime Project.
The retreat was designed to expose those involved in providing online learning to a wide variety of existing standards for online learning. We were provided with advance readings, and spent the majority of the retreat attempting to hammer out the basics necessary for quality online education.
The retreat provided us with an excellent opportunity to begin a reading list for our own campus. The results of the conference were recently released as the proposed guidelines for WAOL courses.
o RESEARCH
Work on Shoreline’s campus did not begin until Winter quarter as it was decided that the input of the Distance Learning Specialist was vital. Karin Roberts was hired after a rigorous selection process in which three team members participated. She has proved invaluable in the process of creating the guidelines, so the team is very pleased we waited until she had arrived on campus.
In early Winter, the entire team was asked to read a set of introductory readings including some of the information Betsey and I learned at the working retreat, but also current writings in applicable journals (see Resource Documents list at the end of this document).
Once the introductory readings had been discussed, the group read about existing standards at other educational institutions with a more critical eye. All members found this research to be helpful in determining what might be important versus what was superfluous, what fit our system versus what did not, and lastly to begin to articulate what was most appropriate for Shoreline Community College. Given prospective expansion of the distance-learning program to incorporate eArmy, we attempted to cover ground at the course level, but also the program as a whole.
o TEAM MEETINGS
The team met on a regular basis over the Winter and Spring quarters to discuss the research readings, formulate and edit guidelines and share information about online teaching and learning. All group members found the meetings to be vital in the formulation of guidelines, but also as a tremendous opportunity to discuss online teaching and gain excellent inputs for improvement. It reinforced the importance of time for online instructors to meet and share ideas, whether that is online or face-to-face.
o GUIDELINES
The attached draft of Quality Guidelines for Online Teaching and Learning for Shoreline Community College (Appendix C) is the result of group discussion on the basis of our past research and the informal polling of colleagues. It was the original hope that these guidelines would be complete and have been reviewed by campus constituencies by the end of the academic year. Given responsibilities of individual team members, involvement in the revision of Master Course Outlines, Curriculum Committee and Self-Study projects, this proved impossible to complete. Plans for completion of this portion of the project are outlined below.
The guidelines are designed to pinpoint three major areas of concern: Course Design, Online Learning Program Support, and Evaluation and Assessment.
Course Design guidelines focus on the process of creating and maintaining online courses: what instructors need to consider, as well as effective pedagogy for online delivery. Special attention is given in this section as to what will create the best learning environment for students. Thus, orientation to online learning and syllabus contents are covered, as well as the expectations for a minimum level of interactivity in online offerings.
Online Learning Program Support considers what factors are necessary to create and maintain a consistent and successful distance-learning program that serves the needs of students in an efficient and professional manner. The section is divided into three main areas. Institutional Support outlines college-wide expectations to insure an ongoing and effective Distance Learning program. The Faculty Support area outlines what faculty requires to teach online effectively, and the Student Support area highlights the necessary components to insure student success in online education.
Evaluation and Assessment needs to occur on two levels. One level examines the program in its entirety. The proposed guidelines entail regular review and revision to insure that students’ needs over the course of degree completion can be successfully met. The second level is at the individual course. Guidelines for reviewing individual courses are also suggested and are designed to blend in with existing procedures. By considering evaluation and assessment at both levels, the team believed that the college would be more likely to maintain a high quality distance-learning program.
The team is confident that the guidelines address almost all of the specific objectives we set for ourselves. Of course, near completion of the project has generated multiple new ideas of tasks to be accomplished in the area of online teaching and learning.
o Additional project detail is contained in Appendix C.
· Information Literacy Assessment
o In this project we launched a pilot assessment related to some of the college’s new Information Literacy (IL) General Education outcomes. Although we did not collect as much information to analyze as was hoped, the project was successful in that we developed a survey instrument and tested implementation and reporting processes that can be used as starting points for future information literacy projects.
o The first step involved collaboration among ourselves to determine which IL outcomes to assess. We selected a set of outcomes related primarily to identifying and accessing relevant resources, the kinds of outcomes taught in classes such as Eng 102, Library 150, as well as the library orientation given by librarian faculty.
o Next, we devised a 16 question survey, with questions in multiple choice and short answer format that could be used to assess student knowledge and learning in these outcomes. This survey was created in Word format and was then placed on the Blackboard Course Management System’s survey function. The Blackboard mounting allowed us a user-friendly web-based interface for students as well as the ability to collect quick percentage information on responses to multiple choice questions.
o Percentage of responses for multiple choice questions is available immediately for two sections, the ones that used Blackboard (Parks), and we will compare pre and post data for these courses. We also devised a point-based scoring rubric for the verbal answer questions and have implemented it for the two sections surveyed on Blackboard. We have already discussed these preliminary findings among ourselves, and will report/discuss in the appropriate programs in the fall (English and LMC) without, however, the illusion these sections speak to the general campus situation.
o Jim James of the Office of Institutional Research and Assessment is working with us to set up a data collection template for the paper-based surveys, and we will report these data as available.
o Based on our discussions of initial findings and responses, as well as information provided by students in a final “feedback” questions offered on the survey, we have drafted some revised survey questions to be used in the next phase of the project.
o The most important accomplishments of this project were to create and pilot the survey, to revise survey questions based on findings, to generate preliminary findings that will be shared as such with appropriate programs, and to identify issues to resolve, such as the hurdle in gaining wider faculty participation in relevant classes.
o Additional project detail is contained in Appendix D.
· Follow-up Performance of Students who have Completed Developmental English & ESL Courses,
o Our project provided information to support the concerns around pre-requisite checking and classroom performance.
o We found that about half of the students who enroll in classes do not meet the communication requirements necessary to perform well in class.
o Students who met the communication requirements for a class tended to perform better than those who didn’t.
o Students who completed the communication pre-requisites more than six months prior to enrolling in the class tended to perform poorly in class.
o Faculty interviews confirmed that most faculty do not check for pre-requisite compliance.
o Faculty said that the earliest they could identify students at risk was after the first or second test.
o Faculty indicated that an important reason why students fail in classes is that they do not have the basic reading and writing skills necessary to do well in the class.
o Additional project detail is contained in Appendix E.
· Interdisciplinary General Education Outcomes Assessment Team Project