Assessment Manual and Report 2004 - 05 5

Self-Assessment Manual

And Report AY 2004 – 2005

Civil Engineering Program

Cleveland State University

Prepared by Walter M. Kocher, Ph.D.

Revised May 30, 2005

Table of Contents

page

SELF-ASSESSMENT REQUIREMENTS 2

NCA Accreditation 2

ABET EC 2000 Accreditation 3

OVERVIEW OF THE ABET SELF-ASSESSMENT PROCESS 4

Senior Exit Survey 4

Faculty Course Reflections & Evaluations 5

FE Exam Performance by Seniors 5

Senior Capstone Design Performance 5

Student Surveys in Key Courses 5

Survey of Employers of Graduates 5

Survey of Alumni 5

Visiting Committee 5

SELF-ASSESSMENT OF OUTCOMES 6

SELF-ASSESSMENT OF OBJECTIVES 6

CLOSING THE LOOP 6

Appendix A. SELF-ASSESSMENT OF OUTCOMES in 2004 7

Appendix B. SELF-ASSESSMENT OF OBJECTIVES in 2004 18

Appendix C CLOSING THE LOOP in 2004 26

Appendix D Attachments from ABET review responses 55

List of Tables and Figures

page

Table 1. Civil Engineering Program Outcomes 27

Table 2. Civil Engineering Program Objectives 27

Figure 1. Academic Assessment Report form 28

Table 3. Schedule of Assessment Measurements 29

Figure 2. Senior Exit Survey form 30

Figure 3. Faculty Course Reflection form 32

Figure 4. Student Course Evaluation form 33

Figure 5. Alumni Survey form 35

Figure 6. Employer and Alumni Annual Survey forms 38

Table 4. Assessment Methods for Outcomes and Objectives 40

Table 5. Assessment Criteria for Meeting Outcomes and Objectives 41

Table 6. Summary of Faculty Reflection Scores 42

Table 7. ABET EC 2000 Curriculum Designed to Meet Outcomes 44

Self-Assessment Manual

Civil Engineering Program

SELF-ASSESSMENT REQUIREMENTS

The Civil Engineering Program must conduct self-assessment procedures for two accreditation bodies, the NCA (North Central Association of Colleges and Schools) accreditation of the entire university, and the ABET accreditation of the Civil Engineering Program.

NCA Accreditation

The Civil and Environmental Engineering Department must submit annual assessment reports on all undergraduate and graduate programs within the department.

The relevant assessment activities identified for all programs within the university include:

·  Surveys (exit interviews, alumni survey, employers survey, best students, student survey, and entrance survey

·  Performance measures (internal exam, external exam, presentation/project, student portfolio, and program portfolio)

·  Judgment assessments (faculty judgments, external judgments of students, external judgment of program, and student self-assessment)

·  Courses (course grade, program grade analysis, GPA required, and special courses

·  Other (pre/post test and teaching evaluation)

The report (Figure 1) must also include:

·  A listing of salient findings from the recent academic year

·  A listing of persons who received those findings and could make informed program decisions based at least in part on the findings

·  A listing of any program/curricular modifications in any way connected to the assessment data

The ABET assessment requirements for the undergraduate (Civil Engineering) program more than satisfy the NCA requirements. Therefore, the ABET assessment process, described in detail within this manual, will also be used to meet the NCA assessment requirements.

However, the graduate programs within the department must conduct separate assessment activities, although the survey and other assessment forms are similar to the forms outlined for the ABET assessment. Faculty evaluations, student exit surveys, plus feedback from several areas including alumni, employers and visiting committee members, are used in a similar manner described for the ABET assessment analysis and development of continual improvement activities.

ABET EC 2000 Accreditation

The new ABET accreditation criteria has moved away from “bean counting” – counting credit hours for designated categories – and now seeks to evaluate engineering programs based upon the particular goals and objectives of each university, college and department. The new Basic Level Accreditation Criteria are:

  1. Students (student quality and performance)
  2. Program Educational Objectives
  3. Program Outcomes and Assessment
  4. Professional Component (ethics, economics, environmental, social, political, etc.)
  5. Faculty (sufficient numbers & competencies to cover program areas, qualifications)
  6. Facilities (classrooms, labs & associated equipment, a “learning environment”)
  7. Institutional Support & Financial Resources (funding, leadership, support functions)
  8. Program Criteria (curriculum requirements & faculty qualifications, criticality)

The “Objectives” have been defined by ABET as statements that describe the expected accomplishments of graduates during the first few (??) years after graduation. Objectives may include expectations that might apply to all graduates as well as expectations that only some graduates might accomplish.

The “Outcomes” are statements that describe what students are expected to know and are able to do by the time of graduation. These outcomes should also support the achievement of the stated program objectives. These outcomes must embrace the ABET required a-k outcomes:

(a)  an ability to apply knowledge of mathematics, science and engineering

(b)  an ability to design and conduct experiments, as well as to analyze and interpret data

(c)  an ability to design a system, component, or process to meet desired needs

(d)  an ability to function on multi-disciplinary teams

(e)  an ability to identify, formulate, and solve engineering problems

(f)  an understanding of professional and ethical responsibility

(g)  an ability to communicate effectively

(h)  the broad education necessary to understand the impact of engineering solutions in a global and social context

(i)  a recognition of the need for, and an ability to engage in life-long learning

(j)  a knowledge of contemporary issues

(k)  an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice

There must be a “system of ongoing evaluation” of the Objectives that demonstrates achievement of these Objectives and uses the results to improve the effectiveness of the program. There must also be in place an “assessment process” to document and demonstrate that the graduates have met the stated Outcomes. The tools used for the “Objectives evaluation” may overlap with the tools used for “Outcomes assessment”, depending upon the constituents (students, faculty, alumni, employers, etc.) involved.

The evaluation/assessment tools that have already been specified by the NCA (North Central Association of Colleges and Schools) self-assessment process and are in use include:

(a)  senior exit survey

(b)  student performance on FE Exam

(c)  student performance in Senior Capstone Design Course

(d)  Survey of employers of graduates

(e)  Survey of alumni

Other tools suggested by ABET include student portfolios, other nationally-normed subject content examinations (GRE, etc.) and documentation of placement data of graduates.

The following process refers specifically to the ABET assessment requirements – the assessment tools used for the graduate programs to meet the NCA requirements are modeled after the ABET process described below.

OVERVIEW OF THE ABET SELF-ASSESSMENT PROCESS

Several assessment methods have been adopted to provide indicators of every program outcome (Table 1) and objective (Table 2) for the Civil Engineering Program. The correlation between assessment methods, outcomes and objectives is summarized in Table 3.

A set of criteria (Table 4) have been established to provide more objective indictors so that each assessment tool either indicates that the program has met that outcome or objective, or reveals an issue that must be addressed by the department to continually improve the program.

The constituents for our program are faculty, students, alumni, our Visiting Committee and employers of our graduates.

The assessment process is administered, and results are collected, by the Department ABET Representative. The results are compiled and an annual report is produced. The department faculty discuss the ABET issues during faculty meetings throughout the year, and the annual report is presented and discussed at the first faculty meeting of each year. Recommendations that involve making curriculum changes are given to the departmental Undergraduate Affairs Committee to investigate possible modifications and recommend improvements to the faculty.

Senior Exit Survey. The seniors are interviewed by the department chairperson shortly before they graduate, and they complete a survey (see appendix) to document their opinions. The response scale ranges from 5 (excellent) to 1 (poor).

Faculty Course Reflections & Evaluations. Faculty self-evaluate each course taught at the end of the semester, rating the level of contribution for each of the outcomes and objectives stated for that course. The rating scale varies from 5 (very strong contribution) to 1 (no contribution).

FE Exam Performance by Seniors. Seniors are strongly encouraged to take the Fundamentals of Engineering (FE) Exam prior to graduation. The percentage of students passing the FE exam (compared to the number of students taking the exam) is used as a metric.

Senior Capstone Design Performance. All students must complete the capstone course to graduate, and all students must participate in preparing the written report and well as in the formal presentation to their peers, faculty and invited guests.

Student Surveys in Key Courses. Students taking a key course (Transportation, Steel Design, Reinforced Concrete Design, Environmental Engineering II, Foundations, plus Construction Planning and Estimating complete a survey. The survey questions are related to the Outcomes and Objectives, but the questions are paraphrased in a manner to be more understandable by the students. The key courses represent a cumulative body of knowledge at the end of a course sequence, and also include significant design content.

Survey of Employers of Graduates. A survey (see appendix) is taken every other year to solicit feedback from the employers of our graduates to review their preparation at CSU for the profession.

Survey of Alumni. A survey (see appendix) is taken every other year to solicit feedback from program alumni regarding their preparation at CSU for the profession, as well as their progress as professionals.

Visiting Committee. This committee is comprised of approximately 20 professionals from all aspects of the civil engineering discipline. They meet annually, and they have met and discussed the ABET outcomes and objectives and the CSU program. Their input is in the form of comments and recommendations recorded as meeting notes. This input will be discussed for each of the relevant outcomes and objectives individually.

The schedule for conducting each of these assessment measurements is outlined in Table 7.

SELF-ASSESSMENT OF OUTCOMES

The self-assessment of outcomes from the most recent ABET Annual Report is included in Appendix A. This appendix details the analysis procedure for the measured data. The values of the collected data may change annually.

The annual report for the upcoming year will reflect the changes described in the body of this manual.

SELF-ASSESSMENT OF OBJECTIVES

The self-assessment of objectives from the most recent ABET Annual Report is included in Appendix B. This appendix details the analysis procedure for the measured data. The values of the collected data may change annually.

The annual report for the upcoming year will reflect the changes described in the body of this manual.

CLOSING THE LOOP

ABET Annual Reports

The purpose of the ABET Annual Reports, initially, was analyze (and later develop a quantitative evaluation tool) the ABET data to determine where we met the ABET criteria, and where we did not.

However, this leaves a gap between meeting the threshold criteria for an accredited program, and the ongoing effort for continual improvement of the program. Therefore, these reports have been expanded to include sections that describe the status of continual improvement efforts over the past year and also identification of the continual improvement efforts that will be pursued during the coming year. This will close the loop to bridge the gap between assessing if the program meets the threshold criteria and planning for continual improvement.

The faculty discuss opportunities and approaches for improvement of the program at every ABET faculty meeting – and at many other faculty meetings as well. These ideas are frequently addressed by a faculty or a faculty sub-committee during the year. When there is a solid plan for action, then it is brought before the faculty for discussion, modification and approval. The ABET report may provide some insight as to where improvements might be made, although there is not an existing deficiency in the program.

Input from our constituents for our program (including faculty, students, alumni, our Visiting Committee and employers of our graduates) will be incorporated into all program discussions.

Appendix A

SELF-ASSESSMENT OF OUTCOMES in AY 2004 - 05

Each program outcome has a specific set of assessment tools used to measure the level of success meeting that outcome. Based upon the metric scales of each assessment tool, a set of threshold criteria (Table 4) has been established to determine the level of success meeting each of the outcomes. The assessment results, discussion and conclusions will be presented for each outcome in the annual report.

A sample summary table of the outcomes scores resulting from the faculty course reflections and evaluation process is provided in Table 5.

Outcome a. An ability to apply knowledge of mathematics, science, and engineering.

Prior to students taking relevant courses within the civil engineering program, several courses are taken from other departments (ESC courses) and colleges (mathematics, chemistry, physics) which are important prerequisites. Later in their studies, their performance in the relevant CVE courses (particularly ESC courses) reflects ability to apply that acquired knowledge. The effectiveness of this foundation is well tested by the engineering courses taken early in their engineering curriculum. The courses and activities that contribute significantly to meeting Outcome a include:

Calculus I

General Chemistry I

General Chemistry Lab I

FORTRAN or ASSI C

Calculus II

University Physics I

Surveying

Multi. Calc. for Engineers

University Physics II

Statics

Differential Eq. for Engineers

Surveying Lab

Strength of Materials

Fluid Mechanics

Eng. Statistics & Probability

Electrical Eng. Concepts

Linear Algebra & Num. Methods

Materials Science

Dynamics

Thermodynamics

Engineering Economy

Co-op experience

Engineering design contests

Assessment Method / Minimum Criteria / Score / Met Criteria
Senior Exit Survey / 3 / 4.3 / Yes
Faculty Course Evaluations / 8 / 15 / Yes
Fundamentals of Engineering Exam / 50% pass / 80 / Yes
Curriculum / Student Records / 95% met all requirements / 100 / Yes

The assessment scores are used to indicate strengths and weaknesses of the program as it pertains to outcome a.

Documented evidence of contributions from specific courses within the curriculum is included in the course binders. The following lists the course binders and materials that have been identified by the faculty as making significant contributions to meeting Outcome a: