UNIVERSITY ASSESSMENT PLAN FOR POLYTECHNIC

Goals and Objectives of Assessment

Assessment is, first and foremost, a process. It is not in and of itself a product, although the process will “produce” a variety of results and documents that are an important part of the process. The process must lead to continuous improvement in the delivery of service to the University’s many constituencies.

In the ABET 2000 era, assessment focuses on identifiable outcomes that demonstrate the achievement of clearly stated missions, goals, and objectives.

Thus, the goals and objectives of the assessment process include:

  1. Documenting the University mission.
  2. Documenting the mission, goals, and objectives of each academic program of the University.
  3. Documenting the mission, goals and objectives of each non-academic unit or function of the University.
  4. Measuring, on a regular and repetitive basis, the achievement of mission, goals, and/or objectives by the University as a whole, and by individual academic and non-academic units.
  5. Identifying shortcomings in the achievement of mission, goals, and/or objectives at all levels of the University.
  6. Development of plans to improve/advance the achievement of mission, goals, and/or objectives of the University as a whole, and of individual academic and non-academic units.
  7. Incorporating feedback on specific plans and actions into subsequent measurements of success against mission, goals, and/or objectives.

It is important to recognize that the entire University community must be involved in the process of assessment and continuous improvement. An Office of Assessment has been established to organize the process, to conduct the many surveys that will be required, to provide a consistent and complete data base for informative statistical analyses, to provide a central location for assessment results and record-keeping, and to assist various units of the University in conducting the assessment process.

The results of assessment, however, must be reviewed and responded to by all affected units of the University. It is important to note that appropriate responses to assessment results are the responsibility of virtually every member of the University community, as assessment will touch every academic and non-academic unit of the Polytechnic.

Organization of the Assessment Process at Polytechnic University

The assessment process at Polytechnic is directed by the Provost as Chair of the Polytechnic Assessment Team (PAT). The PAT serves as a steering committee overseeing various aspects of the assessment process throughout the University.

The membership (Nov 2000) of the PAT consists of:

Ivan Frisch Provost and Chair

Debbie Cassetta Director of Assessment

Dennis Dintino Director of Information Services

Haang Fung Director of Special Services

Dave Gillette Director of Career Services

Ellen Hartigan Vice-President for Student Affairs

Mahnaz Karim Director of Development Information

Jeanette Magee Academic Advisor for Computer Science

Mike Mainiero Director of Institutional Research

Roger Roess Professor

Krisoye Smith Office of Assessment

Nancy Tooney Associate Dean

George Vradis Professor

Cynthia Wolff Library

The membership of this group will change from time to time, but will maintain the same representation as that indicated in the listing.

The specific conduct of assessment activities is supervised by two individuals, one assigned responsibility for all academic aspects of assessment, the other assigned responsibility for non-academic aspects of assessment. These individuals are:

Dr. Roger P. Roess,

Special Assistant to the Dean for Acdemic Assessment

Professor of Transportation Engineering

Mr. Michael Mainiero

Special Assistant to the Provost for Non-Academic Assessment

Director of Institutional Research

The Office of Assessment, established as part of the Office of the Dean of Engineering and Applied Sciences, is responsible for the day-to-day operations of the assessment process, and for maintaining a comprehensive data base for assessment. It is led by a Director of Assessment (Ms. Debbie Cassetta), who reports to the Special Assistant to the Dean for Academic Assessment. It contains a complement of supporting personnel, including a computer data analyst, secretarial assistance, student assistance, and a budget including travel, equipment, and substantial M & S support.

There are two working groups that assist. The Working Group for Academic Assessment includes a representative of every academic department, the library, and special services. It is chaired by Ms. Cassetta as Director of Assessment. The Working Group for Non-Academic Assessment includes representatives of various non-academic units, and is chaired by Mr. Mainiero.

Another group, called the Assessment Advisory Board, consists of representatives of student groups, alumni, and industry. This board performs an advisory role to the Provost, and provides input on surveys, survey results, assessment processes, and other aspects of the process.

Assessment Baseline(s)

There are four base documents that define the overall context of University missions, goals, and objectives against which outcomes are to be measured:

  1. Academic Mission and Objectives for Polytechnic University This document includes a formal statement of the University’s overall mission, and a summary of the mission statements, goals, and objectives of every academic department and inter-departmental program.
  1. Strategic Plan The Polytechnic Strategic Plan, formally approved by the Board of Trustees in Spring 2000, provides an additional set of specific goals and objectives, as well as strategies to achieve them.
  2. Missions and Objectives for Non-Academic Units of Polytechnic This document includes a general mission statement for supporting services and departments at the Polytechnic, and specific missions, goals, and objectives for individual non-academic units.
  1. Course Objectives and ABET Competencies Each academic department maintains a documented set of course objectives for every course offered. Each undergraduate course also maintains a list of the ABET-defined core competencies that the course addresses. These documents are created and maintained by each department, and copies are made available to the Assessment Office. They are also incorporated into each semester’s course evaluation surveys.

Each of these documents is reviewed and modified as appropriate each year as part of the assessment process. While the PAT supervises the effort, all appropriate units and individuals participate in the review and modification process.

Assessment Approaches

A.  Academic Assessment

There are three general approaches employed at Polytechnic in the implementation of academic assessment:

¨  Traditional review and grading of student material.

¨  Data analysis.

¨  Constituent surveys.

Traditional Grading Approaches

While certainly not sufficient, traditional grading of student material continues to be an important and necessary part of assessing both the progress of individual students and the success of curricula and their delivery. Polytechnic has made a substantial effort to insure that a wide variety of assignments are included over the course of a student’s academic career. These include:

¨  Examinations, including numerical, short answer, and essay style questions.

¨  Homework assignments, including numerical and written work.

¨  Term paper and/or project assignments, both for individuals and groups.

¨  Oral presentations of project/term paper results.

¨  Design projects (group and individual) resulting in a fully documented design report.

¨  Laboratory reports on the conduct and results of laboratory experiments.

Each course employs one or more of these techniques to assess the progress of individual students. The quality with which such assignments are completed, however, also provides insight into the effectiveness of the curriculum and its delivery.

Data Analysis

A great deal of insight can be gained by careful analysis of the basic data provided by the University information systems. For students, the Polytechnic information system contains information on admissions, financial aid, academic standing and progress, enrollment status, grades, etc.

Typically, new freshman cohorts are identified each year. These become the basis for analysis of persistence, transfer, and graduation rates. Sub-cohorts based upon gender, ethnicity, H.S. origin, SAT scores, placement exam results, GPA and others can be identified and separately analyzed to reveal problems that are unique to various components of the student population.

Correlations among key variables, such as GPA, placement exam results, SAT scores, persistence, and graduation rates can be explored and examined. Cohorts by major, ethnicity, gender, SAT scores, etc. can be explored and compared. Such analyses are conducted by the Office of Assessment, and the results are widely disseminated to the PAT and Working Groups, and academic and non-academic departments as appropriate.

Constituent Surveys

A number of surveys are conducted each year as part of the assessment process. These are listed below, with a brief annotation of what each consists of and what it hopes to accomplish.

  1. Semester Course Surveys A systematic course and instructor evaluation survey is conducted each semester for all (undergraduate and graduate) courses at Polytechnic. Each survey has several distinct parts, including: a) general and specific questions concerning the course material and its delivery, b) general and specific questions concerning the effectiveness of the instructor and the various approaches used in the classroom and outside it , c) questions concerning the effectiveness of the course/instructor in developing the ABET competencies (only those addressed in the course description and outline are included in the survey), d) questions related to the specific goals and objectives set for the course, and e) special questions related to aspects of a course that might be unique. By vote of the Polytechnic Faculty, participation in the course evaluation survey is required of all undergraduate students in sections of 6 or more. It is optional for all other courses. Grades are withheld until students required to participate in the survey do so. The course survey is conducted electronically over Polytechnic’s intra-net.
  1. Senior Exit Surveys Each semester, graduating students (undergraduate and graduate) are asked to fill out a general exit survey that covers all aspects of their university experience, including curricula and instruction, facilities, supporting services, social environment and a variety of other issues. By vote of the Administrative Council, participation is mandatory. Diplomas are withheld until an exit survey is completed. The senior exit survey is also conducted electronically over Polytechnic’s intra-net.
  1. Alumni Surveys Each year, alumni are surveyed at specified intervals after leaving Polytechnic. Alumni are surveyed one year, five years, ten years, fifteen years, and twenty years after graduation. These surveys seek to establish the placement and career experiences of alumni, as well as their opinions on how well Polytechnic prepared them for their careers. This annual survey is coordinated with the Alumni Office, and is conducted both electronically and with paper (at the discretion of the respondent). As an experiment, some of these surveys will include a companion survey for the recipient’s immediate supervisor.
  1. Employer Surveys Working with the Career Services Office, an annual survey of major employers of Polytechnic graduates is conducted. This survey seeks to establish views of these employers on the preparedness of Polytechnic graduates for the working environment. A related survey of Polytechnic co-op students is also conducted.
  1. Faculty Surveys Every two years, a major survey of Polytechnic faculty is conducted. It is intended to determine what instructional techniques are being used, gather information on faculty training needs, assessing the faculty view of the adequacy of teaching facilities and support services, etc.
  1. Staff Surveys Every two years, a major survey of Polytechnic staff is conducted. It is intended to gather information on the quality of supporting services throughout the University and to elicit the views of staff on various relevant issues.

Each year, as the results of these surveys are analyzed and discussed, the survey instruments themselves are assessed and modified where necessary.

A variety of special surveys may also be conducted on an ad-hoc basis. Some academic units conduct their own detailed surveys (such as the Library, Special Services, and Career Services) of users. The Office of Assessment and PAT always receive copies of these surveys and their results. In some cases, aspects or all of a given survey may be built into one of the regularly-occurring surveys noted previously.

B. Non-Academic Assessment

A parallel process to that for academic assessment is also necessary to assess the performance of “non-academic” offices. Such offices include, for example, the business office, the human resources office, the purchasing office, the facilities office, etc. These and other offices provide essential services to various constituencies, including students, faculty, staff, alumni, and industry. A framework for such assessment is based primarily upon stated goals and objectives in the University’s Strategic Plan, supplemented by annual Management by Objective (MBO) statements from the various vice-presidents.

Because the Strategic Plan is a broad document, some refinement of the general goals and objectives is needed to do a realistic outcomes assessment. Using the Strategic Plan and MBO’s a consolidation of goals and objectives must be performed, and then related to the various non-academic units responsible for carrying them out. This consolidation takes place annually, and is done by the Non-Academic Working Group and reviewed by the PAT. The process includes:

·  Review and organize the tactics and strategies defined for non-academic units.

·  Clarify the outcomes in terms that are conducive to evaluating relative success or failure.

·  Develop appropriate assessment methods with the objective of minimizing the number of separate surveys and/or analyses that must be conducted.

·  Define the process for collection and reporting of results of non-academic assessment.

Because the assessment of non-academic areas is a newer function, and because there are more complicated issues involved, the process for these assessments must itself be clarified and developed over a period of several years. It is also clear that the assessment process for non-academic functions must be carefully reviewed and refined each year.

The process will rely primarily on data analysis, and on constituent surveys.

Constituent Surveys

A good deal of information concerning non-academic functions can be included on surveys that are already a part of the academic assessment process. For items that impact service to students (like financial aid, admissions, registrar, facilities, student life, etc), questions may be included on annual senior exit surveys. The biannual faculty surveys can include questions on services for which they are constituents (payroll, human resources, purchasing, facilities, student record systems, etc). Alumni surveys can include questions on the Alumni Office and its services, for example.

In addition to coordinating appropriate questions on regular academic surveys, it may also be necessary to undertake specific surveys of other constituencies. Other constituencies that are not part of the academic surveys include staff and parents. Regular surveys of these groups, initially once per year, perhaps reducing to once every other year in future years, will be developed and implemented.