The CampusOne Model for Assessment

of a Course of Study

M. Gola, R. Mirandola, L. Modica, A. Squarzoni, E. Stefani, P. Tosi, M. Tronci

The CampusOne Model

Structure of the model

The CampusOne model proposed for self-assessment and assessment of a Course of Study (CS) takes into account the experience that already exists in this field, both nationally and internationally.

The model has the following features:

•a simple structure showing clearly how the entire course of study is run;

•it is broken down into well-defined Aspects and Elements that provide minimal units for assessing the CS and identifying strengths and weaknesses;

•through a series of questions and factors, it permits analysis of the management system and results of the CS, so as to allow them to be checked and for steps to be taken to improve them.

The proposed Model identifies the following five aspects of the assessment:

  • Needs and Objectives
  • Organizational system
  • Resources
  • Educational process
  • Results, Analysis and Improvement

Each aspect is subdivided into elements, which make it possible to focus the assessment on particular features of each aspect. These elements must be assessed individually and, if all are assessed positively, they add up to a positive overall assessment of the aspect concerned.

The elements of each aspect are as follows:

Needs and Objectives

/ Needs of the interested parties
General and policy objectives
Learning objectives
Organizational system / Responsibilities
Management System
Reexamination
Resources / Human resources
Infrastructures
Educational process / Planning
Delivery
Support services
Results, Analysis and Improvement / Results
Analysis and Improvement

For each element of the assessment’s five aspects, the model lists the topics in the form of questions, and also the factors to consider for each one, in order to come to assess that particular element. These factors do not claim to exhaust all details that are useful for a description of the course. The CS may add other factors thought to be useful in assessing each element.

The CS must provide a concise but complete reply to the questions put as regards each element, and emphasize in particular strengths and weaknesses, in order to reach the stated objectives. In this connection it is possible that some (or many) of the activities for which a description is required, so as to cover all elements of a particular aspect, are not directly carried out by the CS but are carried out under the auspices of the Faculty to which the CS belongs; and/or are carried out by the Faculty for groups of CS or even for all CS belonging to a particular Faculty. Such situations must be clearly spelt out by the CS; this does not however exempt the CS from replying to the questions to the extent that they have responsibility in these particular matters.

In both self-assessment and external assessment, the model provides for a mark to be given for each element in the five aspects, which must be assessed for its contribution to the quality of the educational effectiveness of the CS. The assessment must also gauge whether the results obtained are erratic or the result of a systematic plan of action, of an adequate management system. Thus each element is assessed according to the following scale:

0 / Not
assessable / The element is not assessable in that it has not been developed, or if it has been touched on, this has been done without any systematic approach being evident, and/or the information available is not sufficient and/or the results are lacking, skimpy or haphazard.
There are grave inadequacies here.
1 / Acceptable / Reactive approach based on the systematic resolution or correction of problems. Acceptable results. Little information on action to improve and on the results of such action.
Some deficiencies are evident which necessitate significant improvements both as regards results and the management system.
2 / Good / Systematic approach to the management of activities. Results compared with the objectives and systematically assessed. Action to improve begins to be systematic.
It is possible to identify some actions taken to make improvements in some areas, both as regards results and the management system.
3 / Excellent / The approach to the element or the result obtained demonstrates excellence. The management system is suitable and effective. The results match the objectives. The process of improvement has been developed in a systematic and integrated manner.
This may serve as a model for finding solutions or creating new courses, and it is difficult to think of what might be done better.

Both as regards self-assessment and external assessment, the overall assessment of the each aspect is made up of the assessments of all its elements.

In order to be assessable, each area must be made up of elements assessed as being at least acceptable. Aspects containing more than one unassessable element therefore become unassessable themselves.

The aspects are therefore automatically assessed according to the following scale:

0 / Unassessable / If more than one element of the aspect has obtained a mark of 0.
1 / Acceptable / If all the elements of the area have obtained a mark of at least 1 or if at least one element of the area has obtained a mark of 0.
2 / Good / If all the elements of the aspect have obtained a mark of at least 2.
3 / Excellent / If all the elements in the aspect have obtained a mark of 3.

Besides the five aspects, the model offers the CS the opportunity to describe initiatives undertaken to add value to the course of study and make it more interesting and effective for the student. The model presents three examples of elements deemed to be important for a modern university course of study (one of them requiring an obligatory description since it is stipulated in the CampusOne Plan), but the CS can indicate and describe other activities which it thinks necessary for a complete account of the services it has to offer. These initiatives, although an integral part of the model, are not taken into consideration for the purposes of the assessment.

The theoretical approach of the model

Any activity which transforms input into output can be considered a process which normally is always connected with other processes, and can always be subdivided into smaller discrete processes. For example, the following are all processes: planning a course of study (at any level), the delivery of the teaching, the management of resources, the supply of the support services, etc. In general, all organizations must, if they are to function correctly, identify and manage many processes that are interconnected, in which the output of one process often constitutes the direct input for the next output that follows.

The identification and “controlled”[1]management of the various processes characteristic of any organization, and therefore also of a university course of study, and in particular of the interrelations between such processes, constitutes the so-called “process” approach to its operation. The adoption of such an approach is also recommended in the regulations of the ISO 9000 family, since it is considered the most suitable approach for identifying and managing the opportunities for improvement.

Figure 1 is a conceptual representation of the management system for a course of study in a process perspective, and offers a diagrammatic representation both of the interactions between the external/internal worlds (Interested Parties), and of the connections between the five macro-areas:

fig. 1

To sum up, the CS, through the various persons holding responsibility, analyses the need of the Interested Parties (IP) and defines the objectives to be reached (Aspect: “Needs and Objectives”), defines the organization of the CS and applies an appropriate management system (Aspect: “Organizational System”), manages the resources necessary to attain the objectives (Aspect: “Resources”), delivers the educational and support processes (Aspect: “Educational Process”), and the results of these are measured, analyzed and become the object of plans for improvements (Aspect: “Results, Analysis and Improvement”). The needs, objectives, organizational system, resources and processes are then the object of periodic reexamination in order to check on their continuing validity.

Illustration of the model

There follows a description of the model following the logic of the five aspects of the assessment indicated previously. The model has been set up so that it is possible to use not only as a guide for self-assessment and assessment, but also and especially as a guide for the good management of the CS, and especially as a stimulus for continual improvement.

CampusOne Model
Needs and objectives
Organizational system
Resources
Educational process
Results, analysis and improvement
NEEDS AND OBJECTIVES
NEEDS OF THE INTERESTED PARTIES

The purpose of this element is above all to ascertain and assess the capacity of the CS to identify and define in a clear and documented fashion, having regard both for the socio-economic context in which the CS operates and the context into which it is presumed that graduates will fit, the educational needs – cultural, technical and/or scientific – and needs in order to be employable now and in the foreseeable future.

Question / Factors
Have the needs which the CS meets been identified? / Reasons for setting up the CS
Interested parties that have been identified
How contacts with the interested parties have been maintained
Identified needs of the interested parties
Employment prospects
Involvement of the Student Advisory Committee (CampusOne requirement

The Interested Parties (IPs) of the CS have needs which it is expected will be satisfied by the positive action of the CS.

The CS must identify its interested partied clearly and maintain close contact with it in all the management stages of the educational process.
To identify the needs of the IPs the CS can, for instance, make use of information coming from the members of the IPs themselves, of exchanges and meetings with them (for this purpose the CampusOne project requires the active presence of a Student Advisory Committee on which there are representatives of the social, economic and entrepreneurial organizations in CampusOne’s recruitment area), of statistics on the job market in the recruitment area, of research into the student population, etc.

Identification of the needs of the IPs constitutes a primary and indispensable element for defining the general objectives, policies and learning objectives of the CS.

GENERAL AND POLICY OBJECTIVES

The purpose of this element is to ascertain and assess the capacity of the CS to define in clear and documented fashion its general objectives and policies in a way that takes account of the needs that have been identified, and advertising these objectives and policies among staff (teaching and support staff) and the IPs.

Questions / Factors
Has the CS identified the roles for which it proposes to prepare the students in a way that is in accordance with the needs identified? / Type of graduate intended to be produced
Possible jobs in the area
Methods of advertising used
Has the CS defined its policies in a way that is in accordance with the stated general objectives? / Student admissions
Staff recruitment and management
Resource funding and management
Role of the “labor market”

A clear definition of the general objectives is the basis of every other feature of the CS: indeed, before beginning any other activity (planning, staff recruitment, etc.) the CS must define in a clear and comprehensible manner what are the aims of the course of study, and how it meets the professional needs identified with the IPs. The better the definition of the roles for which the CS intends to prepare the students, the better are the planning activities that result.

Through its policies, the CS defines its philosophy, orientation and behavior in the management of all activities connected with the CS.

The policies defined and carried out must fit the general objectives that the CS has set itself so that they can actually be achieved, and they must not present elements that contrast with the policies carried out by the academic grouping to which they belong.

Policies may be defined for specific aspects such as, for example, student admissions, staff management, means of delivering the teaching, the development of the support services, etc.

The general objectives and policies must be understood by all staff members so that each is able to contribute to both their formation and implementation.

LEARNING OBJECTIVES

The purpose of this element is to ascertain and assess the capacity of the CS to define learning objectives that fit the educational needs that have been identified. These learning objectives must be measurable, realistically achievable in the time frame offered by the duration of the CS (with reference in particular to the profile of the average student on entry), capable of being planned in terms of the time allotted and weighting of the various components. The learning objectives must be defined in terms of the things learnt (knowledge), the capacities and skills (know-how) and the behaviors (personal development) expected of the student at the end of the educational process.

Question / Factors
What are the principal characteristics that the CS wishes to produce in its graduates? / Learning objectives in terms of:
- knowledge
- capacities
- skills
- behaviors
Comparison with the learning objectives of other CS of the same type

From the general objectives that have been defined and therefore the roles for which the CS intends to prepare the students, there follow the characteristics that the graduates must have acquired at the end of the learning process.

These characteristics are acquired by the students when the learning objectives are attained. These objectives, defined in respect of the graduate who finally emerges, may be developed to various levels during the learning process. The CS must therefore distribute the learning objectives to be attained by the graduate (once he or she has completed the whole learning process) into learning objectives which the student must have attained after each learning experience (teaching modules, seminars, practical training, thesis, etc.) so that progress can be assessed throughout the whole delivery of the educational process.

The learning objectives must be measurable, realistically achievable in the time frame offered by the duration of CS[2], planned as to time and weighting[3].

The learning objectives must be defined in terms of things learnt (knowledge), capacities and skills (know-how) and behaviors (personal development) expected of the student.

All teaching staff of the CS must be aware of the importance of getting the students to reach the stated learning objectives by the end of their spell of teaching.

ORGANIZATIONAL SYSTEM
RESPONSIBILITIES

The purpose of this element is to ascertain whether the CS has developed an organizational structure of its own with a clear definition of the responsibilities and tasks for managing the CS, whether this structure fits the pursuit of the stated general and learning objectives, and what is the level of commitment in this regard of the various persons responsible.

Question /
Factors
Have the responsibilities for the direction and coordination of the CS been defined and taken on? / Nomination, delegation of responsibility and commitment as regards:
- determining and assessing the needs and objectives
- management of students
- teaching coordination
- administrative coordination
- measures taken to correct, anticipate and improve
- monitoring and data analysis
- internal and external communication
Presence of staff with responsibilities for teaching management
(A CampusONE requirement)

The commitment and involvement of all the management and coordinating staff of the CS are fundamental for the development and maintenance of a management system for the course of study that will satisfy the IPs’ needs.

The CS must therefore identify its organizational structure, assigning responsibilities, specifying the levels of authority and defining the reciprocal relationships for all staff who are carrying out or coordinating activities influencing the quality of the educational process and of the support services.

MANAGEMENT SYSTEM

The purpose of this element is to ascertain whether the CS has developed its own management system with a correct definition and description of the activities, an effective management of documents, and the adoption of an efficient system of communication, so as to assess whether the system fits the stated general and learning objectives.

Questions /
Factors
Have the various actions by means of which the educational process is directed and controlled been identified in a clear and documented manner? / Actions to control[4] the effectiveness and efficiency of the educational process
Actions for monitoring the effectiveness of the interventions made
Documents providing guidelines for activities, and documentation of work and records
Control of the documents
Rules or specific models adopted by the CS as a benchmark for the development of its own management system
Is communication with the CS staff, students and IPs effective? / Methods of communication
Methods of checking the effectiveness of the communication

The CS must set in place, document and run a management system as a means of reaching the general and learning objectives and to ensure that the educational process and support services are delivered in a manner consistent with the needs of the students and other IPs.

Every management system of the CS which leads to positive comment is considered equally effective. The CS is free to equip itself with a quality management system based on rules or precise models.

In fact the objective is not that of emphasizing the “formal” features of the CS’s bureaucratic management, but rather the “substantive” aspects aimed at setting up a management system that makes it possible to satisfy student and IP needs through the identification, planning, implementation and checking both of the individual processes necessary for supplying the teaching and of the integration of these processes.

Thus, with reference to figure 1, the educational process that transforms a student into a graduate can be broken down into a series of processes (course planning, teaching delivery, support services delivery, etc.) that interact and which, when properly managed, each contribute to reaching the general objective.