NEW PERSPECTIVES ON SELF-EVALUATION
Pedro Municio
Universidad Complutense de Madrid
Self-evaluation is one of the concepts and practices that have evolved the most during the past two decades, with the difficulty that people and institutions are permanently on the border between traditional models and recent contributions, between the traditional paradigm, the quality paradigm, and postmodernism. The difficulty stems from the impossibility of finding coherence between different perspectives, especially between the traditional models and the new paradigm with which the various population cultures, social institutions, administrative organizations, and companies are confronted. Very often, people find themselves behaving differently, even contradictorily, depending on the environment. For most, the evolution has been so rapid that only the chaos produced by the change of values can be perceived.
The meaning of self-evaluation varies depending on the different perspectives and it is difficult to give only one definition of it. According to the traditional models, self-evaluation, carried on from the top, consists in a management follow-up whose aim is to make sure that the activity is performed appropriately. The evaluator never performs the task himself. In the quality models, the organization evaluates itself by comparison with an excellence model; the result of which is the starting point for futher action. While in the traditional models the evaluation depends on the individual and subjective criteria of the “owner” of the management model, in the quality models, it is carried out objectively and following quantitative components.
Improvement models are those that directly support the self-evaluation carried out by teams and individuals on their own processes. This is a quality principle but, in practice, only these models try to make it effective. That evaluation is part of the activity of any individual relies basically on the concept of continuous improvement and quality as improvement.
The models derived from the Balanced Score Card are designed to continuous evaluation all levels of the organization or project and, therefore, consider self-evaluation as a follow-up of a complex set of aspects of the activity itself. Therefore, it is the best model to represent self-evaluation, since it forms part of the agent dynamics itself.
Differences between the various models are quite diverse. To begin with, we need to take into account who defines quality, since the implications will greatly vary if quality is established at the top (by the director, the technician or the ideologist), as defined by the traditional model, or if it is established by the user. Among others, this step assumes the existence of quality criteria and the way of establishing descriptors, in order to delimit and measure it. For quality models, the customer becomes an essential component.
TRADITIONAL EVALUATION MODELS
Traditional models, although based on the eighteenth and nineteenth-century social culture, have developed in the twentieth century and can still be traced in most twenty-first-century educational thinking. Traditional models are, still current found especially in the education system, due to culture inertia, and their permanence seems assured in the near future. In accordance with the society that created them, tradicional models are hierarchical thinking models in which evaluation always represents a top-down exercise of the organizational power, even when they adopt the form of self-evaluation.
Any evaluation model is superimposed to an action model since its goal is always to improve the functioning of the action. This means that the history of evaluation models, be they implicit or explicit, is closely linked to control. A quick overview of this evolution helps us notice, for instance, that quality control in education has always depended on who had the power. During the Middle Ages, it was under the pope’s authority that the “licencia ubique docendi” was granted; in the Renaissance, this prerogative was on the hands of the princes, while in the eighteenth century it was granted by the Public Administration. As public organization tries to limit the citizens’ rights, evaluation becomes increasingly rigid and permanent.
Strictly speaking, evaluation models appeared on in the second third of the twentieth century, generally in decentralized systems, particularly in the United States and in large companies that needed to keep tabs on their internal functioning, which was difficult to do directly from management. Ghey are generally, these models are verbal and they stem from the need to verify that the system works as planned. Those who hold the power regulate the functioning of the system and then want to verify whether the plans have been carried out. Somme groups such as military organization, the royal finances and commercial companies were already using them in the sixteenth century when writing their reports. But they only became necessary with the growing of organizations during the nineteenth-century industrialization.
The evaluated quality is internal, and focused on the implementation of activities, the following of procedures and the use of resources. It is, therefore, an internal measure of conformity and compliance, which considers the products only in its most advanced variants. “First party” models, as they are called (where the organization evaluates itself) are the combined result of theories, ideologies, prejudices, experiences, and research. They represent a personal perspective of the organization management and can be called quality control models. Although it may seem strange, many organizations whose managers have little management training still use them.
The last phase of these models focuses on identifying low quality products before they are released. Quality controls were first established at the end of the process but it was soon realized that to the waste costs, you needed to add the control costs, without the generation of a substantial system improvement. As a result of this, controls introduced at the raw material level. Finally, the model was widened in order to control the implementation of activities as the management established them during the whole process.
These models (which are usually not well-defined and generally verbal) include the three variants of the traditional paradigm (Municio, 2000c). In the satisfying variant (characteristic of the Public Administration), models essentially measure activities (paid visits, explained concepts, registered students), the use of resources and the observance of the norms. Activity indicators reflect primarily the functioning, that is, “what” is being done and “how many” times something is being done during the period taken as reference. Resource indicators measure the level (teachers’ degrees) and their use or distribution (time spent in the facilities and cost by registered student). The compliance with the norm is dichotomic and required as a previous condition. The model does not consider the result of the activity, which is understood as an automatic effect of the causal relations that the agent has in mind when he operates.
The political variant, often combined with the satisfying one, is basically concerned with “how” tasks are carried out and relations established. Emphasis is made on the process, considered as the sum of interactions and relations of those who participate in the action. In this variant, models are not very explicit and are dominated by and ideology whose assumptions are the evaluation referents. They are extremely common in the social world, and especially in education.
The optimizing variant can mainly be found in production organizations, although also in education. In this variant, models are concerned with controlling the results (when achieved), whereas “what” is achieved, and “how”, are considered secondary. In any case, they can be used to study the causal relation between entries and processes, and the resulting products. What is important in optimizing models is to measure the achievement of the goals considered at their possible maximum level.
Accreditation and Certification
The failure of the traditional models resulted in accreditation, an evaluation performed by an external “evaluator” who establishes the quality conditions. Evaluation is defined and performed by someone interested in the evaluated activity and who has power over it. That is why this is called “Second Party”. The evaluation is theoretically voluntary but, in practice, the pressure exerted by the other party makes it obligatory.
The first third of the twentieth century saw the appearance of State Inspection, which acts as a warrant, establishing a control system based on administrative norms. In the 1990s, some Latin American countries created, various accreditation agencies, more or less depending on public power. This development was due to the autonomuy of the universities and the abuses this generated. This actually represents a return to administrative control although now confronted with new difficulties when exerting a regulatory action. Coherently with the origin of this accreditation, the results usually are formal and bureaucratic, even though they might produce positive or negative reactions.
The chief development of these models has taken place in the big industry, particularly in the defense sector, nuclear energy, aerospace and automobile construction. In these cases, the evaluators are the clients, who need to ensure that providers can fulfill the demands stated in their contracts as suppliers. This entails that each company evaluatees the production capacity, the planning, the organization, etc. of the hired firm and accepts the system quality of this company to produce those services or products it supplies. Accreditation adopts various names such as authorization, approval, inspection or homologation.
As a result of the high cost of the “Second Party” evaluation due to the large number of evaluations and the variety of requirements entailed in each case, it has become necessary to generalized the use a more simple and normalized system, by means of which an independent entity may perform the evaluation. “Third Party” certification or evaluation is thus born, whose best-known exponent are the ISO norms.
The oldest example of these models can be traced back to North American universities, which established independent agencies capable of verifying compliance with the basic requirements of all the institutions which voluntary underwent evaluation. The agencies created models whose observance was able to guarantee the public that the institutions had the basic resources, could achieve the established procedures, and were able to carry out the activities described in the reference document. The agencies certify before society the compliance with the established requirements.
Various sectors, especially the British industry, had been using for some time common norms of use limited to its members. The need for those norms was not felt until the Second World War, when the Allies realized the necessity of establishing common norms in the armament sector becoming obvious the convenience of international common norms able to transcend borders between nations. The European Community launched the International Standard Organization (ISO), which established the ISO 9000 norms in 1987, and their revision in 1994. At the same time, nations developed specific norms and adapted some of the international ones.
Encouraged by these norms, various countries created independent organizations, which were generally firms specialized in certifying compliance with the norm, that is, ensuring that the norm requirements are being fulfilled. In this type of verification, the evaluated agent establishes in writing what he needs to do, while the certifier verifies that the documentation fulfills the requirements and that the evaluated agent actually executes what is specified in the documentation. The evaluator looks for cases of “nonconformity” with the norms in the organization. Certification models try to rationalize processes and make sure things occur according to what the norm has established. They do not guarantee the organization results, objectives or success, only its good functioning.
The organizations that use this model (approximately 500,000 in the year 2002) part from a philosophy of error prevention in order to attain better results by performing the activities well. On the other hand, the observance of the model allows an open door to other organizations that demand a quality-based processes, in order to accept collaboration and establish alliances.
QUALITY MANAGEMENT MODELS
By the end of the 1970s, traditional models began to prove inadequate. What they were measuring was worthless in the new social and economic context, so new perspectives and solutions began to emerge. What provoked this change was a true revolution in every domain, characterized in its essence by the prominent role of people in opposition to the previous closed and hierarchical systems.
Keys to Social Change
The scientific, technical, geo-political and economic evolution occurred in the last third of the twentieth century has directly impacted people and behavior patterns, producing a change in social values, beliefs and assumptions which leads to the recognition of the importance of the individual, above any organizational, political or economic standpoint. The person ceases to be considered a mere resource and becomes a leading actor in making decisions and searching for results and satisfaction. This implies that political, social and economic institutions need to change their assumptions and teories in management and production in order to serve the people, contrary to what had happened in the previous time period. The recognition of persons as an essential key in any action or decision leads to considering what is necessary in order to fulfill the needs, expectations and interests of all the implied parties in any activity.
The scientific and technical evolution provides means and solutions to the issues of health and well being, making people feel more secure and comfortable, providing them winth improved movement options, an enhanced standard of living and greater perspectives for the future. Geo-politic evolution breaks boundaries between nations and ideologies, and opens for people access to new spaces, products and choices. Economic evolution helps moving from a period in which scarcity (which included famines and epidemics) prevailed, to a new epoch of abundance in which offer exceeds demand and decision-making processes are inverted. The customer has more economic power and finds himself confronted to a variety of offers, which provokes a fracture in the established models. On one hand, this regulates the prices and producers are required to improve processes in order to reduce costs as much as they can; on the other hand, they are required to offer products that respond to the actual needs of the customers. This is how the new concept of quality is born, not subject to the producer or the politician decision-making, but rather depending on the requirements set by the requesting agents. The old paradigm has thus undergone a fatal blow.
If fully people are the essential agents and they, are the source of wealth, with the capacity for decision-making, they necessarily became the center of reality and become the measure of all things. But at the same time, if one accepts the idea that people are not all the same, that they have different values, beliefs and different maturity levels, then one has to acknowledge that they will have different vital needs, expectations and assumptions. Any human group, regardless of its size and characteristics, is conformed by a diversity of individuals, and will therefore be required to respect subjectivity while operating with norms that allow the largest possible degree of liberty. To accept other people’s ideas and attitudes becomes a social good, very different to the regulatory system of the previous period. Quality is not to be considered uniform, nor can it be imposed, but it must represent each individual and the group, when the latter creates common rules.
This axiom brings along the concept of diversity as an external expression of behavior. Denying diversity can be considered as a sign of the previous age: homogenous teaching methods, single study plans (or false diversifications), political impositions stemming from the dominant ideology, etc. But the effects of diversity have become increasingly noticeable and capable of breaking through traditional models. Secondary education becomes chaotic when it attempts to yield adolescent students (who may come from different social backgrounds and have different academic levels and attitudes toward education) to single common studies. The increase of immigration, associated to different socio-cultural values, the economic and commercial globalization, as well as the creation of supranational states, call for continuous contacts and contracts. Diversity has become a new reality, and the acknowledgment and handling of it an indispensable necessity.
Political democracy, practically absent throughout history, has extended outstandingly during the twentieth century. If only 48 nations were democracies in 1900, in the twenty-first century that number reaches 113. The decline of absolutist ideologies, and the fall of the Berlin wall constitute a transcendental step in this process. Slavery, accepted and maintained throughout history, has almost disappeared. By the end of the twentieth century, slavery is no longer officially supported by any nation, although in practice, the estimated number of slaves comes to 27 million. There have been substantial changes throughout the twentieth century in regards to women discrimination and racial segregation. Naturally, in these and other changes, some countries have progressed more than others, and there is to be found a difference between what laws defend and what the actual costumes impose.