Models of Expert Judgement: the Hidden Assumptions of Professional Education

8809

Models of expert judgement: the hidden assumptions of professional education

Nick Boreham, University of Manchester

Abstract

Perspectives on professional education are often based on oversimplified models of expertise. A taxonomy of such models is proposed. Its use in making hidden assumptions explicit is demonstrated by applying it to a current debate about experiential learning.

Introduction

The notion of mental models has become popular in cognitive science for explaining how people understand the world about them. Rather than perceiving events directly, it is proposed that people think in terms of models which filter information from the environment, provide a basis for understanding it, and limit the range of behaviour which they see as possible or desirable. The basic assumption of this paper is that adult educators have implicit models of expertise, which unconsciously bias their perceptions of alternative educational strategies.

One essential point about mental models is that they are simplifications. Because of the limits on the amount of detail the human mind can comprehend at any one time, people understand events only by simplifying them. Another point about mental models is that they are functional. That is, the way in which a mental model simplifies reality can be explained by the goals the individual is trying to achieve. His or her mental models will only contain those aspects of reality which are necessary for achieving those goals. Consequently, when different people stand in different relationships to the same object, they are likely to have different models of it.

A crucial component of expertise in almost all of the professions is judgement, the art of making decisions under uncertainty. This paper focuses on models of expert professional judgement. These, it will be argued, are hidden assumptions which underlie much debate about professional education. To make them explicit, and to encourage educators to replace simple models with more sophisticated ones, are important objectives for teaching and research in profession education.

To contribute to this, the present paper proposes a way of classifying models of expert judgement. The taxonomy contains two dimensions, representing two major sets of assumptions which educators and practitioners are often divided. The first of these concerns the knowledge base which the professional uses in making his or her judgements, and the second concerns the process by which he or she comes to understand the client’s problem.

The knowledge base

Recently, there has been much research into the kind of knowledge which experts use when making professional decisions. Out of these studies has emerged a distinction between two ways of knowing, the data-driven and the concept-driven.

Concept-driven knowing is the systematic, formalised knowledge which in the professions is mainly drawn from basic scientific research. Frequently, it is expressed in mathematical form. This is the familiar everyday material of higher education, where students of banking may be taught mathematical functions linking money supply with other economic indicators; engineers the thermodynamic principles underlying the functioning of engines; and medical students the conditional probabilities of different diseases given certain symptoms, together with Bayes’ Theorem as the means of calculating a diagnosis from this data.

Data-driven knowing consists of the rules-of-thumb which a practitioner acquires over years of experience. These may be very reliable, but often have no theoretical justification. They rarely find their way into textbooks and examination syllabuses, and so are rarely taught in higher education. Much of this knowledge can be verbalised as productions, rules of the form IF (condition) THEN (action). For instance, IF (the engine runs rough) THEN (test the spark plugs). However, much professional know-how cannot be verbalised as easily as this. A great deal of it seems to be based on insight, intuition and pattern recognition, which are hard to put into words. Such knowledge often makes sense only in a particular context, and involves feelings.

The process of understanding the client’s problem

The other dimension in our taxonomy concerns the process by which the professional person comes to understand the client’s problem. An expert practitioner has had much experience. While each new case may be unique, he or she will inevitably construe it in the light of previous cases. How, then, is previous experience brought to bear in the new situation? A study of the literature on professional expertise reveals two main assumptions here.

One very common assumption is that understanding a client’s problem is a process of applying stored knowledge that the expert has a memory store of prototypical situations, and recognises each new case as an instance of something that has been met before. Whether by formal education or by learning on-the-job, the doctor is assumed to have acquired accurate mental representations of diseases; the engineer representations of forms of energy transfer; the banker representations of good and bad investment risks; and so on. These are all assumed to be inner pictures of an outer reality, and the client’s problem is assumed to be understood by categorising it within this existing framework.

Against this stands a quite different view about the kind of understanding which a professional person may achieve of a client’s problem. This is that the understanding is new knowledge, constructed through a process of interaction between the client and the professional. The constructivist thesis is clearly described in Schön’s book , The Reflective Practitioner: ‘the practitioner has built up a repertoire of examples, images, understandings ... when a practitioner makes sense of a situation he perceives to be unique, he [does not] subsume it under a familiar category or rule ... he sees the unfamiliar, unique situation as both similar to and different from the familiar one without at first being able to say similar or different with respect to what’[1]. On an interactionist view, what a professional learns from previous experience is not ready-made explanations for use in new situations, but the capacity to differentiate between alternative states of affairs[2]. One possibility is that recollections of previous cases are fragmentary, permitting them to be assembled in news configurations as the professional attempts to make sense of the client’s problem. This implies a more fluid memory structure than most theories of human judgement have hitherto assumed[3].

A taxonomy of models of expert judgement

Crossing these two dimensions creates a matrix in which each cell represents a different model of expert judgement.

Process of understanding client’s problem.

APPLICATION OF STORED KNOWLEDGE / CONSTRUCTION OF NEW KNOWLEDGE
DATA DRIVEN / Expert system / Facilitation
Knowledge base
CONCEPT DRIVEN / Applied science / Metacognition

Bottom left is the model of applied science or technical rationality, the view that the professional has a scientific theory which explains events in the world of his or her clients, and applies this by encoding each new case within that framework and reading off the solution which the theory prescribes.

Top left is a more recent model of professional judgement the expert system. The new field of knowledge engineering claims that professional expertise can be elicited from experienced practitioners and formalised as production rules, thus storing ‘the compiled experience of human specialists in their domain of skill’[4] for future use.

Both these models assume that expert professional judgements are held in memory in a ready-to-use format, awaiting only the necessary cues to trigger off the appropriate category or production rule. Top left, the knowledge base is data-driven, and bottom left it is concept-driven. But in both cases, it is ready-formed.

If we abandon this for the assumption that understanding is constructed anew in each professional-client interaction, we move to the right hand side of the taxonomy. Sticking with the data-driven knowledge base, we have in the top right cell the model of professional judgement as advocacy, facilitation, or community development. One of my favourite illustrations of this model is George Brown’s definition of how to succeed in politics - ‘find out what people are thinking, then stand up and say it’. More seriously, about fifteen years ago in Scotland there was an experiment in community psychiatry which illustrates this model very clearly. Alleging the inadequacy of scientific psychiatric diagnosis, the practice was evolved of visiting the community which a sick person had come from and simply asking the patient’s relatives and neighbours what had caused his or her breakdown. This was duly accepted by the professionals involved as ‘the diagnosis’.

Staying with the assumption that knowledge is created in the process of interacting with the client, but abandoning data-driven assumptions for the belief that knowing is concept driven, we arrive at the idea of metacognition. This is defined as the ability to monitor one’s cognitive processes during problem solving. This model is manifested in the role of the consultant - someone who is brought in to observe the ongoing activities in an organisation, but who construes these in a broader framework than the regular staff. Consultancy is based on a good conceptual awareness which may be drawn from theoretical studies, but it is not (ideally) a process of providing ready made solutions. Rather, by interacting with the team the consultant seeks to broaden their awareness of what is going on.

Applying the taxonomy to the current debate

Each of these models is an oversimplification of the decision-making procedures followed by most expert professionals. Yet very often, perspectives on professional education seem to assume that just one or other of them encapsulates the whole of judgmental expertise[5]. By analysing current debates in terms of the proposed taxonomy, the limitations imposed by these hidden assumptions can be revealed and hopefully transcended. This will be illustrated by analysis of a current debate about experiential learning.

A popular technique in adult education is the structured experience. This is a role play or group problem solving exercise enacted in the classroom, which claims to provide experiential learning, generally about some interpersonal process. Content is deliberately trivialised to concentrate attention on the process. Structured experiences are designed to achieve a specific objective, the transactions which they permit are closely constrained, and they are published with detailed instructions for use.

But this technique is not without its detractors. Surprisingly, it has been attacked ferociously by Reg Revans, one of the most influential advocates of experiential learning for professional development since the War. Revans’ own technique of action learning differs in significant respects from structured experiences[6]. First, action learning is full of content, being located in the work place and not in the classroom. Second, action learning is not structured by the educator. Apart from arranging for participants to visit each others’ places of work to exchange experiences, no direction of the interaction is given. This is in marked contrast to the structured experience, where rules, checklists, and strict control of timing are of the essence. But the greatest difference is that the designer of a structured experience claims to know in advance what the learner will discover experientially, while the whole rationale of Revans’ approach is that this cannot be stated. Can these differences be explained in terms of different underlying models of the expertise which it is hoped to develop?

Referring to the vertical dimension of our taxonomy, action learning appears to assume a data-driven model of the professional knowledge base. Revans specifically excludes theory from any role in promoting good professional practice. He has described action learning as ‘in total opposition to the academic tradition of exhibiting logical argument, deep understanding and encyclopaedic knowledge’, and - despite having been the Director of one himself - believes that Business Schools should be closed down. Instead, what Revans hopes for is increased openness to experience in professional decision-making, an upward movement of ideas from the client to the professional.

Action learning is also based on the assumption that solutions to really difficult problems cannot be stored and applied to new situations - that professionals cannot be provided with ready-made solutions. Instead, says Revans, these have to be constructed through interaction in the workplace. So, referring to the horizontal dimension, action learning assumes the constructivist view of the judgmental process. This leads us to locate the model of expertise assumed by action learning in the top right cell of our taxonomy.

Structured experiences, by contrast, assume that knowledge is concept-driven. This is implicit in the way that they are usually designed around some key concept. It is also clearly shown in that standard feature of any structured experience, ‘construing the experience’. Following the role play or activity, everyone sits down and analyses what has happened, often being provided with rating sheets for this purpose. An effort is made to verbalise, conceptualise, and then to make connections with the trainees’ experiences ‘back home’.

Revans’ criticism is that structured experiences do not provide experiential learning because they do not take place in the real situation. It is difficult to deny that they are artificial. But I do not think that structured experiences are addressing the same objective as action learning. While Revans’ technique assumes a data-driven model of the professional knowledge base, structured experiences seem to be used mainly for developing the ability to stand back from one’s own experience and construe it - a concept-driven process. The predominant underlying model of expertise is thus the bottom right cell, metacognition. Occasionally, structured experiences may be used as an interesting way of introducing new concepts, to be remembered for future application, which places them in the bottom left cell.

This example shows how the taxonomy might be used to reveal hidden assumptions. Both structured experiences and action learning have been claimed to provide experiential learning in all its complexity. But if my analysis is correct, each technique assumes a different and oversimplified model of professional expertise. The weakness with action learning is the low priority placed on conceptual knowledge, which excludes metacognition and reduces professional horizons to what can be built out of the here and now. The weakness of structured experiences is that in order to concentrate on metacognition, they trade off actual experience. These weaknesses stem from oversimplified mental models of professional expertise, and it ought to be a priority for teaching and research in the field of professional education to promote more complex conceptualisations.

[1] Schön, D.A. (1983). The reflective practitioner. London: Temple Smith.

[2] Bickhard, M.H. and Campb