Assessing Democratic Competence
1
ASSESSING DEMOCRATIC COMPETENCE
Paper presented at the European Conference on Educational Research, University of Lisbon, 11-14 September 2002
Dr Per Gerrevall
School of Education
Växjö University
S-351 95 Växjö
Sweden
Contents
Introduction
Different conceptions of democracy
Different conceptions of competence
Democratic competence as an object for assessment
How democratic competence can be assessed
Assessment tools for evaluating democratic competence
The foundations
The design of the material
The vignette
Assessment tasks
Evaluation
The assessment
Pilot trials
Some preliminary findings
Discussion
Introduction
An important issue for schooling - perhaps the most important - is the development of democratic values and norms. As a consequence, the development of democratic competence is explicitly expressed in the goals of the Swedish national curriculum. But when it comes to assessment and grading of students[1] when leaving school, democratic competence is rarely the focus of interest. Instead, the assessments performed by teachers in compulsory and upper secondary schools in general focus on subject matters, which are easily assessed. And among those easily assessed subject matters you can find knowledge about society and democratic processes as a component within the subject “civics”. The goals related to democratic competence however are much more complex and challenging for the assessment system.[2]
The influence of the assessment system on learning is well documented. It is considered to be an influential part of the hidden curriculum.[3] External examinations have also been found directing what teachers do.[4] “Teach to the test” is a well-known expression, which catches the impact from examinations. Therefore it is of course very important what signs are given when it comes to assessing democratic competence. Assessment procedure always means to consider the classical evaluation questions – what, how, and why. When assessing democratic competence, the relation between these questions becomes extra critical. Since there are different conceptions of democracy, there will be different conceptions on what should be the focus of assessment, and how the assessment procedure should be designed. And accordingly, the purpose of the evaluation is of course as important to be discussed. Will the assessment system have the function of maintaining existing hierarchic structures, or will it serve to develop democratic schooling?
The concept democratic competence can be used in order to summarize the intended integrated outcome of democratic schooling. The concept consists in its turn of two qualities – democratic and competence. Both are the kind of concepts on which we will never fully agree, since they are permeated with different traditions.[5]
Democratic competence with the above stated meaning implies that plain knowledge about society and about democratic processes is not enough. If we add attitudes or values, there is still a dimension missing. Democratic competence is not something that can be shown on your own by answering questionnaires. Perhaps your potentional democratic competence can be shown, but your actual competence is showed when acting democratically in situations where there are different interests at hand, and where you have to come to some sort of a decision. And here we can already point out one of the critical parts in assessing democratic competence.
In two various impressive studies of the outcome of civic education – The IEA Civic Education Study[6], and the American Civics Assessment Framework[7] the focus of interest is potentional civic competence, which is supposed to be caught through questionnaires containing mainly multiple-choice- and short answer questions. In both studies actual performance qualities have been left outside the assessment procedure.
Direct measurement of participator skills, such as participating in school governance or attending a public meeting, is beyond the scope of this assessment.[8]
The reason behind this can be found in the rational behind psychometric test theory, and the interpretation of validity and reliability. According to the psychometric paradigm, it is difficult to develop reliable tests involving personal judgment in complex open-ended situations. The scoring procedure can be at risk. Thus, if we are to assess process qualities related to acting democratically we must challenge the psychometric paradigm, and find different ways in dealing with the concepts of validity and reliability.
In the paper I will firstly discuss different conceptions on democracy and competence and their consequences for the assessment procedure, and secondly describe the foundations behind the new national assessment tools for evaluating democratic competence in Sweden. Finally I will report on the first experiences from pilot trials with teachers and students.
Different conceptions of democracy
To in depth discuss various conceptions of democracy is way beyond the scope of this paper. I will just briefly discuss some aspects of the concept as a background to the “what- and the how-questions” of evaluation, i.e. the content that will be focused upon and the assessment procedure. A more profound discussion of the relationship between democracy and pedagogy can be found in Fritzell (2002), where the theoretical foundations of the project described in the following pages are also presented.[9]
Educational evaluation is naturally closely related to curriculum theory. From this follows, that the perspective on democracy that permeates the curriculum, on one hand decides what competencies students should develop, and on the other gives certain guidelines to the design of the assessment procedure. There are various ways to categorize between different conceptions of democracy. In this project we have made a distinction between representative, participative, and deliberative democracy. All three conceptions can be trailed in the Swedish national curricula of the last decades.
Representative democracy can be seen as functional, and is characterized by the idea that democratic decisions should be made by an assembly, which is constituted on representative grounds. Different elites regularly compete at elections in getting support for their specific programmes. This is of course a very simplified description of representative democracy, but it can give us an idea of what kind of competencies that students in general need to develop.
A normative perspective on democracy constitutes a different point of departure. Here we would like to stress two conceptions – participatory and deliberative democracy. Central for both conceptions is the classical idea, that citizens should take part in democratic decisions as equals. The participatory conception however, stresses the possibility for different groups or individuals to agree upon and act in their specific interest, while the deliberative conception goes beyond specific group interests, and searches for more principally grounded solutions. Deliberative democracy can be described as discursive and communicative. A deliberative discourse can be characterized as argumentative considerations between affected parts, and the procedures should be judged according to reciprocity and equality. According to the deliberative perspective, citizens also are responsible for the consequences of decisions made. An important feature of deliberation is learning. The participants are expected to change or to deepen their original statements through learning. Decision-making according to a deliberative perspective can thus be seen as more qualified than decision-making according to other perspectives.[10]
Accordingly, the demands put on citizens and consequently the demands for schooling differ, depending on what perspective you choose. From being built on representative democracy and after a period built on participatory democracy, the Swedish National Curriculum is now influenced by the deliberative perspective, though it was not very elaborate when the current curriculum was stated in 1994. The National Agency for Education has now proclaimed the deliberative perspective in different documents related to the common values of the school system.
Different conceptions of competence
In parallel, there are various conceptions of competence. Competence can on one hand be seen as an individually or as an inter-subjectively related quality and on the other hand as a personal trait or a contextually related quality, which is illustrated in figure 1 below.
Figure 1 Different conceptions on competence
Individual Inter-subjective
qualityquality
Personal trait
Contextually
related
Competence as a personal trait corresponds to an individual’s general intellectual or cognitive prerequisites. Competence is thus something statically – you either have it or not.[11] From the opposite view, competence is situated or closely related to the specific context. In other words, you are not competent once and for all, but you act more or less competently in various types of situations. In a certain situation in a certain context you have to be able to define and solve a problem, to communicate with others, to plan, carry out and evaluate performed actions.[12]
Assessments in school are concerned with qualities or competencies demonstrated by individuals. But when we discuss learning organizations, or if we evaluate the competence of a football team, competence is looked upon as a quality characterizing an organization or a group. Then competence is seen as a function of the interplay among the individuals in the group. However, it is not enough to sum up the qualities of all the individuals in the group; we have to evaluate the interaction within the group.
Democratic competence as an object for assessment
If we turn back to the conceptions of democracy, competence has different meanings for different conceptions. From a representative perspective the individual becomes the focus of attention. It is important to find those individuals most appropriate for taking part in political decisions. But for the chosen ones it is necessary to be able to argue for ones own position. Therefore, democratic competence from a representative conception can be discussed both in terms of personal traits and as contextually related. The general citizen should be well informed in societal matters and in democratic procedures and demonstrate democratic values. The demands put on the elected representatives would be higher. They have to be able to argue for their own sake, and to communicate with people representing other perspectives. Here, competence would be seen as context related.
The demands put on citizens, in general, are from a normative conception of democracy considerably higher. From a participative conception, the individuals firstly must be well informed, secondly must be able to argue for and defend their positions. This in its turn postulates a moral awareness, since arguments should be justified against common values. The deliberative perspective also considers inter-subjective qualities, or reciprocity in the communication between individuals. It is the way communication is carried out between individuals, that gives democracy its meaning.
Democratic competence should also from a normative view be seen as context related; democratic procedures are always dependent on the type of question that is focused on, the framework at hand, and the participants taking part in decision-making. Therefore, from a normative view, democratic competence cannot be seen as a personal trait. I will discuss the consequences for assessment matters later, but first turn to the how-aspect of evaluation.
How democratic competence can be assessed
In the rich assessment literature a shift of paradigm can be trailed from the psychometric tradition to a more sociocultural perspective. Or perhaps it is more relevant to state that two paradigms occur, which are in conflict with one another. In the striving for equal conditions through standardization and objectification of the assessment procedures, considerable difficulties of sociocultural character have been displayed. It has become obvious that cultural background cannot be reduced to an explanatory factor. It is embedded in the content and the values that permeates the assessment tools, and can therefore not be dealt with separately.[13] To give equal conditions for different individuals, therefore means developing assessment tools, where existing contextual conditions are being integrated in the assessment process.
By this remark, I do not mean that questions concerning test-validity or test-reliability are unimportant. It is of course of the utmost importance, that assessments are focusing the intended content, and in a reliable manner. But we need to fill these old concepts with new meanings. I will return to this question later on.
One important problem with the psychometric tradition has to do with construct validity. It can be called in question, whether it is possible to reduce a complex concept such as democratic competence to a number of underlying sub-concepts, to isolate and define these sub-concepts, and operationalize them in the shape of specific items. A concept then is constituted by a network of underlying qualities, and every one of these qualities is tested separately.
Pellegrino[14] rejects this interpretation of construct validity, and it has led Black to following conclusion.[15]
The conclusion is that the results of tests fashioned and interpreted in the traditional psychometric tradition do not tell us about features that are of central significance in current theories of cognition because they are based on a model of independent ’traits’.[16]
If competence is seen as situated and bounded to discursive practices, and if competence is about mastering complex tasks in specific contexts, then totally new demands are put on the assessment procedures used. Black comes to the conclusion that existing assessment practice cannot fulfil these demands. This quotation also sums up the challenges facing the recent project.
A whole new technology for assessing group contributions will have to be developed. Then teachers will have to be trained both to find ways, with the aid of such technology, to promote and record participatory practices and to find means of collecting evidence to share with peers so that the process of inter-calibration of their methods and criteria (i.e. moderation) can be acceptably rigorous.[17]
In the international evaluations I briefly discussed above concerning civic education, the tests are item based. And the problem mentioned concerning construct validity is thus relevant. They are also based on “paper and pen”-tests, which are taken by individuals, and focusing the individual’s knowledge in civic matters and democratic processes, attitudes to civic phenomena, and in some cases the ability to make morally judgements. Process- or procedural qualities are beyond the object for testing. Only potential democratic competence has been dealt with. The assessments therefore can be described as indirect. It is the statements about what individuals think they would do in a problematic situation, that have been the object of evaluation, not what these individuals actually do.
This is of course a dilemma in education and in assessments in general. It is difficult in a decontextualized setting, which school can be described as, to get in touch with authentic conditions. Therefore we ask students what they would do in a described situation. But school contains many authentic situations and problems related to democratic problems, which demand democratic competence from the students. The ability to make democratic judgement is an important part of teaching in subjects. And the school day also contains lots of situations outside teaching, where different interests are at stake. The distance to authenticity is thus not far. Authenticity is there every day.
Alternative assessment tools have been developed within a framework, which has been called “authentic assessment”.[18] Characteristic for authentic assessment is that real situations are used as triggers, situations as complex and ill structured as they are in real life. The assessments also challenge productive learning, since there could be different solutions to the problem depending on what aspects are focused on. Problem solving is an important part of the tests. The problem must be analyzed and dealt with, and the solution must be evaluated according to basic values and norms. The processing of the problem is thus an important part of the assessment. In the concept of authenticity also lies that students must find the triggers and the problems mirroring the real world and problems meaningful to handle.
Authentic assessments have mainly been developed in vocational settings or in vocational educations. Authentic conditions can either be simulated or be found in actual working life settings. This means that the level of authenticity can vary to a certain degree – some aspects of reality must due to for instance costs or the respect for human integrity be simulated. But as I stated earlier school contains many authentic situations, where democratic competence is needed. Authentic assessment thus seems to have potential benefits if all aspects of democratic competence should be tested.
Assessment tools for evaluating democratic competence
New assessment tools have been developed with the aim to capture democratic competence, as it can be demonstrated in grade 9 in compulsory school, and grade 3 in upper secondary school in Sweden. The material has been developed by a research-team at Växjö University on the commission of the National Agency for Education.
The foundations
The deliberative conception of democracy has been taken as a point of departure but without excluding other conceptions. Deliberation is characterized by certain qualities, such as reciprocity, publicity, and accountability.[19] Different types of problems call for different conceptions of democracy. Deliberation presupposes the existence of a moral conflict, where different value based interests are at stake. The task of a class to decide the destination of a school trip does not necessarily call for a deliberative discourse.
The focus of the assessment procedure is democratic competence. From a deliberative perspective we have defined the concept as follows:
Democratic competence from a deliberative perspective refers to peoples’ capacity, through communication, to critically examine and deepen their cultural positions, and to reach well-founded convictions regarding social and individual justice.
We have also started out from a dynamic and contextually related perspective on competence. Competence is evaluated from how a specific problem is handled in a specific context. Deliberation refers to communication between individuals. From this follows that competence in this respect should be seen as an inter-subjective quality.[20] Competent action thus refers to how the problem is perceived and qualities in the process leading to a statement or a decision according to deliberative principles.
From the situated character of competence follows, that many factors in the context decide the quality of the process. Different problems have different meaning and importance for different people. Different classes form their specific nature, and a school in its turn forms its specific culture. And a school changes with time. New classes come and go, new teachers appear and others leave. Democratic competence must therefore be seen as a dynamic concept.