MATHEMATICS
An introduction to the PISA maths assessment
Are students prepared to meet the challenges of the future? Are they able to analyse, reason and communicate their ideas effectively? How well equipped are they to continue learning throughout life?
The OECD Programme for International Student Assessment (PISA) aims to answer these questions through three-yearly surveys that examine 15-year-old students’ performance in reading, mathematics and science. The first four surveys addressed these subjects in 2000, 2003, 2006 and 2012 respectively. PISA 2012 will focus on mathematics as well assessing performance in science and reading.
PISA focuses on testing the knowledge and skills required for participation in society and assessing the extent to which students can apply skills gained in school in everyday adult life, thus moving beyond the student’s ability to master the school curriculum.
Worldwide, about 470,000 students from 65 countries took part in the study in 2009.
Mathematics assessment
The mathematics assessment seeks to measure the student’s capacity to analyse, reason and communicate effectively as they pose, solve and interpret mathematical problems in a variety of situations that involve quantity, space, probability and other mathematical concepts. In particular, the maths assessment focuses on the following areas and concepts:
• Quantity
• Space and shape
• Change and relationships
• Uncertainty
On the following pages are a selection of questions that show what can be expected during the maths assessment. Questions in PISA cover a wide range of topics and levels of difficulty in order to allow the survey to measure the range of abilities of students as well as to draw comparisons internationally. Details of the level of difficulty and what proportion of students answered the question correctly are included for reference. The tasks are taken from PISA items released by the OECD for public use. Copyright remains with the OECD.
Levels of difficulty as defined by PISA and the assessment of student’s ability
STAIRCASE
The diagram below illustrates a staircase with 14 steps and a total height of 252 cm:
What is the height of each of the 14 steps?
Height: ...... cm.
Scoring
Full Credit: 18
Percentage of correct answers (OECD countries): 78.3%
Comments:
This short open-constructed response item is situated in a daily life context for carpenters and is therefore classified as having an occupational context. One does not need to be a carpenter to understand the relevant information; it is clear that an informed citizen should be able to interpret and solve a problem like this that uses two different representation modes: language, including numbers, and a graphical representation. But the illustration serves a simple and non-essential function: students know what stairs look like. This item is noteworthy because it has redundant information (the depth is 400cm) that is sometimes considered to be confusing by students; but such redundancy is common in real-world problem solving. The context of the stairs places the item in the space and shape content area, but the actual procedure to carry out is simple division. All the required information, and even more than that, is presented in a recognisable situation, and the students can extract the relevant information from a single source. In essence, the item makes use of a single representational mode, and with the application of a basic algorithm, this item fits, although barely, at Level 2.
Carpenter
A carpenter has 32 metres of timber and wants to make a border around a garden
bed. He is considering the following designs for the garden bed.
Circle either “Yes” or “No” for each design to indicate whether the garden bed can be
made with 32 metres of timber.
Scoring
Full Credit: Yes, No, Yes, Yes, in that order.
Percentage of correct answers (OECD countries): 20.2%
Comments
This complex multiple-choice item is situated in an educational context, since it is the kind of quasi-realistic problem that would typically be seen in a mathematics class, rather than being a genuine problem likely to be met in an occupational setting. A small number of such problems have been included in PISA, though they are not typical. That being said, the competencies needed for this problem are certainly relevant and part of mathematical literacy. The item belongs to the space and shape content area. The students need the competence to recognise that the two-dimensional shapes A, C and D have the same perimeter, and therefore they need to decode the visual information and see similarities and differences. The students need to see whether or not a certain border-shape can be made with 32 metres of timber. In three cases this is rather evident because of the rectangular shapes. But the fourth is a parallelogram, requiring more than 32 metres. This use of geometrical insight, argumentation skills and some technical geometrical knowledge puts this item at Level 6.
Exchange Rate
Mei-Ling from Singapore was preparing to go to South Africa for 3 months as an
exchange student. She needed to change some Singapore dollars (SGD) into South
African rand (ZAR).
Question 1: EXCHANGE RATE
Mei-Ling found out that the exchange rate between Singapore dollars and South
African rand was:
1 SGD = 4.2 ZAR
Mei-Ling changed 3000 Singapore dollars into South African rand at this exchange
rate.
How much money in South African rand did Mei-Ling get?
Answer: ......
Scoring
Full Credit: 12 600 ZAR (unit not required).
Percentage of correct answers (OECD countries): 79.9%
Comments
This short open-constructed response item is situated in a public context. Experience in using exchange rates may not be common to all students, but the concept can be seen as belonging to skills and knowledge for citizenship. The mathematics content is restricted to just one of the four basic operations: multiplication. This places the item in the quantity area, and more specifically, in operations with numbers. As far as the competencies are concerned, a very limited form of mathematisation is needed for understanding a simple text and linking the given information to the required calculation. All the required information is explicitly presented. Thus the competency needed to solve this problem can be described as the performance of a routine procedure and/or application of a standard algorithm. The combination of a familiar context, a clearly defined question and a routine procedure places the item at Level 1.
Question 2: EXCHANGE RATE
During these 3 months the exchange rate had changed from 4.2 to 4.0 ZAR per
SGD.
Was it in Mei-Ling’s favour that the exchange rate now was 4.0 ZAR instead of 4.2
ZAR, when she changed her South African rand back to Singapore dollars? Give an
explanation to support your answer.
Answer: ......
Scoring
Full Credit: Yes, with adequate explanation.
Percentage of correct answers (OECD countries): 40.5%
Comments
This open-constructed response item is situated in a public context. As far as the mathematics content is concerned students need to apply procedural knowledge involving number operations: multiplication and division, which along with the quantitative context, place the item in the quantity area. The competencies needed to solve the problem are not trivial. Students need to reflect on the concept of exchange rate and its consequences in this particular situation. The mathematisation required is of a rather high level, although all the required information is explicitly presented: not only is the identification of the relevant mathematics somewhat complex, but the reduction of it to a problem within the mathematical world also places significant demands on the student. The competency needed to solve this problem can be described as using flexible reasoning and reflection. Explaining the results requires some communication skills as well. The combination of familiar context, complex situation, non-routine problem and the need for reasoning, insight and communication places the item at Level 4.
test scores
The diagram below shows the results on a Science test for two groups, labelled as
Group A and Group B. The mean score for Group A is 62.0 and the mean for Group B is 64.5. Students pass this test when their score is 50 or above.
Looking at the diagram, the teacher claims that Group B did better than Group A in
this test. The students in Group A don’t agree with their teacher. They try to convince the teacher that Group B may not necessarily have done better.
Give one mathematical argument, using the graph, that the students in Group A
could use.
Answer: ......
Scoring
Full credit: One valid argument is given. Valid arguments could relate to the number of students passing, the disproportionate influence of the outlier, or the number of students with scores in the highest level.
• More students in Group A than in Group B passed the test.
• If you ignore the weakest Group A student, the students in Group A do better than those in Group B.
• More Group A students than Group B students scored 80 or over.
Percentage of correct answers (OECD countries): 32.7%
Comments
This open-constructed response item is situated in an educational context. The educational context of this item is one that all students are familiar with: comparing test scores. In this case a science test has been administered to two groups of students: A and B. The results are given to the students in two different ways: in words with some data embedded and by means of two graphs in one grid. Students must find arguments that support the statement that Group A actually did better than Group B, given the counter-argument of one teacher that Group B did better – on the grounds of the higher mean for Group B. The item falls into the content area of uncertainty. Knowledge of this area of mathematics is essential, as data and graphical representations play a major role in the media and in other aspects of daily experiences. The students have a choice of at least three arguments here: the first one is that more students in Group A pass the test; a second one is the distorting effect of the outlier in the results of Group A; and a final argument is that Group A has more students that scored 80 or above. Students who are successful have applied statistical knowledge in a problem situation that is somewhat structured and where the mathematical representation is partially apparent. They need reasoning and insight to interpret and analyse the given information, and they must communicate their reasons and arguments. Therefore the item clearly illustrates Level 5.
Here are a few more past PISA numeracy questions.
LITTER
For a homework assignment on the environment, students collected information on the decomposition time of several types of litter that people throw away:
Type of Litter / Decomposition timeBanana peel / 1–3 years
Orange peel / 1–3 years
Cardboard boxes / 0.5 year
Chewing gum / 20–25 years
Newspapers / A few days
Polystyrene cups / Over 100 years
Question 1: LITTER
A student thinks of displaying the results in a bar graph. Give one reason why a bar graph is unsuitable for displaying these data.
EARTHQUAKE
Question 2: EARTHQUAKE
A documentary was broadcast about earthquakes and how often earthquakes occur. It included a discussion about the predictability of earthquakes. A geologist stated: “In the next twenty years, the chance that an earthquake will occur in Zed City is two out of three”.
Which of the following best reflects the meaning of the geologist’s statement?
A 2/3 x 20 = 13.3, so between 13 and 14 years from now there will be an earthquake in Zed City.
B 2/3 is more than ½, so you can be sure there will be an earthquake in Zed City at some time during the next 20 years.
C The likelihood that there will be an earthquake in Zed City at some time during the next 20 years is higher than the likelihood of no earthquake.
D You cannot tell what will happen, because nobody can be sure when an earthquake will occur.
GROWING UP
YOUTH GROWS TALLER
In 1998 the average height of both young males and young females in the Netherlands is represented in this graph.
Question 3: GROWING UP
Since 1980 the average height of 20-year-old females has increased by 2.3 cm, to 170.6 cm. What was the average height of a 20-year-old female in 1980?
Answer: ...... cm
Question 4: GROWING UP
Explain how the graph shows that on average the growth rate for girls slows down after 12 years of age.
......
......
......
Question 5: GROWING UP
According to this graph, on average, during which period in their life are females taller than males of the same age?
......
......
LICHEN
A result of global warming is that the ice of some glaciers is melting. Twelve years after the ice disappears, tiny plants, called lichen, start to grow on the rocks. Each lichen grows approximately in the shape of a circle.
The relationship between the diameter of this circle and the age of the lichen can be approximated with the formula:
d =7.0×√(t − 12) for t ≥ 12
where d represents the diameter of the lichen in millimetres, and t represents the number of years after the ice has disappeared.
Question 6: LICHEN
Using the formula, calculate the diameter of the lichen, 16 years after the ice disappeared. Show your calculation.
Question 7: LICHEN
Ann measured the diameter of some lichen and found it was 35 millimetres. How many years ago did the ice disappear at this spot? Show your calculation.
LIGHTHOUSE
A lighthouse beacon sends out light flashes with a regular fixed pattern. Every lighthouse has its own pattern.
In the diagram below you see the pattern of a certain lighthouse. The light flashes alternate with dark periods. /
light
dark
0 1 2 3 4 5 6 7 8 9 10 11 12 13