Table of Contents

1.Purpose of this codebook

2.About Students’ Conceptions of Assessment

References

Overview

SCoA v1

SCoA v2

SCoA v3

SCoA v4

SCoA v5

SCoA v6

Multi-Site Invariance Studies

Coverage

Methodology

Response Format

Missing data analysis

Privacy, security, or confidentiality issues

Coding Scheme

3.List of Data Sets

4.Data Source Variables

Summary Table

5.School Measures

Summary Table

6.Participant Demographic Measures

Summary Table

7.Assessment Practice Checklist

Summary Table

8.Students’ Conception of Assessment Version Six and Five

Summary Table

9.Students’ Conception of Assessment Version Five Additional Items

Summary Table

10.Students’ Conception of Assessment Version Four

Summary Table

11.Students’ Conception of Assessment Version Three

Summary Table

12.Students’ Conception of Assessment Version Two

Summary Table

13.Students’ Conception of Assessment Version One

Summary Table

1.Purpose of this codebook

This codebook or data dictionary provides users of the Students’ Conceptions of Assessment Inventory details of the structure of the data files used to analyse the New Zealand data. This codebook needs to be read in conjunction with the following complementary files:

  1. The Students’ Conceptions of Assessment Questionnaire or Inventory
  2. Version 6
  3. Version 5
  4. Version 4
  5. Version 3
  6. Version 2
  7. Form 1
  8. Form 2
  9. Version 1
  10. Form A
  11. Form B
  12. Form C
  13. Form D
  14. The New Zealand data files, in SPSS format
  15. Version 6
  16. Version 5
  17. Version 4
  18. Version 3
  19. Version 2
  20. Version 1
  21. The confirmatory factor analysis input files for all versions
  22. AMOS graphic input system

2.About Students’ Conceptions of Assessment

The Students’ Conceptions of Assessment (SCoA) inventory is a self-administered, self-report opinion, attitude, or belief questionnaire designed in New Zealand by Gavin T L Brown (2003-2008) after his PhD dissertation research into Teacher Conceptions of Assessment. That work was carried out with New Zealand secondary school students. It started on the assumption that student conceptions of assessment may be parallel to teacher conceptions. Very quickly it morphed into a model, now informed by self-regulation of learning theories about how students can improve their performance by using assessment to regulate their own learning practices.

The questionnaire has been published in six gradually improved and developed versions. See list below for publications related to each version. Comparison studies between New Zealand data and responses of students in other jurisdictions has also been conducted and published.

The most recent version of the inventory aggregates student beliefs about assessment around four major ideas:

  1. Assessment is used by students to improve their learning and by teachers to guide students into improved learning. [Improvement]
  2. Assessment is linked to external attributes of the student such as their future performance or job, their intelligence, and the quality of the school they attend. [External Attribution]
  3. Assessment requires and builds on the quality of student cooperation and classroom climate and elicits strong positive emotion of enjoyment [Affect]
  4. Assessment is oppressive, inaccurate, and ignored by the student [Irrelevance]

In Version 5, these four ideas were created by 8 1st order factors which grouped in pairs into the four factors. In Version 6, one of the contributing 1st-order factors within Irrelevance had to be removed with the items loading directly on the main factor. The four major factors are inter-correlated. This means that the inventory is multi-dimensional and there is no single overall score. Reading of the listed references will provide the user with a sufficient background to appreciate the meaning and intent of the TCoA.

References

Overview

Brown, G. T. L. (2011). Self-regulation of assessment beliefs and attitudes: A review of the Students’ Conceptions of Assessment inventory. Educational Psychology, 31(6), 731-748. doi:10.1080/01443410.2011.599836

SCoA v1

Brown, G. T., & Hirschfield, G. H. (2007). Students' Conceptions of Assessment and Mathematics: Self-Regulation Raises Achievement.Australian Journal of Educational & Developmental Psychology,7, 63-74.

SCoA v2

Brown, G. T. L., & Hirschfeld, G. H. F. (2008). Students’ conceptions of assessment: Links to outcomes. Assessment in Education: Principles, Policy and Practice, 15(1), 3-17. doi:10.1080/09695940701876003

Hirschfeld, G. H. F., & Brown, G. T. L. (2009). Students’ conceptions of assessment: Factorial and structural invariance of the SCoA across sex, age, and ethnicity. European Journal of Psychological Assessment, 25(1), 30-38. doi:10.1027/1015-5759.25.1.30

Walton, K. F. (2009). Secondary students’ conceptions of assessment mediated by self-motivational attitudes: Effects on academic performance. (Unpublished M.Ed. thesis), University of Auckland, Auckland, NZ.

SCoA v3

Brown, G. T. L. & Hirschfeld, G. H. F. (2005). Secondary school students’ conceptions of assessment. Conceptions of Assessment and Feedback Project Report #4. Auckland: University of Auckland.

SCoA v4

Brown, G. T. L. (2006, September). Secondary school students’ conceptions of assessment: A Survey of Four Schools. Conceptions of Assessment and Feedback Project Report #5. Auckland, NZ: University of Auckland.

SCoA v5

Brown, G. T., Irving, S. E., Peterson, E. R., & Hirschfeld, G. H. (2009). Use of interactive–informal assessment practices: New Zealand secondary students' conceptions of assessment.Learning and Instruction,19(2), 97-111.doi:10.1016/j.learninstruc.2008.02.003

Weekers, A. M., Brown, G. T. L., & Veldkamp, B. P. (2009). Analyzing the dimensionality of the Students' Conceptions of Assessment (SCoA) inventory. In D. M. McInerney, G. T. L. Brown & G. A. D. Liem (Eds.), Student perspectives on assessment: What students can tell us about assessment for learning. (pp. 133-157). Charlotte, NC US: Information Age Publishing.

SCoA v6

Brown, G. T. L., Peterson, E. R., & Irving, S. E. (2009). Beliefs that make a difference: Adaptive and maladaptive self-regulation in students’ conceptions of assessment. In D. M. McInerney, G. T. L. Brown, & G. A. D. Liem (Eds.), Student Perspectives on Assessment: What Students can Tell us about Assessment for Learning (pp. 159-186). Charlotte, NC: Information Age Publishing.

AlQassab, M. (2012). Differential item functioning according to subject motivation: Psychometric properties of the Student Conceptions of Assessment (unpublished master's thesis), The University of Auckland, Auckland, NZ.

Multi-Site Invariance Studies

Brown, G. T. L. (2013). Student conceptions of assessment across cultural and contextual differences: University student perspectives of assessment from Brazil, China, Hong Kong, and New Zealand. In G.A.D. Liem & A. B. I. Bernardo (Eds.), Advancing Cross-cultural Perspectives on Educational Psychology: A Festschrift for Dennis McInerney(pp. 143-167). Charlotte, NC: Information Age Publishing.

Matos, D. A. S., Cirino, S. D., Brown, G. T. L., & Leite, W. L. (2013). A avaliação no ensino superior: Concepções múltiplas de estudantes Brasileiros [Assessment in highereducation: MultipleconceptionsofBrazilianstudents]. Estudos em Avaliação Educacional, 24(54), 172-193. doi:10.18222/eae245420131907 ·

Matos, D. A. S., & Brown, G. T. L. (2015). Comparing university student conceptions of assessment: Brazilian and New Zealand beliefs. In C. Carvalho & J. Conboy (Eds.). Feedback, Identidade, Trajetórias Escolares: Dinâmicas e Consequências(pp. 177-194). Lisbon, Portugal: Universidade de Lisboa, Instituto de Educação.

Coverage

This manual applies to the New Zealand secondary student data files only and analysis files obtained from surveys carried out between 2003 and 2009 as described in the reference list. Only the SCoA items and accompanying checklist of assessment practices (used in Versions 4-6 only) are included.

Methodology

The SCoA inventory involves two parts each with its own response format. The SCoA main inventory (Part A) consists of 33 items which probe student agreement about the four main purposes of assessment. Part B is a checklist which allows students to identify the types of practices they were thinking about when answering the main part.

Response Format

Each statement of the SCoA required the participant to respond on a six-point positively-packed rating scale. Options were scored:

  • Strongly Disagree=1;
  • Mostly Disagree=2;
  • Slightly Agree=3;
  • Moderately Agree=4;
  • Mostly Agree=5; and
  • Strongly Agree=6.

The Assessment Practices Checklist required a binary Yes/No response. Participants were allowed to indicate as many as they considered relevant.

Missing data analysis

Prior to imputation, all cases with more than 10% missing were removed from further analysis. The remaining cases and variables had the remainder of missing values imputed using the expectation maximisation procedure. All imputed values <1.00 and >6.00 were corrected to either 1.00 or 6.00 respectively. All values between 1.00 and 6.00 were left uncorrected on the assumption that the response scale created a continuous variable.

Privacy, security, or confidentiality issues

All data files have been anonymized so that individuals and their schools cannot be identified. All data were obtained under various institutional research board authorisations at the University of Auckland.

Coding Scheme

Throughout this document, the data sets, and the excel master spreadsheet, SCoA statements have been coded to help align data across all six versions of SCoA.

Statements have been assigned a two or three letter code to capture the content focus of the item plus a number. The lettering refers to the variable group or variable subgroup a statement belongs to. Lower numbers have been assigned to newer version of SCoA (6 and 5) and higher numbers are most likely aligned with earlier versions of SCoA.

Where a statement has been repeated in different versions the coding is exactly the same.This also applies for statements where the wording was slightly different but which was intended to measure the same content. This is mainly seen in Version 1, which was the pilot version. Please see that section for further details.

The content code structure is:

Code / Code Translation
bd / Bad
ce / Class environment
ig / Ignore
ir / Irrelevant
pe / Personal enjoyment
sf / Student future
si / Student improvement
sq / School quality
sr / Self-regulate
sta / Student accountability
ti / Teacher improvement
val / Valid

3.List of Data Sets

The following data files capture the New Zealand data from the Students’ Conceptions of Assessment questionnaires carried out and collected from 2003 to 2009.

SCoA Master:

The following Excel spreadsheet contains all items from all six versions and shows the history of items as they were added, deleted, or carried through. It also shows the factor to which each items was aligned.

  • NZ Master SCoA.xlsx

SCoA-VI Data Sets:

The following SPSS data file contains data information that was collected in 2007from large samples of students in a few Auckland region schools for Version 6 of SCoA. Note that Version 6 is a reanalysis of the data collected for Version 5. This SPSS file contains all relevant variables and codes as outlined within this codebook/data dictionary.

  • NZ SCoA-VI.sav

The following AMOS file is the confirmatory factor analysis input for the SCoA-VI data set.

  • NZ SCoA-VI.amw

SCoA-V Data Sets:

The SPSS data file contains data information that was collected in a national survey in 2006of NZ secondary students, primarily in Years 9 and 10, for Version 5 of SCoA. This SPSS file contains all relevant variables and codes as outlined within this codebook/data dictionary.

  • NZ SCoA-V.sav

The following AMOS file is the confirmatory factor analysis input for the SCoA-V data set.

  • NZ SCoA-V.amw

SCoA-IV Data Sets:

The SPSS data file contains data information that was collected in 2006 from a survey of students in 4 Auckland regional secondary schools participating in the Conceptions of Assessment and Feedback Project. This SPSS file contains all relevant variables and codes as outlined within this codebook/data dictionary.

  • NZ SCoA-IV.sav

The following AMOS file is the confirmatory factor analysis input for the SCoA-IV data set.

  • NZ SCoA-IV.amw

SCoA-III Data Sets:

The SPSS data file contains data information that was collected in 2005from NZ secondary students in 4 Auckland regional school participating in the Conceptions of Assessment and Feedback Project. This SPSS file contains all relevant variables and codes as outlined within this codebook/data dictionary.

  • NZ SCoA-III.sav

The following AMOS file is the confirmatory factor analysis input for the SCoA-III data set.

  • NZ SCoA-III.amw

SCoA-II Data Sets:

The SPSS data file contains data information that was collected in 2004from a national survey conducted to evaluate new reading comprehension test items as part of the Assessment Tools for Teaching and Learning Project. Items were split into 2 forms to be completed by the students after they had finished the trial tests. Only 11 items were common between the two forms. Thus analysis by Brown & Hirschfeld (2008) and Hirschfeld and Brown (2009) applies to those common items. Walton (2009) provides an overview of all the items.

This SPSS file contains all relevant variables and codes as outlined within this codebook/data dictionary.

  • NZ SCoA-II.sav

The following AMOS file is the confirmatory factor analysis input for the SCoA-II data set.

  • NZ SCoA-II.amw

SCoA-I Data Sets:

The SPSS data file contains data information that was collected in 2003 from a national survey conducted to evaluate new mathematics test items as part of the Assessment Tools for Teaching and Learning Project. Items were split into 4 forms to be completed by the students after they had finished the trial tests. There were no common items across the forms.

This SPSS file contains all relevant variables and codes as outlined within this codebook/data dictionary.

  • NZ SCoA-I.sav

The following AMOS files are the confirmatory factor analysis input for the SCoA-I data set.

  • NZ SCoA-IA.amw (Form A – negative/bad)
  • NZ SCoA-IB.amw (Form B – useful)
  • NZ SCoA-IC.amw (Form C – improvement)
  • NZ SCoA-ID.amw (Form D – accountability)

4.Data Source Variables

These variables provide a unique identification number for each participant and identify what form was filled out where applicable.

Summary Table

Variable Code / Label
ID / Case identity number
CoA_Form / Form filled out

Variable Name: ID

Description: A sequential identification number for each participant.

Variable Name: CoA_Form

Description: The form filled out by participant.

In the SPSS data file for version 1:

1 = accountability items (Form D)

2 = improvement items (Form C)

3 = irrelevance items (Form A)

4 = validity items (Form B)

Note. This variable is only in version 1.

5.School Measures

These variables refer to school characteristics rather than individual participants. Not all versions contain this information, this is highlighted in the following section.

Summary Table

Variable Code / Label
Schl_Type / School Type
Schl_Gen_Type / School Gender Type
Schl_Region / School Region
Main_Region / School Main Region
Schl_Decile_Rate / School Decile Rating
Schl_Decile_Grp / School Decile Group
Schl_Size / School Size
Schl_Cluster / School Cluster
Schl_Authority / School Authority

Variable Name: Schl_Type

Description: Type of school where participant attended.

Note. This variable is only in the SPSS data files for versions 5, 2 and 1.

In the SPSS data file for version 5:

1 = Year 9 to 15

2 = Year 7 to 15

3 = Area

In the SPSS data files for version 2 and 1:

-1 = Missing data

1 = Primary/Intermediate

2 = Secondary

3 = Other

Variable Name: Schl_Gen_Type

Description: School type for secondary schools. In New Zealand secondary schooling there is a tradition of separate schools for boys and girls, often associated with higher academic status schools, such as grammar schools. Otherwise, most public schools are co-educational.

Note. This variable is only used within the SPSS data files for versions 5, 2 and 1.

In the SPSS data file for version 5:

0 = Co-educational

1 = Female

2 = Male

In the SPSS data file for version 2:

-1 = Missing data

1 = Co-Ed

2 = Single Sex Boys

3 = Single Sex Girls

In the SPSS data file for version 1:

-1 = Missing data

1 = Co-Ed

2 = Single Sex

Variable Name: Schl_Region

Description: National region of participants’ school

Note. This variable is only used in the version 5 SPSS data fie.

In the SPSS data file for version 5:

1 = Northland

2 = Auckland

3 = Waikato

4 = Bay of Plenty

5 = Gisborne

6 = Hawkes Bay

7 = Taranaki

8 = Manawatu-Wanganui

9 = Wellington

10 = Nelson

11 = Marlborough

12 = Tasman

13 = West Coast

14 = Canterbury

15 = Otago

16 = Southland

Variable Name: Main_Region

Description: Main region that the participants’ school belongs to

Note. This variable is only used in the version 5 SPSS data fie.

In the SPSS data file for version 5:

1 = Upper North Island

2 = Lower North Island

3 = South Island

From the above variable “Schl_Region” codes 10 to 16 belong in the South Island.

Variable Name: Schl_Decile_Rate

Description: School decile rating range from 1 to 10. Decile 1 is meant to indicate a school has children who come from the lowest 10% (decile) of the socio-economic stratum of NZ. The decile is assigned by the Ministry of Education in response to updated census information from Statistics New Zealand. The index is determined by linking the addresses of children attending the school to the mesh block deprivation index for those addresses.

Note.This variable is only used in the SPSS data file for version 5.

In the SPSS data file for version 5:

1 = Decile 1

2 = Decile 2

3 = Decile 3

4 = Decile 4

5 = Decile 5

6 = Decile 6

7 = Decile 7

8 = Decile 8

9 = Decile 9

10 = Decile 10

Variable Name: Schl_Decile_Grp

Description: School decile group, Low, Medium or High. The Low decile grouping included decile 1, 2, and 3. The Medium decile grouping included decile 4, 5, 6, and 7. The High decile grouping included decile 8, 9, and 10.

Note. This variable is only used in the SPSS data files for versions 5, 2 and 1.

In the SPSS data file for version 5:

1 = Low

2 = Medium

3 = High

In the SPSS data files for versions 2 and 1:

1 = Low

2 = Low

3 = Low

4 = Medium

5 = Medium

6 = Medium

7 = Medium

8 = High

9 = High

10 = High

Variable Name: Schl_Size

Description: School size as categorized by small (<= 120), medium (<= 350), and large (> 350). Data for this were obtained from the Ministry of Education schools database.

Note. This variable is only used in the SPSS data file for version 5.

In the SPSS data fil for version 5:

1 = Small

2 = Medium

3 = Large

Variable Name: Schl_Cluster

Description: School asTTle Cluster

These are the clusters of schools developed for the asTTle test system. Values correspond to the clusters reported in Hattie, J. (2002). Schools like mine: Cluster analysis of New Zealand schools (asTTle Tech. Rep. #14). Auckland, NZ: University of Auckland, Project asTTle. Only uses in Primary School measures.

Note. This variable is only used in the SPSS data files for versions 2 and 1.

Primary, Kura & Special