Assessment Exemplar with Guide for Instructors

ASSESSMENT EXEMPLAR WITH GUIDE FOR INSTRUCTORS:

THE SCIENTIFIC METHOD

Topic:Level: Core concepts:

☐Microeconomics
☐Macroeconomics
☒Statistics/Econometrics / ☐Bachelors
☐Masters
☒Both/Either / The scientific method; empirical modelling in line with tenets of economic theory; measurement

Learning domains:Type(s) of task (can be more than one)

☒Knowledge
☒Application
☒Data analysis
☒Communication
☒Reflection / ☒Constructed response given stimulus material (clip/narrative/etc.)
☐Constructed response with no stimulus material
☒Quantitative analysis
☒Independent data sourcing
☒Independent research (not data sourcing)
☒Written presentation
☐Oral presentation
☒Group collaboration
☒Use of nominated maths/stats software (specify): Excel, Stata, etc.
☐Use of other technology (specify):
☐Other (specify): ______

The core statistical concept underpinning this assessment item is the scientific method. The item asks students to engage with ideas related to excess returns in a market, as they simulate progressing a research frontier. Students are asked to apply their quantitative skills to perform appropriate analysis themselves of a relevant data set using relevant software in order to answer a particular question about the possibility of excess returns. The instructions to students can be customised to include the expectation that students explain why the premium they are testing for might be a predicted by, or a puzzle for, economic theory. Reflection on the existing literature, the present analysis, the concept at play, the politics of research dissemination, and/or the suitability of the research question to the role of economists in society can all be explicitly requested.

The callouts to the learning standards assessed by different components of the items appear in square brackets within the items, and should be removed before the items are used.

Bachelors level task:

Masters level task:

Research Question: Is there a marriage premium?
Labour economists have long noted that married people tend to earn more money, conditional on many other characteristics, than unmarried people. This notion is referred to as a ‘marriage premium’.
Some evidence for this premium exists {cite existing (ideally recent) academic evidence for the existence of the marriage premium}, but it is notoriously difficult to identify because married people could be genuinely more productive than unmarried people. A colleague of yours suggests that this problem might be able to be circumvented using data on basketball players.
To find out, she amasses the data set {data set name} that provides information drawn from three National Basketball League teams about the numbers of points, blocked shots, steals, assists, rebounds, and turnovers that accrued to each player in the team across all games during a particular season. The data also include marital status (married, divorced/widowed, or never married) and salary information for each player.
PART A: Investigate
After discussing the idea at lunch one day, the two of you decide to pursue the project together. While she hunts down additional data, you perform an initial investigation of the research question using her existing data in {software package}.[AM3]
In your investigation, be sure to:
(i)Describe your data set.[KM1,CM1]
(ii)Explain your empirical approach.[AM1,AM2,DM1,CM1]
(iii)Clearly present your findings, both graphically and numerically.[KB1,CM1]
PART B: Write a first draft
On reviewing your initial findings, your colleague is pleased. The next step is to write up your results in the form of a draft article and position your work in the broader literature related to the marriage premium. Your colleague is still busy gathering additional data, so you write a first draft.
In your write-up, be sure to:
(i)Explain the rationale behind the statement above that “[the marriage premium] is notoriously difficult to identify because married people could be genuinely more productive than unmarried people”. How would using data on basketball players be better than using data on other types of workers to investigate this research question? [KM1,CM1]
(ii)Interpret your findings using the economic framing provided in the question.[KM1,AM1,AM2,CM1]
(iii)Compare your results to existing results in the literature, and hypothesize about possible reasons for any differences.[RM1,RM2,CM1]
(iv)Discuss the limitations of your analysis, and sketch what further research could be done to improve our understanding of the importance of the marriage premium.[RM1,RM2,CM1]
PART C: Re-evaluate
Your colleague returns with more data, from three more NBA teams, available in {data set name}. Re-do Parts A and B above with your larger, pooled, data set. Interpret any changes you see in your results. [AM3,DM2,CM1]
Given these changes, what ethical dilemmas do you face with regard to how you make your research public? Suppose your colleague is up for tenure in six months. Does this change your dissemination
plan? If so, how, and if not, why not? [RM3,CM1]

Advice to assessors

This assessment is intended to weight heavily the more academic side of students’ development, and involves a simulation of data analysis tasks that could be used as the cornerstone of a research paper. Masters students are asked to do the analysis twice, with the idea that adding the second round of data to the first will generate much less convincing results, causing the researchers to face an ethical dilemma that the student must handle.
Students may be given reading material about the marriage premium by the teacher, or can instead be required to do their own literature search. For a further extension, Masters students could be tasked with finding additional data sources that could be used as input into statistical analyses that would help determine whether there is a marriage premium.
The question frame can be used as a graft for other questions that query students’ skills in performing other types of basic empirical research about wage premia that would pose a puzzle for traditional economic theory – for example, an examination of the wage returns to gender, parental status, religion, sexual orientation, or other worker characteristics.
Learning domain / Learning outcomes
Bachelor Degree / Masters Degree
Knowledge / KB1Bachelor graduates will be able to identify, coherently explain and synthesise core economic concepts / KM1Masters graduates will be able to identify, coherently explain and synthesise core and advanced economic concepts, including recent developments in the discipline
Application / Bachelor graduates will be able to:
AB1• frame problems in terms of core economic concepts and principles
AB2• apply economic reasoning and analytical skills, in order to make informed judgments and decisions / Masters graduates will be able to:
AM1• frame and critically analyse problems in terms of core and advanced economic concepts and principles
AM2• apply advanced economic reasoning and analytical skills, including quantitative techniques where appropriate, in order to make informed judgments and decisions
AM3• plan and execute a research-based project
Data analysis / Bachelor graduates will be:
DB1• able to use economic data to address typical problems faced by economists
DB2• aware of, and able to implement, basic empirical techniques and interpret the results / Masters graduates will be able to:
DM1• select and apply an appropriate empirical method to address typical problems faced by economists
DM2• critically evaluate the results
Communication / CB1Bachelor graduates will be able to present a clear and coherent exposition of economic knowledge, ideas and empirical evidence both orally and in writing, individually or in collaborative contexts / CM1Masters graduates will be able to communicate complex ideas clearly and coherently, in written form and interactive oral form to expert and non-expert audiences, individually or in collaborative contexts
Reflection / Bachelor graduates will be able to reflect on:
RB1• the nature and implications of assumptions and value judgments in economic analysis and policy
RB2• interactions between economic thinking and economic events, both historical and contemporary
RB3• the responsibilities of economists and their role in sociey / Masters graduates will be able to reflect on and evaluate:
RM1• the nature and implications of assumptions and value judgments in economic analysis and policy
RM2• interactions between economic thinking and economic events, both historical and contemporary
RM3• the responsibilities of economists and their role in society