Designing Assessment and Using Data to Document Individual Student Learning in a Music

Designing Assessment and Using Data to Document Individual Student Learning in a Music Performance

Class

Presented by: Leslie Manfredo

ISU Band Director Workshop - July 16-17, 2013

Packet contents:

Pages

Tips for writing good test questions

Assessment Task and Questions to Inform the Next Steps of Instruction

Mahomet-Seymour HS Chorus Curriculum (Power Standards)

Learning Objectives Mastery Roster Example from Mastery Manager and Excel

Excel Spreadsheets showing student data

Data showing individual student growth for learning targets 4.6, 4.8, 4.9

Data showing individual student mastery for learning targets 4.6, 4.8, 4.9

Final (Narrative) Report Data of Individual Student Growth

Today’s Agenda

Introduction - Who are we?

Part 1: Setting the Foundation

◦Curriculum and Standards

◦What IS assessment

Formative and Summative assessment

Types of assessment

What should you assess?

Decisions you must make about assessment

◦Assessment of Content Knowledge

◦Writing assessment questions

Part 2: The Practical

◦Group work – an assessment task

Interpreting assessment data

Asking and answering assessment questions

Wrap up

The Goals (or learning objectives) for this session:

To clarify the relationship between standards, curriculum, learning objectives, and assessment.

To give you practical, hands-on experience with individual student data of a content knowledge pre-test and post-test.

To give you some assessment ideas you can use in your teaching THIS SCHOOL YEAR.

To share the “administrative-speak” vocabulary/ideas of general education assessment methodology so that you can explain (using data) that your individual music students ARE learning to your administrator (even when he/she says, “I don’t know anything about music”).

To take the fear out of assessment and statistics (because I don’t really know that much) and look at student data from a practitioner’s point of view (not from that of a theoretician).

Assessment Task and Questions to Inform the Next Steps of Instruction

Tips for writing good test questions

Multiple Choice.

-The grammar you use in your answer choices may be influencing how your students choose an answer. Make sure all your choices are grammatically parallel. In other words, if you ask, “Where does a rabbit live?”, phrase all your answers as prepositional phrases.
A. In a house
B. In a car
C. Under a bridge
D. In a warren.
If you offered choice E. dangerously close to the highway, it will obviously be incorrect because it is phrased differently than the other choices.

-Though every education student has probably heard it, watch your vowels. Always give the possibility of either a or an when asking a question. You can include the choice either in the question itself or in the answers.
What does a rabbit eat for breakfast?
A.a carrot
B. a protein shake
C. an egg
D. a piece of toast,
(or a rabbit eats a(n)______for breakfast).

-Keep all your answers around the same length. If one answer is significantly longer or shorter than all the rest, it will likely be the best choice or an easy elimination. Try to write about the same amount for each answer option to avoid give away answers.

-Don't use excessive wording when creating the test question stem. Be clear and concise in your word and phrase choices.

-Make sure that there is only one clearly correct answer from the options given to the student.

-Provide between 3-5 plausible choices for the student to select from as their answer

-Minimize the use of 'all of the above' or 'none of the above' question answers.

-Randomly distribute the correct answer options i.e. A, B, C, D etc so that there is not a clear pattern that becomes obvious to the student

-Be sure to use test questions that test knowledge, application, comprehension, analysis and evaluation throughout your test to get the best overall sense of the student's understanding and mastery of a subject matter

Fill in the Blank:

-There is a difference between recognition knowledge of a word and recall knowledge of a word. Recognition knowledge means you understand the word when you hear it or read it. Recall knowledge means you can and will use the word in your own speech or writing. Everyone no matter what his language has a greater recognition vocabulary than a recall vocabulary. Test your students’ ability to understand the material you have presented, their recall knowledge, by providing them with a word bank. The word bank can have more words or the same number that your students will need to fill in the blanks. Because providing a word bank tests their recall knowledge, it is a better measure of what they have learned than testing their recall knowledge and asking them to elicit vocabulary words on their own.

-Ensure that there is only one possible correct answer to avoid confusion and difficulty grading

-Blanks should come at the end or as close to the end of the question or statement as possible

-Questions should recall important information taught within the lesson plans

True/False.

-Do not try to trick your students with questions that have minor changes in them to make them false. When testing on literature, do not make up false names for characters or change letters in words to make them incorrect. This is confusing for your students and does not give you a useful gauge on their comprehension of the material. Instead, try to test general comprehension of the material.

-If you really want to know what your students have learned, have them correct the false statements to make them true. This will eliminate random guessing and also give you a better idea what concepts your students might not understand yet. You may also find that you have to cover specific material again or explain a concept in another way.

-Make sure that the answer is clear and that it could not be either or

-Try not to use negative questions such as 'this novel was not written by...." but instead use 'this novel was written by...."

-Use a random order of true and false responses with your test questions to avoid creating a pattern

-Use more false questions than true questions as they have been proven to cater towards higher cognitive level students

-Avoid unnecessary modifiers, especially absolutes (e.g. always, never, etc.).

Open-Ended Questions

-Open ended or essay format questions are excellent for measuring higher level cognitive learning and overall comprehension of a subject. They allow the student to select content for their response, to organize their thoughts in a logical manner and to present their ideas on a given subject matter. Overall, these types of test questions allow the teacher to test the student's broader understanding of a subject matter. And, these types of questions are often more applicable to real life situations that the student may be presented with in the future.

-When writing good open-ended questions, keep the following tips in mind:

-Be sure that the test question clearly states the answer that you are seeking from the student. For example, 'discuss the recent election outcome' is a poor test question. But, worded as 'describe the potential positive and negative impacts that Barack Obama's recent election win for president could have on the US's economy' is a better test question as it clearly gives the student something to compare and contrast within a focused area, the US economy.

-If you are requiring the student to prepare a longer essay (2-3 pages), include several questions that are intended to be in addition to the primary question for the student to respond to rather than only a single question to answer.

-If you are looking to test comprehension, a good opening line for the test question is, 'Explain the following..."

-If you are seeking to test the student's ability to analyze a concept, a good opening phrase for your test question is, 'compare and contrast....."

-Don't give students the option to pick 2 or 3 questions from among 5. This can add confusion for the students and complexity for the teacher when grading for a classroom. How can you accurately compare students to each other when they have answered different test questions?

Difficulty Level

Start your test with the easiest questions and move toward those that are more difficult. Though you might want to mix up the order to challenge your students, going from easier to harder questions alleviates stress for your students and makes for a better testing experience. Not to mention, standardized tests like the SAT and TOEFL follow this organization, so structuring your tests that way will help your students on these important tests in the future.

Test multiple learning levels. The majority of your questions should target the lower learning levels of recall, comprehension and application. Do not be afraid, however, to add one or two questions testing higher levels of learning like analysis, synthesis and evaluation (see Bloom’s taxonomy of learning for more information on learning levels). Test these higher levels sparingly, especially if you have not spent a lot of time during class on these types of activities.

General

Give your students experience with the types of questions with which you will be testing. The test should not be the first time they have seen a fill in the blank or made corrections to an incorrect sentence. Your students should have practice with the form so you can test the content and not the form.

Tell them ahead of time what will be on the test both in structure and in content. It does not hurt you as a teacher or skew your test results to give your students a heads up about the type of questions that will be on the test. Giving your students this information ahead of time means you cannot write your test the night before, but that makes for a higher quality piece of evaluation anyway. Also, give them some idea what content will be covered on the test though you do not have to give specifics. Page numbers, lecture dates or book chapters are sufficient.

Include the points each section is worth. This way students can budget their time to be most impactful for them. Neither you nor they want them to spend ten minutes struggling to answer a question of minimal importance. Let them have full knowledge of what is weightiest as they take the test so they can prioritize as they take it.

Expect your students to have “foreign” handwriting. Even though English is taught in places all around the world, handwriting is not the same everywhere. Do not be surprised if your students consistently write in a penmanship style that is challenging for you to read.

Avoid providing cues to correct answer in the stem.
Avoid providing clues to the answer of one question in another question.

Disaggregating and Interpreting Assessment Data

Task: Determine which standards/learning objectives from the M-S Curriculum were assessed in the 2 sections. Then complete the individual and class statistical information, as indicated.

Assessment Worksheet

Section 1 Standard and Learning Objective Number from M-S Curriculum ______(ie 6f)

Section 2 Standard and Learning Objective Number from M-S Curriculum ______(ie 6f)

Individual Student Scores Statistical Data – Complete the chart

Student / Pre-Test % Section 1 / Post-Test %
Section 1 / Difference % / Mastery
Attained (90%)? / Pre-Test % Section 2 / Post-Test %
Section 2 / Difference % / Mastery
Attained (90%)?
Ashley / 55% / 87% / 50% / 90%
Katie / 77% / 100% / 50% / 100%
Leah / 66% / 93% / 50% / 100%
Natalie / 91% / 80% / 50% / 90%
Jacob / 68% / 67% / 50% / 80%

Class Statistical Data – Complete the following:

Mean Score of Pre-Test of Section 1 = ______

Mean Score of Post-Test of Section 1 = ______

Difference of Means = ______

Percentage who have attained mastery = ______

Mean Score of Pre-Test of Section 2 = ______

Mean Score of Post-Test of Section 2 = ______

Difference of Means = ______

Percentage who have attained mastery = ______

Questions to think about/consider to inform the next steps of instruction:

After comparing pre-test to post-test results, what does the data tell you about your students’ achievement/learning in the content areas focused upon in these two sections of the assessment?

What types of supports would you have planned for student learning in each section after viewing the pre-test scores?

Was mastery attained by all students in both areas? How do you know? If mastery was not attained, give some possible reasons why not. If mastery was not attained, what could be the next step(s) for the student.How can you assist and support their learning?

Where would you go from here in terms of these learning objectives with these students?

Section 1 from Pre-Test

Part 3: Choose the correct major key from the following choices:

A: G majorB: Bb majorC: F majorD: C majorE: D major

11. 12. 13. 14. 15.

Part 4: Choose the correct relative minor key from the following choices:

A: e minorB: g minorC: d minorD: a minorE: b minor

16. 17. 18. 19. 20.

Section 2 from Pre-Test

Musical terms. Choose your answer from the following terms:

A: GenreB: TempoC: KeyD: FormE: Style

41. The musical term describing the “speed” at which a song is sung. An example is “andante” or “slow.”

42. The musical term describing how a song is put together. An example is ABA.

43. The musical term describing the type of articulation used to perform a song. An example is “legato.”

44. The musical term describing the “type” of song. An example is “spiritual.”

45. The musical term that gives information about where “DO” is located. An example is “Bb major.”

Section 1 from Post Test

37. Which key has 1 sharp?