Frary, R. B. (1995): How Difficult Should a Test Be? Educational Resources Information Center: Briefing paper.
Replies to This Discussion
Reply by Callie Grubb
I had never thought much about the grading of tests/assignment until this week's class discussion on zeroes. All to often grades are presumed to show achievement of students--the higher the grade, the higher the achievement. However, this may or may not reflect the knowledge gained by the student. A student who scores a 50 may have learned more than a student who scores a 90. They may have just started at different points in the learning process. Children with more cultural capital often learn at a faster pace than those with less. Because no two children are the same, I really like the idea of self assessment and the graphing of a student's progress. This way the learning/progress of a student can be more readily seen from start to finish.
This article helped reinforce that test construction and grading are not necessarily straightforward tasks. To paraphrase Frary, a teacher can make a test as hard or easy, long or short as he/she likes. Due to the arbitrariness of test building, exact levels of achievement may not be shown. It is interesting to me that maximum effectiveness of a test, in regards to student ranking, is gained when the average score is between 50-60%. In today's educational world, a 50 is thought to be failing. I also found Frary's comment that teachers should limit the use of items that are answered correctly by more than 90% of test takers to be very perplexing. I had always thought that all questions needed to be answered correctly by all students 100% of the time. I never stepped back to consider if the test was fair or if the questions were difficult enough.
I must admit that as a classroom teacher, I did rank my students based on their grades. The higher the grade, the smarter I considered the student to be. Also, if everyone did well on the test, I thought I did an excellent job of teaching. However, maybe I made my test to easy or maybe all of my questions required only lower-level thinking skills. This article makes me think that maybe we need to reconfigure our grading scale. What makes a 60 an F and 100 an A? After reading Chapter 3 in Popham's Assessment for Educational Leaders, the one thing that should be clear is the fact that all tests need to benefit the instruction of students.
______
Reply by Cyndi Austin
Robert Frary’s article, How Difficult Should a Test Be, made some good points concerning desirable passing scores. However, I doubt that the average individual would follow the statistical presentation of “bits.” With that said, I also don’t think that there are too many students or parents who would even question the grading system. They will question the grade but not the system from which the grade was produced. Apparently the arbitrary grading scale that has been “adopted” by most educational institutions (A.S.U. included) has been ingrained in society’s psyche as the embodiment of a statistically accurate measurement of achievement. Although, as an educator I have observed the shortcomings within the system and have manipulated the scores and testing items (bonus, extra credit etc), to reflect what I felt was the student’s true measurement of achievement, I now realize that my actions were not contributing to the accuracy of the measurement but rather conforming to the inequitable grading scale.
The class discussion over the credence of the zero highlighted the problems associated with the disproportionate grading scale. Still a firm believer in the zero measuring zero evidence of work or achievement, I acknowledge how such a grade can skew a student’s true measurement of academic achievement. After much thought on the subject, I have decided to adopt Dr. Olson’s suggested 0-5 grading policy. I believe this grading policy focuses on achievement and removes the focus of rank that is produced by the 100 pt. scale. With this policy students are always within a realistic range of success and teachers are able to accurately chart achievement. I realize that this is a rubric assessment format and rubrics are generally reserved for performance based assessment, however, I am beginning to realize that all assessments are performance based assessment and should be scored accordingly. If nothing else, I can stop embedding those questions, which Frary warns are too easy (90% answered correctly), for the purpose of increasing a student’s score.
Although NC Wise and the general population are not ready for this grading policy, I believe that a 0-5 grading score will result in a more accurate measurement of a student’s achievement and will convert to the 100 pt. scale with the same accuracy. Additionally, I wouldn’t have to create and grade a last minute, bogus, extra credit activity that a student (or parent) would complete in order to raise his/her grade.
______
Reply by Michael Robbins
The ideas presented in Frary's article seem legitimate enough; however, I question the reduction or elimination of "easier" items on a test. I think that it is very important for students to experience some sort of sense of success. Isn't this one of the most effective ways to motivate students, especially low achieving students? If we present all students with a test that only fires higher order thinking and difficult questions at them, they may quickly become discouraged and then that horrendous feeling of hopelessness begins to set in. It's easy enough to give such a test and "adjust" the percentile rankings to better fit the grading policies already in place. This piece of the article makes sense. Sure, we, as educators, may gain a better feel of the skills our students are mastering and the progress they are making by taking those less difficult items out of the picture, but are we destroying feelings of hope and accomplishment for some of our students? Personally, I find that differentiation of assessments is a better approach. Students that are demonstrating acceleration in a particular tested area are provided chances to expand and apply their skills to more difficult items and problem solving opportunities. At the same time, students that are just becoming comfortable with the same skills are provided with items that allow them to demonstrate their level of mastery in a fair and appropriate manner.
______
Reply by Jennifer Blankenship
After reading Frary's article, I thought a lot about the way I write and use tests. I am one of those educators that put items on a multiple choice test thinking that all students will surely be able to answer this question. I believed that tests should have a variety of low, middle, and high level thinking skills questions. I also thought that tests should have “easy” (for lack of a better word) questions so the student’s level of confidence would increase. While reading, I thought back to our class discussion on grades and giving zeros. As with this article, I still think parents would have a difficult time accepting a student bringing home a 70 thinking that was a good grade showing the student is achieving what is expected. I realize this would have to be communicated with the students and parents but it would require a complete adjust in how we perceive good and bad grades.
I do believe that tests should measure a students’ achievement in regards to “learning targets”.I do use test grades as a way to rank students. I, also, go over the tests with students discussing commonly missed questions. However, I have found that when students get test papers back with grades of 70, 60, or below many will just turn their paper over, not caring to look at what is missed. Just by looking at the grade they feel that they failed. This gives me great concern over eliminating the “easy” questions from tests. What happens when students who normally make As and Bs go to making Cs and Ds simply because I assume everyone knows the basics? I guess if I make a test with all high level questions I could grade using a 50 point scale and just double the end score to better fit into the accepted and expected grading scale. I do like the idea of students charting their own progress and I do this with my remedial Language group. Students get a confidence booster when they see that bar graph continue to rise.
I carry away from our readings of the articles and book that I do need to look a lot closer to what I am assessing and how am I using those assessments. I need to use assessments as a way to guide my teaching and a tool for students to setup their own learning goals.
______
Reply by Michelle Baker on
This article was not at all what I thought it would be when I first began the reading. I thought Frary, although suggesting revolutionary assessment strategies with bits and reducing the passing score to 50%, actually advocated for assessment of students in mesures contradictory to AFL. Frary believes strongly in eductors not presenting questions that 90% of the students could answer. If teachers are going to involve students in the assessment process, focusing instruction on helping students understand LHTL, then I would expect 90% or greater of those students to score in the traditional passing range of 70% or higher. For example, through this entire coursework to date, I am seeing the importance of giving students opportunity to voice what an assessment should look like, how an assessment should be scored, what questions and components of the target goal need to be assessed. If I am truly partnering with my students on creating assessments, then those assessments should reveal a more accurate picture of what all students have learned. Perhaps there is still validity to Frary'sarguement, but I would utilize his assessment ideas in formats such as pre-tests or pre-assessments. It is for these assessments that I could rationalize eliminating the "knowns" to determine the "needs more understanding" content of a target goal. However, final assessments should tests what all students should now know after the unit of study. When I give a final assessment, it is not a trick! I am not going to ask questions that I think most of my students do not know the answers. It is not that I think tests should be easy; teachers can ask difficult questions. However, students should be prepared for the questions, should have helped develop those concepts, and should have been given strategies for solving those quetions before they appear on an assessment. Difficult questions should not be surprise questions! Assessments should equate to a success level for students, and if success is still measured widescale by the traditional grading percentages, then students, parents, and teachers must consider this as key when relating measures of success to stakeholders. Assessments can be authentic, grading can be nontraditional, rubrics can be designed by a one-four point system, but when it is time to communicate success, I feel those scores must translate to traditional norms. In conclusion, I obviously did not agree with Frary's mindset of how to develop an assessment to determine success of students. I will remain open to others' points-of -view, but I may have to take the position to respectfully agree to disgaree on this article.
______
Reply by Anna Lankford
When asked the question "What is wrong with 70% as a passing score?", my immediate response is yes that is passing but have they mastered the material? Probably not. As a parent and knowing the expectations we have set for our children, 70% would not be acceptable. I think you would find this true with most parents as the traditional 7 point grading system is ingrained in us. When I have used rubrics in the past with a scale of 0-5 for each criteria and total points adding to 20, parents have "freaked" out because they thought their child had failed with that score. It takes a lot of communication between teacher and parents when other types of scoring systems are used.
I also found it interesting the article recommended limiting the use of test items that are answered correctly by more than 90 percent of the students. I usually tried to balance the types (easy, medium, and difficult) of questions I placed on the assessments. I found that when students missed the easy questions, it was typically a careless mistake and a good teaching opportunity to discuss with the student about checking thier work. I think removing the easier questions would also be setting some of the lower students up for failure before the assessment was ever handed to them.
I found the timing of this article interesting since we just received 3rd grade Pretest Reading Scores and have all been shocked across the state at the extremely low numbers of students proficient. The state has raised the bar from somewhere around 38% to be proficient to 70%. When my testing director told me this figure this week this article immediately came to mind. We always expect to see a drop in scores when the test is re-normed but have never seen a drop like this. How do we explain this to parents so they understand when we as educators have a hard time understanding it ourselves?
______
Reply by Michael Robbins on
Michelle and I seem to agree with "disagreeing" on Frary's idea of assessment. This idea, indeed, seems to contrast the realm of AfL. If students are to play a role in developing assessments and learning targets, then we are providing a real disservice to them by creating a test that they can only be 50% successful with. In order for students to take ownership of their learning, then we, as educators, must provide opportunities for them to feel comfortable and successful in the process.
______
Reply by Dustin Farmer
As I read this article the first time I was not for sure I understood what Frary was advocating. I could not understand why as a teacher I would want the average score of a test to be 50%. As I skimmed the article I began to understand Frary's take on testing and the actual reasoning behind it. It is hard to argue with a fellow Hokie. However, I do not fully agree with Frary's reasoning behind his testing method.
I agree with what has been said concerning the use of easier questions. I think it is very motivating for those struggling learners to get off to a good start on a test. I'll never forget a test I took at Virginia Tech. There were only three questions on the test. I looked at the first one and knew I could not complete it. I moved on to the second question and found that I could not fully answer it either. I then moved on to the third question and immediately returned to the first question and began the test. For the first time in my career I had ran into a test that actually challenged me, and as a result made me feel like a failure. The 63 I made was failing by most standards, but was an average grade for the class on that particular test. It made me realize that I not only needed to study harder, but that I needed to go back and review some of the basic concepts that I was expected to know to complete the course. We have to remember when making tests that some of our students are coming to us with limited skills. Tests that are too difficult just reinforce what they have already come to know, they are failures. We must also be aware that our upper level students must be challenged, therefore we have to try to have a happy medium. I am afraid that if our tests are too difficult that it will not only have a negative effect on those students who are "destined" to fail, but also on those students who are high achievers.
Frary makes a compelling argument and I can definitely see the reasoning behind it. However, if we as teachers are going to conform our tests to these standards, we must reculturalize our students and parents. We must make them realize that our job as teachers are to challenge their thinking and in the process produce more effective learners. I believe the most important aspect of this process is the evaluation of each student following the assessment, and addressing any issues that seemed to be a real bump in the road. Failure to address consistent problems within the class, along with needed encouragement and active student participation in the testing and grading process is where difficult assessments can lead to discouraging environment.
This article, along with the other articles and the textbook, has made me take a harder look at the assessments I have given in my own class. It is so easy to teach the material and give the test that we have been using for several years. However, I do believe it is important that we challenge our students through assessments, and use them to not only give grades, but to evaluate our students progress and our actual instructional methods.
______
Reply by Travis Richardson
Reading this article really made me stop and think about some of the tests I have given in the past. Many times I have felt better about myself as a teacher when all of my students have done well on a test. I have also wondered what I could have done differently when many of my students have done poorly on a test. I had never really thought about what Frary said about placing items on tests that 90% of the students should be able to answer with out any trouble. The statement he makes is true. It could be considered a waste of time in grading those questions, because we already know that a great number of the students should be able to answer these types of questions. As teachers we really aren't learning anything about our students by giving them these "easy" questions. I understand the point that many of you have made on this comment about how these types ofquestions can build the confidence of students who might have trouble on the test, believe me I have been there many times myself. I think we really need to ask ourselves what type of information do you want to gain from giving students this test and go from there. Are we just giving this test to get a grade for the grade book, or are we giving this test to really find out what our students know and what do they need to improve on? I think the type of test that actually shows what the students know helps both the students and the teacher to evaluate where they are and challenges both students and the teacher to improve what they are doing. Although, I know many teachers will still give tests just so they will have some grades to average for the grade book.