Once again the recently released NAEP results reveal that American student achievement in writing is far worse than local report cards would have us believe. If the new assessments for Common Core are going to be as demanding as NAEP tests are – a likely bet – then we have a disaster in the making: scores are going to be bad and there is going to be hell to pay politically (since NAEP is not district-level reported and typically flies below the layperson radar).
Just so we’re clear on the problem, let’s compare 8th grade writing results on NAEP with results on a state writing test (Pennsylvania), in four varied districts.
Here is a table with comparisons for 8th grade writing:
|
|
Advanced |
Proficient |
Basic |
Below Basic |
| USA – NAEP 8th gr | 2 | 29 | 57 | 13 |
| PA – NAEP 8th gr | 1 | 35 | 55 | 9 |
| PA – PSSA 8th gr | 10.3 | 64.8 | 22.6 | 2.3 |
| Bellwood-Antis | 46.4 | 48.8 | 4.8 | 0.0 |
| Quakertown | 22.5 | 61.7 | 14.2 | 1.6 |
| Philadelphia | 2.9 | 45.3 | 42.5 | 9.3 |
| Wilkinsburg Borough | 0.0 | 20.9 | 61.2 | 17.9 |
Notice that the PA state average scores on the PSSA (3rd row) are much higher than PA NAEP scores: Pennsylvania claims that 75.1% of its 8th-grade students can perform at the Proficient or Advanced levels while NAEP results suggest the number is less than half that – 36%. (A close look at the anchor papers reveals that Pennsylvania scoring standards are indeed lower).
Compare, further, the gap between this already-low PSSA state standard with the results in Wilkinsburg Borough. Only 20.9% of their students are proficient (and none are advanced). Compared to NAEP results, it is likely, therefore, that only about 10% of their students are truly proficient in writing.
SCORES VS. GRADES. Now here’s my big rhetorical question: do those students and their teachers realize this? Because we can safely predict what letter grades students in ELA and English earned from their teachers in Wilkinsburg Borough (or Philadelphia): a typical spread of grades on a curve. Now, do a little thought experiment by looking at the chart: consider what letter grades the students in each district should receive from their teachers if we sought to communicate honestly to students where they really stand.
Which means that most students in weaker districts are utterly deceived as to where they really stand in the fundamental skill of writing.
Teacher grading everywhere is thoughtless because it assumes that an isolated individual educator, with no training in measurement, using no anchors or calibration scheme, can reach a valid result. Worse, we happily permit and even expect a bell curve of results. In other words, building-level norms determine grades, not any objective standard of quality. Typical single-letter-grade evaluations are thus indefensible in a world of standards as well as being poor feedback.
Put differently, an “A” on a transcript in Bellwood-Antis or Quakertown is a much more credible grade than an “A” from Wilkinsburg Borough, once we compare PSSA scores to grades. This is, of course, a big reason why the SAT and AP’s were invented: to help sort out the meaning of unmoored local letter grades.
Look, we educators all know why norm-referenced grading happens. Teachers cannot politically or morally fail most of their students even if their work is weak. Over the years we have allowed local grades to reflect effort and potential, and tacitly assumed that the work of our best kids in excellent, regardless of wider-world excellence. Nor am I in any way in favor of shaming students (or teachers) about this. I only know, though, that if we make students think for years that their performance is objectively of high quality when it is not then we are setting those kids up for failure and despair later on.
SOLUTION: BETTER REPORTING. The solution is to demand 1) that report cards contain standards-based scores in which teachers compare student work to state or national anchor papers, not just norm-referenced letter grades; 2) the state should publish the school and district letter grade distribution in its school report cards so everyone can see the differences; and 3) the state should require that districts audit the validity of standards-based scoring done locally to ensure that the local scores are indeed predictive of state and national test scores. The motto here is: no unpleasant surprises come test and college admission day.
So, then a “B” student in a weak district would be more accurately told: “Your work ethic and assignment completion are really good, so you earned a “B” from your English teacher. But in terms of your standing in the state, you are a 1 on a scale of 4, so we have plenty of work still to do on your writing to get it up to standard.” This is the same thing we do in sports and in the arts; why can’t we do it in academics?
I have worked in one district in Pennsylvania that has been doing standards-based reporting for a few years, and I put their data in the chart above: Quakertown. Their writing scores have improved steadily for the past 5 years as a result of this effort. Yet, they get foolish pushback for their efforts from nervous nellie parents and board members, and few other districts have followed suit. Don’t these educators see the trouble coming down the road? Apparently not. [NOTE: Look at the comment posted by an educator from Quakertown on their history with standards-based grading, containing a link to their site].
In a perfect world, we would not need standardized tests. Schools would do such a good job of reporting out student performance against objective criteria and standards and there would be such good local quality control in the assessment and grading process that the transcript, by itself would be completely trustworthy as a record of student achievement. Alas, we’re not there by a long shot.
In short, most school systems deserve a “Below Basic” rating for their grading system. Is anyone concerned about THAT result?
12 Responses
Your points are well taken. The whole issue of grading is central to helping kids develop a sense of what it means to “learn.” I see kids turn off to challenging work as early as 1st grade because they would rather get a good grade by choosing easier work. But the question remains, how do we start early to communiciate to learners the idea of growth or improvement or even what we mean to learn, As you suggest, we need a system other than letter grades or perhaps in addition to letter grades. I’m hoping that the growing interest in formative assessment will help students and teachers to know where they are in the “process” of learning.
I think we need to reconsider our current testing scheme…if we return 40-50 years in the past, we had grades for classroom efforts, etc, and standardized test scores, so you had some idea of where your student stood against large numbers of students (CAT, Iowas, etc). Seems that between both of these, we had a decent picture of what students both could do and were doing. Isn’t that what we are really looking for?
Most state standardized tests are not very good, but they do allow parents and students to see how they stack up compared to the rest of the state. Many students in low performing districts will have high grades and a state test report labelling them “proficient”, but also indicating that they scored in the bottom 25% in the state.
It is not difficult to determine whether your child ranks relatively high or low compared to the rest of the state – it is right there on the report. But, when most states have ridiculously low standards for “proficient”, it is also not difficult to ignore the fact that your child ranks very low compared to the rest of the state.
We really don’t need more reporting. We need the states to be more realistic with their labeling of “proficient” and we need parents that will open their eyes and look objectively at their children’s achievment.
Bravo. I’ve been advocating for criterion-referenced over norm-referenced grading for as long as I’ve understood the difference. But your argument here takes the issue to a whole new level….thank you.
Love the post, however as a tenured teacher I have the ability to grade “harder” than my new colleagues. The administration in my district (not just my school) will mandate that teachers – especially new ones – raise grades so that they do not have such high failure rates &/or low grades in general in their class. As a teacher of primarily juniors – I do make a point to say, if my students were getting an A in my class, their standardized test scores would be much higher. The students have to have better accountability than just the individualized grades of teachers, which as mentioned can be inconsistent.
I guess I need to clarify my earlier response, especially in light of those subsequent to it. I think we need both criterion-referenced grades and performance-based grades…together they give us a picture; separately, it is more like one of Picasso’s cubist efforts: wonderful in itself (yes, individual aesthetics), but not necessarily a true picture for us to utilize.
I think this comes around as I generate weekly/daily “I can/will…” statements for my students, rather than thinking about what I going to “cover.” As educators, even with these tests, the real question is what we have our students actually DO, and how well they retain it in their futures.
[…] This blog post discusses what criterion referenced assessment reveals for schools using norm referenced assessments – good thoughts here with good explanations, examples and challenges to think about. Now that most states are implementing rigorous standards (CCSS) and teachers are able to choose the best curriculum, methods and tools to teach with, how will we give the most meaningful feedback to bring students' work up to the proficient level? Further, how will we communicate the importance to push through difficulties and obstacles that students are likely to encounter when they may not be used to this? Building student investment and resourcefullness in the importance of meeting academic challenges is as important as building their self-esteem and protecting their feelings when they don't perform. […]
We were fortunate enough to have Dr. Wiggins visit Quakertown. After four labor intensive years, we are finally comfortable as a district with SBG, albeit professional development and refinement never cease. Is has been a long, difficult path to get to this point. QCSD contracted with a private statistician to analyze the correlation between our student grading and our state test results and found a strong positive correlation at every grade level in reading and math. For parents, we report two “grades:” proficiency on learning targets (aligned to state standards…soon to be CC standards…our objective criteria for now.) and a citizenship grade. The extraneous factors such as compliance, extra credit, attitude, homework (unless it is an assessment or evidence), averages, etc., are not part of the proficiency grade. These affective traits show up in the citizenship grade. Parents still struggle because they are more used to averaging and behaviors as part of grading. Some parents also do not like that “other” children can re-test or re-do to attain proficiency when their child got it on the first try! We remain committed to ALL students learning at high levels with deep understanding no matter how long it takes or how much effort we ALL must invest. Visit http://www.qcsd.org and find the Standards Based Grading page for more info.
Even in a standards-based school, “grades” become a problem due to a lack of inter-rater reliability. Also, there is a misconception that standards = grades.
Once per year, our faculty of 27 teachers grades 3-5 blindly score open-ended writing samples against an identical 6-traits rubric. Each year, teachers are told to “score based on rubric descriptors” and “go back to the language of the scoring guide”, teachers still mark scores relative to others in the grade.
You can hear, “This is a ‘5’ paper. It’s really good for a 3rd grader.” The standard for a “5” is that the paper is clear, focused, detailed, and insightful. No inter-rater reliability exists when teachers disregard rubric descriptors in favor of a normative judgement.
It’s okay if NO 3rd graders get a ‘5’. A ‘5’ does not equate to an ‘A’. The 3rd grade teachers can get together and decide what standards correlate with a grade.
In other words, you can assess standards and then assign grades based on standards met. You cannot assign grades and assume those grades equate to standards.
Ran across your post while researching SGB. Apparently Quakertown is going back to the future and abandoning SBG despite increases in ACT and other scores. http://www.wfmz.com/news/news-regional-southeasternpa/quakertown-school-district-will-welcome-new-grading-system-next-year/-/121434/22147024/-/vo5vst/-/index.html
Not surprised. I spoke to the Board and there was strong feeling against by three members then. Too bad.
There is an E.D.Hirsh, “back, to basics,” remediation is an “entitlement,” sort and select, element to the discussions surrounding SBG in QCSD. SBG is hard….hard for students, teachers, and parents. It demands students keep at it until they demonstrate proficiency. They cannot take the “0.” It creates more work for teachers who are already stressed. Students can no longer earn an “A” by being compliant or adequate…they must demonstrate a high depth of knowledge. Unfortunately, there is pressure from parents that an “A” is too hard. In turn, there is pressure to the board. The most disheartening comments from both the board and parents is that giving kids multiple chances to demonstrate proficiency somehow takes away from those who may not need as much help. I have heard “there are no second chances in life” multiple times. I guess everyone only has one opportunity to pass their drivers exam, the CPA or Bar exam, create a design, or write an essay. We are working to address the issues and you can read about that on our website http://www.qcsd.org with a link from our home page. Most ironically, SBG has been blamed for students not doing well by parents when in fact QCSD has the highest SATs, ACTs, AP exam scores, the number of students taking AP courses, and lowest dropout rate in the history of the school. Whenever a student or teacher is not successful, you have to blame something. SBG has become the unwarranted victim in QCSD. So much for being on the current (bleeding) edge.