In a previous post I argued that the dreary and un-improving results on tests of reading comprehension mean we need to take a radical look at what we are and are not doing, especially in middle and high school. I argued in the first post that we know far too little about what real readers actually do when they face challenging text – a predictable problem of great importance in light of Common Core and college.
Here, in Part 2 of a series, I summarize the research on comprehension.
I spent three weeks reading almost everything of note written on the subject since 2000, plus seminal pieces from the 80’s. My chief reference was The Handbook of Research on Reading Comprehension (2009) in which most of the top researchers in literacy summarize the state of the art of current and past research on comprehension.
Except for the bracketed comments by me, every paragraph below is a direct quote from the Handbook (sometimes shortened for readability).
What we know
Skilled comprehenders use metacognitive strategies significantly more often than less skilled readers.
Less skilled comprehenders were significantly less likely to make inferences from text even with the equal background knowledge… This supports the notion that comprehension requires flexible simultaneous consideration of multiple elements.
Younger readers have little awareness that they must attempt to make sense of text; they think that reading is decoding – reading as word understanding. Older readers [5th and 7th graders] knew what to do but they did not do it. Older students were no less likely to classify their difficulties at the word level than younger students [as opposed to the sentence or paragraph level].
The extent to which children slow down their reading on encountering inconsistent information is a significant predictor of comprehension.
Comprehension monitoring accounted for unique variance, once working memory and other background factors were controlled.
Rather than teaching students how to become self regulated learners, teachers seem to expect behaviors would naturally developed through prompted questions. There is of course no evidence that such prompting leads to anything like active self-regulated use of comprehension strategies.
Improving performance is possible. However there is less evidence that comprehension focused interventions produce either autonomous use of comprehension strategies or longer-term improvements in comprehension proficiencies.
The instructional frameworks described (Question the Author, Reciprocal Teaching, Concept oriented Reading Instruction, Collaborative Strategic Reading, Transactional Strategy Instruction) have demonstrated great potential for developing higher-order comprehension. That said, research continues to describe few classes where children are benefiting from these or similar approaches…. While the research evidence in favor of comprehension instruction piles up, the gap between research and practice remains stubbornly wide.
It is difficult for many teachers to understand the necessity of keeping the content of the text at the forefront while teaching strategies… This [problem] occurs, for example, when teachers only ask students questions about which strategies they used and why, instead of asking questions about the content of the selection.
Decades later Pressley et al. (2006) found essentially the same thing [re: Durkin’s famous study in 1979 that, despite teacher enthusiasm for developing real understanding of texts, no instruction in comprehension was occurring.]. Despite these well-understood findings about the value of teaching strategies, comprehension instruction continues to receive less attention than other skills or content.
[Grant’s note: there is little justification in the Handbook for the most common long lists of Strategies I critiqued in an earlier post. Almost all the Landmark Studies discussed in the research focus on only 3-4 strategies to be taught and used simultaneously, in order to develop a self-regulated repertoire via transfer of learning thru gradual release: Preview, monitor comprehension for breakdown at more than the word level and re-read as needed, draw inferences across the text, identify the main idea of the text, ask questions of the text, summarize.
Note that these are precisely the abilities students do poorly on in tests of reading comprehension.]
What we don’t know
We lack research demonstrating how the ‘automatic pilot’ [of student internalization and self-regulation of comprehension monitoring] develops.
Remarkably few studies focus on typically developing middle-grades readers. Given the unique challenges presented by young adolescent readers and their teachers, there is a great deal of work to be done to further our understanding of these students.
Research in content area literacy/Adolescent literacy has rarely if ever addressed comprehension strategies. Previous research on comprehension strategies has been limited by its focus on younger readers with only very simple tasks.
Too few studies involve classroom teachers providing comprehension instruction, especially over longer periods of time. As Guthrie (2004) has noted, we have many, many studies involving a small number of children being taught a single strategy over a few days or weeks… But only one or two studies are available where public school teachers have provided instruction to large numbers of students on multiple comprehension strategies across 1 or more years.
In our view there is too little research to provide useful guidance for developing complex proficiencies in struggling readers across the school career.
Reading comprehension is an acquisition that involves many components and develops over an extended period of time, with the case made that a comprehensive theory of reading development must be one that includes many more components and many more years than the theories that we currently have. Relatively little work has investigated how expert reading processes develop.
The lack of evidence [about whether strategy instruction transfers] stems from the heavy reliance on smaller sample sizes and shorter-term intervention designs as well as limited attention to a ‘gold standard’ of transfer of training to autonomous use.
Too often traditional accounts assume students want to be higher or high achievers…. The most obvious direction needed is a solid base of empirical studies on self regulated comprehension. Contrary to what many believe, there is not a solid research based now.
3rd and 5th grade students relied almost exclusively on word-level criteria for evaluating their understanding, replicating the findings of Baker more than 20 years earlier. Similarly, Eme et al. found that third graders’ conception of a good reader was one who reads quickly without making mistakes, replicating the findings of Myers and Paris 30 years earlier. [Thus] research findings still look a great deal like they did originally which is quite troubling. Change is slow.
Some thoughts of mine, based on these findings:
1. There is no mention in any of the research to support E D Hirsch’s claims about the key to reading comprehension being better background content knowledge. Indeed, his name is not mentioned in this 600+ page reference book.
2. The lack of transfer of learning of the strategies seems to be a function of inadequate teaching for transfer, and a failure to understand the principles behind the strategies.
3. There is little or no teaching of comprehension/metacognition strategies in 6th through 12th grade because curriculum is written around books to be assigned and the ASSUMPTION that all a teacher needs to do is assign and assess, not teach reading. I once had a high school teacher say to me bluntly – after he complained about poor literacy levels – It’s not my problem: I teach English, not reading.
4. There is really only one piece of research that frames the entire debate about what is possible in adolescent literacy: the work of Palinscar & Brown in the mid-80s on so-called Reciprocal Teaching. This is the most-referenced study, and all of the highly-successful research projects of the past 20 years derive their approach from that study. Every teacher of reading and English should read it carefully, therefore. You can find it here. I also highly recommend Questioning the Author by Beck et al – one of the top-rated approaches – because it provides the most accessible and practical way of showing how to teach students to read more carefully and critically.
5. It seems reasonable to say, from the data and the research, that the reason SES correlates highly with reading scores AND that reading scores are too low and flat over time, is that we are not teaching people to read effectively. The data are clear: except in small research studies and outlier classrooms, all the time we put into literacy does not result in highly-literate students – in spite of what research DOES reveal about how to improve comprehension. The research could not be more consistent on one basic theme: comprehension and self-regulation CAN be improved in ALL learners of ALL backgrounds. That reading results haven’t improved much beyond decoding says clearly, however, that we have not designed ‘backward’ from in-depth comprehension, using what research tells us will work.
Categories:
Tags:
36 Responses
Thanks, Grant – good summary of what we know. I observe that teachers seldom teacher/model strategies like reciprocal teaching. Do they know how to use it?
I have the advantage of working with very small groups of students. The teachers are newbies, but they are willing learn.
Reblogged this on A Teaching Life and commented:
Wonderful article by Grant Wiggins:
I will have to start looking in to this style of reciprocal teaching. I am not familiar with it. Thanks for sharing.
The name is a tad misleading. It’s about training kids to work in groups, where kids become teachers of one another as they work to infer key aspects of the task. It goes from teacher modeling to complete kid independence in the now-called “Gradual Release of Responsibility” approach.
Like the “Monitor” schools?
??
Grant, I wonder if the same can be said about math, substituting problem solving for comprehension and skills for decoding. Certainly your conclusion about the test scores is the same. When our mission is focused on “What is___” and not “What if…” our children learn to be mechanics and not thinkers.
Art
I am not only with you, it links directly to kid inability to handle word problems in some research. And there are numerous references to the science misconception literature as being linked to a similar failure. I will post some of this domain-specific research along with a few more extended quotes on the key role of ‘cognitive flexibility’ shortly.
I love this post…what an excellent summary of the research on comprehension. I am also wondering if perhaps the inconsistency of the definition of ‘strategy instruction’ contributes to the gap between research and practice. There is a tendency to lump both the cognitive strategies that you mention (monitoring, summarizing, previewing, etc.) together with the instructional techniques designed to teach these strategies (QtA, Reciprocal Teaching, etc.). While I really appreciated the “Put Reading First” document, I think it added to this confusion. I work with many teachers who consider the use of graphic organizers as strategy instruction without reflecting on the cognitive strategies that students use as a result of having the graphic organizer. I have read some of Scott Paris and Kay Stahl’s work on the difference between cognitive strategies and instructional techniques, but I see it used interchangeably in classroom practice.
My dissertation work is related to middle level reading comprehension and the content area strategies that are most implemented outside the English Language Arts classroom. I liked your ideas about transfer and the lack of teaching for this transfer–I am also wondering if the systemic lack of teaching and applying literacy strategies outside of the ELA class contributes to this challenge as well.
Anyway–thanks for the summary–it will help guide my reading as I continue my work!
I didn’t dwell on the literature on cognitive vs. comprehension strategies but your point is spot on: there is confusion here, and it plays out in instruction.
Great post as usual. Seems to me that one of the reasons RTI hasn’t put a dent in reading achievement is because it tends to focus on isolated skills, such as fluency. Comprehension can’t be taught as an isolated skill. What do you think? And is the SPED approach to reading also void of effectiveness ?
There is a lot of research saying that fluency is NOT a good predictor of comprehension.
Grant– Seems that we spend too much time and effort on intervention and not enough getting Tier One instruction done right. Do you agree?
I also wonder how many students appear to be OK readers in primary grades (K-2) but then falter as read for meaning in upper elementary and into MS.
The research seems very clear, consistent, and of longstanding: too little valid instruction that culminates in self-regulation. So, in a word: YES.
In my next post on this topic I want to highlight the right and wrong ways to do instruction in comprehension/metacognition strategies, according to all the research.
Can you cite the research on fluency not being an indicator of comprehension? It would be most helpful for the discussions at the high school where I teach. Thank you.
Will do, in my next post
Thank you as always for sharing your expertise. I am wondering about reading comprehension tests themselves. How good of a measure are they of reading comprehension?
Good question! I plan to address this soon. There is a chapter on reading assessment research in the Handbook.
In compiled observations of English language arts classrooms, Stanford Professor Pam Grossman found very little evidence that strategy instruction and modelling and demonstration actually happens. See graphic from the Measures of Effective Teaching report and link at https://twitter.com/thor_winnipeg/status/475708708882616320
Even if we’ve known that reciprocal teaching works for many year, it hasn’t translated into teachers delivering that kind of reading instruction in classrooms.
Agreed – just reading M. Presley’s book Reading Instruction That Works and he makes same point – even when observing highly-recommended classrooms. More on this in my next post on this topic.
Thanks for this discussion and the recommended references. I plan to read them immediately! As a part-time ESL teacher of adults (and then only in retirement. I’m not a life long teacher), I’m afraid I’m another one of those teachers who hasn’t thought much about reading comprehension. I am far from experienced in the teaching of reading. (“I teach English, not reading.” ha ha.) But, like it or not, I now have a student in a small intermediate ESL class whose first language would probably be labelled as English. That is, his parents are Salvadoran, but he was born and educated (through high school) in the U.S. Hence he speaks English fluently. Since graduating from high school, he has attempted some community college classes, but says he couldn’t do the math – and someone apparently assessed his issues as reading related.
He seems to have a lot of self-awareness about what he refers to as his “disability.”
And says that “some words I just can’t even begin to figure out” and “maybe if I read something many times, I can finally do it.” (And probably by “I can finally do it,” he means “I can finally understand it,” because when asked to read a paragraph for the first time, his reading sounds reasonably “fluent,” to me anyway.) Stumbles over a few words, but not that many…
It seems to me that his tuition to this private language school would likely be better spent on time with a reading specialist of some sort. But the school accepted him, (guess his money was green), so I am trying to figure out what he needs and how best to help him – in the context of a class with 4 other “real” ESL students – 3 Spanish speakers and 1 Japanese speaker. His mother’s goal apparently is that he “have something to DO.” great. Talking to him, his goal seems to be to “improve my reading” so “I can try college again.”
Anyway, as a result of working with this student over the last two weeks, I’ve been thinking more about strategies for reading comprehension! So I’m glad I came upon the discussion.
Of course, if anyone has any suggestions for me, I’d love to hear them. That is, beyond the fairly obvious “he’s in the wrong class.”
I think some of my recommendations will help. I also recommend Reading Instruction That Works by Michael Pressley and What Readers really Do by Barnhouse & Vinton. I’ll also get me ESL friends to reply.
In “She’s My Best Reader, She Just Can’t Understand”, Applegate, Applegate, and Modla (2009) tested 171 ‘fluent’ readers and found a third were poor comprehenders. It seems that early on (K-2) fluency IS a good predictor of comprehension, but the predictive dimension of fluency quickly loses strength such that by grade four or six it is common to find fluent readers with low comprehension.
http://www.jstor.org/discover/10.2307/20464457?sid=21106174194063&uid=4&uid=3739408&uid=2&uid=3737720
Great title. Yes, the lack of fluency predicting is mentioned in a number of the articles in the Handbook. Not at all surprising, really, if you think that reading is word accuracy and speed in reading as many students believe (according to the research). Can you shoot me a copy of this article? I don’t have a login for JSTOR.
Thank you for your summary on the research on reading. I am going to hold tightly to your comment: “comprehension and self-regulation CAN be improved in ALL learners of ALL backgrounds.”
If anyone is looking for ideas and support in teaching reading, I suggest Mosaic of Thought by Keene and Zimmerman and Readicide by Kelly Gallagher. These books have helped me tremendously with understanding both skilled and unskilled readers. And both books also have a sense of style–which is always a boon for a reader. 🙂
I’m a fan of both (and I know Kelly – great guy and good teacher). I also find that I have learned the most from Beers’ When Kids Can’t Read and Winehouse & Hinton What readers really do. I am also halfway through Pressley et al Reading Instruction that works, which I find pretty good at balancing all the research with readability. But, as you’ll see in the next post, I find some of the advice in many of the literacy books re: comprehension shaky and often not reflective of what research reveals, as some of the quotes form the Handbook in the previous column suggest.
So, what book or PD should I start with in regards to comprehension & strategies that will give me the most bang for my buck? There is so much out there it can be pretty overwhelming.
I’ll make some annotated recommendations in the last post. For now, I really would start with a fluoride of the Palinscar-Brown paper on Reciprocal Teaching that I attached to the post. If you want a readable mainstream text on comprehension and metacognitive strategies for older kids, Kylene Beers’ When Kids Can’t Read is a great start.
G’day Grant,
This is a fantastic post on literacy and one I would be keen to continue reading about in further posts.
I would also really like to share a positive story about the impact of a literacy program I developed for a secondary school here in Australia had on our students. It is inline with your discussion on literacy and could provide your readers with valueable insight and possible solutions to address the points you’ve raised.
I would share it as a reply to your post but it is rather long and needs some graphics to support the concepts discussed. Is there an email address I can send it to?
Cheers,
Andrew Nicholls
Yes, please, Andrew! gwigginsATauthenticeducation.org if it seems apt, we can have you do it as a guest post?
G’day Grant,
Story has been emailed to you. Hope you find it interesting and apt.
Cheers,
Andrew Nicholls
Thanks for wading through the literacy research for us and providing this helpful digest. As you note, that research doesn’t seem to support the powerful link between background knowledge and reading comprehension claimed by some, like E.D. Hirsch. However, cognitive scientists, at least some of them, do seem convinced that this link is real and important. From Dan Willingham’s latest book (I saw this excerpt on the Core Knowledge Blog which I follow along with your blog, leading to occasional bouts of cognitive dissonance!):
“Most parents want their children to be solid general readers. They aren’t worried about their kids reading professional journals for butterfly collectors, but they expect their kids to be able to read the New York Times, National Geographic, or other materials written for the thoughtful layperson. A writer for the New York Times will not assume deep knowledge about postage stamps, or African geography, or Elizabethan playwrights— but she will assume some knowledge about each. To be a good general reader, your child needs knowledge of the world that’s a million miles wide and an inch deep—wide enough to recognize the titles The Jew of Malta and The Merchant of Venice, for example, but not that the former may have inspired the latter. Enough to know that rare stamps can be very valuable, but not the going price of the rare Inverted Jenny stamp of 1918.
If being a “good reader” actually means “knowing a little bit about a lot of stuff,” then reading tests don’t work quite the way most people think they do. Reading tests purport to measure a student’s ability to read, and “ability to read” sounds like a general skill. Once I know your ability to read, I ought to be able (roughly) to predict your comprehension of any text I hand you. But I’ve just said that reading comprehension depends heavily on how much you happen to know about the topic of the text , because that determines your ability to make up for the information the writer felt free to omit. Perhaps, then, reading comprehension tests are really knowledge tests in disguise.
There is reason to think that’s true. In one study, researchers measured the reading ability of eleventh graders with a standard reading test and also administered tests of what they called “cultural literacy”—students’ knowledge of mainstream culture. There were tests of the names of artists, entertainers, military leaders, musicians, philosophers, and scientists, as well as separate tests of factual knowledge of science, history, and literature. The researchers found robust correlations between scores on the reading test and scores on the various cultural literacy tests—correlations between 0.55 and 0.90.”
Then there’s this from the CCSS, which one observer (Robert Pondiscio) called the “57 most important words in education reform. Ever.”
“By reading texts in history/social studies, science, and other disciplines, students build a foundation of knowledge in these fields that will also give them the background to be better readers in all content areas. Students can only gain this foundation when the curriculum is intentionally and coherently structured to develop rich content knowledge within and across grades.By reading texts in history/social studies, science, and other disciplines, students build a foundation of knowledge in these fields that will also give them the background to be better readers in all content areas. Students can only gain this foundation when the curriculum is intentionally and coherently structured to develop rich content knowledge within and across grades.”
So what is going on here? Do we have a major disagreement between two different groups (cognitive scientists and literacy researchers) and/or different approaches to reading research? If it’s not an outright disagreement, how do we reconcile what appears on the surface to be a significant difference regarding the basis of reading comprehension? Most importantly, what do educators working in schools and making curricular decisions right now do with this seemingly contradictory guidance when it comes to the role of content in helping students become strong readers (on hand we’re told to focus on reading comprehension as a highly complex facility, the development of which will not be significantly aided through broad content knowledge acquisition; on the other that there really is no such thing as reading comprehension skill and that the best thing we can do to do build capable readers is to “ensure strong decoding skills at the earliest possible moment, then pile on rich and meaty reading in all subject areas” – from Robert Pondiscio’s recent article in US news entitled “Don’t Start Reading Tests in Third Grade. End Them There.”)?
I don’t think the disagreement is as wide as it seems. I have had lots of back and forth with Pondiscio. He, like Hirsch, simply deny that extensive instruction in reading strategies has value; they place way too much emphasis on background knowledge as the key to comprehension. It’s clear necessary – but not sufficient. And a moment’s thought reveals why: by their argument, comprehension just ‘happens’ if you have lots of background knowledge. All you have to do is think of HS and college English to realize this isn’t true. Lots of background knowledge about Romeo and Juliet is not the key to understanding the play! Nor can they explain how it would be possible for anyone to ever read science friction or Philosophy and make any sense of it. (Hirsch, in a series of emails with me, curtly dismisses Philosophy as “knowledge-free” writing.)
And as my review showed, the few studies at the middle school level show conclusively that even factoring out processing speed and background knowledge there is still a great gap in comprehension across readers, based on how well readers are metacognitive and discispined about slowing down, thinking, etc. Self-monitoring is the KEY variable in the upper-grades research. Neither fluency nor background knowledge predicts comprehension as well as meta-cognition. That, of course doesn’t mean that both are unimportant. But the Hirschians keep claiming, with no data to support them, that comprehension is more or less dependent upon factual knowledge. (Even that idea is flawed because lots of research shows that it is not ‘background knowledge’ per se but the ability to find the RELEVANT background knowledge in reading a new text – the transfer problem. Hirsch denies that the transfer problem is a problem – reclaims it’s just a problem of instruction in gaining background knowledge. Clearly too narrow.
Willingham also acknowledges that comprehension strategy instruction is important, unlike his colleague Hirsch, though he qualifies it a bit.
In sum – I think their case is very weak and confuses causality with correlation, the same problem that undermines the case for fluency as a key predictor in the early grades. It turns out not to be a key predictor of higher-level comprehension. (I could teach you years of ‘soccer’ content and background knowledge, but that won’t make you a better soccer player…)
More on this soon….
You’re doing a bit of violence to my views, Grant. I have always acknowledged the value of some instruction in comprehension strategies. But what I think you have failed to recognize is that in far too many schools, skills and strategy instruction is what reading instruction is. As a result, too many children (especially low-SES children, ELLs, etc.) never have the opportunity to build up the background knowledge that their more fortunate peers tend to accumulate through their daily lives, enrichment opportunities, etc. Since there is no evidence I’m aware of that continuing to practice reading strategies makes a difference, year after year after year, it seems not unreasonable to suggest that the long tedious hours spent in mind-numbing strategies work could be more profitably spent reading coherently across the curriculum. Not sure how that translates into denying the value of strategies instruction.
It’s encouraging to hear (after dumping all over Hirsch and now me) that you say that background knowledge is “clearly necessary.” That in itself is a validation of Hirsch’s work. Now take the next logical step and ask, well, where does all the “clearly necessary” knowledge come from? Osmosis?
Background knowledge is necessary, not sufficient. (So is decoding ability) What’s the next step? Teach people how to read, as the research makes clear.
Reblogged this on The education of a teacher. and commented:
Extremely important analysis of the historical and current research on teaching literacy, and also, importantly, what is lacking in the literacy learning scholarship.
Key takeaways:
1) Students are not able to “transfer” the literacy strategies learned into deeper, independent comprehension.
2) All teachers, in all subject areas and grade levels need to be aware that teaching literacy explicitly or through content knowledge is essential.
3) Reading results, as measured by standardized tests, have not improved despite the wealth of research on strategies that improve deeper comprehension.