[UPDATE February 2015: Over the past few years, numerous people have commented on my last paragraph as being an overstated and overheated conclusion, unwarranted by the data and of no help in advancing reform. Fair enough: I have come to think that they are correct. So, a new concluding comment is attached, with the old concluding paragraph available for inspection. I agree with my critics: there is no need to pile on teachers in this era of teacher-bashing – and it was not my point. My point was to say: we can improve learning, so let’s do it.]
[UPDATE 11/2014: There have been recent reports suggesting that some of Hattie’s math is flawed. I am not in a position yet to judge the validity of the critique nor, perhaps more importantly, to determine the impact of that critique on the overall findings. The author of the statistics critique in fact suggests that the math used to explain effect size may not impact the conclusions. I will keep abreast of the issue and update this post as warranted.]
I have been a fan of John Hattie’s work ever since I encountered Visible Learning. Hattie has done the most exhaustive meta-analysis in education. Thanks to him, we can gauge not only the relative effectiveness of almost every educational intervention under the sun but we can compare these interventions on an absolute scale of effect size.
Perhaps most importantly, Hattie was able to identify a ‘hinge point’ (as he calls it) from exhaustively comparing everything: the effect size of .40. Anything above such an effect size has more of an impact than just a typical year of academic experience and student growth. And an effect size of 1.0 or better is equivalent to advancing the student’s achievement level by approximately a full grade.
The caveat in any meta-anlysis, of course, is that we have little idea as to the validity of the underlying research. In a summary of all research we are agnostic as to how ‘good’ the research is. (For a good critique of Hattie’s approach in particular and meta-analysis in education in general, read this. [alas, the post was removed and is now only available for a fee. Here is some background from NZ on the controversy and the rationale for the critique. Here is a different critique.
Fans of the book may be unaware that a brand new Hattie book has just been released entitled Visible Learning for Teachers: Maximizing Impact on Learning. In this slim but jam-packed book, Hattie takes us through the planning and teaching process, based on what works according to research. He provides a comprehensive set of checklists that reflect what best practice tells us we should consider in planning and teaching. And in an Appendix he provides a simple way for all teachers to gauge effect size of their teaching. Alas, the text is a bit too dense for the average teacher-reader, I think. But there are countless good pieces of advice, if one persists through the tiny print, lack of white space, and lots of data.(You can also hear and see Hattie discussing his research and its import here.)
As in Visible Learning, the (updated) rank order of those factors that have the greatest effect size in student achievement will be of interest to every teacher, administrator, and education professor.
Here is the rank-ordered list of the top effect sizes, with a half-dozen removed by me because they either refer to programs unknown outside of Australia & New Zealand – Hattie’s home base – or they refer to sub-sets of students (e.g. the learning disabled). And I am going to provide a bit of suspense with this list. I want you to guess which two factors come next after what is listed below; you’ll see why I wanted to add a bit of intrigue by the end. (I have also starred the factors that have an effect size of .7 or greater since these are significant gains):
- Student self-assessment/self-grading*
- Response to intervention*
- Teacher credibility*
- Providing formative assessments*
- Classroom discussion*
- Teacher clarity*
- Feedback*
- Reciprocal teaching*
- Teacher-student relationships fostered*
- Spaced vs. mass practice*
- Meta-cognitive strategies taught and used
- Acceleration
- Classroom behavioral techniques
- Vocabulary programs
- Repeated reading programs
- Creativity programs
- Student prior achievement
- Self-questioning by students
- Study skills
- Problem-solving teaching
- Not labeling students
- Concept mapping
- Cooperative vs individualistic learning
- Direct instruction
- Tactile stimulation programs
- Mastery learning
- Worked examples
- Visual-perception programs
- Peer tutoring
- Cooperative vs competitive learning
- Phonics instruction
- Student-centered teaching
- Classroom cohesion
- Pre-term birth weight
- Peer influences
- Classroom management techniques
- Outdoor-adventure programs
Can you guess the next two items on the rank order list?
“Home environment” and “socio-economic status.”
In other words, everything on the list has a greater effect on student achievement than the student’s background – despite the endless fatalism of so many teachers on this point (especially in the upper grades).
Co-incidentally, Jay Matthews in a recent Washington Post article discusses the remarkable gains in Arlington, VA, in which the achievement gap was greatly narrowed by sustained focused effort by district leaders. And the Gates Foundation released a preliminary report on its Measures of Effective teaching project that shows convincingly what any of us who have worked in schools for years knows: good teachers make a considerable value-added difference.
Revised concluding comment: So, let’s change what is in our control and stop talking as if improvement of teacher and student performance is not possible and all that matters is what happens outside of school. That’s not what the data or common sense about good teaching suggest.
[Original over-the-top concluding comment: It is thus high time that we call teacher fatalism about their ability to achieve gains with poor or unmotivated students what it is: unprofessional, passive, and cynical thinking that has no place in school. It is a form of prejudice that becomes a self-fulfilling prophecy.]
70 Responses
I will be checking out Hattie’s books but to have this list at my finger tips will be a wonderful tool!!
Thanks for the heads up on the book for teachers. I just bought it. I love VISIBLE LEARNING… I end up looking at it frequently as a type of resource manual. Thanks
Lori Cullen
http://attheprincipalsoffice.wordpress.com
Glad to see this. I had been hoping that you would weigh-in on Hattie.
Steve Hunsaker
HI, Grant,
Could you help me understand this further? I thought effect size was a measure of the effect a “technique” had when used with students. I thought it was a measure of the change from before student self-assessment, for example, to after using it. Is that not how it works? If it is, could you explain how “home environment” is measured in that way?
If that’s not how effect size works, would you mind explaining it a bit further?
Thanks.
Good question, Russ. You are correct in the sense that when we do deliberate research or intervention we are looking at the net result of effect size. But Hattie was after something vaster – the effect size of any factor (as he put it “all influences on student achievement” – including factors that were neither deliberate nor under our control. Look at ETS’ own data on the SAT, for example: the effect size of SES of the parents is far greater than the effect size of the students’ GPAs in predicting SAT scores. Let me also paraphrase from Hattie: everything has an effect. Therefore the only thing that matters is the size of the effect. Here is a nice summary of effect size. Does that help?
Hattie’s meta-analysis is fascinating in its sheer breadth. What still concerns me is the “fragmentation effect” it seems to encourage, at least as I’ve witnessed in being used. While noble in intent, by providing a check-list of “strategies that work,” it continues to de-center the more macro notion of instructional design.
“unprofessional, passive, and cynical”…
Wow! You sure are the sharp end of the corporate reformer stick. I bet that’ll inspire a lot of people to do better!
As someone who works in the trenches, I think you represent everything wrong about the corporate reform movement. As for your absolute certainty about home environment and socio-economic status not being important, check out the following:
http://educationnext.org/the-mystery-of-good-teaching/
“We found that the vast majority (about 60 percent) of the differences in student test scores are explained by individual and family background characteristics. All the influences of a school, including school-, teacher-, and class-level variables, both measurable and immeasurable, were found to account for approximately 21 percent of the variation in student achievement.”
I think all teachers want to make a difference, but not in an environment which is punitive and extremist.
You clearly know little about who I am: 15 years of teaching kids, 30 years of working with teachers in schools all over the world including many schools in poor areas, and have no corporate bent; I vote Democratic and my views fit right in line with recent NY Times editorials on reform of teaching – hardly an ‘extreme’ newspaper. Some teachers are ineffective; some are fatalists. That’s a poor combination for kids. Unlike you I believe that kids matter more than the feelings of teachers – I have felt that way since I began teaching 40 years ago. BTW: your research says the same thing Hattie’s says so I miss your point. SES matters but it is not decisive.
A telling comment you make: “all teachers want to make a difference.” Sure they do! But that’s not why they are paid. They are paid to actually make a difference. I’ll bet you haven’t been to a school where poor kids really engage and learn or you wouldn’t say such stuff. I have: visit Ron Clark Academy in Atlanta or Central Park East back in the day. Doug Reeves in his 90/90/90 work. 30 years of research by Ron Edmunds and Larry Lezotte. “Extremist”? Hardly. It’s extremist to give up on kids and get away with it because the adults have the power. Why not do a little observing at outlier schools, then come back with your comments?
PS: today’s column from Nick Kristoff, hardly a corporate extremist:
A landmark new research paper underscores that the difference between a strong teacher and a weak teacher lasts a lifetime. Having a good fourth-grade teacher makes a student 1.25 percent more likely to go to college, the research suggests, and 1.25 percent less likely to get pregnant as a teenager. Each of the students will go on as an adult to earn, on average, $25,000 more over a lifetime — or about $700,000 in gains for an average size class — all attributable to that ace teacher back in the fourth grade.
As I read this I find both of you making good points. I have some questions/thoughts for Mr. Wiggins:
1. To what extent do/should we as an educational community work to train teachers in the above-mentioned effective techniques, and, much more importantly, give them time to develop their craft? My personal experience has taught me that patience and, guidance from administration is a supremely important factor in creating a good teacher. I feel embarrassed when I look back on my first years as a teacher (I would be considered a “bad” teacher), and eternally to my administrators for letting me fail and develop. Not all (I would argue most) teachers are incapable of making a difference until they have developed. Isn’t there a big problem with putting an underdeveloped teacher in a situation of widespread and crushing poverty (for example in the Detroit Public Schools) and then calling them “fatalist” or “unprofessional, passive, and cynical”?
2. I find the claim that home environment and SOS have little influence to be highly dubious. I teach on the other end of the spectrum: at a private Catholic prep school for mostly well-off white students. The parental and community involvement is significant in the development of our students. Specifically, I have had many students who would have given up, stopped trying, or even failed had their parents not made concerted efforts to intervene in their situation.
Of course, this is all anecdotal, but sometimes data-driven claims (as oppose to, say, data-informed) can be as cloudy as anecdotal claims. In the world outside of data, how can any educator claim that real poverty not be a significant factor education? How can we lay this problem at the hands of ill-equipped (see above) and even “good” teachers with no support and frown on their performance dismissive of real-life issues?
No one is saying it has little influence. On the contrary, an effect size of .57 is very high. The point of this post is that a large number of powerful effects have a higher effect size, almost all of which are under the control of teachers and administrators.
Few teachers are thoroughly trained in the techniques at the top of the list, and most training is too hit-or-miss and/or theoretical. Read Doug Lemov’s new book on Perfect Practice to see how we should be training people to ensure that new habits take root.
While I continue to believe there is a better way of educating children than through current learning forms–and find it troubling that there are those that will misinterpret this kind of data to suggest that socio-economic status and initial literacy “don’t matter”–I know that’s not the point of the post, so I’ll get back on track because I’m interested in your thoughts about the findings.
I found some of the nomenclature difficult to understand or distinguish from other terms. I am unsure what “Piagetian programs” are (though I can imagine), nor “Quality Teaching” (.44 ES).
Also, while the “meta” function of the analysis is what makes it powerful, it also makes me wonder–how can Individualized Instruction only demonstrate a .22 ES? There must be “degrees” of individualization, so that saying “Individualized Instruction” is like saying “pizza”: what kind? With 1185 listed effects, the sample size seems large enough that you’d think an honest picture of what I.I. looked like would emerge.
Problem-based Learning a .15 ES? This makes me worry about how many may abuse this data–and that kind of brings me to the most significant issue with information like this: the same institutions (public schools) that are not able to conduct effective research of their own are now expected to re-interpret this incredible data load and apply it without compromising its integrity. I have personally witnessed PDs where, in the middle of a staff meeting, “Hattie” has been tossed down in the middle of every table in the library and teachers are told to “come up with lessons” that use those strategies that appear in the “top 10.” Then, on walk-throughs for the next month, teachers are constantly asked about “reciprocal teaching” (.74 ES after all), while project-based and inquiry-based learning with diverse assessment forms and constant meta-cognitive support is met with silence (as said administrator flips through Hattie’s book to “check the effect size of my strategies).
If you consider the analogy of a restaurant, Hattie’s book is like a big book of cooking practices that have been shown to be effective within certain contexts: Use of Microwave (.11 ES) Chefs Academic Training (.23 ES), Use of Fresh Ingredients (.98). The problem is, without the macro-picture of instructional design, they are simply contextual-less, singular items. If they are used for teachers to consider while planning instruction, that’s great, but that’s not how I’ve typically seen them used–not in schools I’ve worked in anyway. They are items to check, along with learning target, essential question, and evidence of data use–all available in a big binder that needs to be near the door so district and state folks can check for compliance when they buzz through. The same lack of careful attention and analysis that brought a school or community to a place where they need to be shown how to be effective educators makes this much data almost hurtful.
While I leave it up to Hattie and those left-brain folks way smarter than I am to make sense of the numbers, I wonder how the effect of problem-based learning cab be measured independently of other factors (assessment design, teacher feedback, family structure, and so on). Also, it can also be difficult to untangle on strategy (inquiry-based learning) from another (inductive teaching).
“Cooperative Learning” (.41 ES) less effective than “Direct Instruction” (.59 ES)–interesting. While I’ve always believed “lecture” (or what I call accountable talks) get a bad rap, clearly this data doesn’t imply to entirely forgo collaborative learning in favor of direct instruction, but that is a low-hanging mis-interpretation.
Also, Teacher Content Knowledge: .09? Thoughts here?
Totally agree: caveat lector. We don’t know the quality of the individual research, we don’t know how study topics got categorized or labeled as they did, we don’t know what measures were used and how intellectually valid they are, and we don’t know what to do with vast numbers of very different studies on a vaguely-defined general element. That said, the weakness of meta-analysis is no different, really, than the weakness of fine little studies that follow 6 kids in something the author chooses to call “student-led discussion”. It’s not that Hattie’s work is the be-all end-all, any more than Marzano’s or Slavin’s or anyone else’s is. Rather, the trend in the data is clear: some things work better than others, there is a pattern to what works, and many things can overcome the SES of the family.
The next step, I think, is to do further finely-grained research on each of the squishy-sounding topics. A good example is studies of ‘feedback’. Anyone who has read my writing over the years knows that I believe most people use the term improperly. Feedback is DIFFERENT from advice. But hundreds of studies discuss ‘feedback’ as if it were advice. And far too many people think ‘feedback’ has to come from people when video games prove this to be false.
PS, I disagree with you on ‘lecture’. What a good meta-analysis can do it sort out ‘lectures’ from ‘good lectures’ by looking at effect size. So, while some specific approaches to lecturing may have a good effect, ‘lectures’ in general are not very effective. That is the kind of useful finding meta-analysis can give.
Great thoughts – thanks.
My friend (who posted the above rant on the way schools can pervert sound thinking and practice in the name of homogeneity, but did so anonymously for obvious reasons) asked me to clarify for him that he was referring to “accountable talks” rather than a pure college-style lecture, claiming it to be a sound way to pitch an idea, standard, concept, or challenge to students in lieu of its teacher-centered-ness and reliance on a brief degree of sage-on-stage.
Thanks always for your willingness to support understanding, Grant.
Very well said! I think you hit on the inherent problem of our reform-crazy system: anything that shows an effect size and is labeled “scientificly-based research” is embraced by policy-makers and administrators with mindless, surface-level implementation, which results in (as you stated) “compromising its integrity.” I have seen many examples where policy makers attempt to mandate the replication of effective practices on ‘ineffective’ schools or districts. This rarely, if ever works. Most cases of reform that work, work precisely because they are AUTHENTIC – invented, created, tested, and executed by the very educators who make them take flight. I am so tired of policy makers treating educational research as if it were a pill – if that pill worked for you, it must work for everyone. Every child holds with-in them thousands of variables that can play on their learning. Every educational community holds many variables within its culture, background, resources, etc. This makes pure replication elusive. I am not saying that teachers do not make a difference – they most certainly do. And many of them do make huge impacts by implementing effective practices such as Hattie promotes by his research – because they already have an inherent belief in the fundementals of the specific practice. But trying to force this mountain of data on the education system is what creates a mountain of unauthentic application = confusion.
Mountains of data exacerbate our problems – Finland is a great example in this respect. They do not need sheets and sheets of data because their teachers are well-trained, well compensated, and loop the grade levels so that the same teacher works with the same students for a number of years. These teachers then, do not need data sheets to tell them that Johnnie doesn’t have decent decoding skills. They know from years of working with Johnnie (sorry, I’m sure that’s not a very good Finnish name) what his strengths and weaknesses are.
Education policy makers and politicians seem to think that the more data we have the closer to perfection we will approach. . . ??? On the contrary. The more data we have seen, the more programs that are instituted in the name of “data-proven” research the more confusing and muddied the waters become. Leonardi DaVinci said it best: “Simplicity is the ultimate sophistication.”
By the way, I wrote about some of these ideas a few weeks ago, where I basically made the case for “instructional design” over piece-meal grand-and-go planning. http://www.teachthought.com/?p=2108
[…] as an opportunity to emphasize broader notions of instructional design. A few weeks later, Grant Wiggins also wrote about Hattie’s work in the context of accountability and “value-added” accountability. A […]
Still a little puzzled by the low ES ranking of SES. I totally support the corporate reform movement and consider myself part of that and I also fully agree with Grant/Hattie’s central message here. However, in my own parenting experience, I feel strongly about the SES beyond the school/district – role modeling of the parents/siblings, exposure to extracurricular activities, extra tutoring, availability of tools beyond what school can provide, etc. Aren’t those parts of the SES or closed correlated to SES? Would like to get some clarification and feedback from Grant.
No, the SES ranking is HIGH – well above most educational interventions. But it is not the top. There are 30 interventions that have a greater effect – that’s the point. Another way of making both your and my point: in a typical school, where there is really very little best practice used, the SES will trump school effects. Only in a school dedicated to best practice will overcome its effects – and those are the outlier schools that show up in every accountability report in a district.
I think that’s part of the “problem” with the work. It’s important to realize that the point is to let the data speak for itself, rather than interpreting the data as subjective recommendations. That said, the ES is “louder” here than the analysis of what actually constitutes the strategy or factor being measured. Thus some of the head-scratchers.
[…] Grant Wiggins recently blogged about John Hattie’s work on visible learning that triggered some parallels for me. The identification in the research into the meta-analysis of learning and teaching reported some interesting data. I’ve short listed accordingly from this post but you get checkout the full list on Grant’s blog and also in Hattie’s publications. So the top learning and teaching strategies and pedagogical approaches were: […]
I think socio-economic status, background and home environment are fundamentally important to success and should not be discounted. The fact that certain types and subgroups of students perform better on tests when all students receive the same instruction from the teacher means that some students come to school with a certain mindset to learn, while others are unsure about the value of education in their lives.
Not discounting them. Asking everyone to consider the fact that 32 interventions have a greater effect. We know they matter – but really effective teachers and schools overcome it, as the outlier schools and teachers all indicate in the TN value-added studies, teachers like Jaime Escalante, and the 90/90/90 schools that Reeves highlighted. Look at all the top schools in NYCF in bad neighborhoods – they are outliers that show what is possible by really excellent staff. My take on the SES data – including SAT data – is that all the SES data shows is that in general schools are not very effective. But all the outlier schools show what is possible when a truly team group of pros don’t take no for an answer. I continue to be puzzled why so few teachers think that their influence should be so minimal. I have seen dozens of great teachers in my career, in some of the worst schools in America, all making a difference that gets reflected in grad. rates and test scores. Why are so many educators unwilling to except the fact that SOME teachers trump SES while most do not?
You say “really effective schools” overcome poverty, but you only name one school that is still in existence (Ron Clark). That school is extraordinarily well funded, it is highly selective of its students, and it is a private school. It seems like a wonderful school, but it is hardly proof that poverty can be overcome on a schoolwde level. I don’t doubt that poverty COULD be overcome, but it is pretty amazing that I’ve never yet found a school that actually did it. The other group of schools you mention (Reeves’s 90/90/90 schools; I don’t know what the “nycf” schools are) is also pretty unconvincing. I had a lot of trouble finding names of actual 90/90/90 schools, and so have other people (http://blogs.edweek.org/edweek/on_performance/2011/05/909090_schools_revisited.html). If you know of these miracle schools, we would all love to hear about them.
Hattie’s work is interesting, and I believe that good teaching and good schools are very important for poor kids, just as they are for rich kids, but you have not made a very convincing case here. Could you please list ten schools that neither cherry pick their students nor spend way more time or money than other schools and still educate mostly very poor kids to a very high level? I’d love to learn from these schools, but I am afraid they don’t exist.
Doug Reeves’ 90-90-90 work is being implemented through School Improvement services at the Leadership and Learning Center. Please check it out.http://www.leadandlearn.com/services/school-improvement
Those amazing outlier schools we speak of here need not be outliers – they could and should be the norm!
I thought it would be a good exercise to Google on outlier schools, in light of Lorne’s challenge. Here are some examples:
http://www.startribune.com/newsgraphics/129810153.html
http://www.minnpost.com/learning-curve/2011/09/what-are-successful-outlier-schools-doing-close-achievement-gap
http://www.chicagomag.com/Chicago-Magazine/The-312/September-2012/Poverty-and-Graduation-Rates-in-Chicago-and-the-US/
http://www.minnpost.com/community-voices/2011/06/what-did-cincinnati-public-schools-do-close-high-school-graduation-gap
http://www.houstonisd.org/HISDConnectEnglish/Images/PDF/StudentAchievementReport.pdf
I challenge other readers to report on outlier schools they know of.
To repeat: I am NOT saying SES doesn’t matter. It matters more than most things. I am saying that there are outlier schools and teachers and it is from them we must learn, along with implementing Hattie’s data.
I think readers need not just to google the success stories, but actually look into them and see if they are true. For example, a counterpoint to the Cincinnati story you cited: http://www.citybeat.com/cincinnati/article-24967-miracle_or_mirage.html
More students are graduating, yes, but they are getting 15’s on the ACT.
Jumoke Academy in Hartford, CT has a high percentage of low income students and their standardized test scores are similar to the state as a whole. There are none of the usual signs of test score manipulation and they have a very long waiting list to get in (lottery admission).
The Mathews column on the Arlington District is extremely misleading. Over the last 10 years or so, the gap in Arlington narrowed by roughly the same amount as it did for the entire state. The main reason for the huge reduction in the gap is that Virginia made it much easier to pass the state tests. In fact, the gap in the passing percentages for the economically disadvantaged is larger in Arlington than it is for the state as a whole. It is of course possible that Arlington is doing a lot of great things in their schools that narrow the achievement gap that are not reflected in the standardized test results.
Data is here: https://p1pe.doe.virginia.gov/reportcard/
What does student achievement even look like? What does that mean? What does learning look like? Is it the same for all kids everywhere? In every classroom? Teaching is essentially a human process and cannot be bottled up and sold or quantified or listed in order of importance.
Give every kid a 35 thousand dollar a year education with programs and curriculum that top private schools like Exeter, Friends, and Waldorf school use and don’t worry about test scores and silly measurements.
Ok, I’m going to break it down for you because it really is simple:
1) socio economic status matters. To say it is on the bottom of the totem pole shows how little you understand this segment of society. This is the same line Rhee and Duncan try to use and it is just ridiculous to hear. Really laughable.
2) just because socio-economic status and family background matters does not mean we should focus our energies on that. Rather, we create great schools like The Met in Providence or like private schools (the Met is a public school but what does it have in common with private schools? Small class sizes and collaborative learning!) and we take the focus off of testing.
3) we give meaningful tests to assess where our students are at but use it as only one small measurement of success of a school.
SES does matter to the extent that it is just a proxy for IQ. It’s in the genes, all of the money in the world won’t change it. We are all different, externally and internally.
So all poor people are stupid? Come on! Shame on you! Some of the greatest minds in our country came out of poverty and low SES. It used to be that poor kids did often have lower IQ due to lead paint issues, lack of stim early on, etc. SES is more about a culture of poverty. This can have a lot of negative effects on education. There are many, many things that can help reduce the effects of low SES. We just choose to make them worse via high stakes testing. Any child at any economic, cultural, genetic, etc level can be taught and learn. How much they can learn? We don’t know. We do know that if we don’t try, they won’t learn. And if they come to school with one hand tied behind their back, they won’t be able to learn as much. And that is what it is like to be from a low SES family.
Example? I’ve seen many kids who come from homes that do not have books. They just don’t have a need for them. Their everyday struggles and experiences do not require books. Having kids be given books when they are born could expose them to words and help them be ready for school experiences. Babies just play with books. Parents would be encourage to read to them. They could be encouraged to allow annual book gifts as birthday presents from their doctors. This could be one step in helping them to prepare for school.
Very interesting article, thank you for sharing this information. This whole subject is new to me, as my question will most likely reveal.
If a school were to enact reform based on choosing interventions with the highest effect sizes, how could one project the combined impact these would have on student performance? For example, if all students were exposed to the same 3 interventions and one had an effect size of .5, another .3, and another .2 would the student population experience the same impact of an itervention with a effect size of 1.0, or would only the largest effect size be noticed? For example, let’s say a school were to set a new focus on formative assessment, combined with differentiated instruction and increased feedback. How would you project the impact of this: using just formative assessment which probably has the highest effect size of the three, or with the accumulated effect sizes of all three strategies?
I’ve been reading a little more on this and it sounds like I would use a weighted average to determined a general prediction for the combined effect. But how do you determine which weights to assign to each intervention? Any thoughts would be greatly appeciated.
There are greater differences among the skill set of teachers at any given school than there are between schools. This shows you how much teachers matter.
For the teachers who want to say that SES is more important than quality teaching, my question is simple: “How many public schools with 100% of their kids from poverty, that are high-performing, sending over 90% of their kids to college, do you need see or hear about before you admit the truth? I’m serious about this question. If there was only one school in the country, you could dismiss these schools as aberrations. But there are many of them. When any teacher tells me, “It can’t be done,” they are talking more about their own limitations than those of kids.
Strong teaching trumps low SES every time. Average teachers simply don’t have the skill sets to teach better classroom behaviors, induce stronger effort, upgrade student attitudes or build cognitive capacity. If you go years of not being able to perform a task, you’re pretty likely to think it can’t be done.
I’m not saying it’s easy. But it’s doable. If you tell me it’s hard to do it; I’d agree. But don’t post inaccurate information on Grant’s website such as, “SES matters more than teachers.”
All Grant’s saying is, weak teaching will lose out and strong teaching trumps low SES every time.
Indeed, that is all I am saying. I think it is a sad ‘profession’ when many of its practitioners actually think that even the best among us cannot achieve success. No one said it is easy; no one said that ANY teacher ought to be able to succeed in schools of poverty. But the obvious fact that many teachers do succeed – side by side teachers in the same school who do not succeed – should cause all the naysayers to reflect carefully. Not only did Jaime Escalante succeed in AP Calculus, but after he left there were more passing kids at Garfield in AP Govt than there had been in math!
We were just in a turnaround middle school in a large urban system near Washington DC. We saw a few excellent teachers: total control of the kids, engaging work, fast-paced and lively culture. On the other hand we saw some horrific teaching – and yelling at kids. That to me is the tipoff: when there is quality control in any system, be it sports, manufacturing, glee club or ELA class you see minimal differences across teachers and outcomes. In a middle school it means you see few disciplinary problems and lots of good teaching across teachers. When there is NO quality control, you see vast ranges of quality. That tells me most of what I need to know when visiting schools – and about the fatalism of those who think SES trumps all.
The missing key is to attract great teachers. How can we do that with the current carrot and stick approach with little carrot and a big stick? The ceiling on salary is also not very attractive to smart students choosing college degrees. We just do not have enough teachers who choose teaching out of the goodness of their heart. We must provide more to get more. People argue teachers get paid enough- entry level pay is adequate for certain. It’s the low ceiling, the lack of adequate support in the first few years, the struggle for respect, the lack of being referenced as an expert in ones field, etc… the list goes on and on.
And while there are no easy and quick solutions to low SES problems, we must start coming up with some. These are children living in poverty. Many are homeless. Making it easier for homeless children to have regular meals, attend the same school (no matter what shelter they stay at or eventual place they live), have clean clothes and a way to get a decent night sleep each night would help tremendously. Why isn’t this a huge priority? I cannot ever say “we’ll never end poverty so why argue that”- we may never end it completely but we can do a lot better and need to.
I always enjoy Googling the schools in some of the “leading education reformist” books and finding out they are selective admission, lottery, etc. OBVIOUSLY you’ll get better achievement. If you don’t, time to find a new profession. I also enjoyed the national conference I attended and had a “leading reformist” talk about a school, with his efforts, that improved performance. I used my smartphone to Google the school and, through various links, found out the state had renormed the test and all schools had seen increased performance and that the school he discussed had recently turned into one of those selective schools with smaller class sizes, lottery system, etc. So, just to be a jerk, I asked him, “were there any other happenings that may have influenced the research results or strictly, from what you can tell, just those events you’re describing.” He went on about the research and stated that nothing else meaningful, in his recollection, had occurred during the time of the research. It was then I pulled the ultimate trump card and asked if the websites I had on my phone were false. His response, noticeably angry, nervous, etc was, “well, I would have to see what you’re discussing but I am not aware of what you’re talking about.” Strangely, I wasn’t escorted from the conference. It’s these events that always make me question things. I believe an education SYSTEM can overcome poverty to an extent. I also believe poverty, given a desire to learn, has little consequence on education. I chuckle each time I see “student disposition to learn” with a smaller effect size than “feedback.” Given a student with little disposition to learn I’d love to see feedback actually helping them. In order for feedback to work, a teacher needs to GET PAST the STUDENT’S disposition NOT to learn. That’s the biggest key to student success in my view. If kids think you’re a tool, ain’t nothing gonna work!
“student disposition to learn” is evaluated separately in http://www.educationalleaders.govt.nz/Pedagogy-and-assessment/Building-effective-learning-environments/Teachers-Make-a-Difference-What-is-the-Research-Evidence
Here, it is clear that the student accounts for 50% of outcomes, and teachers can only influence 30%. More particularly, even though teachers might like to think they can influence the student’s 50%, Hattie points out that they cannot.
What does this all mean for Inquiry based Education? Are Inquiry, Project based learning, Problem based learning dead?
I was initially puzzled by this, too. But you need to read the whole book to see how he defines these terms. By problem-based learning he means it as a method of pedagogy used where the goal is still to develop content and skill knowledge. In other words, PBL is not ideal for learning basic skills – that’s what various studies show. But that doesn’t mean it has no value: its value is in developing problem-solving abilities and transfer of learning, things not typically tested, alas. Nor is he equating problem-based projects or inquiries with Problem-Based Learning (PBL) as used in med school and business school.
You suggested reading all of Visible Learning for Teachers 2012 by John Hattie to find the definitions. I have done this but I do not see definitions for enquiry-based (inquiry-based) teaching, problem-based learning, problem-solving teaching and play programs. The definitions are important to understanding his work. Can you help me out here? Are they listed in his book Visible Learning, 2009?
Grant, you state: “PBL is not ideal for learning basic skills – that’s what various studies show”. This makes some sense to me but can you provide me with the names of a couple of studies that you are referring to.
I continue to follow your work, that of Ben Levin and Michael Fullan, amongst others. As a Literacy Consultant and author working with teachers and school districts around the world I see firsthand the difference your collective work is making. . Thank you!
Reblogged this on Kathleen McNamara, Tuxedo Park School and commented:
I just discovered this posting as we are working collaboratively on our paper at Columbia. Very interesting and succinct! Thanks, Tom Batty!
[…] that could probably be implemented which would achieve greater results. Grant Wiggins has a nice post about Hattie’s research on his […]
[…] I have been a fan of John Hattie’s work ever since I encountered Visible Learning. Hattie has done the most exhaustive meta-analysis in education. Thanks to him, we can gauge not only the relative … […]
[…] of these sessions is when I leave knowing something I didn’t know before. I first heard about Hattie’s effect sizes in an after school training session – in that case someone stood at the front and talked […]
[…] mind and proceed with activities only when those goals and criteria are clear.) Last year, Wiggins commented on his blog in what is now a noteworthy (at least in educator circles) post about a new book by Australian researcher John Hattie. (Hattie performed a meta-analysis, examining […]
Your bias is showing. “Student background” is not the same as “home environment” which should be clearly defined. Curiously, you ignore almost all other student variables. Cognitive ability (indicated by, for example, IQ) has the greatest effect on measures student learning performance, much higher than any instructional variable (such as those on your list). We can’t test “home environment” or background or SES with an experiment like we can most of the instructional variables your list, so we are less sure of causal effects and effect sizes of the former.
Let’s also be clear that we don’t have enough data for reliable effect sizes for most of the variables you list. Factors of study design can be more responsible for measured effect sizes than the independent variables investigated. If few studies are done on a specific variable, we really don’t have a valid and reliable effect size for that variable. Your imply that we have rock solid meta analyses for all the variables you list. We don’t, so we are not sure of the actual effect sizes. And, we can’t be confident unless we have a lot of high-quality studies in the meta-analysis (and we don’t even for variables that have been studied extensively).
Overall, you are simply demonstrating confirmation bias in finding support for your ideology.
[…] John Hattie http://growthmindseteaz.org/johnhattie.html What Works in Education – Grant Wiggins http://grantwiggins.wordpress.com/2012/01/07/what-works-in-education-hatties-list-of-the-greatest-ef… Part 1 and 2 – *Please watch John Hattie’s Video Presentation (Visible Learning for Teachers) […]
I know that everytime I want to place the failure of a lesson, test, activity on the child’s attitude, I cannot. I have to look further – what else could I have done, said, presented, etc. The other thing I believe is defeatist is when an administrator ( translate: principal) tells me that I am dreaming if I believe kids will want to come into class looking forward to and/or excited about math!!! I refuse to give up!
You may be interested in this article about assessment and the environmental conditions for making the most of feedback (Hattie’s #1). It covers a concept I call Learning Intelligence, or LQ and suggests a way of moving the focus of assessment feedback from grades or scores to a focus on what has or has not been learnt. The link is: http://wp.me/2LphS
For more on LQ go back to August 11th, there are now 14 articles in total covering LQ and the link to:
• learning teams
• initiative
• resilience
• creativity
• empathy
• design (problem solving)
• developing LQ
• the learner
• the environment and
• the LQ approach
Kev
I love your last thought on this topic about shadowing our students. It seems that the major push is engagement, however, it is not “true engagement”. Teachers look good to administrators because their students are compliant rather than engaged. The pressures of teacher observations and high-stakes testing only seem to make this situation worse. When teachers step outside of the “norm” and truly engage students and challenge their thinking, those teachers are often looked at differently (not always positively). Like Dylan Thomas wrote — “we should not go gentle into the good night” but “rage, rage against”…we need to do what’s right for our kids. Thank you for being an advocate for our children.
[…] of the past decade or so is what we might term “Hattie’s Law”, after researcher John Hattie. Most educational interventions have some effect. Doing something is usually better than doing […]
I do not know why educators drool over news that parents do not matter. Parents clearly do matter. Hattie’s methodology just did not measure socio-economic status and home environment affect on children before they enter school. Here is a quote from Hattie in his own book! How could he be misinterpreted by so many? “It is not a book about what cannot be influenced in schools — thus critical discussions about class, poverty, resources in families, health in families, and nutrition are not included — but this is NOT because they are unimportant, indeed they may be more important than many of the influences discussed in this book. It is just that I have not included these topics in my orbit.” To anyone with common sense, home environment does matter. [p ix]
So when my eval this year incorporates student test scores who have currently missed more than 25% of the school year (of which I can’t count the total students on my fingers and toes combined) I’ll be sure to remind the principal student attendance wasn’t part of this list and I should be fired. Also, who do I file a lawsuit against when I’m rated in this way? Seems like a great civil case.
I totally agree.
I agree totally. I’m not trying to make excuses at all. I am simply stating fact. I think about myself as a child and when I didn’t sleep well or came to school in a bad mood, it did have an effect on my learning. Think about adults today in every workforce. If there is substantial issues going on in a person’s life, it does affect productivity in any field.
[…] Attention to what works for effective learning is the necessary glue. Grant Wiggins provides a summary of John Hattie’s research. […]
[…] research and studies that have been done over the years). Here is the “Top 20″ list, as Grant Wiggins broke it down to take out a few factors that were not relevant to most teachers. I’ve bolded the items that […]
[…] what actually affects student achievement. Here’s a good summary of the book by Grant Wiggins-http://grantwiggins.wordpress.com/2012/01/07/what-works-in-education-hatties-list-of-the-greatest-ef…. Visible Learning is a decent place to start for forming a definition for what “good […]
[…] What works in education? (This supports my conversations on how we cannot claim victim with all that we are asked/required to do. There are many things that we CAN control in education.) https://grantwiggins.wordpress.com/2012/01/07/what-works-in-education-hatties-list-of-the-greatest-e… […]
[…] deeply researched book, Visible Learning For Teachers: Maximizing Impact on Learning, the number one factor that improves students learning is self-assessment. When students take the time to assess the work they are doing and to think about the process of […]
[…] Grant- https://grantwiggins.wordpress.com/2012/01/07/what-works-in-education-hatties-list-of-the-greatest-e… stating “Hattie has done the most exhaustive meta-analysis in education. Thanks to him, we can […]
[…] G. Wiggins (2012, Jan 7). What works in education: Hattie’s list of the greatest effects and why it matters. Retrieved from http://grantwiggins.wordpress.com/2012/01/07/what-works-in-education-hatties-list-of-the-greatest-ef… […]
[…] I was listening to a presentation by Cris Tovani, who shared a blog post by Grant Wiggins entitled “What works in education—Hattie’s list of greatest effects and why it matters” which is an anlysis of John Hattie’s book Visible Learning–an exhaustive research of […]
[…] What works in education – Hattie’s list of the greatest effects and why it matters – Blog post by Grant Wiggins […]
While the student’s “home environment” and socio-economic status do not impact all that much on student outcomes, in one of Hattie’s other works, called “Teachers Make a Difference: What is the research evidence?” it is clear that students are 50% responsible for outcomes, and teachers only 30%. Hattie warns against “double dipping”, that is, assuming that teachers can have an indirect effect on the 50% component. http://www.educationalleaders.govt.nz/Pedagogy-and-assessment/Building-effective-learning-environments/Teachers-Make-a-Difference-What-is-the-Research-Evidence
[…] “What Works in Education – Hattie’s List of the Greatest Effects and Why It Matters.”… Granted and. 2012. Web. 17 May 2016. […]
[…] is why John Hattie titled his important work Visible Learning because the best academic achievement results from explicit and […]
[…] anything higher is “not bad,” and anything lower is “not good.” More precisely, Grant Wiggins aggregated all of the strategies that resulted in a .7 or better–what is considered […]
[…] Grant Wiggins wrote on article back in 2012 on Hattie’s work, and there are a few editorial co… […]