This episode of the Tips for Teachers podcast is proudly supported by Arc Maths
You can download an mp3 of the podcast here.
Dylan Wiliam’s tips:
- Make feedback into detective work (3:27)
- Make detention work fit the crime (9:40)
- Make question planning part of lesson planning (13:40)
- We have little insight into our learning (20:02)
- Don’t let “Don’t know” be the end of the conversation (28:19)
Links and resources
- Dylan Wiliam on Twitter
- Dylan’s website
- Adam Boxer’s book: Teaching secondary science
Subscribe to the podcast
- Subscribe on Apple Podcasts
- Subscribe on Spotify
- Subscribe on Google Podcasts
- Subscribe on Stitcher
Watch the videos of Dylan’s tips
Podcast transcript
Craig Barton 0:00
Hello, my name is Craig Barton and welcome to the tips for teachers podcast. The show that helps you supercharge your teaching one idea at a time. Each episode I invite our guests from the wonderful world of education to share five tips for teachers to try both inside or maybe even outside of the classroom. With each tip, the challenge is always to ask yourself, what would I have to do or change to make this work for me, my situation and my students, experimentation and frustration may follow, but hopefully something good will come out of it. Now remember to check out our website tips for teachers.co.uk, where you’ll find all the podcasts as well as the links resources and audio transcriptions from each episode. But better than that, you’ll also find a selection of video tips, some taken directly from the podcast and others recorded by me. These clips can be used to spark discussion between colleagues at departmental meeting at twilight inside and so on. Now, just before we dive into today’s episode, a quick word of thanks for our lovely sponsor, because this episode of the tips for teachers podcast is proudly supported by arc maths. Math is an innovative app created by teachers to help students remember all those crucial skills needed to succeed at maths. Art. Math is built around research into the power of retrieval practice and space practice on memory. And here’s how it works. Students Krakow Kerbal averse students crack open the art class app and are given a 12 question quiz with follow up practice questions on anything they got wrong, not just straight away, but the next day three days later, a week later, and so on until they have it secure in long term memory. The more time they spend on the app, the better Ark will get to know your students and with no teacher input required, you can spend more of your time inspiring your students with new ideas. So do check out art maths and remember that’s arc with a C knots okay. Okay, back to the show. And let’s get learning with today’s guest the wonderful Dylan William spoiler alert. Here are Dylan’s five tips. Tip one, make feedback into detective work. Tip two, make detention work fit the crime. Tip Three, make it question planning part of lesson planning. Tip four word of caution here we have little insight into our learning. And Tip Five, don’t let don’t know be the end of the conversation. I’ll tell you why. This is an absolute classic. Now if you look at the episode description on your podcast player or visit the episode page on tips for teachers dot code at UK you’ll see I’ve timestamps each of the tips so you can jump straight to anyone that you want to listen to first, or when you revisit this you can dive straight into any tip you want to read listen to enjoy the show.
Well, it gives me great pleasure to welcome Mr. Dylan William to the tips for teachers podcast. Hello, Dylan. How are you? Very good. Thanks. How are you? Great. Very good. Thank you. I’m Dylan, please, could you just tell us a little bit about yourself ideally in a sentence.
Dylan Wiliam 3:09
A former math and science teacher who then went on to train teachers. It’s a research, did some things in university administration, and then oddly left university life to concentrate on teaching and research.
Craig Barton 3:23
Fantastic, brilliant stuff. Right down. And let’s dive straight in. What’s your first tip for us today?
Dylan Wiliam 3:29
My first tip relates to feedback. And there’s a lot of stuff that teachers are told about feedback that really isn’t supported by the research. And there’s a really important review by two American researchers Kluger and NEC. And everybody cites this research. But hardly anybody reads what these two guys said. On the last page of their paper, which is a very, very dense paper. They did a meta analysis of the effects of feedback found the average effect size was point four. But then they said, we don’t think that’s important. We don’t think that’s as important because the effect size or feedback is irrelevant. If you get a high effect size by making the students more dependent on the feedback, what they pointed out was that a feedback intervention that makes students need more feedback in the future is actually not very helpful. And so what they suggested was, rather than looking at the effect size of feedback, we should be looking at what the students do with the feedback. They pointed out, they can change behaviour, change the goal, abandon the goal or reject the feedback. And so this was the first paper. This is 25 years ago now, that really said we should worry less about the kind of feedback and more about what the students do with it. And so I say to teachers, good feedback is feedback that students use. And so the bug isn’t getting getting used by the students. It’s completely irrelevant. And yet still people’s problems figure out what the effect size of feedback is, when it’s a kind of silly question. So recently, a lot of People have started looking at what some researchers call recipients processes for putting in the effort not into getting the feedback perfect, but getting the relationship between the students and the teachers, right to the students act on the feedback. And so I’ve been working on this for about 10 years now trying to think about how we can make feedback more accessible to students. And so, you know, I do think there’s a case for teachers writing comments on students work, I don’t think it should be the primary form of feedback. So I’ve advocated what I call four quarters marking 25%, detailed feedback, 25%, whole class marking 25% peer assessment, 25% self assessment, but I want to focus on that event where the teachers do write comments on students work. And we had this crazy thing in England, which is called triple marking. Teachers wrote on the kids work, then the kids responded to it, and the teacher didn’t check that the students had responded to it. Here’s my point. I think that if you’re going to take time to write individual comments on Stephens work, which is basically one to one tuition, I don’t know a single teacher of a mark two books at the same time. It’s the most expensive form of education we have. So let’s make sure something happens. So I will say to teachers, if it’s worth your while, taking time to write comments on students work, it’s worthwhile taking class time for the students to respond. So I think we should just completely change the way we think about feedback. And if you’re getting feedback, the next 10 minutes, when those students are in the classroom with you, they’ll be responding to the feedback that you give them. But to make it even more effective, I think we have to kind of make that feedback, something that invites a response. So this was triggered by looking at the work of an English teacher named Sharla. Kerrigan, who rather watching comments on her students work. Your 10 class, during an essay on Shakespeare play, they’d read the comments on strips of paper. And each group of four students got back their four essays, and the four strips of paper. And their task was match the comments to the essays. maths teacher said to us, this is all very well for history in English, we can’t do comment on the market in maths, if you take 15 of these equations as correct and put across next to five others, the students could figure out for themselves, they’ve got 15 out of 20, or 75%. So we suggested Well, why not just tell them five of these roles, you find them, you fix them. And so here’s the big idea. Rather than thinking about feedback as information, think about feedback, as detective work, the idea is that the feedback should cause a puzzle or a challenge for the students to engage in. So rather than saying, Remember to use the correct grammatical gender there, D das in Germany, which I call feedback as nagging about, there are five places in this piece of writing where you’ve used the incorrect grammatical gender for dirty does, I’ve highlighted two of them, see if you can find the other three. And so by actually creating kind of invitation to respond to the feedback, I think it makes it far more likely that students respond to the feedback in a positive way. So Tip one is make feedback into detective work.
Craig Barton 8:20
I love it, Dale. And just just one thing on that it was a low point in my teaching career was those Sunday afternoons or went into Sunday evenings, marking through a pile of 30 books, the comments get worse and worse as I go through like child 26. They’re lucky if they get in anything in that book. So I’ve exhausted all my all my efforts and enthusiasm. And then you give it back to the kids. And then you’d have to hand out the purple pen and then have to do that, then you’d respond to the purple with a green pen. And the irony I always found was that the more detailed I made the comments, the less impact it had, because it was so supportive for the kids, they didn’t have to do any thinking they were like, alright, I can see it all there and so on. And whenever I first heard you mentioned this, this feedback as detective work, I loved it for a couple of reasons. I loved it primarily because it made the kids think more and it was much more active part of the of the process. But also it’s less work for the teacher as well in a good way, because I can just do as a math teacher, tick, tick, tick, cross cross. And of course, I’m making a note of general trends in the class, which are the problematic questions, any common misconceptions, but I don’t have to write all these flippin big long comments, give it back to the kids. And they’d like the honest gamification of trying to find where those wrong ones are. It is brilliant. I absolutely love it. It’s great. It’s great. Right down, there’s a danger. You’ve peaked too soon. That’s that’s a brilliant. Well, what’s your what’s your second tip you’ve got for us? Well, I
Dylan Wiliam 9:43
was trying to come up with a series of tips at different levels of specificity and different aspects of teachers work. So the second one concerns detention. Now there’s a big debate about whether detentions are useful or not. But these days, they’re often the only sanction that a teacher has. And so The question for me is what should kids do in detention? So obviously, there’s there is the challenge of getting students to turn up. And that depends on the school policy. So some schools actually have a very clear policy that a teacher issues, student detention. And the student doesn’t turn up, that automatically gets delegated elevated up to a deputy head. And they’re there in the deputy heads detention. So students know that if they don’t set up a detention, it’s gonna get more serious. I think that’s really important. The school culture is important. But I think the really important point for me is what do students do due attention? I think there’s two things. The first is, you must make it absolutely clear that this is not any bother at all for you. So whenever I had kids intention, I always had a stack of marking to do. And so it was a kind of like, I’m gonna be here marking his work, and is not inconveniencing me at all. The other thing you have to watch out is that English teachers don’t like students being made to write lines because it destroys handwriting, they also don’t like them to do be asked to do some kind of writing about what they did wrong, because become broad writing becomes a punishment. And so I think if students have actually not done the work, and the detention is there to help them catch up, then I think it’s appropriate for students to do the work they missed. But I don’t think they should be doing maths questions, because it makes them work, the punishment, I think the big message here is why schoolwork is not a punishment. schoolwork is a privilege that people in other countries who would love to have the opportunity to learn and advance their skills. So I think we should be very careful about the messages we send to different detention. And my personal preference is for students to do absolutely nothing in detention. It doesn’t sit there. And you know, it’s really, really boring, anything you give to do, makes it slightly less boring. And therefore, for me, you know, make the punishment fit the crime to quote Gilbert and Sullivan, if it’s lack of work, then it’s appropriate to the work. But for the rest of the time, I think, just two things, make sure that you they understand that you’d be there anyway, during the marking, and don’t give them work that makes it harder for other teachers make it actually as boring as possible.
Craig Barton 12:25
That’s really interesting. Well, I’ll tell you now I fall into two traps there, I’ve done the opposite of what you’ve said for many years. So first is I dread, when I’m on detention duty, and I make it very clear, when I walk in, I’m fed up, I don’t want to be there. I think I’ll even say that sometimes I don’t want to be a you don’t want to be. So that’s that’s an error straightaway. So I’ve made a note about that one. But the second thing is, I owe it all always turned into almost like an intervention class. Because if the kids are doing maths, and they know I’m a maths teacher, I barely get to sit down and it’s hands on, can you help with this? And then the classic thing is, of course, if they say, if you don’t help them, well, they say, Well, I’m not doing anything, I’m stuck, I can’t help and then the behaviour starts to unravel, and so on. So I’m gonna find it harder think, for them to do nothing. But I like the logic of it. It’s I’m going to find it hard to enact, but I’m on board with a logic I like that.
Dylan Wiliam 13:14
It’s only appropriate if they’re punishing being punished for bad behaviour. If therefore, they’re not doing their work or not doing their homework or whatever, then it’s entirely appropriate for them to make that up. In homework.
Craig Barton 13:27
Make sense? Okay, Dylan, tip number three, please.
Dylan Wiliam 13:33
So in your conversation with Adam boxer, which I thought was absolutely brilliant. He talked a lot about me whiteboards. And so he actually exemplified one aspect of the fundamental principle of classroom formative assessment, better evidence leads to better decisions leads to better learning. Now, of course, as David data has pointed out, we can’t be sure that what they’ve been doing in the lesson results in long term learning. But you can be sure that if they have got the wrong end of the stick in the lesson, you find out about it before the lesson ends. And Adam talked very eloquently about many whiteboards as a way of planning out what is what is going on. And he was rightly criticised critical of cold call. cold call is better than asking the usual suspects. You know, if the smartest kid in the room gives you the correct answer, it doesn’t mean that anybody else has got it. And the staggering thing for me is how prevalent that practice is of relying on evidence from the smartest kid in the room to make a decision about the learning needs of the other 39 kids in the class in the case of Adams class. And so, you know, cold call is better because you’re hearing from somebody who isn’t necessarily the smartest kid in the room. But it’s still weak. You’re only hearing from one student, so many whiteboards. I actually prefer multiple choice questions. We’ve talked about this before. because it makes the job simpler reading 40 minute whiteboards is very difficult scanning 40 ABCD cards or even students, one for a two for B three for sleeve for 35. Three, that makes it much simpler. So multiple choice questions have the benefit of pre processing, that that data analysis tells me for the teacher. So Adam talks eloquently about the breadth of the evidence. I think to go with that we have to talk about the depth of the evidence. So is this a question that is worth asking? And what we’ve discovered in our work with teachers is that good questions are really hard to come up with. So the big idea here is you should plan the question as part of your lesson plan. If I was a head teacher, and we had a policy of looking at teachers lesson plans, which I’m not sure pretty sensible idea, but I’d be very happy with the teacher saying it here’s the objectives of this lesson. And here’s the questions I’m going to use to find out, there’s no point in making teachers lay out the script they’re going to use because they’re going to, they’re going to do what they’ve been doing for 20 years. But I think the question that they asked to find out if the students have been successful, is crucial. And many teachers ask questions where the student can get the right answer with the wrong thinking. So the big idea here is make the questions that you’re going to be asking to check on understanding part of a lesson planning process. The idea is you build that into the lesson, you always build Plan B into Plan A. I’m gonna get to this point in the lesson, I’m going to check to see whether the students are with me. And here’s the question I’m going to ask at that point. I scripted word for word. One of the teachers in the Como FIP project in from Medway Dave toughen. He often if he had a sixth form class in the morning, he would post up with a question that he’d plan to use with his urate class in the afternoon, say, and the class would often discuss whether that was a good question to be using. So making these questions kind of things that we discuss them, maybe changing the word here or there can make it work slightly more effectively, that idea of refining and polishing your questions, and including was part of the lesson plan. That’s my third tip, make it part of lesson planning?
Craig Barton 17:12
I like it just to two things on that one. I think we both agree. It’s hard, isn’t it, as you’ve said that it’s writing these questions is tough. And for many years, as a teacher, that was the last thing I thought I’d be my plan, my plan was all about the activity all about the Bright Shiny Paper wrap round it and so on that one a very rarely check from the stem. And so let’s just put that on the table. But then when I when I did learn that checking for understand was a good thing. I almost thought, well, I’ll just make the question up on the spot, it will be fine. And again, having having written however many 1000 diagnostic questions, you realise how hard it is to write a good question and how planning those in advance and working with perhaps more experienced colleagues and making a collaborative process is a really important part of this. So that’s the first thing I wanted to reflect on. Secondly, I was doing some work just yesterday, actually in a school, and they were looking to improve their scheme of work. And the classic thing they had on this scheme of work, and you see this on every scheme of work, is it so I’d like you know, that too, we in these two weeks, we’re teaching percentages, and these are the objectives. So they’ve got kids have got to be able to find a percentage of an amount without a calculator, percentage of that amount with a calculator, reverse percentages, blah, blah, blah. And the point I was making with the head of department is it’d be so much more powerful if you have examples of the type of questions you wanted kids to be able to answer. And they could be used as those hinge point questions so that every teacher, at some point will ask that question. And it may be at different points if kids are working faster, and so on. But there’s your well thought through hinge point question. And if the kids do well at it, okay, we crack on if they don’t, you’ve got this plan B as you speak about, but I very rarely see, I don’t know, if you do in schemes of work in schemes of learning, kind of examples of questions. And for me, it feels like one of the most important things to put in there. Absolutely. What
Dylan Wiliam 18:52
would it mean to be successful? Yeah. And the thing is, this is a bit of a jargon phrase, but one of my favourite phrases is assessments, operationalize constructs. So we can talk about what it means to be able to add fractions. But the question is, which fractions do you do mean? And so the assessments put flesh on the bones of those ideas. And we might think we agree about what it means to be able to rank fractures in order of magnitude. But until we talk about the questions we’re going to use to determine that. We don’t know that. And in fact, often we find out that we have very different ideas of what it would be to do that in practice. So assessments forced you to get off the fence and say, exactly, you know, if my teaching has been successful, then my student will be able to answer this question correctly. That’s why assessments are so powerful.
Craig Barton 19:43
Yeah, I agree. And that definitely the last thing on this is, it helps the kids as well, right, because you have these lesson objects and they can’t teachers can’t interpret them. The kids certainly don’t have a clue what’s going on with them. Whereas there’s something real concrete about a question, can you answer this question or not? And that that’s the kind of good hinge point Oh, I love that Devin. That’s great. Okay, tip number four, please.
Dylan Wiliam 20:04
Tip number four is more a caution than a tip. But the big idea here comes from the distinction that psychologists make between learning and performance. So Robert Bjork has done a lot of work in this area. And he’s shown that students really like being successful in completing learning activities. But that is often a very poor guide, to whether they’re going to remember it in two or three weeks time, as their performance is the performance in the task is designed to teach them something. And learning is the long term changes in capability that result. And the point is that they’re not the same. There’s some evidence that there’s a slight negative relationship between these two things. Now, obviously, if the task is so challenging, the students give up, little learning will take place, but probably awkward invoke this phrase desirable difficulties, we need to struggle a little bit in the task. Obviously, if we struggled too much, we might be unsuccessful. But I think the thing that is really important for me, is, we have very little insight into our learning. So we are very bad at predicting whether we’re going to remember something in the future. And obviously, if there’s a very strong emotional resonance, I will never forget where I was, when I heard about the death of Princess Diana, more of the assassination of JFK, or the attacks on the Twin Towers in New York. I mean, I remember where I was when I heard that. But most of school life isn’t like that. And so we’re actually not very good at predicting whether we’re going to learn something, or we’re gonna remember, I’ve lost track of the number of times, I put something in my freezer. And I haven’t bothered to label it, because I know, I’m gonna remember what it is. And six weeks later, I haven’t got a clue what it is. The fact that I forgot is not interesting. What is interesting is how certain I was that I would remember, we don’t have very good insights into our own learning. And we often use the word learn in a very kind of unhelpful scents. So I’d a leaking tap in the wash basin in our house. And American taps are different from British tap. So I watched the video. And I learned how to change the tap. But I didn’t learn it. Because if I needed to do it again, I would need to watch the video again. So we often use this, I learned how to do this in a way that actually says I learned how to follow a set of instructions to achieve the desired result. And so we often use the word learning when would performance would be more appropriate. I think it’s really important to remember that students because I know this now I’ve got this. First of all, we don’t know whether it’s gonna get through to long term memory. And we also need to remember the Dunning Kruger effect, the less you know about something, the more likely you are to overestimate your achievement. You know, this is why self reports is so inaccurate 93% of American car drivers believe they’re better than average. They can’t be right, can they? And we used to think that the reason for this was because they didn’t want to admit that they didn’t know. And part that is probably the case. But the biggest reason why so many people think they’re good drivers is they don’t know enough about good car driving to know that they’re not very good at car driving. As David Dunning himself says rule, one of Dunning Kruger club is you don’t know you’re in Dunning Kruger club. And so I think we teachers just need to be constantly sceptical about where the students say I get this now. And there’s this thing about student voice and asking students to say what they what they like in learning. They don’t know they are novices. They’re not experts. And so, you know, I think we should listen to our students, we should trust their insights into their own learning. I think that’s the important thing. I’m a big believer in self assessment. self self assessment often make students sharper and clearer in asking for help. But we shouldn’t trust those students saying I understand this means they understand it, because they may not know enough about what it means to understand, to actually really understand it at the level that you want them to understand. So Tip Four is we have lifted, all human beings have little insights into their own learning. And you need to be vigilant to be focusing on the long term learning, not just the improvements in task performance.
Craig Barton 24:19
That’s brilliant that Dell and just a couple of thoughts on that. It goes back to something you said earlier on. Often this is used and I’ve read David died as arguments almost kind of argument kind of not against formative assessment is too strong, but a cautionary tale about formative assessment because of this learning performance. Division. But as you said earlier on, if if you ask a good formative assessment, question, whether it’s a diagnostic question or whatever it is, and the kids can’t do it, you can be pretty sure they’re not going to learn it. So it’s it’s almost kind of a bit of a checking point, isn’t it? It’s not the end of the story, but it’s it’s a necessary step on the path to path muscling. The other thing that I just wanted to raise there is what I’ve started doing now, for a start. It took me about 12 years to realise The distinction between learning and performance, which is error number one again, but when I did learn it event took me another few years to realise the importance of sharing this with the kids, because it’s one thing for teachers to be aware of it. But it’s frustrating for students, right? If they if they think, Oh, I nailed this today. And then next week, they forgotten it or next month. So making them aware of the distinction themselves. And sometimes I will show them a diagram of the forgetting curve just to show students how quickly things go. But the positive side of that is if we retrieve it and think hard about it, then we start to flatten out this forgetting curve, I think visuals like that. And I’m kind of bringing students and kind of behind the curtain on how memory works and things like that, that feels, to me quite important to get that buy in, and also kind of negate some of the frustration they may feel, if that makes sense.
Dylan Wiliam 25:47
Absolutely. I mean, I want to talk about this as a user manual for the human brain, we actually have quite a lot of insights into how learning works. And what’s interesting is, it’s not how most people think that learning works. So John Donne Loski, and his colleagues did a review for the Association for Psychological Science on students self study strategies. And what is interesting is people think that rereading or summarising or highlighting is an effective review technique. And it really isn’t. And I think this distinction between of performance and learning is really important when it comes to revision. Because students read something that they read yesterday, and they think they know it. Yeah, I know this, because I read it yesterday. The point is, probably ox work allows us to say, yes, it’s familiar because retrieval strength is high. You retrieved it from your memory yesterday. So yes, it’s available to you right now. It doesn’t mean you will actually learned it. And so getting students to be much more kind of self critical. Yes, it feels familiar. But, you know, can I close the book and not look at the book and retrieve what’s you know, what’s in it? Can I can I give myself some revision practice from something I last read two weeks ago, rather than yesterday, just getting students to understand how easy is to be seduced by this familiarity? I’ve got it now I know it. Yes, that’s retrieval strength. It’s not storage strength. And of course, retrieval strength is good for passing exams. But if you want long term learning, then we also have to focus on story strength, you know how well it connected is connected to everything else in your memory.
Craig Barton 27:22
Absolutely. Five final point on this gentleman, it just thinking of videos, when you mentioned your story about about the tap there. I see this a lot with kids. I can only speak of maths here. But there’s 1000s of maths videos on YouTube. And kids will often say, Oh, I revised last night because I watched the video on adding fractions or whatever. And this is your classic familiarity. But you can nod your way through a video thing and I get this, I get this, I get this. And at the end of it, you think you’ve understood it, but you don’t have a clue. So I’ll often say to get to two things. One, obviously, the best way to learn maths is to practice math. So as you say, make sure you can do it with no queues around and you’ve got questions and so on. But if you are going to watch videos, keep pausing. And just just asking yourself, what’s just happened there? And what do I think’s going to happen next, just to make it a bit more of an active part of the process, as opposed to just let’s just watch a five minute video not our way through and then we think we’ve learned so there are little tricks out there if the kids are aware how memory works and stuff that we can teach them. Right, Dylan? Tip number five, please.
Dylan Wiliam 28:22
So with things like a cold call, when you pick on a student who hasn’t raised their hand, the instant reaction is done? No. Yes. So whenever we want to engage more of the classroom, a lot of students will say don’t know. And don’t know is student code four, go away and leave me alone. So the question is how you’re going to react to that. And so one way, if you’ve asked a higher order question when which might have different answers, you might say, Okay, if you didn’t know, I’ll come back to you. And then go around the class and get three or more answers or other members of the class and then say, okay, so which one of those answers do you like best?
So now rather than having to construct, so they just have to select from other people’s answers,
but you’re making them actually respond in a way to the question. Obviously, if it’s a loaded question, where there’s only one correct answer, that strategy can’t be used, but if it’s a Multiple Choice format, you can say to them, okay, so if you don’t know which one of these four options is correct, are any of them definitely incorrect? Can you make the question go 5050, which suggests some other techniques like phone a friend, or ask the audience. And so I’m very happy if a student is saying don’t know, you want a phone a friend, you want to ask the audience? Because the really important point here is you mustn’t let don’t know be the end of the conversation. If students think they can get rid of you by saying don’t know they will use it every single time. So If you say don’t know, I’m going to keep on going, you are going to say something, even if it’s just to repeat something that somebody else has said, don’t know will never be the end of the conversation. And a tip I got from an educator called Ellen keen in the United States, if you’re really sure a student is saying, don’t know, because they can’t be bothered. And a really good technique is yes, but if you did, no, what would you say? And it’s amazing how often students come up with something because they realise that they’ve been rumbled, they can’t be bothered to think that they’re gonna have to think they’ll come up with something. And so don’t let don’t know by the end of the conversation is a really important thing. But the other way to avoid that is just to not ask so many questions. So rather than asking questions, make statements. So this is the work of James Dillon, an American researcher. And what he’s shown is that when teachers make statements rather than asking questions, just to students responses, they tend to give longer and more thoughtful replies. So, you know, you might do two dozen American politics, and you say, what are Democrats believe in? And the student says, Democrats believe in progressive taxation, the teacher might say, but most Republicans also believe in progressive taxation. So you actually making a statement. So it could be just a kind of point back. What you just said, seems to criticise seems to contradict what ginger said, that hasn’t been my experience, or that has been my experience. Students says, you know, lithium, sodium, and potassium all have a single electron, in addition to saying that the first three elements in group one all have a single electron in the outer shell. It’s that idea of a reflective restatement, trying to move the conversation on. And here’s my hunch about why that works. When you’re asked a question, you could be wrong. But you can’t be wrong responding to a statement. Yes. And so it’s just an attempt to encourage the child to say more. And so that that’s why, you know, if you get into don’t know, that’s, in a way, that’s already the problem. And so I think one of the things that I encourage teachers to do is to think more reflectively, about how you’re going to get the students talking more. And what’s interesting is, as soon as you think about this as a conversation, rather than a q&a, then a lot more kind of moves become available. So I think the was under appreciated, and very, very rarely reflect on somebody to do it naturally. But I think it should be a focus of all of our reflections is just the power of nonverbal signal. Or even just a hand gesture to to invite another student to the conversation. And just getting away from this, it has to be another question. Don’t rely entirely on questions. Sometimes making statements can lead to longer, more thoughtful responses. Because, as I said, My hunch is you can’t be wrong. Responding to a statement, you can be wrong. Andrea question?
Craig Barton 33:12
That’s fascinating that I’m going to need to reflect about how I can make that work for maths but I sent it I sense he could do I don’t see any reason. Because mathematics is one of those things where he just had just question after question after question, but I can’t see any reason why I can’t do the statements. Let harlot hits ginger.
Dylan Wiliam 33:29
I mean, so. So when I give an example, when I work with teachers, I do a workshop and Adam boxer reflected on this in his talk with you that He now uses these classroom formative assessment techniques in his professional development sessions. Yes. So I often asked teachers, he has five options that might have happened in his research study. Which one of these things do you think was the result? And I make? I do think about a one variety. And I always say, you chose a
Craig Barton 33:58
Yeah, okay.
Dylan Wiliam 34:00
You chose B. Just the difference is just, it’s very subtle. Yeah, it could be really profound. It’s just getting it just trying to get into out of the habit of asking the questions that we always ask. And it’s really, really hard, and now much better than I used to be. So now when I’m wearing a t shirt workshop, I hardly ever ask any questions in response to something said by a participant, it’s always a kind of statement, just to try to lubricate that conversation. Just making it more like an adult, a normal adult conversation, rather than a q&a session.
Craig Barton 34:38
That’s amazing. I’m gonna definitely try that out. Let me ask you one more thing on this. I’ve been wanting to ask you this for about five years. So now now’s the perfect time, right? cold call. I really like the idea of cold call. But the problem I always have with it, and this this came out of my conversation with Adam, and it’s come out in this conversation here. I can’t see the argument why if you’ve got a really good question that you want To ask kids, why you wouldn’t want first every child to be thinking about the answer. And I think with cold call, I don’t think you can guarantee every child’s thinking about the answer. I think a lot of kids kind of play a bit of a gamble and think there’s 30 kids in this class, what are the chances of him asking me? And if he does, I’ll start my thinking there. And then so I think that’s potentially problematic. And also, if I can collect all the responses, whether it’s on mini whiteboards, or diagnostic questions, or whatever, I can then choose which responses I discuss with the whole class as opposed to again, I’m doing a bit of a gamble that if I asked me, Dylan, what do you think you may, you know, you may have the right answer. But actually, there may be some really interesting wrong answers that I’ve not heard from from the rest of the class. So my long rambling thing there is, we’re under what scenario as a teacher, would you choose to cold call versus whole class assessments and collecting all responses?
Dylan Wiliam 35:49
Well, first of all, I think they can be combined. So I’ll get to that in a minute. But but sometimes a child just hasn’t said anything. And so John, I’d be really interested in your reactions to what Tracy just said. Yes. So I just think that kind of thing. Because I know you personally, I know your experiences have been different from Tracy. So I value. So that’s the thing with Doug Lomov talks about that kind of cold call. But I’m with you, really, I think cold call is a stepping stone from asking the usual suspects to hope or what, what I call all student response systems. Yes. But I think what’s really interesting, I mean, so a lovely example of this in the English teacher in the classroom experiment, Melissa overy. She did a lesson on media reporting of emotionally charged events. And she chose her high school state and tragedy. And so the students read some clips about the high school stadium tragedy. And then she asked the students who was to blame Liverpool fan with the Ventus, farmers, the police, the football authorities, or the stadium authorities ABCD. And they all had cards and they held up the car, they thought she said, leave your choices showing on the desk. So then she was able to have a whole class discussion. You thought Liverpool fans were to blame? Tell us what you thought you went as far as what to blame. And so the teacher they would have a much more organised discussion by bringing students into the conversation at the right time. And then, at the end of the lesson, she decided to ask the students to vote again to see if anything had changed. And this time, every student held up more than one card. Wow. So their views got much more subtle and nuanced. And what I like about this story is you couldn’t have done that with an electronic voting system, you’re gonna have one right answer. And one girl, Katie was waving all five cards in here. Now, Katie is not an angel. And therefore it’s entirely possible she was being silly. Katie, why are you waving all five cards in the air. And Katie says, because everyone had some responsibility here. And so I think that idea of moving of using these old student response systems not just as a way of checking on understanding, not just a way of doing retrieval practice, but it’s giving the teacher information about what kinds of follow ups will be appropriate, so you can bring students into the conversation at the right time. That seems to me to be a very powerful idea.
Craig Barton 38:13
That’s brilliant Dylan, just before we go down, is there anything that you would recommend listeners or viewers check out any anything of your work that you’d want to point them towards or anything that’s caught your eye recently?
Dylan Wiliam 38:26
And unboxes book on teaching science, it’s a bit nerdy, but I mean, that’s just seems to be the best thing that I’ve read about teaching science. It’s so comprehensive, so thoughtful. So yeah, that would be that’s top of my list at the moment.
Craig Barton 38:39
That’s a great recommendation. Well, Dylan, it’s always a pleasure speaking to you, and this has been absolutely brilliant. Thank you so much for your time.
Dylan Wiliam 38:44
You’re very welcome. It’s been fun.