Desirable difficulties and four key retrieval opportunities

Contents

Diagnosis

What retrieval opportunities do you provide your students?
What challenges do you face?

Evidence

The never-ending cycle

We have all been there:

We teach something to our students
They seem to get it in the moment
Next lesson, next week, next term, whenever it may be, they have forgotten it
So we teach it to them again

And so the cycle continues.

This is why our curriculum often seems too crowded, and why many of our students get so frustrated.

Why we forget

But this is all to be expected.

In A New Theory of Disuse, Bjork and Bjork explain:

As impressive as we appear to be in terms of getting information into long-term memory, we are far less impressive at getting information out of the system. The retrieval process is erratic, highly fallible, and heavily cue-dependent. Information (such as a name, phone number, or street address) that is recallable on one occasion without apparent effort can be impossible to recall on another occasion. Even the most highly overlearned and frequently accessed information, such as a street address or phone number, eventually becomes non-retrievable over years of disuse, though it is a simple matter via tests of relearning or recognition to demonstrate that such items still exist in storage.

Our brains receive so much daily stimuli and information that it would be impossible to keep it all at the forefront of our minds. So, our brain prioritizes the information we regularly access, whilst the rest remains in storage but becomes less and less accessible over time.

Solution steps

Consider retrieval practice a core part of learning

Fortunately, there is a solution, and a simple one at that.

Each time we access one of our memories, we signal to our brain that it is important. Technically, this gives that memory a boost in storage strength—how deeply embedded the momentary is—and retrieval strength—how accessible the moment is. Here is a great paper on the role and distinction between these two memory features.

A nice way to visualise this is the classic Ebbinghaus Forgetting Curve diagram:

Each successful retrieval opportunity flattens out the curve, thus slowing down the rate of forgetting.

My favourite summary of all of this comes from Carl Hendrick. Carl explains:

You don’t learn something when you encounter it, you learn it when you forget it-remember it-forget it-remember-link it to something else you know and so on… Think of learning as something which happens over 6-12 months not in a single lesson:

We should view retrieval as a learning event. Retrieval practice is not so much about testing whether someone knows something, but rather a core part of learning that something. It is a process which occurs across time and space.

So, when should this retrieval practice happen? I think there are four key retrieval opportunities teachers can take advantage of.

What do I mean by a retrieval opportunity?

There are two broad categories of concepts we can challenge students to retrieve:

Retrieval of concepts related to the content of the lesson
Retrieval of concepts unrelated to the content of the lesson

In this section, I am talking about the latter.

As I discuss in the Building Blocks of a Lesson section, I believe that students should be given the opportunity to retrieve concepts related to the content of the lesson during the Atomisation phase. This is where such prerequisite knowledge is assessed. However, if this is the only retrieval opportunity we give our students, then they will only retrieve a subset of the curriculum. Sure, they will have lots of chances to retrieve knowledge of negative numbers, fractions and decimals, as these are prerequisites to lots of new concepts. But what about transformations, histograms, and repeat percentages? What are they prerequisite knowledge for?

So, we must provide opportunities for students to be challenged to learn any concepts they have encountered in the past. This is the only way to ensure that there are no gaps in our students’ knowledge. Hence, when discussing retrieval opportunities below, I am talking about things unrelated to the lesson’s content.

Retrieval opportunity #1 – The Do Now

Teachers often use the start of the lesson to revisit concepts students have met in the past.

Main pros:

When done regularly, this can form part of a routine that settles students and sets them up for success for the rest of the lesson.
You can cover many concepts each week.

Main cons:

If students do not take the Do Now seriously, valuable lesson time is wasted
Latecomers to the lesson may miss out

Read the Do Now section for more on this phase of a lesson.

Retrieval opportunity #2 – A Low-Stakes Quiz

Teachers may set aside a significant portion of a lesson each week or fortnight to do a quiz on concepts students have met in the past.

Main pros:

The longer time compared to a Do Now not only means more topics can be covered, but also topics that require specialist equipment such as protractors, compasses and calculators.
Students may also take this more seriously than a Do Now.

Main cons:

It eats into curriculum time
If students miss the Low-Stakes Quiz lesson, then they miss a big chunk of their retrieval practice.

Retrieval opportunity #3 – Woven into the current topic

Teachers often skillfully weave topics students have encountered in the past into the current unit. Negative numbers into a unit on averages, or solving equations into a unit on angle facts, for example.

Main pros:

This feels less of a bolt-on than retrieval during the Do Now and Low-Stakes Quiz, helping students see the interconnected nature of mathematics.
It does not eat into curriculum time

Main cons:

It is tricky to weave in certain topics. Try weaving rotations into a unit on cumulative frequency diagrams, or simultaneous equations into a lesson on percentage increase. This means students may not get an opportunity to revisit some concepts.
If students are struggling, it can be difficult to diagnose why. Is the issue with the new topic or the old one?

Retrieval opportunity #4. Homework

The best homeworks I see do not just assess students’ understanding of the topic they have just studied, but also includes questions on subjects students have encountered in the past.

Main pro:

This is the only retrieval opportunity that does not take up precious lesson time.
As a result, students can spend as long as they need on any given question.

Main cons:

Teachers have less knowledge about who has done the retrieving.
The simple fact is that many students – often the ones we most need to do retrieval – do not do their homework.

Track topics across each retrieval opportunity

If we want students to remember something, we need to provide retrieval opportunities. Therefore, it is a good idea to track what is included in each of the four retrieval opportunities. Without this, you risk gaps, especially of material that is harder to test (constructions, angle measuring, etc).

This can be as simple as printing off a list of objectives from your scheme or working and ticking off each time one is included in a retrieval opportunity.

Or you can go one stage further and indicate how successful students were at that retrieval opportunity.

This will flag whether a topic needs re-teaching, re-testing, or if it can be left for a while.

Desirable difficulties: Introduction

Making learning harder than it needs to be sounds like a counterintuitive idea when you first encounter it. But in other walks of life, we know that often, harder in the short term can be better for us in the long run.

When I interviewed leading researcher Nick Soderstrom, he used the analogy of stairs versus escalators:

Sure, taking the escalator is easier than the effort required to climb the stairs. But in the long run, regularly taking the stairs will build up fitness that will serve you well for the rest of your life. Short-term pain, for long-term gain.

Hence, we have the idea of desirable difficulties – strategies that slow down performance but enhance long-term learning.

The word desirable is key here. We are not making things difficult for the sake of it. Presenting new material in a confusing way or not assessing students’ background knowledge will make learning more difficult for our students, but not in a way that will benefit their long-term learning.

Here is my favourite paper on desirable difficulties, and here is my favourite paper on the key distinction between performance and learning.

Let’s now look at how we can make learning desirably difficult.

Desirable difficulty #1: Testing

What is testing?

Asking students to retrieve from their long-term memories, as opposed to asking them to re-read.

Why is testing effective for learning?

Retrieval is a “memory modifier.” Each act of retrieval is not a neutral event. It boosts the retrieval and storage strength of a memory, making it more deeply embedded and more accessible.

What is the key research to read?

Testing (Quizzing) Boosts Classroom Learning

A large-scale, cross-subject, cross-phase meta-analysis found strong evidence that testing significantly improves student achievement, with an average effect size of 0.499.

What are the implications for the classroom?

1. Test frequently over long periods

The more occasions class content is tested, the larger the learning gains. And the longer the time period over which testing occurs, the larger the learning gains.

2. Design tests so students feel successful

Students must be able to retrieve some of the content successfully to benefit from retrieval and also keep motivation levels sufficiently high.

When designing retrieval opportunities, I aim for the average score in the class to be 80%. This means students feel successful but also have something to think hard about. I aim for a success rate of 80%. There are two variables to consider here:

The difficulty of the content
The gap between when the material was last taught or assessed

3. Ensure students are given corrective feedback

This does not mean written feedback, but simply that students need to know if their answers are right or wrong.

4. The format of the test does not matter

Test-enhanced learning generalizes to a variety of test formats. So, feel free to use matching, Fill-in-the-blank, Short answer, Multiple-choice, Cued recall, Free recall.

5. Scaffold initially and then reduce cues

When students are first asked to retrieve content, you might provide cues. That knowledge has not been embedded, and it is important to manage motivation levels. Using different formats, like Fill in the Blanks or Banckwards-faded examples, can help here.

However, if students successfully retrieve on the first occasion, reduce these cues in subsequent retrieval opportunities, eventually moving to no cues at all.

6. Carefully manage the stakes of the test

This is an interesting one:

Low-stakes quizzes have the same effect on learning. You don’t need to assign grades to benefit from retrieval.
Indeed, higher-stakes quizzes may cause test anxiety that reduces the impact of retrieval.
However, if the stakes are too low, students may lack motivation.

It is about knowing your students well enough and adjusting the stakes accordingly to ensure they are motivated enough to put in the effort required to benefit from retrieval. I discuss the formats of my Low-Stakes Quizzes here (LINK).

Desirable difficulty #2: Spacing

What is spacing?

Spreading out the opportunities for students to retrieve knowledge over time, instead of completing these in immediate succession.

Why is spacing effective for learning?

Spacing is thought to enhance learning via two explanatory mechanisms:

The delay between initial exposure to an idea and subsequent retrieval opportunities induces forgetting. The increased cognitive effort required to retrieve when students are on the point of forgetting leads to a bigger boost of retrieval and storage strength.
Spacing may also be effective because the rest between initial exposure and retrieval opportunity allows depleted working memory resources to recover.

What is the key research to read?

Spacing and Interleaving of Study and Practice

To help students retain information over the long term, teachers should implement spaced reviews of previously learned material

Spacing effects in learning: A temporal ridgeline of optimal retention

The timing of learning sessions can have a powerful effect on long-term student retention. Teachers should consider this when planning lessons and scheduling reviews.

Spacing and Interleaving Effects Require Distinct Theoretical Bases

This effect may occur because rest allows students to restore depleted working memory resources. Spaced practice can be implemented by distributing learning sessions across multiple days or months, or by inserting rest periods between learning sessions.

What are the implications for the classroom?

1. Have a short gap between the initial expose and the first retrieval opportunity, and then space out the gaps

The slope of the forgetting curve is at its steepest after the initial exposure of an idea.

Therefore, we need to provide an early retrieval opportunity. This may be in the Do Now the very next lesson.

If retrieval is successful, then we can wait a little longer for subsequent opportunities, perhaps including the next one in that week’s homework, and then a month later in a Low-Stakes Quiz.

2. Don’t sweat too much about the optimal spacing schedule

This paper sought to find an optimal spacing schedule, and does come up with some concrete suggestions:

Science Teacher, Damian Benney, discusses here how he applied this research in a blog post here, and in a video here.

But use this with caution. There are so many factors that could influence things:

The quality of the initial instruction
The attention of the students
The complexity of the material to be retrieved
The conditions under which it is retrieved

For what it is worth, I take two lessons from research into spacing

Try to include several opportunities to retrieve content over time
Make that initial opportunity come soon, then increase the time intervals

3. Have a curriculum that explicitly includes spaced retrieval opportunities

Providing questions to assess prerequisite knowledge is a good way to ensure a significant proportion of content is revisited.

However, this will not capture concepts that are not prerequisites for others, such as transformations, density, etc. That is where having a suite of retrieval opportunities available and tracking content across them is important.

4. Set lagged homework

If you have written topic-based homework, or have online homework platforms that can only set one topic, then consider lagging those homeworks. In other words, don’t set homework on the current topic being taught but on the topic you taught, say, 3 weeks ago.

Here is a telling the time homework my little boy was set, 4 weeks after it was studied in class:

Desirable difficulty #3: Interleaving

What is interleaving?

Mixing together different types of questions in a single practice session, as opposed to practising several types of the same question in a row

Why is interleaving effective for learning?

There are three explanatory mechanisms through which interleaving is thought to enhance learning:

Retrieval practice – students have to constantly reload different memories as they switch between questions, with each reload giving a boost to retrieval and storage strength
Attention attenuation – when question types constantly change across an exercise, students must pay more attention than if they are answering the same type of question repeatedly.
Discriminative contrast – interleaving increases the opportunities to compare or contrast questions, making the differences and similarities between them more obvious, thus helping develop students’ abilities to discriminate between them.

What is the key research to read?

A randomized control trial of interleaved maths practice

Simply mixing up the order in which practice content is given to students can enhance long-term retention.

Spacing and Interleaving Effects Require Distinct Theoretical Bases

Interleaving is most effective when the interleaved topics are similar and require students to learn to discriminate between them. When topics are very dissimilar, interleaving may not be effective.

What are the implications for the classroom?

1. For complex material, do blocked practice first

If I have just taught students, say, how to solve simultaneuosu equations, then I want the immediate practice that follows to be on simultaneous equations and nothing else. Hence, I want a period of blocked, not interleaved, practice.

That is why in my various lesson structures, Consolidation comes after the We Do:

Once students have obtained a degree of confidence and competence, then we can begin to interleave.

2. Have mixed-topic homework

An interleaving easy-win is to include a Revision section on each homework.

This allows students to practise task switching and benefit from the re-loading of memories.

3. Use materials that support discrimination

To really tap into the power of the discriminative contrast mechanisms of interleaving, you want to give students materials that require them to think hard about the similarities and differences. That is exactly why I designed SSDD Problems—sets of questions with Similar Surface features but Different Deep structures.

Here is a set of SSDD Problems I would use after the consolidation of Pythagoras’ theorem:

Students can retrieve how to calculate the area of a triangle, form and solve equations, work out angle facts, and apply Pythagoras’ theorem. They must also be on the ball when working through each problem, constantly switching topics and strategies. Finally, students are forced to ask themselves, for example, why do the top-left and bottom-right questions require different strategies, even though they both have an isosceles triangle and ask for the area?

I write more about SSDD Problems here.

4. Look for opportunities to interweave

Mathematics lends itself to a specific type of interleaving, where concepts are mixed together. This is called interweaving.

I like this example from @MiddletonMaths:

Interweaving has all the advantages of interleaving, helping students see mathematics’s interconnected nature.

5. Consider curriculum ordering

It is important that students get practice discriminating between similar-looking procedures and concepts. If they don’t, they will calculate the area instead of the perimeter, use the algorithm for multiplying fractions instead of adding fractions, and so on.

So, while there is a good argument for teaching similar concepts separately at first to enable students to be secure in them, students must also be given opportunities to practice discriminating between these concepts.

Desirable difficulty #4: Varying the conditions of practice

What is varying the conditions of practice?

When learning becomes contextualised, the material may be easily retrieved in the context in which it was learned, but students may struggle to retrieve it if tested in a different context.

Why is varying the conditions of practice effective for learning?

Varying the conditions means memories are not tied to cues. They become more robust and accessible in a wider variety of settings.

What is the key research to read?

Effects of initial context processing on long-term memory

The authors found that memory recall was better when the testing environment matched the learning environment. The findings suggest that environmental context plays a significant role in memory recall, particularly when retention intervals are long.

What are the implications for the classroom?

1. Make use of homework

Ensuring students have an opportunity to consolidate and solve problems both in the classroom and at home ensures they do not come to rely on one particular setting.

2. Vary conditions in the classroom

If the culture and behaviour in the class permit it, you could regularly change the seating plan so students do not come to rely on familiar surroundings. Likewise, consider alternating between silent independent work, paired collaboration and group work.

3. Ensure students experience conditions similar to their final exam

We have all taught students who are successful in class but then go to pieces in exams. Often this is because they are not used to it. Counteract this by ensuring students experience the conditions in which they will do their final exam. This could involve have a lesson in rthew exam hall itself, or creating the silent, timed conditions of an exam.

4. Vary the types of questions we ask

If we always check for understanding by asking the same question format, students’ ability to retrieve may become dependent on this structure.

For example, all of these questions challenge students to calculate the area of a trapezium in much the same way.

My favourite way of ensuring I ask a variety of question types to check students’ understanding is to follow the taxonomy proposed by Poimton and Sanguin (LINK):

Here is how from @MiddletonMaths has applied this taxonomy for the area of a trapezium:

5. Vary the structure of practice materials

Similarly, we want to ensure students have an opportunity to consolidate their understanding in a variety of ways. If all practice material looks the same, students may become dependent on a certain format or way of asking a question, leading to problems if this changes in the exam.

A set of high-value activity structures is a good way to achieve this. Completion tables and Venn Diagrams force students to constantly vary their problem-solving approaches, strengthening their understanding.

Desirable difficulty #5: Pre-testing

What is pre-testing?

Testing students on material they have not yet been taught before they are taught it

Why is pre-testing effective for learning?

There are two explanatory mechanisms through which pre-testing is thought to work:

Pre-testing increases attention during teaching – students may be more motivated to learn how to do the thing they were unable to do in the test
Increases studying outside of lessons – older students in particular may be more likely to put in extra effort to study the material they were unable to complete in the test.

What is the key research to read?

Presting enhances learning in the classroom

The authors conducted a study in an undergraduate research methods course where students took short, competitive multiple-choice pretests before some lectures. They then analyzed the students’ performance on a final exam and also administered a questionnaire to gauge student perceptions of the pretests. The results of the study found that pretesting significantly improved student learning, both for the material tested and for related material that was not pretested

What are the implications for the classroom?

The implication is clear:

Test students on material they have not yet encountered

This could take the form of a Low-Stakes Quiz on the upcoming topic. It could also be a problem you ask students to solve before they have been taught the necessary tools.

I’ll be honest—out of all the desirable difficulties, this is the only one I have an issue with. I understand the logic—testing students on something they have not seen before may make some students curious and increase their motivation and, hence, their attention when we subsequently teach them. But some students are the key.

We have all taught students that this would not be an effective strategy. These tend to be students who lack confidence in the subject and have experienced little success. For them, the struggle is not motivating. Struggle is not a short-term feeling that is a precursor to understanding. Instead, struggle is a common occurrence that perhaps never gets resolved. So, if fact, pre-testing may have the opposite effect on such students, causing them to pay less attention when the concept is being taught because they have given up, hence decreasing learning.

I have the same issue with the research into productive struggle (LINK). It works great with confident, achieving students but not so great with students for whom the struggle is rarely productive.

I discussed all this with Nick Soderstrom in our podcast conversation. He understood my concerns and assured me that further research is being done on school-aged children to see how wide the effect transfers.

Do with it what you will!

Want to know more?

Check out these two podcast episodes with some of the world’s leading researchers in memory and retrieval:

#023 Robert and Elizabeth Bjork: Memory, Forgetting, Testing, Desirable Difficulties
#183 How to help students remember things with Nick Soderstrom

The book Desirable Difficulties in Action is well worth a read

Implementation planning

???