- Title: The pretesting effect is robust throughout adulthood, but metacognitive beliefs about pretesting and challenge differ
- Authors: Veronica X. Yan, Stephany Duany Rea, Faria Sana
- Access the original paper here
- Listen to a deep-dive podcast:
Paper summary
This paper explores the concept of pretesting, where individuals attempt to answer questions on a topic before formally learning the material, as a learning strategy that falls under desirable difficulties. The research, conducted across a wide age range, found that the pretesting effect—the learning benefit from making initial guesses, even incorrect ones—remained consistent throughout adulthood. However, the study highlights a significant difference in metacognitive beliefs about this strategy, revealing that while younger adults tend to see the value in pretesting and making errors, older adults are less likely to appreciate the benefits of this errorful learning. These findings suggest the importance of not only promoting effective learning tools like pretesting but also fostering a more positive attitude towards challenges and mistakes in learning, particularly among older learners, to support successful lifelong learning.
If teachers are to remember one thing from this study, it should be…
Promoting effective learning strategies like pretesting while also addressing learners’ beliefs and attitudes towards errors and challenge, especially for older adults
***Paper Deep Dive***
Define any technical terms used in the paper
Based on the sources, here are definitions for some of the technical terms used:
z-standardized/standardized: A data transformation process where variables are converted to have a mean of zero and a standard deviation of one. This makes it easier to interpret and compare coefficients from variables measured on different scales.
Pretesting effect: This is the finding that asking people to try to answer questions about a topic before they study it results in better memory of that topic compared to simply studying without a pretest. It is also sometimes called the prequestion effect or errorful generation effect. The benefit arises from how the pre-questions prepare learners to process the information they encounter afterward.
Metacognitive beliefs/judgments: These refer to learners’ beliefs and evaluations about learning strategies and how effective they perceive them to be. These judgments are important because they influence whether learners choose to use specific strategies, especially in self-regulated learning contexts. Examples include judging the effectiveness of pretesting compared to studying only or broader beliefs about the role of errors in learning.
Desirable difficulties: These are conditions or strategies for learning and practice that require more effortful processing and may lead to increased perceived difficulty or more mistakes during the learning process. However, because they foster deeper engagement, they ultimately result in better long-term retention and understanding. Pretesting is considered one of these strategies.
Retrieval practice/testing: A learning strategy that involves asking learners to recall information from their long-term memories. This process strengthens the knowledge and makes it more accessible in the future. It is described as one of the most robust “desirably difficult” strategies for long-term learning.
Errorful learning: Learning techniques or conditions where the learner is likely to make errors or incorrect guesses during the study or practice phase. Pretesting, by its nature, often involves making erroneous guesses.
Semantic activation: One theoretical explanation for the pretesting effect, suggesting that being presented with questions related to upcoming material activates relevant information in a learner’s memory. This activation is thought to facilitate the encoding of the correct answer when it is later presented.
Attentional theories: Theories that propose that the pretesting effect occurs because pretesting directs learners’ attention to the specific concepts or information in the material that follows. While attention plays a role, it is not considered the sole mechanism.
Paired associates paradigm: A type of experimental task frequently used in memory research, where participants are presented with pairs of items (e.g., cue word and target word) and asked to learn the association between them. This study used weakly associated word pairs in a paired associates paradigm to investigate the pretesting effect.
Errorless learning: A learning condition where participants are immediately provided with the correct answers or responses, designed to prevent them from making errors. Some earlier research suggested this was beneficial for older adults and clinical populations.
Metacognitive mismatch: The situation where there is a discrepancy between the actual effectiveness of a learning strategy and learners’ beliefs or judgments about that strategy’s effectiveness. This mismatch can lead learners to avoid using beneficial strategies.
Working memory: A cognitive system involved in holding and manipulating a small amount of information for a short period. It is a cognitive ability that tends to decline with age.
Interference: Cognitive processes where previously learned information (proactive interference) or subsequently learned information (retroactive interference) impairs the memory for other information. Older adults have been found to exhibit greater interference effects in some tasks. Pretesting might potentially protect against retroactive interference.
Semantic knowledge/memory: Refers to general knowledge about the world, concepts, facts, and word meanings. This type of memory tends to be preserved or even enhanced in older adulthood. The pretesting effect, particularly with semantically related items, may rely on drawing upon this prior knowledge.
Associative memory: The ability to form and retrieve links or associations between different pieces of information. Older adults often show deficits in associative memory tasks.
Source monitoring: The cognitive process involved in remembering the origin or context of information. Older adults may experience inaccuracies in source monitoring.
Hypercorrection effect: This effect describes the phenomenon where errors made with high confidence are more likely to be corrected after feedback than errors made with low confidence. (Mentioned in relation to semantic activation theories).
Judgments of learning (JOLs): Predictions made by learners, typically during the study phase, about how well they expect to remember specific items or information on a future test.
Global metacognitive judgments: Overall evaluations made by learners about the effectiveness of different learning strategies after completing a learning task. These judgments are thought to significantly influence learners’ future strategy choices.
Challenge beliefs: Learners’ attitudes and perceptions about various aspects of challenge encountered during learning, such as making errors, experiencing failure (viewed as debilitating or enhancing), or facing difficulty (viewed as impossibility, importance, or potential for improvement). These beliefs can influence how learners engage with difficult tasks or strategies.
Self-regulated learning: Learning where individuals direct and manage their own learning process, including choosing and applying learning strategies. Metacognitive beliefs are crucial for effective self-regulated learning.
Multifactorial Memory Questionnaire (MMQ): Contentment subscale: A component of a questionnaire designed to assess subjective reports about memory, specifically focusing on a person’s level of satisfaction or concern regarding their memory abilities. It was adapted to be used across different age groups in this study.
Generalized linear mixed models (GLMMs): A statistical modeling technique used for analyzing data structures where observations are not independent (e.g., multiple responses from the same participant). It allows researchers to examine how predictor variables (fixed effects) influence an outcome while accounting for variability between individuals or items (random effects). It was used to determine if the pretesting effect was moderated by age or memory contentment.
Multinomial logistic regression: A statistical method used when the outcome variable is categorical with three or more options. It predicts the probability of being in each category based on one or more predictor variables. It was used to analyze participants’ choices about which learning strategies were perceived as most effective.
Bi-factor model: A statistical model used in factor analysis to assess whether a set of items reflects both a general overarching factor and specific, narrower factors. It was applied to the challenge belief scales to determine if there was a general factor underlying beliefs about challenge.
Cross-sectional design: A research approach where different groups of participants (e.g., different age groups) are studied at a single point in time. This study used a cross-sectional design to examine age differences.
Longitudinal variable: A variable measured repeatedly over time within the same individuals. The study notes that a longitudinal design would track individuals’ changes over time, contrasting with their cross-sectional approach which observes differences between groups of different ages at one point in time.
Cohort differences: Differences between age groups that are attributable to the unique historical, social, or environmental experiences shared by individuals born within a specific period (a cohort), rather than changes that occur with aging itself. The study acknowledges that observed age differences could reflect cohort differences.
Developmental differences: Changes or differences observed between age groups that are due to biological, cognitive, or psychological changes that naturally occur as individuals grow older.
Qualitative coding/thematic approach: A method for analyzing qualitative data, such as open-ended text responses, by systematically identifying, analyzing, and reporting recurring patterns or themes within the data. An inductive thematic approach was used to analyze explanations for pretesting scenario choices.
Confirmatory factor analysis (CFA): A statistical technique used to evaluate whether a proposed factor structure (i.e., how latent variables relate to observed variables) fits the collected data. It was used to confirm the underlying structure of the challenge belief scales.
Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), Root Mean Square Error Approximation (RMSEA): Statistical indices used to assess how well a theoretical model (like a factor analysis model) fits the observed data. Higher CFI and TLI generally indicate better fit, while lower RMSEA indicates better fit.
Akaike Information Criterion (AIC): A measure used to compare the relative quality of different statistical models for a given set of data. It penalizes models for having more parameters, and the model with the lowest AIC is typically preferred.
Cohen’s d: A standardized measure used to express the magnitude of the difference between two groups or conditions. It quantifies the effect size in standard deviation units.
Effect size (g): A standardized measure indicating the magnitude of a research finding. Hedges’ g is a common effect size metric used in meta-analyses and quantifies the difference between groups or conditions in standard deviation units, with a correction for small sample sizes.
Log odds: In logistic regression, the natural logarithm of the odds of an event occurring. The regression coefficients are typically expressed in log odds.
Odds ratio: A measure used in logistic regression that represents the ratio of the odds of an outcome occurring in one group compared to another group. It indicates how much the odds of the outcome change with a one-unit increase in a predictor variable.
What does this paper add to the current field of research?
This paper significantly contributes to the current field of research, particularly concerning the pretesting effect and how learning strategies are perceived across the adult lifespan.
Here are some key contributions the paper makes:
Methodological Approach: The study’s use of a wide age range, treating age as a continuous variable, and employing a mixed-methods approach (quantitative performance/judgments and qualitative responses) are valuable contributions to studying cognitive and metacognitive processes across the lifespan.
Robustness of the Pretesting Effect Across Adulthood: While previous research had established the pretesting effect’s robustness in young adults and explored it in older adults (often using categorical age comparisons), this study provides strong evidence that the pretesting benefit is remarkably consistent across the entire range of adulthood, from 18 to 82 years. Using age as a continuous variable rather than a categorical one allows for a more sensitive examination of how the effect might change with age, and this study found it was not moderated by age or memory contentment. This extends prior findings, suggesting pretesting is a useful learning strategy for adults of all ages, at least for semantically related content where prior knowledge can be drawn upon.
Age-Related Differences in Metacognitive Beliefs: Critically, despite the pretesting effect being cognitively beneficial across age groups, the study found that metacognitive beliefs about pretesting and the value of errors do differ with age. This contrasts with much of the existing literature on metacognitive beliefs about pretesting, which has focused almost exclusively on younger adults who tend to underestimate its benefits. This paper shows that:
Younger participants were more likely to appreciate the benefits of pretesting than middle-aged and older adults.
In a general context, older participants were less likely to appreciate the benefits of errors and were less likely to endorse learning techniques that involve making some errors compared to techniques with few or no errors.
Older adults reported less positive beliefs about challenge overall (learning from errors, failure-as-enhancing, difficulty-as-importance, difficulty-as-improvement). This finding highlights that older adults may hold a less positive or more neutral view towards the potential upsides of challenge compared to younger adults.
Highlighting Metacognitive Mismatch Across Age: The findings reveal a metacognitive mismatch where older adults underestimate the effectiveness of a strategy (pretesting/errorful learning) that is actually beneficial for them. This is particularly important because most learning, especially beyond formal education, is self-regulated, and learners’ beliefs determine which strategies they choose to use.
Insights into Beliefs About Errors and Challenge: The study provides deeper insights into how beliefs about errors and challenge vary with age and across different learning contexts (e.g., trivia vs. vocabulary). The qualitative analysis revealed that beliefs about the value of guessing often depend on whether participants feel they can draw on prior knowledge or context.
Practical Implications for Lifelong Learning: The results have significant practical implications for supporting lifelong learning. Since older adults can benefit from pretesting but are less likely to believe it’s effective or appreciate the role of errors, instructional interventions should not only promote effective strategies like pretesting but also actively address and reshape learners’ attitudes towards errors and challenges.
What are the characteristics of the participants in the study?
Based on the sources, the participants in the study had the following characteristics:
Representativeness Limitation: The authors note that recruitment through platforms like Prolific tends to yield participants who are healthy, technologically active, and well-educated, particularly among older adults. This may limit the generalizability of findings to the broader population, potentially including less cognitively healthy older adults. Despite this, even these potentially more cognitively healthy older adults were less likely to positively view errors and challenges in learning.
Overall Participant Count: The research involved two studies. Study 1 included data from 273 participants after exclusions. Study 2 included data from 296 participants after exclusions.
Age Range: Across both studies, the participants spanned a wide range of adulthood.
In Study 1, the age range was 18 to 82 years.
In Study 2, the age range was 18 to 83 years.
The mean age was similar across studies: 48.26 years (SD = 17.08) in Study 1 and 48.50 years (SD = 17.32) in Study 2.
Age Distribution: Participants were recruited with stratified sampling to ensure an even distribution across different age groups: 18–29, 30–39, 40–49, 50–59, 60–69, and 70+ years, with fifty participants initially recruited in each group for both studies.
Gender:
Study 1 included 146 women, 125 men, and 2 non-binary participants.
Study 2 included 153 women, 142 men, and 1 non-binary participant.
In Study 1, women were, on average, older than men, but gender did not differ on memory contentment. Gender was related to age and final test performance, so subsequent analyses controlled for gender.
Race/Ethnicity:
In Study 1, participants were 88% White/Caucasian, 6% Hispanic or Latino/a, 3% Asian, 1% African American or Black, 1% Middle Eastern or North African, and 2% other classifications.
In Study 2, participants were 74% White/Caucasian, 8% Hispanic or Latino/a, 2% Asian, 12.5% African American or Black, 2% Middle Eastern or North African, and 4% other classifications.
Education Level: Education levels varied among participants in both studies.
In Study 1, 15% had a high school diploma or lower, 29% had education beyond high school but no Bachelor’s degree, 29% had a Bachelor’s degree, and 25% had a postgraduate degree.
In Study 2, 15% had a high school diploma or lower, 23% had education beyond high school but no Bachelor’s degree, 40% had a Bachelor’s degree, and 21% had a postgraduate degree.
Language: Participants in both studies were required to be fluent English speakers.
Recruitment: Participants were recruited via the online platform Prolific.co.
What are the key implications for teachers in the classroom?
Drawing on the sources and our conversation history, the study offers several key implications for teachers and instructional designers in various learning contexts, particularly those involving adult learners:
- Implement Pretesting as a Strategy Across the Adult Lifespan: Teachers should feel confident in using pretesting as a learning strategy for adults of all ages, from 18 to over 80. The study found that the cognitive benefit of pretesting is robust and not significantly moderated by age or memory contentment, at least for semantically related material where learners can draw on prior knowledge. This means that asking learners to attempt to answer questions before presenting the correct information is a tool that can enhance memory and learning across the entire adult range.
- Explicitly Address and Reshape Metacognitive Beliefs About Errors and Challenge: This is perhaps the most critical implication. While pretesting is cognitively beneficial for all adult age groups, their beliefs about its effectiveness and the value of errors differ significantly with age. Younger adults are more likely to appreciate pretesting and errorful learning, whereas middle-aged and older adults are less likely to appreciate the benefits of errors and may hold more neutral views towards challenge.
- Teachers should actively teach learners why making mistakes can be beneficial for learning.
- Instruction should aim to cultivate a positive mindset towards errors and challenges. This involves explaining that errors are not simply “bad” but can be a valuable part of the learning process that fosters deeper engagement and prepares the learner for subsequent correct information.
- For older learners, in particular, interventions might need to specifically address potential anxieties or negative perceptions related to memory and the role of errors. Emphasize that difficulty can signal importance and is integral to improvement.
- Teach Conditional Knowledge About Strategy Use: The study suggests that learners’ beliefs about when guessing or making errors is useful often depend on whether they feel they can draw upon prior knowledge or context. Teachers can help learners develop better metacognitive awareness by discussing how strategies like pretesting might be more effective for certain types of content (e.g., trivia, concepts where prior knowledge is relevant) compared to others (e.g., rote memorization of unrelated facts), aligning with learners’ intuitive understanding that guessing is beneficial when there’s a basis for it. This helps learners make informed choices when regulating their own learning.
- Frame Effortful Strategies as “Desirable Difficulties”: Teachers should frame strategies like pretesting as “desirable difficulties” – techniques that may increase effort or lead to errors in the short term but result in stronger, more lasting learning in the long term. This can help overcome the natural preference many learners have for “easy” learning methods, like simply restudying correct answers, which they often believe is more effective despite evidence to the contrary.
- Consider Context, While Recognizing Underlying Principles: While the study used a specific paired-associates task, which is more controlled than typical classroom learning, the findings on the robustness of the pretesting effect and, more importantly, the age-related differences in metacognitive beliefs about errors and challenge are likely applicable to broader educational settings. Teachers should consider how these principles apply to learning from text, lectures, or problem-solving, recognizing that the nature of pretesting and error generation might differ but the underlying cognitive benefits (e.g., activating prior knowledge, directing attention) and the need to address beliefs remain relevant.
In essence, the study highlights that for effective lifelong learning, teachers need to provide adult learners not only with effective tools like pretesting but also with the right mindset to embrace the challenges and errors inherent in deep learning.
Why might teachers exercise caution before applying these findings in their classroom?
Teachers might exercise caution before directly applying these findings in their classroom for several important reasons, primarily related to the specific context and characteristics of the study compared to real-world educational settings:
- Participant Representativeness: The participants were recruited via an online platform (Prolific.co), and the authors note that this source tends to yield healthy, technologically active, and well-educated participants, especially among older adults. This selection bias means the findings might not fully generalize to a broader population of adult learners, particularly those who are less cognitively healthy, less technologically literate, or have lower educational attainment. While the study found that even these potentially more cognitively healthy older adults had different beliefs about errors, the effects might be different or larger in a less selected group.
- Nature of the Learning Task: The primary task used in Study 1 was a paired-associates task with weakly related word pairs (e.g., Whale: Mammal). While this is a controlled method often used in pretesting research, it is described as “artificial” and not fully representative of the complexity of real-world educational content like integrating information from lectures, reading textbooks, or solving problems. The study itself acknowledges that the nature of the pretesting effect in these more complex settings (which engage higher-order cognitive processes) often differs from paired-associate paradigms. Teachers in typical classrooms deal with a much wider variety of content and learning activities than simple paired associations.
- Limited Generalizability of Stimuli: While the study deliberately chose semantically related basic nouns to ensure content was similar across the wide age range, this means the findings are specifically tied to learning content where learners can draw on prior knowledge or context. The qualitative data supported that learners’ beliefs about the value of guessing depended on having a basis for the guess. The findings might not apply as strongly to learning unrelated facts or highly novel material where learners have no relevant prior knowledge to activate during pretesting. Future research is needed to confirm these effects with more conceptual information and other types of stimuli.
- Cross-Sectional Design: The study uses a cross-sectional design, comparing people of different ages at a single point in time. This means the observed age-related differences in metacognitive beliefs about errors and challenge could reflect cohort differences (differences between generations based on their unique experiences, including different educational pedagogies) rather than purely developmental changes that happen naturally with age. Teachers need to consider that older adults’ views on errors might be shaped by educational experiences in earlier decades, which may have been less likely to embrace challenge and errors compared to more recent pedagogies. This suggests that the beliefs might shift over time in the population, even if the cognitive benefit of pretesting remains stable across ages.
- Inter-Individual Variation in Benefit: Although the study found a robust average pretesting benefit, there was substantial variation among participants. Approximately 25% of participants in Study 1 actually scored lower on the pretested items compared to study-only items. The study could not identify whether age or memory contentment explained this variation. This highlights that pretesting is not universally beneficial for every single learner in all instances, and teachers lack specific markers from this study to predict which individual students might not benefit or might even be hindered by the strategy. There are likely other, unmeasured moderators at play.
In summary, while the study provides valuable insights into the robustness of the pretesting effect and age-related differences in metacognitive beliefs, its findings are based on a specific type of task and participant pool, and the design doesn’t fully disentangle age from cohort effects. Teachers should consider these limitations and potentially adapt or supplement the pretesting strategy, particularly by actively addressing learner beliefs about errors and challenge, while being mindful that individual student outcomes may vary.
What is a single quote that summarises the key findings from the paper?
These findings highlight the importance of shifting attitudes towards challenge to support “desirably difficult” approaches to learning, thus ensuring that learners have both the right mindset and the right toolset
