EPPP Statistics Study Aid: Differentiating Threats to Internal Validity

The EPPP loves to test your ability to differentiate between closely related threats to internal validity. If you want to maximize your score, it’s helpful to know how to tell these four commonly confused threats apart when they are buried in a clinical research scenario.

⏱️ 1. Maturation vs. History (The Time Traps)

Both of these threats involve changes that happen over time, but the cause of the change is entirely different.

  • Maturation (Internal Change): This is a change within the participants due to the mere passage of time. Think biological or psychological growth, fatigue, or boredom.

    • Example: Evaluating a reading intervention for 1st graders over a year. If their reading improves, is it your intervention, or did they just naturally grow and mature?

  • History (External Event): This is an unpredicted, external event occurring outside the study that influences the participants' scores.

    • Example: You are testing an anxiety reduction app. Mid-study, a major economic crash or a natural disaster happens. The external event alters everyone's anxiety levels, and alters your data.

📉 2. Statistical Regression vs. Attrition (The Participant Traps)

These threats both alter your group averages, but they do so through different statistical mechanisms.

  • Statistical Regression to the Mean: This happens automatically when participants are selected for a study specifically because of their extreme scores (either incredibly high or incredibly low). On re-testing, their scores will naturally drift back toward the average.

    • Example: Testing a depression treatment using only patients in the top 2% of severe depression. Even without treatment, their scores on a second test are likely to decrease simply due to statistical probability.

  • Attrition / Experimental Mortality: This occurs when a specific type of participant drops out of the study before it ends, systematically biasing your final sample.

    • Example: You run a highly intensive, 12-week CBT program for panic disorder. The most severely anxious patients drop out by week 3 because it’s too stressful. Your post-test data looks amazing, but only because your sickest participants left.

📊 How to Spot Them

  • Look for words like "extreme scores" or "selected because of baseline severity" ➡️ Think Regression to the Mean.

  • Look for words like "dropout rate," "lost to follow-up," or "non-compliance" ➡️ Think Attrition.

  • Look for long-term studies with kids/elderly or mentions of "fatigue/boredom" ➡️ Think Maturation.

  • Look for unexpected "real-world events" occurring during the treatment period ➡️ Think History.