Sample size refers to the number of observations or participants included in a study. A properly chosen sample size ensures that the findings are reliable and representative of the larger population.
Statistical power, on the other hand, is the probability that a study will detect a true effect or difference when it exists. High statistical power reduces the risk of false negatives (Type II errors), increasing confidence in the study’s results.
Determining the right sample size and power is crucial for producing valid and meaningful research outcomes. If the sample size is too small, the study may fail to detect real effects; if too large, it may waste resources without adding significant value.
Six Rules of Thumb for Determining Sample Size and Statistical Power: #
- Smaller effect size needs larger sample: Detecting subtle differences requires more participants.
- Lower variability means smaller sample: Less variation in the data allows for reliable results with fewer observations.
- Higher power requires larger samples: To achieve 80–90% power (common standards), more data may be necessary.
- Lower significance level increases sample size: Stricter significance thresholds (e.g., p<0.01) demand larger samples.
- More complex analyses need more data: Studies with multiple variables or subgroup analyses typically require bigger samples.
- Resource constraints matter: Practical limits such as budget, time, and accessibility should be considered in planning.
Understanding sample size and statistical power is essential for designing strong, credible studies that yield actionable insights.
List of recommended resources #
For a broad overview #
Guidance on Conducting Sample Size and Power Calculations
This slide deck from an Applied Statistics Seminar Series by Lutfiyya N. Muhammad gives a broad overview of power analysis via software and simulations as well as different types of effect sizes.
Statistical Power and Why It Matters | A Simple Introduction
This blog post by Pritha Bhandari for Scribbr gives a brief overview of statistical power and why it matters in hypothesis testing. The blog also discusses factors that affect power and how to increase power.
For in-depth understanding #
This article by J. Uttley presents a case of sample size and power analysis by discussing the challenges in lighting research, emphasizing that small sample sizes and low statistical power limit the detection of small effects. The report also notes the lack of effect size reporting, which hinders cumulative research and complicates power analysis for future studies.
This paper aims to explain the importance of sample size and its relationship to effect size and statistical significance. Furthermore, it aims to assist researchers planning to perform sample size estimations by suggesting and elucidating available alternative software, guidelines and references that will serve different scientific purposes.
Case study #
This report discusses the ongoing debate over mass drug administration (MDA) for deworming, highlighting that while some studies, like Taylor-Robinson et al. (2015), found limited impact, updated analyses reveal that these findings suffer from low statistical power. The revised analysis shows that MDA leads to significant weight gains in children, particularly in high-prevalence areas, making it a highly cost-effective intervention compared to alternatives like school feeding programs.
This report by Christopher Woodruff and David McKenzie discusses the impact of business training programs, noting that many evaluations are limited by small sample sizes, short follow-up periods, and measurement challenges. While modest improvements in business practices are observed, significant effects on profits or sales are rare, except in studies with higher statistical power. The report also highlights the need for further research to clarify the mechanisms behind observed outcomes and inform policy decisions.
