Bootstrap

On this page
  1. Definition
  2. How to compute
  3. Pitfalls

Definition

The bootstrap estimates the distribution of a statistic by repeatedly resampling the data with replacement. Instead of deriving a formula for the standard error, the empirical distribution is built from BB resamples of the same size. It works for statistics with no convenient asymptotics: medians, percentiles, ratio metrics, composite business measures.

How to compute

Draw BB resamples of nn observations with replacement, compute the statistic on each, and collect the distribution. The confidence interval is the percentile interval of that distribution (say 2.5 and 97.5), or BCa to correct for bias and skew. BB is typically 2000–10000.

Pitfalls

The percentile interval is biased on small samples and skewed statistics. Data are dependent by user, not by row, so you must resample users or the interval comes out falsely narrow. The bootstrap does not fix SRM and does not replace a split check. It refines the standard error but does not raise power dramatically.

Related terms

Mentioned in

Connections map

Interactive subgraph: 9 nodes and 18 links around this page. Each node is keyboard-focusable; press Enter or Space to open its page. A full list of nodes and their links follows the graphic.

  • Causal / Uplift — CUPED and Individual Treatment Effects : Correlation and causation (Explicit link), Bootstrap (Explicit link), Uplift modelling (Explicit link), LinkedIn Content Analytics (Lateral link)
  • LinkedIn Content Analytics : Bootstrap (Explicit link), Causal / Uplift — CUPED and Individual Treatment Effects (Lateral link)
  • Bootstrap : LinkedIn Content Analytics (Explicit link), p-value (Explicit link), Uplift modelling (Explicit link), Causal / Uplift — CUPED and Individual Treatment Effects (Explicit link), Confidence interval (Explicit link), Correlation and causation (Explicit link), Sample size (Explicit link), ab-testing (Thematic link)
  • Confidence interval : p-value (Explicit link), Bootstrap (Explicit link), ab-testing (Thematic link)
  • Correlation and causation : Causal / Uplift — CUPED and Individual Treatment Effects (Explicit link), Uplift modelling (Explicit link), Bootstrap (Explicit link), ab-testing (Thematic link)
  • p-value : Bootstrap (Explicit link), Confidence interval (Explicit link), ab-testing (Thematic link)
  • Sample size : Bootstrap (Explicit link), ab-testing (Thematic link)
  • Uplift modelling : Bootstrap (Explicit link), Correlation and causation (Explicit link), Causal / Uplift — CUPED and Individual Treatment Effects (Explicit link), ab-testing (Thematic link)
  • ab-testing : Bootstrap (Thematic link), Confidence interval (Thematic link), Correlation and causation (Thematic link), p-value (Thematic link), Sample size (Thematic link), Uplift modelling (Thematic link)

Terms, articles and projects connected to this concept. Tap or click opens a node.

← All terms