V The V Lab

Propensity Score Matching in Depth

From the causal estimand, design stage, and overlap to balance diagnostics, effect estimation, and transparent reporting

The V Lab Home Biostatistics Foundations Medical Statistical Tests Linear Regression Logistic Regression Multilevel Modelling Causal Inference Propensity Score Matching Survival Analysis AFT and Cox PH 中文版

AudienceLearners in public health, epidemiology, medicine, and health data science

Study timeApproximately 150–210 minutes

PrerequisitesRegression, confidence intervals, causal inference fundamentals, and basic R

This page is the practical deep dive into PSM This tutorial focuses on the design, implementation, and diagnosis of propensity score matching (PSM). See Causal Inference in Depth for the full framework covering potential outcomes, target trials, DAGs, standardization, IPW, and AIPW. Rather than repeating every method, this page carries one PSM analysis from the question through the report.

About the simulated data Every participant, treatment, outcome, and counterfactual truth is generated with a fixed random seed and contains no real personal health information. A real study cannot observe both potential outcomes for the same person. The “truth” shown here is only a teaching check of the workflow, not real-world evidence.

How to use this tutorial

Work through the material in this order:

Define the causal question and estimand → lock the pretreatment variables → estimate propensity scores → match → check overlap, sample flow, and balance → freeze the design → analyze outcomes → conduct sensitivity analyses and report

This is a “design first, outcomes later” workflow. All primary code relies only on base R, MatchIt, and knitr for rendering the page. The example outcome is a continuous six-month clinical measure for which lower values are better.

Learning objectives

After completing this tutorial, you should be able to:

• explain the propensity score and its balancing role without mistaking it for a causal effect;
• distinguish ATE, ATT, and ATC, and explain how matching failures or discarding units change the target population;
• state consistency, exchangeability, positivity, and no-interference assumptions;
• select variables from temporal order and causal structure while avoiding posttreatment variables and colliders;
• use glm() to understand the PS model and MatchIt::matchit() to perform 1:1 nearest-neighbor matching;
• correctly interpret link = "linear.logit", a 0.2 SD caliper, replacement, and ratio;
• audit sample flow, propensity-score overlap, SMDs, variance ratios, and eCDFs before and after matching;
• retain pair structure with subclass and estimate the matched-sample ATT with a confidence interval;
• separate design diagnostics from outcome analysis and avoid choosing a design by significance or outcomes;
• adapt PSM to binary, count, and time-to-event outcomes while recognizing their different inferential requirements;
• explain why unmeasured confounding, missing data, and time-varying treatment lie beyond ordinary baseline PSM; and
• write an auditable report that covers the target population, sample loss, balance, and limitations.

1 What problem does PSM address?

1.1 From potential outcomes to the estimand

Let $A\in\{0,1\}$ indicate receipt of the intervention, and let $Y(1)$ and $Y(0)$ be the potential outcomes for the same individual under the two strategies. Common targets include:

$$\operatorname{ATE}=E\{Y(1)-Y(0)\}, \qquad \operatorname{ATT}=E\{Y(1)-Y(0)\mid A=1\}.$$

The ATE concerns the full target population, the ATT concerns those who actually received treatment, and the ATC concerns those who actually received control. They can differ when effects are heterogeneous or regions of support differ. PSM is not an estimand-free button: the direction of matching, whether units are discarded, and whether controls can be reused must all serve the prespecified target.

The primary question in this tutorial is:

Among observed intervention recipients for whom a suitable control can be found, how much would their mean six-month outcome differ under the intervention rather than control management?

The initial intent is an ATT. Because the caliper leaves some intervention recipients unmatched, the final identifiable target is more precisely the ATT among retained intervention recipients, or matched-sample ATT. This narrowing of the target population must be reported; writing only “we estimated the ATT” is not enough.

1.2 Definition and balancing role of the propensity score

For pretreatment covariates $X$, the propensity score is:

$$e(X)=P(A=1\mid X).$$

If treatment assignment is exchangeable with the potential outcomes given $X$, and positivity holds, outcomes can be compared after propensity scores are used correctly to form comparable groups. Their central role is to compress a set of variables involved in treatment assignment to help construct a balanced design. A PS neither predicts who “should” be treated nor represents an individual causal effect.

PSM usually pairs treated and control units with similar $e(X)$ or logit $\{e(X)\}$. Success is not defined by the PS model’s AUC, likelihood-ratio test, or coefficient p-values. It is defined by adequate postmatch balance of pretreatment covariates, a clear target sample, and credible overlap.

Check your understanding: Is a PS model better when it predicts treatment more accurately?

Not necessarily. Very strong discrimination can indicate that the groups have almost no common support. The PS model is a design tool; evaluation should emphasize covariate balance and overlap after matching rather than maximizing predictive accuracy.

1.3 Identification assumptions cannot be tested by software

Assumption	Meaning in this question	What you can do
Consistency	“Intervention” and “control” are sufficiently well-defined versions, and the observed outcome equals the potential outcome under the strategy actually received	Specify dose, start time, adherence, and treatment versions
Conditional exchangeability	After conditioning on selected pretreatment variables, no remaining common cause affects both treatment and outcome	Use temporal order, DAGs, subject-matter knowledge, and sensitivity analyses
Positivity	Both strategies can occur for covariate profiles in the target population	Plot overlap, inspect extreme PS values, and restrict the target population when needed
No interference	One person’s treatment does not change another person’s outcome	Explain whether transmission, institutional, or network effects may occur
Adequate measurement and modeling	Treatment, outcome, and key covariates are measured adequately, and the design model can form balanced groups	Audit data, add defensible flexible terms, and diagnose balance

Matching can address baseline confounding that is measured and used correctly. It cannot create randomization or test whether every confounder was measured.

2 Design stage: keep outcomes hidden

2.1 Select variables from causal structure, not p-values

After defining time zero, a PS model should use only variables known before treatment starts. Prioritize:

• common causes of treatment and outcome;
• strong outcome predictors, even when their sample association with treatment is weak; and
• reliable proxies for common causes, clinical decisions, and enrollment mechanisms.

Use caution with or avoid:

• posttreatment mediators;
• colliders jointly affected by treatment and causes of the outcome;
• strong instrument-like variables that affect treatment but barely affect the outcome, because they can worsen overlap and amplify residual confounding; and
• variables retained only because of a small univariable p-value, convenient missingness pattern, or stepwise selection.

This example uses age, BMI, smoking, baseline severity, and a pretreatment biomarker. The outcome is absent from psm_design_data and will not be used to select the caliper, ratio, or PS formula.

stopifnot(
  nrow(psm_design_data) == 1200,
  all(psm_design_data$treatment_num %in% c(0, 1)),
  !anyNA(psm_design_data),
  !"outcome" %in% names(psm_design_data)
)

data_preview <- transform(
  head(psm_design_data, 6),
  treatment = ifelse(treatment == "Intervention", "Intervention", "Control"),
  smoker = ifelse(smoker == "Yes", "Yes", "No"),
  severity = c(
    Mild = "Mild", Moderate = "Moderate", Severe = "Severe"
  )[as.character(severity)]
)

knitr::kable(
  data_preview[, c(
    "id", "treatment", "age", "bmi", "smoker", "severity", "biomarker"
  )],
  col.names = c(
    "ID", "Observed strategy", "Age", "BMI", "Current smoker",
    "Baseline severity", "Standardized biomarker"
  ),
  caption = "First six rows of the design-stage data, with no outcome"
)

First six rows of the design-stage data, with no outcome

ID	Observed strategy	Age	BMI	Current smoker	Baseline severity	Standardized biomarker
1	Control	55.9	18.8	Yes	Mild	0.91
2	Intervention	66.5	23.4	Yes	Mild	0.15
3	Control	51.8	40.4	Yes	Moderate	0.62
4	Control	59.4	23.9	Yes	Mild	-0.19
5	Intervention	75.5	20.4	No	Mild	0.21
6	Control	65.4	30.9	No	Severe	-2.38

cohort_audit <- data.frame(
  `Total sample` = nrow(psm_design_data),
  Control = sum(psm_design_data$treatment_num == 0),
  Intervention = sum(psm_design_data$treatment_num == 1),
  `Intervention proportion` = pct(mean(psm_design_data$treatment_num == 1)),
  check.names = FALSE
)

knitr::kable(cohort_audit, caption = "Prematch cohort audit")

Prematch cohort audit

Total sample	Control	Intervention	Intervention proportion
1200	710	490	40.8%

The cohort contains 490 intervention recipients and 710 controls. Matching should begin only after confirming variable meanings, units, coding, temporal order, duplicate records, and missingness.

2.2 Estimate the PS with logistic regression

The most common working model is:

$$\operatorname{logit}\{e(X)\} =\log\frac{e(X)}{1-e(X)} =\beta_0+\beta^T X.$$

Fitting an ordinary glm() first makes the estimated quantity transparent. matchit() will then use the same formula to construct the distance.

ps_model <- glm(
  treatment_num ~ age + bmi + smoker + severity + biomarker,
  data = psm_design_data,
  family = binomial()
)

psm_design_data$ps_probability <- predict(ps_model, type = "response")
psm_design_data$ps_logit <- predict(ps_model, type = "link")

ps_coefficient_table <- data.frame(
  Term = names(coef(ps_model)),
  `Logit coefficient` = unname(coef(ps_model)),
  check.names = FALSE
)

knitr::kable(
  ps_coefficient_table,
  digits = 3,
  caption = "Coefficients from the logistic working model for treatment assignment"
)

Coefficients from the logistic working model for treatment assignment

Term	Logit coefficient
(Intercept)	-7.602
age	0.066
bmi	0.093
smokerYes	0.806
severityModerate	0.413
severitySevere	1.171
biomarker	0.270

Linearity on the logit scale for continuous variables, interactions, and missing-data handling are all modeling choices. If subject-matter knowledge or postmatch covariate diagnostics reveal inadequate balance, prespecified defensible nonlinear terms such as I(age^2) or interactions among pretreatment variables can be added without viewing outcomes. Do not remove variables because treatment-model coefficients are nonsignificant.

Do not tune matching to the outcome If formulas, calipers, or algorithms are tried repeatedly and the design with the largest outcome difference or smallest p-value is selected, the outcome has contaminated the design stage. Freeze the design using covariate balance, overlap, sample retention, and prespecified clinical comparability; then reveal the outcome once.

3 Perform 1:1 nearest-neighbor matching in R

3.1 Primary specification: ATT, nearest neighbor, and a 0.2 SD caliper

The primary design uses:

• estimand = "ATT": start from each intervention recipient and search for a control;
• method = "nearest": nearest-neighbor matching;
• ratio = 1: match one control to each retained intervention recipient;
• replace = FALSE: do not reuse a control;
• link = "linear.logit": match on the linear predictor, or logit, of the PS;
• caliper = 0.20, std.caliper = TRUE: require two units’ logit distance to be no greater than 0.2 standard deviations of that distance; and
• m.order = "closest": in greedy nearest-neighbor matching, prioritize currently closer candidate pairs. This is not optimal matching that minimizes total pair distance globally.

m.out <- MatchIt::matchit(
  treatment_num ~ age + bmi + smoker + severity + biomarker,
  data = psm_design_data,
  method = "nearest",
  distance = "glm",
  link = "linear.logit",
  estimand = "ATT",
  ratio = 1,
  replace = FALSE,
  caliper = 0.20,
  std.caliper = TRUE,
  m.order = "closest"
)

stopifnot(
  isTRUE(all.equal(unname(m.out$distance), psm_design_data$ps_logit))
)

match_summary <- summary(m.out, un = TRUE, standardize = TRUE)
matched_design <- MatchIt::match_data(m.out, data = psm_design_data)

Here m.out$distance is the logit linear predictor, not a probability between 0 and 1. Use plogis(m.out$distance) when a probability is needed. The caliper therefore also operates on the logit scale. Changing link changes the distance scale, so the value 0.2 should never be copied without its definition.

3.2 Inspect common support before outcomes

ps_probability <- plogis(m.out$distance)
control_density <- density(
  ps_probability[psm_design_data$treatment_num == 0],
  from = 0,
  to = 1
)
treated_density <- density(
  ps_probability[psm_design_data$treatment_num == 1],
  from = 0,
  to = 1
)

plot(
  control_density,
  col = palette_psm["orange"],
  lwd = 2.4,
  xlim = c(0, 1),
  ylim = c(0, max(control_density$y, treated_density$y)),
  xlab = "Estimated propensity score",
  ylab = "Density",
  main = "Common support before matching",
  las = 1
)
lines(treated_density, col = palette_psm["teal"], lwd = 2.4)
legend(
  "topright",
  legend = c("Control", "Intervention"),
  col = c(palette_psm["orange"], palette_psm["teal"]),
  lwd = 2.4,
  bty = "n"
)

Estimated propensity-score distributions for intervention and control groups before matching. The groups share support, but the intervention distribution is shifted toward higher probabilities, making units in the tails harder to compare.

Having overlapping PS ranges is only a minimum condition; it does not establish credible positivity everywhere. Assess densities, local sample sizes, specific covariate combinations, and the characteristics of excluded units. If a few controls in a tail must support many intervention recipients, inference may rely on fragile extrapolation even when the ranges overlap.

3.3 Sample flow and matching distance

sample_flow_raw <- match_summary$nn[
  c("All", "Matched", "Unmatched", "Discarded"),
  ,
  drop = FALSE
]
sample_flow <- data.frame(
  Stage = c(
    "Original sample", "Successfully matched",
    "Unmatched", "Discarded before matching"
  ),
  Control = sample_flow_raw[, "Control"],
  Intervention = sample_flow_raw[, "Treated"],
  check.names = FALSE
)

knitr::kable(sample_flow, caption = "Sample flow for 1:1 nearest-neighbor matching")

Sample flow for 1:1 nearest-neighbor matching

	Stage	Control	Intervention
All	Original sample	710	490
Matched	Successfully matched	375	375
Unmatched	Unmatched	335	115
Discarded	Discarded before matching	0	0

pair_groups_design <- split(matched_design, matched_design$subclass)
pair_logit_differences <- vapply(
  pair_groups_design,
  function(x) {
    abs(
      x$distance[x$treatment_num == 1] -
        x$distance[x$treatment_num == 0]
    )
  },
  numeric(1)
)
caliper_width <- 0.20 * sd(m.out$distance)

distance_audit <- data.frame(
  Pairs = length(pair_logit_differences),
  `Caliper width` = caliper_width,
  `Largest observed pair distance` = max(pair_logit_differences),
  `Median observed pair distance` = median(pair_logit_differences),
  check.names = FALSE
)

knitr::kable(
  distance_audit,
  digits = 3,
  caption = "Caliper and observed pair distances on the logit PS scale"
)

Caliper and observed pair distances on the logit PS scale

Pairs	Caliper width	Largest observed pair distance	Median observed pair distance
375	0.203	0.202	0.002

The primary specification forms 375 pairs and retains 375 intervention recipients with the same number of controls. 115 intervention recipients have no available control within the caliper. The largest observed distance is 0.202, below the caliper of 0.203.

Sample loss changes more than precision The 115 unmatched intervention recipients are no longer represented by the primary comparison. The primary result therefore concerns the 375 retained recipients; it does not automatically generalize to all 490 intervention recipients. Compare the baseline characteristics of retained and unmatched recipients, and identify the target population explicitly in titles, tables, and discussion.

retained_treated_ids <- matched_design$id[matched_design$treatment_num == 1]
treated_population_table <- rbind(
  `All intervention recipients` = c(
    n = sum(psm_design_data$treatment_num == 1),
    mean_age = mean(psm_design_data$age[psm_design_data$treatment_num == 1]),
    mean_bmi = mean(psm_design_data$bmi[psm_design_data$treatment_num == 1]),
    severe = mean(
      psm_design_data$severity[psm_design_data$treatment_num == 1] == "Severe"
    )
  ),
  `Retained intervention recipients` = c(
    n = length(retained_treated_ids),
    mean_age = mean(matched_design$age[matched_design$treatment_num == 1]),
    mean_bmi = mean(matched_design$bmi[matched_design$treatment_num == 1]),
    severe = mean(
      matched_design$severity[matched_design$treatment_num == 1] == "Severe"
    )
  ),
  `Unmatched intervention recipients` = {
    u <- psm_design_data$treatment_num == 1 &
      !psm_design_data$id %in% retained_treated_ids
    c(
      n = sum(u),
      mean_age = mean(psm_design_data$age[u]),
      mean_bmi = mean(psm_design_data$bmi[u]),
      severe = mean(psm_design_data$severity[u] == "Severe")
    )
  }
)

treated_population_display <- data.frame(
  Population = rownames(treated_population_table),
  Number = treated_population_table[, "n"],
  `Mean age` = treated_population_table[, "mean_age"],
  `Mean BMI` = treated_population_table[, "mean_bmi"],
  `Severe proportion` = pct(treated_population_table[, "severe"]),
  check.names = FALSE
)

knitr::kable(
  treated_population_display,
  digits = 1,
  caption = "Baseline characteristics of all, retained, and unmatched intervention recipients"
)

Baseline characteristics of all, retained, and unmatched intervention recipients

	Population	Number	Mean age	Mean BMI	Severe proportion
All intervention recipients	All intervention recipients	490	62.9	29.2	23.7%
Retained intervention recipients	Retained intervention recipients	375	61.0	28.7	20.3%
Unmatched intervention recipients	Unmatched intervention recipients	115	69.0	30.8	34.8%

4 Balance diagnostics: did matching actually work?

4.1 SMDs are central; baseline p-values are not

For a continuous variable, the standardized mean difference can be written as:

$$\operatorname{SMD} =\frac{\bar X_1-\bar X_0}{s^*},$$

where $s^*$ is a predefined standardization scale. For the ATT design on this page, MatchIt uses the standard deviation in the original intervention group as the SMD denominator and retains the same denominator before and after matching. The reference scale can differ for ATE or ATC designs. Categorical variables are usually expanded into indicator variables and checked separately. An SMD does not shrink mechanically with sample size, so it is useful for describing design balance. A baseline significance test asks whether population means could be equal, not whether an observed difference is large enough to confound the analysis.

An absolute SMD below 0.1 is a common heuristic, not an automatic certificate. Important variables may require a stricter standard, and mean balance does not guarantee balance in tails, nonlinear terms, or interactions.

balance_terms <- setdiff(rownames(match_summary$sum.all), "distance")
balance_labels <- c(
  age = "Age",
  bmi = "BMI",
  smokerNo = "Nonsmoker",
  smokerYes = "Smoker",
  severityMild = "Mild",
  severityModerate = "Moderate",
  severitySevere = "Severe",
  biomarker = "Biomarker"
)

balance_table <- data.frame(
  Covariate = unname(balance_labels[balance_terms]),
  `SMD before` = match_summary$sum.all[balance_terms, "Std. Mean Diff."],
  `SMD after` = match_summary$sum.matched[
    balance_terms, "Std. Mean Diff."
  ],
  `Variance ratio before` = match_summary$sum.all[balance_terms, "Var. Ratio"],
  `Variance ratio after` = match_summary$sum.matched[
    balance_terms, "Var. Ratio"
  ],
  `Maximum eCDF difference before` = match_summary$sum.all[
    balance_terms, "eCDF Max"
  ],
  `Maximum eCDF difference after` = match_summary$sum.matched[
    balance_terms, "eCDF Max"
  ],
  check.names = FALSE
)

max_smd_before <- max(abs(balance_table$`SMD before`), na.rm = TRUE)
max_smd_after <- max(abs(balance_table$`SMD after`), na.rm = TRUE)

knitr::kable(
  balance_table,
  digits = 3,
  caption = "Balance diagnostics for pretreatment covariates before and after matching"
)

Balance diagnostics for pretreatment covariates before and after matching

	Covariate	SMD before	SMD after	Variance ratio before	Variance ratio after	Maximum eCDF difference before	Maximum eCDF difference after
age	Age	0.566	-0.017	0.992	0.958	0.237	0.035
bmi	BMI	0.371	-0.009	1.049	1.052	0.148	0.040
smokerNo	Nonsmoker	-0.312	0.000	NA	NA	0.151	0.000
smokerYes	Smoker	0.312	0.000	NA	NA	0.151	0.000
severityMild	Mild	-0.296	-0.066	NA	NA	0.144	0.032
severityModerate	Moderate	0.076	0.033	NA	NA	0.037	0.016
severitySevere	Severe	0.252	0.038	NA	NA	0.107	0.016
biomarker	Biomarker	0.250	-0.031	0.991	1.052	0.131	0.040

The largest absolute SMD among pretreatment covariates falls from 0.566 to 0.066. The distance row is intentionally excluded from this “largest covariate” calculation: balancing the PS itself cannot replace diagnostics for every covariate.

love_before <- abs(balance_table$`SMD before`)
love_after <- abs(balance_table$`SMD after`)
love_order <- order(love_before)
love_y <- seq_along(love_order)

plot(
  love_before[love_order],
  love_y,
  pch = 16,
  col = palette_psm["orange"],
  xlim = c(0, max(love_before) * 1.08),
  yaxt = "n",
  xlab = "Absolute standardized mean difference",
  ylab = "",
  main = "Covariate balance before and after matching",
  las = 1
)
points(
  love_after[love_order],
  love_y,
  pch = 17,
  col = palette_psm["teal"]
)
axis(2, at = love_y, labels = balance_table$Covariate[love_order], las = 1)
abline(v = 0.10, lty = 2, col = palette_psm["vermillion"])
legend(
  "bottomright",
  legend = c("Before matching", "After matching", "|SMD|=0.10"),
  col = c(
    palette_psm["orange"],
    palette_psm["teal"],
    palette_psm["vermillion"]
  ),
  pch = c(16, 17, NA),
  lty = c(NA, NA, 2),
  bty = "n"
)

Absolute standardized mean differences for pretreatment covariates before and after matching (Love plot). The dashed 0.10 line is a common heuristic, not a sufficient causal condition.

4.2 Variance ratios and eCDFs supplement mean balance

The variance ratio (VR) compares dispersion between groups. Here MatchIt reports the intervention-group variance divided by the control-group variance and applies the corresponding matching weights after matching. The ideal value for a continuous variable is near 1. Ranges such as 0.5–2, or the stricter 0.8–1.25, can flag concerns but should not determine causal validity mechanically. A binary indicator’s variance is determined by its mean, so its VR may be blank in the table; that is not a computational failure.

Empirical cumulative distribution functions (eCDFs) compare distributions across the entire observed range. eCDF Max is the largest vertical separation between the two eCDFs; values closer to zero are better. It can detect settings in which means agree but distributions do not.

old_par <- par(mfrow = c(1, 2), mar = c(4.2, 4.2, 3, 1))

plot(
  ecdf(psm_design_data$age[psm_design_data$treatment_num == 0]),
  col = palette_psm["orange"],
  lwd = 2.2,
  verticals = TRUE,
  do.points = FALSE,
  main = "Before matching",
  xlab = "Age",
  ylab = "Empirical cumulative probability",
  las = 1
)
lines(
  ecdf(psm_design_data$age[psm_design_data$treatment_num == 1]),
  col = palette_psm["teal"],
  lwd = 2.2,
  verticals = TRUE,
  do.points = FALSE
)

plot(
  ecdf(matched_design$age[matched_design$treatment_num == 0]),
  col = palette_psm["orange"],
  lwd = 2.2,
  verticals = TRUE,
  do.points = FALSE,
  main = "After matching",
  xlab = "Age",
  ylab = "Empirical cumulative probability",
  las = 1
)
lines(
  ecdf(matched_design$age[matched_design$treatment_num == 1]),
  col = palette_psm["teal"],
  lwd = 2.2,
  verticals = TRUE,
  do.points = FALSE
)
legend(
  "bottomright",
  legend = c("Control", "Intervention"),
  col = c(palette_psm["orange"], palette_psm["teal"]),
  lwd = 2.2,
  bty = "n"
)

Empirical cumulative distributions of age before and after matching. The two step curves are much closer after matching, supplementing the mean-level SMD diagnostic.

par(old_par)

continuous_vr_before <- balance_table$`Variance ratio before`[
  is.finite(balance_table$`Variance ratio before`)
]
continuous_vr_after <- balance_table$`Variance ratio after`[
  is.finite(balance_table$`Variance ratio after`)
]

balance_summary <- data.frame(
  Diagnostic = c(
    "Largest covariate |SMD|",
    "Continuous-variable variance-ratio range",
    "Largest covariate eCDF Max"
  ),
  Before = c(
    sprintf("%.3f", max_smd_before),
    sprintf(
      "%.3f–%.3f",
      min(continuous_vr_before),
      max(continuous_vr_before)
    ),
    sprintf("%.3f", max(balance_table$`Maximum eCDF difference before`))
  ),
  After = c(
    sprintf("%.3f", max_smd_after),
    sprintf(
      "%.3f–%.3f",
      min(continuous_vr_after),
      max(continuous_vr_after)
    ),
    sprintf("%.3f", max(balance_table$`Maximum eCDF difference after`))
  ),
  check.names = FALSE
)

knitr::kable(balance_summary, caption = "Overall balance summary for the primary specification")

Overall balance summary for the primary specification

Diagnostic	Before	After
Largest covariate |SMD|	0.566	0.066
Continuous-variable variance-ratio range	0.991–1.049	0.958–1.052
Largest covariate eCDF Max	0.237	0.040

4.3 PS distributions after matching

matched_probability <- plogis(matched_design$distance)
matched_control_density <- density(
  matched_probability[matched_design$treatment_num == 0],
  from = 0,
  to = 1
)
matched_treated_density <- density(
  matched_probability[matched_design$treatment_num == 1],
  from = 0,
  to = 1
)

plot(
  matched_control_density,
  col = palette_psm["orange"],
  lwd = 2.4,
  xlim = c(0, 1),
  ylim = c(0, max(matched_control_density$y, matched_treated_density$y)),
  xlab = "Estimated propensity score",
  ylab = "Density",
  main = "Region of support after matching",
  las = 1
)
lines(
  matched_treated_density,
  col = palette_psm["teal"],
  lwd = 2.4
)
legend(
  "topright",
  legend = c("Matched controls", "Retained intervention recipients"),
  col = c(palette_psm["orange"], palette_psm["teal"]),
  lwd = 2.4,
  bty = "n"
)

Estimated propensity-score distributions for intervention and control groups in the matched sample. The distributions are highly similar within the retained region of support.

Check your understanding: If the PS is balanced, can the original covariates be ignored?

No. Different covariate combinations can share a similar PS. Every prespecified pretreatment covariate must be checked individually, along with nonlinear terms, interactions, tails, and clinically important strata when appropriate.

5 Analyze outcomes only after freezing the design

5.1 Retain subclass and weights

MatchIt::match_data() returns the original variables plus:

• distance: the matching distance;
• weights: analysis weights produced by the matching design; and
• subclass: the matched-set identifier.

The primary design uses 1:1 matching without replacement, so every retained unit has weight 1 and each subclass contains exactly one intervention recipient and one control. Under a different ratio, replacement, or another matching method, weights cannot be assumed to equal 1 and subclass cannot be discarded in favor of an ordinary independent-samples analysis.

# Balance diagnostics have passed and the design is frozen. Only now merge the
# outcome into the matched sample.
matched_outcome <- MatchIt::match_data(m.out, data = psm_data)

subclass_audit <- aggregate(
  treatment_num ~ subclass,
  data = matched_outcome,
  FUN = function(z) c(n = length(z), treated = sum(z))
)

stopifnot(
  nrow(matched_outcome) == 750,
  all(matched_outcome$weights == 1),
  all(vapply(
    split(matched_outcome$treatment_num, matched_outcome$subclass),
    function(z) length(z) == 2 && sum(z) == 1,
    logical(1)
  ))
)

matched_preview <- transform(
  head(matched_outcome[order(matched_outcome$subclass), ], 8),
  treatment = ifelse(treatment == "Intervention", "Intervention", "Control")
)

knitr::kable(
  matched_preview[, c(
    "id", "treatment", "outcome", "distance", "weights", "subclass"
  )],
  col.names = c(
    "ID", "Strategy", "Six-month outcome", "Logit PS", "Weight", "Pair ID"
  ),
  digits = 3,
  caption = "Matched outcome data revealed after freezing the design"
)

Matched outcome data revealed after freezing the design

	ID	Strategy	Six-month outcome	Logit PS	Weight	Pair ID
2	2	Intervention	128	-0.170	1	1
32	32	Control	134	-0.199	1	1
5	5	Intervention	124	-0.640	1	2
810	810	Control	128	-0.601	1	2
11	11	Intervention	132	-0.845	1	3
725	725	Control	140	-0.845	1	3
14	14	Intervention	134	0.032	1	4
791	791	Control	158	-0.031	1	4

5.2 1:1 paired ATT and confidence interval

For matched pair $j$, calculate:

$$D_j=Y_{1j}-Y_{0j}, \qquad \widehat{ATT}_{matched}=\frac{1}{J}\sum_{j=1}^{J}D_j.$$

Treating the pair as the independent unit gives a standard error of $s_D/\sqrt J$ for the mean. This simple inference applies to the present 1:1 design without replacement; it cannot be copied unchanged to matching with replacement, many-to-one matching, or weighted matching.

matched_pairs <- split(matched_outcome, matched_outcome$subclass)
pair_differences <- vapply(
  matched_pairs,
  function(x) {
    x$outcome[x$treatment_num == 1] -
      x$outcome[x$treatment_num == 0]
  },
  numeric(1)
)

matched_att <- mean(pair_differences)
matched_att_se <- sd(pair_differences) / sqrt(length(pair_differences))
matched_att_ci <- matched_att +
  qt(c(0.025, 0.975), df = length(pair_differences) - 1) * matched_att_se
matched_att_p <- 2 * pt(
  -abs(matched_att / matched_att_se),
  df = length(pair_differences) - 1
)

naive_difference <- with(
  psm_data,
  mean(outcome[treatment_num == 1]) - mean(outcome[treatment_num == 0])
)

true_full_att <- mean(
  psm_data$individual_effect[psm_data$treatment_num == 1]
)
true_matched_att <- mean(
  matched_outcome$individual_effect[matched_outcome$treatment_num == 1]
)

effect_table <- data.frame(
  Analysis = c(
    "Unadjusted raw mean difference",
    "Matched-sample ATT after 1:1 pairing",
    "Simulated truth: ATT among all intervention recipients",
    "Simulated truth: ATT among retained intervention recipients"
  ),
  `Estimate or truth` = c(
    naive_difference,
    matched_att,
    true_full_att,
    true_matched_att
  ),
  `95% CI lower` = c(NA, matched_att_ci[1], NA, NA),
  `95% CI upper` = c(NA, matched_att_ci[2], NA, NA),
  check.names = FALSE
)

knitr::kable(
  effect_table,
  digits = 2,
  caption = "Unadjusted comparison, matched estimate, and simulated truths shown only for teaching"
)

Unadjusted comparison, matched estimate, and simulated truths shown only for teaching

Analysis	Estimate or truth	95% CI lower	95% CI upper
Unadjusted raw mean difference	-0.54	NA	NA
Matched-sample ATT after 1:1 pairing	-6.27	-7.26	-5.28
Simulated truth: ATT among all intervention recipients	-6.12	NA	NA
Simulated truth: ATT among retained intervention recipients	-6.18	NA	NA

The unadjusted mean difference is only -0.54: higher-risk participants are more likely to receive the intervention, substantially obscuring its outcome-lowering effect. The matched estimate is -6.27 (95% CI -7.26 to -5.28; p <0.001), close to the simulated truth of -6.18 among retained intervention recipients.

Under the assumptions that measured variables are sufficient for conditional exchangeability, positivity holds in the retained postmatch population, treatment and outcome are correctly defined, and interference is absent, receiving the intervention lowers the mean six-month outcome among retained intervention recipients by approximately 6.27 units. Lower outcomes are better, so a negative difference favors the intervention. This conclusion concerns successfully matched recipients; it does not automatically represent unmatched intervention recipients or the full original cohort.

5.2.1 The confidence interval does not contain every source of uncertainty

The paired t interval treats the realized pairs as fixed and relies on independence across pairs plus a mean-based approximation. It does not automatically quantify:

• uncertainty in the PS model and design choices;
• unmeasured confounding and measurement error;
• extrapolation uncertainty from excluding people without overlap;
• additional clustering by institution, household, or clinician; or
• selection uncertainty from trying matching specifications repeatedly in a data-driven way.

Do not translate a narrow confidence interval into “confounding has been eliminated.” For complex matching, clustered data, or small samples, prespecify robust or clustered inference that is compatible with the design and seek statistical consultation when needed.

6 Design choices and sensitivity analyses

6.1 Tradeoffs among caliper, ratio, and replacement

Choice	Main benefit	Main cost and inferential caution
Narrower caliper	Produces closer pairs and often improves local comparability	Leaves more intervention recipients unmatched, narrows the target population, and reduces precision
Wider caliper	Retains more people	May accept poorer pairs and increase residual imbalance
ratio > 1	Uses more controls per intervention recipient and may improve precision	Additional controls are usually farther away; weights can differ when set sizes differ and must be used as returned by the software
Replacement	Allows scarce high-quality controls to be reused and can improve comparability	Reduces effective sample size and requires attention to repeated controls, clustering, and weights
Exact matching	Forces key categorical variables to agree exactly	Can discard many units; all other variables still require diagnosis

Do not choose these options from outcomes. The comparison below uses only prespecified design diagnostics and reads no outcomes.

fit_design <- function(caliper = 0.20, ratio = 1, replace = FALSE) {
  MatchIt::matchit(
    treatment_num ~ age + bmi + smoker + severity + biomarker,
    data = psm_design_data,
    method = "nearest",
    distance = "glm",
    link = "linear.logit",
    estimand = "ATT",
    ratio = ratio,
    replace = replace,
    caliper = caliper,
    std.caliper = TRUE,
    m.order = "closest"
  )
}

design_fits <- list(
  `1:1, no replacement, 0.10 SD` = fit_design(0.10, 1, FALSE),
  `1:1, no replacement, 0.20 SD (primary)` = m.out,
  `1:1, no replacement, 0.30 SD` = fit_design(0.30, 1, FALSE),
  `Up to 1:2, no replacement, 0.20 SD` = suppressWarnings(
    fit_design(0.20, 2, FALSE)
  ),
  `1:1, with replacement, 0.20 SD` = fit_design(0.20, 1, TRUE)
)

extract_design_diagnostics <- function(fit) {
  s <- summary(fit, un = TRUE, standardize = TRUE)
  covariate_rows <- setdiff(rownames(s$sum.matched), "distance")
  c(
    retained_treated = s$nn["Matched", "Treated"],
    used_controls = sum(
      fit$weights[psm_design_data$treatment_num == 0] > 0
    ),
    max_smd = max(
      abs(s$sum.matched[covariate_rows, "Std. Mean Diff."]),
      na.rm = TRUE
    ),
    max_weight = max(fit$weights)
  )
}

design_sensitivity_matrix <- do.call(
  rbind,
  lapply(design_fits, extract_design_diagnostics)
)
design_sensitivity_table <- data.frame(
  `Design specification` = rownames(design_sensitivity_matrix),
  `Retained intervention recipients` = design_sensitivity_matrix[
    , "retained_treated"
  ],
  `Distinct controls used` = design_sensitivity_matrix[, "used_controls"],
  `Largest covariate absolute SMD` = design_sensitivity_matrix[, "max_smd"],
  `Largest matching weight` = design_sensitivity_matrix[, "max_weight"],
  check.names = FALSE
)

knitr::kable(
  design_sensitivity_table,
  digits = 3,
  caption = "Comparison of prespecified matching designs without viewing outcomes"
)

Comparison of prespecified matching designs without viewing outcomes

	Design specification	Retained intervention recipients	Distinct controls used	Largest covariate absolute SMD	Largest matching weight
1:1, no replacement, 0.10 SD	1:1, no replacement, 0.10 SD	372	372	0.066	1.00
1:1, no replacement, 0.20 SD (primary)	1:1, no replacement, 0.20 SD (primary)	375	375	0.066	1.00
1:1, no replacement, 0.30 SD	1:1, no replacement, 0.30 SD	379	379	0.060	1.00
Up to 1:2, no replacement, 0.20 SD	Up to 1:2, no replacement, 0.20 SD	375	509	0.046	1.36
1:1, with replacement, 0.20 SD	1:1, with replacement, 0.20 SD	489	270	0.055	6.07

Each specification could be a reasonable design; the choice depends on the estimand, key covariates, sample retention, and precision. Under the “up to 1:2” specification, the software retains a 1:1 match when only one control is available within the caliper, so it does not guarantee exactly two controls in every subclass. The primary specification achieves balance inside the prespecified 0.10 SMD reference while retaining 375 intervention recipients. Outcome analysis occurs only after the design is frozen.

6.2 Use weights correctly under different ratios and replacement

The weights in match_data() encode the target sample constructed by MatchIt. Under 1:k matching, multiple controls in a subclass usually share the control weight. With replacement, the same original control may serve several intervention recipients. A common error is to run one unweighted lm() on extracted data and thereby discard the original design.

# Example: retain MatchIt weights and subclass after matching up to 1:2.
m_ratio2 <- design_fits[["Up to 1:2, no replacement, 0.20 SD"]]
d_ratio2 <- MatchIt::match_data(m_ratio2, data = psm_data)

# Matching weights can be used for the point estimate. The standard error must
# remain compatible with subclass and any reuse structure.
fit_ratio2 <- lm(outcome ~ treatment_num,
                 data = d_ratio2,
                 weights = weights)

# With replacement, get_matches() can expand every use in a matched set; the
# same source_row can therefore appear repeatedly.
m_replace <- design_fits[["1:1, with replacement, 0.20 SD"]]
d_replace <- MatchIt::get_matches(
  m_replace,
  data = psm_data,
  id = "source_row"
)

The lm() above only illustrates how to retain weights for the point estimate; it does not automatically provide a valid design-compatible standard error. Real analyses need subclass- or person-clustered robust variance, or other inference appropriate for the matching method. A reused control must never be treated as several independent people.

6.3 What if balance is still inadequate?

While outcomes remain hidden, work through these steps:

1. Recheck variable coding, units, timing, and missingness.
2. Identify the specific variables, tails, or interactions that remain imbalanced.
3. Use subject-matter knowledge to add nonlinear terms or interactions to the PS.
4. Use exact matching for key categorical variables, or revise the caliper or matching method.
5. If support is fundamentally absent, narrow the target population and report that choice rather than forcing matches.
6. Repeat the full balance and sample-flow assessment before freezing the design.

Deleting an imbalanced variable from the table does not “improve” balance. If defensible design changes still cannot form comparable groups, PSM may not be suitable for these data.

7 Adapting PSM to different outcome types

Matching designs comparable groups; it does not determine the outcome model. The outcome scale must follow the research question.

7.1 Continuous outcomes

The present 1:1 pairs can be analyzed directly through pair differences. For a severely skewed outcome, report the distribution of paired differences, a robust location measure, or a prespecified transformation. Do not switch estimands after a significance test of normality.

7.2 Binary outcomes

Within each pair, $Y_1-Y_0\in\{-1,0,1\}$, and its mean is the risk-difference ATT among retained intervention recipients. McNemar’s test focuses on discordant pairs, but its p-value does not replace a risk difference and confidence interval.

# Assume binary_outcome has been merged into matched_outcome after the design
# was frozen.
binary_pairs <- split(matched_outcome, matched_outcome$subclass)
pair_risk_differences <- vapply(
  binary_pairs,
  function(x) {
    x$binary_outcome[x$treatment_num == 1] -
      x$binary_outcome[x$treatment_num == 0]
  },
  numeric(1)
)
mean(pair_risk_differences)  # Matched-sample ATT risk difference

# Form a 2-by-2 table across pairs for a paired binary-outcome test.
paired_binary <- do.call(
  rbind,
  lapply(binary_pairs, function(x) c(
    treated = x$binary_outcome[x$treatment_num == 1],
    control = x$binary_outcome[x$treatment_num == 0]
  ))
)
mcnemar.test(table(paired_binary[, "treated"], paired_binary[, "control"]))

7.3 Count and time-to-event outcomes

Count outcomes require a clear observation time and attention to overdispersion. Use a Poisson or negative-binomial model compatible with matching weights and clustering. Time-to-event outcomes also involve censoring, time zero, competing risks, and proportional hazards. Even after matching, use an appropriate survival model and account for pairing or repeated individuals. See Survival Analysis in Depth.

# Requires the survival package; illustrates a stratified Cox model for 1:1
# pairs.
survival::coxph(
  survival::Surv(time, event) ~ treatment_num + strata(subclass),
  data = matched_outcome,
  ties = "efron"
)

# Matching weights, replacement, or multiple rows per person require a
# compatible robust variance.

Matching does not fix informative censoring or turn an HR into a risk difference. Report a fixed-time risk, RMST, HR, or another effect measure that answers the question, and diagnose the assumptions of the survival analysis itself.

8 Boundaries of PSM

8.1 Unmeasured confounding

A perfect Love plot establishes only that the distributions of included variables are similar. Unmeasured disease severity, clinician preference, access to care, or treatment contraindications may still affect both treatment and outcome. Analysts should:

• list plausible unmeasured common causes systematically during the design stage;
• use negative controls, quantitative bias analysis, or sensitivity methods appropriate to the effect scale;
• compare defensible covariate sets rather than reporting only the most favorable specification; and
• distinguish clearly between a causal estimate “under no unmeasured confounding and other assumptions” and an established causal effect.

Sensitivity analysis cannot prove that bias is absent, but it can describe how strong an unmeasured relationship would need to be to alter the conclusion.

8.2 Deleting missing observations is not a complete strategy

Complete-case PSM can induce selection bias when missingness is related to both treatment and outcome, and it changes the target population. A plan should:

1. Distinguish unmeasured variables, structurally missing values, and missing follow-up outcomes.
2. Describe missingness proportions and patterns in each group.
3. Perform multiple imputation or another strategy compatible with the analysis target before estimating the PS.
4. Preserve the “design without viewing outcomes” principle while making the imputation model respect causal temporal order.
5. Complete the design and estimation in every imputed data set, combine results using appropriate rules, and report that matched units may change.
6. Conduct sensitivity analyses for MNAR values or loss to follow-up.

Simply coding NA as an “unknown” category does not necessarily remove bias, and single mean imputation distorts variances and relationships.

8.3 Time-varying treatment and confounding

Ordinary baseline PSM assumes that treatment strategies are defined at a shared time zero. If treatment can start, stop, or switch during follow-up, and current health both affects the next treatment decision and is affected by prior treatment, baseline PS methods cannot resolve the resulting time-varying confounding.

Instead, emulate a target trial, construct person-period data, and consider the g-formula, marginal structural models, or other longitudinal causal methods. Back-filling “ever treated during follow-up” as a baseline variable uses future information and creates immortal-time bias. See Causal Inference in Depth for more on identification and the boundaries of longitudinal methods.

8.4 Matching is not a randomized trial

PSM does not automatically solve:

• measurement error in treatment, outcome, or covariates;
• selection into the cohort or bias from loss to follow-up;
• inconsistent intervention versions, adherence problems, or treatment crossover;
• institutional or network interference;
• inconsistent outcome definitions, follow-up windows, or time zero; or
• excessive extrapolation to unmatched people, other regions, or other eras.

9 Complete application results and reporting

9.1 Results summary

case_summary <- data.frame(
  Item = c(
    "Original sample",
    "Primary design",
    "Matched sample",
    "Largest covariate |SMD|",
    "Unadjusted outcome mean difference",
    "Postmatch matched-sample ATT",
    "Simulated truth (teaching only)"
  ),
  Result = c(
    sprintf(
      "%d intervention recipients; %d controls",
      sum(psm_design_data$treatment_num == 1),
      sum(psm_design_data$treatment_num == 0)
    ),
    "ATT; 1:1 nearest neighbor; no replacement; logit PS; 0.20 SD caliper",
    sprintf(
      "%d pairs; %d intervention recipients unmatched",
      nlevels(matched_outcome$subclass),
      sum(psm_design_data$treatment_num == 1) -
        sum(matched_outcome$treatment_num == 1)
    ),
    sprintf("%.3f → %.3f", max_smd_before, max_smd_after),
    sprintf("%.2f", naive_difference),
    sprintf(
      "%.2f (95%% CI %.2f to %.2f)",
      matched_att,
      matched_att_ci[1],
      matched_att_ci[2]
    ),
    sprintf("ATT among retained recipients = %.2f", true_matched_att)
  ),
  check.names = FALSE
)

knitr::kable(case_summary, caption = "Auditable summary of the primary PSM analysis")

Auditable summary of the primary PSM analysis

Item	Result
Original sample	490 intervention recipients; 710 controls
Primary design	ATT; 1:1 nearest neighbor; no replacement; logit PS; 0.20 SD caliper
Matched sample	375 pairs; 115 intervention recipients unmatched
Largest covariate |SMD|	0.566 → 0.066
Unadjusted outcome mean difference	-0.54
Postmatch matched-sample ATT	-6.27 (95% CI -7.26 to -5.28)
Simulated truth (teaching only)	ATT among retained recipients = -6.18

The simulated cohort included 1,200 people: 490 received the intervention and 710 received control management. Using a prespecified logistic propensity score, we conducted 1:1 nearest-neighbor ATT matching without replacement and used a caliper of 0.20 standard deviations of the logit PS. A control was found for 375 intervention recipients, while 115 were unmatched. The largest absolute SMD among pretreatment covariates fell from 0.566 to 0.066. After freezing the design, the mean paired outcome difference across 375 pairs was -6.27 (95% CI -7.26 to -5.28). This estimate concerns retained intervention recipients and relies on no unmeasured confounding, positivity, consistency, no interference, and related assumptions. The simulated truth for the retained population was -6.18 and is shown only as a teaching check.

9.2 Auditable reporting template

In [target population, place, and period], we estimated the [ATE, ATT, ATC, or postmatch target]. Treatment was defined as [strategies and time zero], and the outcome was [time point and scale]. The propensity score included the pretreatment variables [list and functional forms], selected from [DAG, subject-matter knowledge, and protocol]; outcomes were not used during design. We used [matching method, direction, ratio, replacement, caliper, and exact-matching conditions]. The original sample included [group sizes], and the matched sample included [people or matched sets]; [number] remained unmatched, so the target population [did or did not change]. The largest absolute SMD changed from [value] to [value], and we assessed [VRs, eCDFs, overlap, and key interactions]. Using [weights, subclass, and variance method], the estimated [effect measure] was [estimate and 95% CI]. Results rely on [identification assumptions], with primary limitations from [unmeasured confounding, missingness, overlap, measurement, or transportability].

9.3 Minimum audit materials to retain

• The complete analysis protocol, estimand, and target-trial description;
• the PS formula, software and version, random seed, and matching parameters;
• group sizes before and after matching, reasons for unmatched units, and any target-population change;
• SMDs, VRs, eCDFs, and distribution plots for every prespecified covariate;
• PS overlap plots, the caliper scale, and pair distances;
• weights, subclass, original IDs, and matched counterparts;
• when outcomes were revealed, primary effect code, and the uncertainty method; and
• every design specification and sensitivity analysis, not just the most favorable result.

10 Common mistakes at a glance

Mistake	Why it is wrong	Better approach
Treating PSM as an automatic deconfounding button	Unmeasured confounding and design defects remain	Define the target trial and identification assumptions first
Starting matching without an estimand	Matching direction and sample loss can change the question	Prespecify ATE, ATT, or ATC and the target population
Choosing a PS formula from outcomes or p-values	Contaminates the design and creates selective results	Hide outcomes during design and choose by balance and support
Removing covariates by treatment-model p-values	Predictive significance is not a confounding criterion	Use temporal order, DAGs, and subject-matter knowledge
Including posttreatment variables	Can block effects or open collider paths	Use only variables measured before time zero
Reporting only the PS model AUC	Predicting treatment is not the same as creating balance	Report postmatch diagnostics for every covariate
Evaluating balance with baseline t tests	p-values depend heavily on sample size	Emphasize SMDs, VRs, eCDFs, and distributions
Looking only at the average SMD	Can hide one severely imbalanced covariate	Report every variable and the largest absolute value
Ignoring covariates once the PS is balanced	Similar PS values can represent different covariate profiles	Diagnose individual variables, nonlinear terms, and interactions
Failing to define the caliper scale	A value of 0.2 differs on probability and logit scales	Report the link, standardization, and actual width
Ignoring unmatched intervention recipients	The estimand and transport population have changed	Report sample flow and describe excluded people
Dropping weights or subclass	The estimate or standard error no longer matches the design	Preserve the complete output from match_data()
Treating reused controls as independent after replacement	Understates uncertainty	Retain IDs, weights, and clustering structure
Running an ordinary independent t test after matching	Ignores matched dependence	Analyze pair differences for 1:1 matching without replacement
Failing to report missingness after complete-case analysis	Can induce selection bias and change the population	Address missingness prospectively and analyze mechanism sensitivity
Declaring a definite causal result from a good Love plot	Covers only measured covariates	Use conditional causal language and discuss unmeasured confounding

11 Exercises and answers

1. A study asks for the mean outcome difference if people who actually received surgery had instead received medication. Is its target closer to the ATE or ATT?
2. A known strong outcome predictor has p=0.40 in the treatment model. Should it be removed from the PS?
3. After 1:1 caliper matching, 30% of treated participants remain unmatched. Does the primary result still represent every treated participant?
4. The largest postmatch SMD is 0.06, but the age eCDFs still differ substantially in the tail. What should the analyst do?
5. In matching with replacement, one control is used 12 times. Can that person be analyzed as 12 independent controls?
6. During follow-up, current health affects the next medication decision and is itself affected by prior medication. Is baseline PSM adequate?
7. Why should an outcome p-value not be used to select among calipers of 0.1, 0.2, and 0.3?
8. For a matched binary outcome, is reporting only a McNemar p-value sufficient?

Show exercise answers

1. ATT, because the target population is the people who actually received surgery.
2. No. Variable selection follows temporal order, causal structure, and prognostic value, not a p-value alone.
3. Not automatically. The more accurate target is the ATT among retained treated participants with common support; unmatched participants must be described.
4. Without viewing outcomes, examine the functional form, key interactions, caliper, or matching method and then repeat every diagnostic. The 0.1 line is only a heuristic.
5. No. That remains one person; retain the ID, number of uses or weight, and a compatible variance method.
6. Usually not. This is time-varying confounding affected by prior treatment and calls for a longitudinal target trial and methods such as g-methods.
7. Doing so lets random outcome differences determine the design and creates selective inference. Choose by balance, support, sample retention, and the protocol.
8. No. Also report a risk difference, risk ratio, or another prespecified effect measure with a confidence interval while preserving pair or weight structure.

12 Quick reference

12.1 Core formulas and R entry points

Target	Formula or code	Interpretation
Propensity score	$e(X)=P(A=1\mid X)$	Probability of treatment given pretreatment variables
ATT	$E\{Y(1)-Y(0)\mid A=1\}$	Average causal effect among those actually treated
Logit PS	$\log[e/(1-e)]$	Common matching-distance scale
SMD	$(\bar X_1-\bar X_0)/s^*$	Mean-balance measure with little dependence on sample size
Nearest-neighbor matching	matchit(..., method="nearest")	Finds the closest comparison units for the target group
0.2 SD caliper	caliper=.2, std.caliper=TRUE	Also report the distance and link scale
Extract design sample	match_data(m.out, data=d)	Retains distance, weights, and subclass
Expand matches with replacement	get_matches(m.out, data=d)	An original control may recur; preserve its source ID
Balance summary	summary(m.out, un=TRUE, standardize=TRUE)	Returns prematch and postmatch SMDs, VRs, eCDFs, and counts
1:1 pair difference	mean(Y_treated - Y_control)	Matched-sample ATT in the present primary design

12.2 Minimal workflow from raw data to results

# 1. Design data with the outcome hidden
design_data <- d[, c("A", "age", "bmi", "smoking", "severity")]

# 2. Prespecified 1:1 nearest-neighbor matching for the ATT
m <- MatchIt::matchit(
  A ~ age + bmi + smoking + severity,
  data = design_data,
  method = "nearest",
  distance = "glm",
  link = "linear.logit",
  estimand = "ATT",
  ratio = 1,
  replace = FALSE,
  caliper = 0.20,
  std.caliper = TRUE,
  m.order = "closest"
)

# 3. While outcomes remain hidden, assess sample flow, overlap, and balance for
# every covariate.
summary(m, un = TRUE, standardize = TRUE)

# 4. Freeze the design, then merge outcomes while retaining weights and subclass.
matched <- MatchIt::match_data(m, data = d)

# 5. Use effect and variance estimators compatible with the matching structure
# and outcome type.

12.3 Final checklist

• Are the target trial, time zero, treatment strategies, outcome, and estimand explicit?
• Has the target population been renamed accurately after any matching failure?
• Does the PS use only pretreatment variables selected for reasons other than p-values?
• Was the outcome completely excluded during the design stage?
• Are the link, distance, caliper, ratio, replacement, order, and exact-matching conditions reproducible?
• Are the complete sample flow and characteristics of unmatched people reported?
• Were PS overlap and support for specific covariate combinations assessed?
• Are SMDs reported for every variable, with supplementary VR, eCDF, nonlinear-term, and interaction diagnostics?
• Were weights, subclass, and original IDs retained and used correctly?
• Is the confidence interval compatible with 1:1, many-to-one, replacement, or clustered structure as applicable?
• Were missingness, measurement error, loss to follow-up, and time-varying treatment addressed separately?
• Were sensitivity analyses conducted for unmeasured confounding and transport beyond the target population?
• Does the conclusion use conditional causal language tied to its identification assumptions?

Where to go next

Next topics include full matching, optimal matching, coarsened exact matching, Mahalanobis distance within PS calipers, generalized propensity scores, overlap weighting, doubly robust outcome models, matching after multiple imputation, robust variance for matched designs, quantitative bias analysis for unmeasured confounding, and g-methods for longitudinal treatment. See Causal Inference in Depth for the full causal framework and other estimators.

sessionInfo()

## R version 4.6.1 (2026-06-24)
## Platform: aarch64-apple-darwin23
## Running under: macOS Tahoe 26.5.1
##
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.6/Resources/lib/libRblas.0.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.6/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1
##
## locale:
## [1] C.UTF-8/C.UTF-8/C.UTF-8/C/C.UTF-8/C.UTF-8
##
## time zone: America/Edmonton
## tzcode source: internal
##
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base
##
## loaded via a namespace (and not attached):
##  [1] backports_1.5.1 digest_0.6.39   R6_2.6.1        fastmap_1.2.0
##  [5] xfun_0.60       cachem_1.1.0    knitr_1.51      htmltools_0.5.9
##  [9] rmarkdown_2.31  lifecycle_1.0.5 cli_3.6.6       sass_0.4.10
## [13] jquerylib_0.1.4 compiler_4.6.1  chk_0.10.0      tools_4.6.1
## [17] evaluate_1.0.5  bslib_0.12.0    Rcpp_1.1.2      yaml_2.3.12
## [21] MatchIt_4.7.2   rlang_1.3.0     jsonlite_2.0.0

Teaching use: This module teaches the design, diagnosis, estimation, and reporting of propensity score matching. It does not replace a study-specific causal analysis plan, data audit, or professional statistical consultation. All data and numerical results are simulated.