DecisionDrift provides tools for detecting, decomposing,
and stress-testing temporal drift in repeated binary
decision systems. It is designed for settings in which an institutional
or algorithmic system issues a binary decision — approve or deny, flag
or pass, intervene or not — repeatedly over time for the same or
overlapping set of units.
The package does not estimate causal effects and does not require full specification of the underlying decision rule. Instead, it treats the decision system as a time-indexed stochastic process and asks whether that process changed over time — and if so, how, when, and for whom.
Let:
The decision system is defined as the stochastic process:
\[S = \{ D_{it} : i = 1, \ldots, N;\ t = 1, \ldots, T \}\]
Critically, \(S\) is treated as a generator of decisions over time, not as a parametric model. This makes the framework applicable to hybrid human–AI systems where the decision rule \(f_t\) is only partially observed.
In many applied settings, decisions are produced by a joint human–AI process:
\[D_{it} = f_t(X_{it},\ H_{it})\]
where \(X_{it}\) are inputs (features, signals) available to the system, \(H_{it}\) represents human influence (override, discretion, intervention), and \(f_t\) is the time-varying decision mechanism emerging from the interaction of algorithmic and human components.
Drift in this system means \(f_t \neq f_{t'}\), or equivalently:
\[\Pr(D_{it} = 1 \mid X_{it}, H_{it}) \text{ changes over time}\]
DecisionDrift audits changes in \(f_t\) without requiring its full
specification. This is the core methodological
contribution.
Marginal drift (level shift): a change in the marginal distribution of decisions across waves.
\[\text{Drift exists if } \exists\ t_1 \neq t_2 \text{ such that } P_{t_1} \neq P_{t_2}\]
where \(P_t(d) = \Pr(D_{it} = d)\).
Transition drift (dynamic instability): a change in the conditional distribution given the prior decision.
\[\text{Transition drift exists if } P_t(d_t \mid d_{t-1}) \text{ varies with } t\]
The distinction matters. A system can have stable prevalence
but unstable dynamics (transition drift without marginal
drift), or shifting prevalence with stable dynamics
(marginal drift without transition drift). DecisionDrift
separates these two phenomena.
The DDI estimates the linear trend in wave-level decision prevalence:
\[p_t = \frac{1}{N} \sum_{i=1}^{N} D_{it}\]
\[\text{DDI} = \frac{\hat{\beta}}{\text{SD}(p_t)}\]
where \(\hat{\beta}\) is the OLS slope of \(p_t\) on \(t\). Standardising by \(\text{SD}(p_t)\) makes the DDI scale-invariant and comparable across datasets.
Properties:
For the step from wave \(s\) to \(s+1\), define uptake and reversal:
\[u_s = \Pr(D_{i,s+1} = 1 \mid D_{is} = 0), \qquad r_s = \Pr(D_{i,s+1} = 0 \mid D_{is} = 1).\]
Because each row of the transition matrix sums to one, \(u_s\) and \(r_s\) describe it completely. Transition drift between consecutive steps is
\[D^T_s = \tfrac12\left(|u_s - u_{s-1}| + |r_s - r_{s-1}|\right),\]
and the TDI is its mean over the \(T-2\) pairs of consecutive steps:
\[\text{TDI} = \frac{1}{T-2}\sum_{s=2}^{T-1} D^T_s.\]
Properties:
dd_global_test() for a formal testLet \(\text{DDI}^{(g)}\) denote the DDI estimated within group \(g\). For groups \(A\) and \(B\):
\[\text{GDD}_{AB} = \text{DDI}^{(A)} - \text{DDI}^{(B)}\]
GDD measures inequality in system evolution, not static disparity. A positive \(\text{GDD}_{AB}\) indicates the system became more permissive for group \(A\) faster than for group \(B\). A value near zero indicates parallel drift regardless of any baseline disparity.
\[\text{CDB} = \sum_{t=2}^{T} |p_t - p_{t-1}|\]
CDB captures total accumulated volatility in the decision rate, regardless of direction. Unlike the DDI, CDB is non-zero even when drift oscillates (e.g., the system alternates between permissive and restrictive periods). A high CDB with a near-zero DDI indicates instability without a directional trend.
The five core modules each estimate one aspect of drift:
| Module | Stochastic object estimated | Index |
|---|---|---|
dd_prevalence() |
\(\Pr(D_t = 1)\) over time | DDI |
dd_transition() |
\(\Pr(D_t \mid D_{t-1})\) over time | TDI |
dd_entropy_trend() |
Complexity of path distribution | — |
dd_group_drift() |
Heterogeneity in \(\Pr(D_t)\) across \(G\) | GDD |
dd_changepoint() |
Structural break in process | — |
dd_decompose() |
Inflow and outflow behind each change in \(\Pr(D_t = 1)\) | NEC, CMB, CTB |
dd_group_decompose() |
Within-group vs. composition change | \(D^C\) |
dd_global_test() |
Constancy of uptake, reversal, prevalence | bootstrap test |
Together they answer: Did the system change, and
how? The robustness and sensitivity modules then ask:
Is that conclusion credible? dd_simulate()
generates panels with known drift for checking methods.
# Simulate a system that became more permissive after wave 4.
# Group A drifts faster than Group B.
n_units <- 80
n_waves <- 8
probs_A <- c(rep(0.20, 4), rep(0.60, 4)) # sharp increase at wave 5
probs_B <- c(rep(0.25, 4), rep(0.40, 4)) # modest increase at wave 5
dat <- data.frame(
id = rep(seq_len(n_units), each = n_waves),
time = rep(seq_len(n_waves), times = n_units),
decision = c(
rbinom(40 * n_waves, 1, rep(probs_A, 40)),
rbinom(40 * n_waves, 1, rep(probs_B, 40))
),
group = rep(c("A", "B"), each = 40 * n_waves)
)
dp <- dd_build(dat, id, time, decision,
group = group,
event_time = 5L)
print(dp)
#>
#> ── DecisionDrift Panel Object ──────────────────────────────────────────────────
#> • Units: 80
#> • Waves: 8 (1–8)
#> • Balanced: TRUE
#> • Group var: group
#> • Event time: 5
#> • Units dropped (min_waves): 0prev <- dd_prevalence(dp)
print(prev)
#>
#> ── Prevalence Drift (Module 1) ─────────────────────────────────────────────────
#> • Mean decision rate: 0.359
#> • Trend slope: 0.0557 (p = 0.0191)
#> • R²: 0.627
#> • Cumulative change: 0.4
#> • DDI: 0.323
plot(prev)The DDI is positive and the slope is statistically significant, confirming that the overall decision rate increased over time.
tr <- dd_transition(dp)
print(tr)
#>
#> ── Transition Drift (Module 2) ─────────────────────────────────────────────────
#> • TDI (Transition Drift Index): 0.1416
#> • Uptake drift mean |Delta P(1|0)|: 0.1513
#> • Reversal drift mean |Delta P(0|1)|: 0.1319
#>
#> ── Linear trends in transition probabilities ──
#>
#> p11: slope = 0.0566, p = 0.1089
#> p10: slope = -0.0566, p = 0.1089
#> p01: slope = 0.0465, p = 0.1163
#> p00: slope = -0.0465, p = 0.1163
plot(tr)The TDI captures the instability in switching and persistence dynamics that accompanies the structural break at wave 5.
et <- dd_entropy_trend(dp, window = 3L)
print(et)
#>
#> ── Entropy & Stability Trend (Module 3) ────────────────────────────────────────
#> • Method: binary
#> • Window: 3 waves
#> • Entropy trend slope: 0.0618 (p = 0.0107)
#> • Switching trend slope: 0.0296 (p = 0.0402)
plot(et)gd <- dd_group_drift(dp)
print(gd)
#>
#> ── Group-Differential Drift (Module 4) ─────────────────────────────────────────
#>
#> ── Per-group trend slopes ──
#>
#> # A tibble: 2 × 5
#> group slope se p_value r_squared
#> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 A 0.0729 0.0282 0.0412 0.528
#> 2 B 0.0384 0.0135 0.0297 0.573
#> ── GDD table ──
#> # A tibble: 1 × 4
#> group_a group_b GDD direction
#> <chr> <chr> <dbl> <chr>
#> 1 A B 0.0345 A faster
#> • Gap trajectory: diverging (slope = 0.0345, p = 0.2468)
plot(gd)Group A experienced faster drift than Group B (positive GDD). The gap trajectory shows divergence over time.
cp <- dd_changepoint(dp)
print(cp)
#>
#> ── Change-Point & Regime Detection (Module 5) ──────────────────────────────────
#>
#> ── CUSUM ──
#>
#> • Detected break wave: 4
#> • Max |CUSUM|: -3.631
#>
#> ── Segmented regression ──
#>
#> • Best breakpoint: 6
#> • delta_AIC (null - seg): 1.58
#>
#> ── Event alignment ──
#>
#> • Event wave: 5
#> • Pre-mean: 0.203
#> • Post-mean: 0.516
#> • Change: 0.312
#> • p-value: 5e-04
#> • p < 0.05: TRUE
plot(cp)Both the CUSUM and segmented regression methods identify a structural break near wave 5, consistent with the known event time.
The audit runs every module, one shared cluster bootstrap, and the
global stability test. It does not attach labels such as “strong drift”.
Its conclusion states the test result, and any practical cut-offs are
supplied by the user through thresholds.
aud <- dd_audit(dp, include_robustness = TRUE, R = 199, seed = 1,
thresholds = c(TDI = 0.05, NEC = 0.10), verbose = FALSE)
print(aud)
#>
#> ── DecisionDrift Audit Report ──────────────────────────────────────────────────
#> • Units: 80 | Waves: 8 | Balanced: TRUE
#> • Group var: group
#> • Event time: 5
#>
#> ── Summary Indices ──
#>
#> ── DecisionDrift Summary Indices ───────────────────────────────────────────────
#> • DDI (Decision Drift Index): 0.3233
#> • TDI (Transition Drift Index): 0.1416
#> • GDD (Group Differential Drift): 0.0345
#> • CDB (Cumulative Drift Burden): 0.6125
#>
#> ── Prevalence Drift ──
#>
#> • DDI = 0.323 | slope = 0.0557 | p = 0.0191
#>
#> ── Transition Drift ──
#>
#> • TDI = 0.1416 | uptake drift = 0.1513 | reversal drift = 0.1319
#>
#> ── Flow Decomposition ──
#>
#> • NEC = 0.2625 [0.125, 0.363]
#> • CMB = 0.6125 [0.499, 1.100]
#> • turnover_total = 2.8375 [2.524, 3.089]
#> • CTB = 0.8495 [0.741, 1.408]
#>
#> ── Change-point Detection ──
#>
#> • CUSUM break: wave 4
#> • Segmented break: wave 6 (delta_AIC = 1.58)
#>
#> ── Group-Differential Drift ──
#>
#> # A tibble: 1 × 4
#> group_a group_b GDD direction
#> <chr> <chr> <dbl> <chr>
#> 1 A B 0.0345 A faster
#> • Within-group change (total): 0.2625
#> • Composition change (total): 0.0000
#>
#> ── Global Stability Test ──
#>
#> • max statistic = 5.044, bootstrap p = 0.005 (R = 199)
#> • Contrasts whose simultaneous band excludes 0: 1 of 12
#>
#> ── User Thresholds ──
#>
#> • TDI: |0.1416| >= 0.05
#> • NEC: |0.2625| >= 0.1
#>
#> ── Robustness ──
#>
#> • Fragility index: 0
#> ────────────────────────────────────────────────────────────────────────────────
#> ℹ Stability of uptake and reversal rejected at alpha = 0.05 (bootstrap p = 0.005).
#> ────────────────────────────────────────────────────────────────────────────────Prevalence can stay flat while the process underneath changes. The marginal-transition identity makes this exact:
\[\pi_{s+1} - \pi_s = \underbrace{(1-\pi_s)\,u_s}_{\text{inflow}} - \underbrace{\pi_s\,r_s}_{\text{outflow}}.\]
If inflow and outflow rise together, prevalence does not move. The
"masked" scenario in dd_simulate() does
exactly this: uptake and reversal both rise from 0.08 to 0.32 over five
steps while the positive rate stays near 0.5.
masked <- dd_simulate(n = 600, waves = 6, scenario = "masked", seed = 11)
dpm <- dd_build(masked, id, time, decision)
round(tapply(masked$decision, masked$time, mean), 3)
#> 1 2 3 4 5 6
#> 0.483 0.493 0.492 0.485 0.503 0.520fl <- dd_decompose(dpm, R = 199, seed = 2)
print(fl)
#>
#> ── Flow Decomposition: change = inflow - outflow ───────────────────────────────
#> step n uptake reversal inflow outflow net turnover
#> 1->2 600 0.097 0.083 0.050 0.040 0.010 0.090
#> 2->3 600 0.128 0.135 0.065 0.067 -0.002 0.132
#> 3->4 600 0.174 0.193 0.088 0.095 -0.007 0.183
#> 4->5 600 0.227 0.203 0.117 0.098 0.018 0.215
#> 5->6 600 0.332 0.295 0.165 0.148 0.017 0.313
#>
#> ── Summary ──
#>
#> • NEC: 0.0367 [0.0017, 0.0783]
#> • CMB: 0.0533 [0.0349, 0.1334]
#> • CTB: 0.2237 [0.1892, 0.3010]
#> • TDI: 0.0559 [0.0473, 0.0753]
#> • turnover_total: 0.9333 [0.8716, 1.0221]
#> • net_flow_total: 0.0533 [0.0349, 0.1334]
#> 95% cluster-bootstrap percentile intervals, R = 199.
plot(fl)Net change (net) stays near zero at every step while
turnover more than triples. In the summary, net endpoint change (NEC)
and cumulative marginal burden (CMB) are small, but total turnover and
cumulative transition burden (CTB) are large. Intervals come from a
unit-level cluster bootstrap, which keeps each unit’s full history
together. Intervals for quantities defined as absolute values (NEC, CMB,
CTB, TDI) cannot include zero and are biased upward when the true value
is near zero; use the global test, not these intervals, to decide
whether the process changed.
dd_global_test() tests whether uptake and reversal were
constant over all steps. It uses the same centred cluster bootstrap and
reports familywise-adjusted p-values and simultaneous bands for each
consecutive change, so it also shows where the process changed.
gt <- dd_global_test(dpm, R = 199, seed = 3)
print(gt)
#>
#> ── Global Test of Decision Process Stability ───────────────────────────────────
#> • H0: uptake, reversal constant across waves
#> • Statistic: max = 3.358 (consecutive contrasts)
#> • Bootstrap p-value: 0.01 (R = 199)
#> • Contrasts tested: 8
#>
#> ── Contrasts (familywise-adjusted) ──
#>
#> component group contrast estimate lower upper p_adj
#> uptake (5->6) vs (4->5) 0.1057 0.0215 0.1899 0.010
#> reversal (5->6) vs (4->5) 0.0920 0.0007 0.1832 0.045
#> reversal (2->3) vs (1->2) 0.0524 -0.0224 0.1271 0.330
#> reversal (3->4) vs (2->3) 0.0581 -0.0249 0.1410 0.330
#> uptake (4->5) vs (3->4) 0.0528 -0.0343 0.1398 0.460
#> uptake (3->4) vs (2->3) 0.0455 -0.0363 0.1273 0.595
#> uptake (2->3) vs (1->2) 0.0315 -0.0364 0.0994 0.775
#> reversal (4->5) vs (3->4) 0.0095 -0.0799 0.0989 1.000
#> Bands are 95% simultaneous (max-t critical value 2.68).
plot(gt)Compare the prevalence trend test from
dd_prevalence():
Expected prevalence is constant in this scenario, so a prevalence test has nothing to find; the transition test targets the change that did happen. A rejection says the decision-generating process was not constant. It does not say why, and it is not a causal effect.
By default each step is compared with the one before it, which
localises abrupt change. For gradual drift,
contrast = "baseline" compares each step with the first
one; small step-to-step changes then add up, so this version has more
power against steady trends.
The overall rate is a weighted average of group rates, \(\pi_t = \sum_g w_{gt}\pi_{gt}\). A change
in \(\pi_t\) can come from the group
rates (within-group change) or from the group shares (composition
change). dd_group_decompose() uses a symmetric midpoint
decomposition,
\[\Delta\pi_t = \sum_g \bar w_{gt}\,\Delta\pi_{gt} + \sum_g \bar\pi_{gt}\,\Delta w_{gt},\]
which is exact and does not depend on which wave is the baseline.
In the "composition" scenario, both groups follow stable
processes, but the higher-rate group A enters the panel over time, so
its share grows.
comp <- dd_simulate(n = 600, waves = 6, scenario = "composition", seed = 12)
dpc <- dd_build(comp, id, time, decision, group = group)
gdc <- dd_group_decompose(dpc, R = 199, seed = 4)
print(gdc)
#>
#> ── Within-Group and Composition Decomposition ──────────────────────────────────
#> step delta_pi within composition D_C
#> 1->2 0.0702 0.0165 0.0537 0.1365
#> 2->3 0.0296 -0.0084 0.0380 0.0988
#> 3->4 -0.0267 -0.0267 0.0000 0.0000
#> 4->5 0.0133 0.0133 0.0000 0.0000
#> 5->6 0.0183 0.0183 0.0000 0.0000
#>
#> ── Totals ──
#>
#> • within_total: 0.0131 [-0.0354, 0.0661]
#> • composition_total: 0.0917 [0.0748, 0.1108]
#>
#> ── Group-specific uptake and reversal (mean over steps) ──
#>
#> • A: uptake = 0.323, reversal = 0.140
#> • B: uptake = 0.160, reversal = 0.341
plot(gdc)Almost all of the rise in the overall rate is composition change. A prevalence-only audit would attribute it to a change in the decision rule.
The reverse also happens: group rates can move in opposite directions
while the overall rate stays flat. Testing group-specific parameters
with by_group = TRUE checks for that. Because the
divergence is gradual, baseline contrasts are used.
gm <- dd_simulate(n = 800, waves = 6, scenario = "group_masked", seed = 13)
dpg <- dd_build(gm, id, time, decision, group = group)
round(tapply(gm$decision, gm$time, mean), 3) # pooled rate
#> 1 2 3 4 5 6
#> 0.489 0.496 0.494 0.505 0.478 0.479
gtg <- dd_global_test(dpg, components = "prevalence", by_group = TRUE,
contrast = "baseline", R = 199, seed = 5)
print(gtg)
#>
#> ── Global Test of Decision Process Stability ───────────────────────────────────
#> • H0: prevalence constant across waves (within each group)
#> • Statistic: max = 3.072 (baseline contrasts)
#> • Bootstrap p-value: 0.03 (R = 199)
#> • Contrasts tested: 10
#>
#> ── Contrasts (familywise-adjusted) ──
#>
#> component group contrast estimate lower upper p_adj
#> prevalence B 6 vs 1 -0.1025 -0.2002 -0.0048 0.030
#> prevalence A 6 vs 1 0.0825 -0.0150 0.1800 0.110
#> prevalence B 5 vs 1 -0.0750 -0.1704 0.0204 0.140
#> prevalence A 4 vs 1 0.0700 -0.0248 0.1648 0.185
#> prevalence A 5 vs 1 0.0525 -0.0463 0.1513 0.550
#> prevalence B 4 vs 1 -0.0375 -0.1181 0.0431 0.700
#> prevalence A 2 vs 1 0.0050 -0.0579 0.0679 1.000
#> prevalence A 3 vs 1 0.0125 -0.0738 0.0988 1.000
#> prevalence B 2 vs 1 0.0100 -0.0543 0.0743 1.000
#> prevalence B 3 vs 1 -0.0025 -0.0822 0.0772 1.000
#> Bands are 95% simultaneous (max-t critical value 2.93).dd_simulate() provides stable, marginal, masked, abrupt,
group-masked, and composition-change panels, plus custom uptake and
reversal sequences. The script in
system.file("simulation", "mc_study.R", package = "DecisionDrift")
runs a Monte Carlo study comparing the prevalence trend test with the
global stability test across the first four scenarios.
A key claim of the DecisionDrift framework is that
observed drift is not an artefact of analytic choices or data problems.
The robustness and sensitivity modules test this claim directly.
Robustness (dd_robustness()) tests
whether the DDI sign and magnitude are stable across:
A fragility index summarises the proportion of analytic variants in which the slope changes sign. Values near zero indicate robust conclusions.
Sensitivity (dd_sensitivity()) probes
vulnerability to plausible data problems:
A tipping-point estimate identifies the smallest perturbation level at which more than 50 per cent of resampled conclusions flip sign.
rob <- dd_robustness(dp, variants = c("lopo", "min_waves"))
print(rob)
#>
#> ── Robustness Analysis (Module 6) ──────────────────────────────────────────────
#> • Baseline DDI: 0.323
#> • Fragility index: 0
#>
#> Robustness table:
#> # A tibble: 12 × 6
#> variant DDI slope p_value n_waves n_units
#> * <chr> <dbl> <dbl> <dbl> <int> <int>
#> 1 Baseline 0.323 0.0557 0.0191 8 80
#> 2 Leave out wave 1 0.341 0.0571 0.0591 7 80
#> 3 Leave out wave 2 0.313 0.0540 0.0499 7 80
#> 4 Leave out wave 3 0.303 0.0540 0.0399 7 80
#> 5 Leave out wave 4 0.312 0.0539 0.0228 7 80
#> 6 Leave out wave 5 0.331 0.0531 0.0103 7 80
#> 7 Leave out wave 6 0.305 0.0515 0.0376 7 80
#> 8 Leave out wave 7 0.314 0.0545 0.0487 7 80
#> 9 Leave out wave 8 0.396 0.0723 0.0143 7 80
#> 10 Min waves >= 2 0.323 0.0557 0.0191 8 80
#> 11 Min waves >= 3 0.323 0.0557 0.0191 8 80
#> 12 Min waves >= 4 0.323 0.0557 0.0191 8 80
plot(rob)DecisionDrift is designed to occupy a specific and
currently underserved position in the audit toolbox:
We define temporal drift as a change in the decision-generating process over time and introduce a set of summary indices that quantify level shifts, dynamic instability, subgroup divergence, and cumulative change. By implicitly treating decision systems as evolving stochastic processes, the framework enables longitudinal auditing of hybrid human–AI systems without requiring full specification of the underlying decision rule.
This positions the package as semi-parametric auditing of dynamic systems — distinct from full Markov modelling, time-series estimation, and cross-sectional fairness testing.
decisionpathsdecisionpaths constructs and summarises longitudinal
decision paths, answering what happened.
DecisionDrift tests whether the system that generated those
paths changed over time, answering did the system change. A
third package in this ecosystem, AIBias, will ask whether
inequality accumulated or amplified over repeated decisions.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.
Hait, S. (2026). Artificial intelligence as decision infrastructure: Rethinking institutional decision processes. Unpublished manuscript.
Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27(3), 379–423.