Causal inference & experiment design
Separate what a change caused from what merely moved alongside it.
32 of 388 tools.
Audit AI code change evidence integrity
Prove that aggregate AI-assisted coding evidence comes from prospectively registered, nonoverlapping treatment/control studies with immutable assignment, configuration, trace and mature-outcome denominators before anyone estimates an effect.
Audit AI configuration release integrity
Audit that the exact immutable AI configuration bundle evaluated and approved is the bundle exposed in every staged rollout, with consecutive parent lineage, complete blast-radius declaration, effective runtime controls, monotone traffic and a tested prior-version rollback path.
Audit AI routing experiment integrity
Audit prospectively registered AI-route experiments at the randomization-unit/period/route grain, reconciling logged propensities, allocation fidelity, pre-assignment balance, crossover, outcome maturity, simultaneous-experiment overlap and unique value at risk before any causal effect is reported.
Audit causal claim negative controls
Gate a causal effect claim using prespecified negative outcome/exposure controls, Benjamini-Hochberg multiplicity control, and an omnibus chi-square falsification test.
Audit cluster randomization integrity
Audit cluster-randomized experiments for practical baseline imbalance and differential outcome observation, with cluster-size-weighted standardized differences and assignment permutation diagnostics.
Audit growth incrementality experiment integrity
Audit aggregate randomized growth experiments before anyone trusts channel incrementality: enforce unique experimental units, nondegenerate logged propensities, both arms, control-spend discipline, baseline balance, spillover and evidence gates; then estimate propensity-weighted baseline-adjusted contribution, cluster-unit bootstrap uncertainty and incremental return on spend.
Audit root cause traceback evidence integrity
Audit whether an anomaly traceback is complete, point-in-time, multiplicity-controlled and honestly labeled as temporal or causal, including every upstream candidate, path lag, edge identification basis and later root-recovery validation.
Audit staggered rollout identification
Audit staggered team-by-team adoption with not-yet-treated controls and require every simultaneous pre-period interval to fit inside a governed equivalence margin.
Calculate analytics portfolio realized ROI
Reconcile the analytics portfolio's realized ROI from unique finance-owned incremental benefit sources, causal-evidence weights, non-overlapping function allocations, implementation/recurring/shared costs and coherent joint scenarios, with positive-value probability and CVaR loss gates.
Design balanced stepped wedge rollout
Assign teams or other aggregate clusters to capacity-constrained rollout waves with pair-exchange optimization of cumulative causal balance, represented population, and rollout risk.
Estimate causal value of execution fidelity
Estimate how much outcome value an additional unit of implementation fidelity causes by using randomized enablement as an encouragement instrument, with whole-cluster arm bootstrap, first-stage, balance, negative-control and ratio-stability gates.
Estimate feature incremental value
Estimate rollout value from segment-level treated/control outcomes with beta-binomial uplift posteriors, finance-owned contribution economics, and a probability-of-positive-value gate.
Estimate lee bounds under attrition
Partially identify a randomized treatment effect under differential outcome attrition using direction-aware fractional Lee trimming and bootstrap outer bounds.
Estimate network direct and spillover effects
Estimate direct, neighbor-spillover, and total effects under Bernoulli-randomized network interference using exact exposure probabilities and randomization inference.
Estimate randomized causal mediation
Decompose a randomized intervention into natural direct and mediated effects with optional treatment-mediator interaction, bootstrap intervals, and total-effect reconciliation.
Estimate staggered policy rollout effects
Estimate cohort-aware dynamic effects of a team-by-team policy rollout against not-yet-treated controls, with a simultaneous pretrend identification gate.
Estimate switchback policy effect
Estimate randomized operational switchback effects with unit and period fixed effects, declared washout exclusions, distributed carryover lags, overlap enforcement, and whole-unit bootstrap uncertainty.
Fit anchor regression shift robust model
Fit anchor regression across declared operating environments, penalizing residual variation predictable from environment anchors over a governed gamma path; choose robustness strength only on held-out worst-environment RMSE; and expose average fit, environment bias, coefficients, and leave-one-environment stability without claiming generic or causal invariance.
Forecast organizational change second order effects
Forecast the incremental capacity, review, knowledge, quality, backlog, recovery and financial distribution of a submitted departure, hire, restructure, reassignment, PTO, AI rollout, framework migration or contractor scenario using company-local completed episodes and a dependency DAG.
Infer revealed policy preferences maxent irl
Infer aggregate linear state-feature rewards and their implied stochastic policy from sequential demonstrations using finite-horizon maximum-causal-entropy inverse reinforcement learning.
Optimize AI configuration rollout portfolio
Select one current or staged rollout plan per AI configuration release, maximizing expected value minus CVaR regret under hard controls, failure ceilings, application concurrency, dependencies, budget and shared resources while computing overlapping application blast-radius loss once from joint survival.
Optimize causal release assurance portfolio
Select one release, assurance or hold option per change from prospectively identified Beta-binomial relative-risk effects while pricing delay, failure and shared common-mode loss under budget, scarce resources, mandatory controls, expected-failure and CVaR constraints with an exact or disclosed beam-search Pareto frontier.
Optimize CI assurance portfolio
Choose one baseline, cache, shard, test-selection, flaky-repair, mutation, integration-suite or runner-scale option per assurance unit using only prospective randomized/known-propensity fault-detection and feedback evidence, common random scenarios, controls, relations, budget, implementation and runner capacity, undetected-fault, latency and CVaR gates.
Optimize incident learning portfolio
Choose immediate remediation or a predeclared experiment-contingent action for each failure mode using Bayesian value of information, prospective test accuracy and causal remediation effects, common scenarios, shared-loss accounting, Pareto search and hard cost, capacity and tail-risk gates.
Optimize model averaged joint outcome decision
Choose a governed aggregate engineering action across competing plausible Bayesian-network structures using pseudo-Bayesian out-of-time model weights, coherent joint outcome worlds, causal-identification mass, weighted CVaR, worst-model regret, decision stability and the expected value of resolving model uncertainty.
Optimize org health intervention portfolio
Select an anti-Goodhart intervention portfolio using conservative causal lower bounds on real operating loss, never score movement, with design/transport/fidelity shrinkage, negative controls, interference, shared-loss, equity, resource and CVaR constraints.
Optimize organizational change mitigation portfolio
Choose a dependency-safe portfolio of documentation, cross-training, review redistribution, onboarding, staffing buffers, staged rollout, rollback or migration-support mitigations that minimizes change loss under nonlinear overlap, common risk, budget, scarce skills, recovery deadlines and CVaR.
Optimize retention interventions by principal strata
Estimate who an optional retention intervention can actually help—not merely who looks likely to leave—from randomized principal strata, then allocate scarce capacity by conservative net value under harmed-stratum sensitivity, budget and fairness constraints.
Recommend stop continue scale decisions
Recommend stop, continue learning, or scale for aggregate initiatives using beta-binomial posterior rollout economics, independent harm gates, simulation precision, sampling cost, and opportunity decay.
Solve robust policy across causal models
Choose an aggregate policy across competing interventional causal models and a bounded posterior credal set: derive model-by-action expected utility from outcome probabilities, compute exact lower/upper utility and adversarial model weights, minimize worst-case regret, expose model disagreement and value of perfect model information, and fail closed when any causal evidence gate fails.
Stress test causal effect robustness
Quantify the omitted-confounder partial-R² strength required to erase a causal point estimate or its statistical significance, benchmarked against observed covariates.
Value AI assistant rollout ROI
Value an aggregate AI-assistant rollout from aligned joint causal-effect draws, preserving delivery/time/defect/incident dependence while enforcing identification, out-of-time, overlap, metric-integrity, effective-sample, quality-harm, NPV, ROI, and payback gates.