Estimate causal value of execution fidelity
Estimate how much outcome value an additional unit of implementation fidelity causes by using randomized enablement as an encouragement instrument, with whole-cluster arm bootstrap, first-stage, balance, negative-control and ratio-stability gates.
What it's for
Lets Gitrevio answer a rare but decisive question: did better execution itself create value, or did apparently successful execution merely accompany better strategies and easier projects?
What you give it
Inputs split into evidence read from your connected systems, calibration your team owns, and numerical controls that affect precision but never the result's meaning.
| Field | Type | Role | Required |
|---|---|---|---|
| bootstrap_draws | integer ≥ 200, ≤ 20000 | Numerical control | Optional |
| confidence_level | number ≥ 0.5, < 1 | Your calibration | Optional |
| max_detail_rows | integer ≥ 1, ≤ 500 | Numerical control | Optional |
| maximum_absolute_baseline_imbalance | number ≥ 0 | Your calibration | Optional |
| maximum_absolute_negative_control_effect | number ≥ 0 | Your calibration | Optional |
| maximum_invalid_ratio_draw_fraction | number ≥ 0, ≤ 1 | Your calibration | Optional |
| minimum_causal_value_per_fidelity_unit | number | Your calibration | Optional |
| minimum_cluster_count | integer ≥ 10, ≤ 100000 | Your calibration | Optional |
| minimum_first_stage_fidelity_effect | number > 0, ≤ 1 | Your calibration | Optional |
| observations | array of objects (9 fields) ≥ 40 items | Evidence | Yes |
| seed | integer ≥ 0, ≤ 2147483647 | Numerical control | Optional |
Each observations
record
| Field | Type | Required |
|---|---|---|
| assigned_enablement | boolean | Yes |
| baseline_outcome | number | Yes |
| cluster_id | string (non-empty) | Yes |
| id | string (non-empty) | Yes |
| implementation_fidelity | number (≥ 0, ≤ 1) | Yes |
| negative_control_outcome | number | Yes |
| outcome_value | number | Yes |
| program_id | string (non-empty) | Yes |
| weight | number (> 0) | Yes |
{
"bootstrap_draws": 200,
"confidence_level": 0.8,
"observations": [
{
"assigned_enablement": true,
"baseline_outcome": 0,
"cluster_id": "delivery-unit-00",
"id": "enablement-0",
"implementation_fidelity": 0.8,
"negative_control_outcome": 0,
"outcome_value": 8,
"program_id": "implementation-enablement",
"weight": 1
},
{
"assigned_enablement": true,
"baseline_outcome": 0,
"cluster_id": "delivery-unit-01",
"id": "enablement-1",
"implementation_fidelity": 0.8,
"negative_control_outcome": 0,
"outcome_value": 8,
"program_id": "implementation-enablement",
"weight": 1
},
{
"assigned_enablement": true,
"baseline_outcome": 0,
"cluster_id": "delivery-unit-02",
"id": "enablement-2",
"implementation_fidelity": 0.8,
"negative_control_outcome": 0,
"outcome_value": 8,
"program_id": "implementation-enablement",
"weight": 1
},
{
"assigned_enablement": true,
"baseline_outcome": 0,
"cluster_id": "delivery-unit-03",
"id": "enablement-3",
"implementation_fidelity": 0.8,
"negative_control_outcome": 0,Truncated for display — the full payload is 446 lines.
What you get back
This is the actual output of running the example above — computed by the same function the platform calls, not an illustration.
{
"configuration": {
"bonferroni_one_sided_tail_probability": 0.04,
"bootstrap_draws": 200,
"confidence_level": 0.8,
"maximum_absolute_baseline_imbalance": 0.1,
"maximum_absolute_negative_control_effect": 0.1,
"maximum_invalid_ratio_draw_fraction": 0.05,
"minimum_causal_value_per_fidelity_unit": 0,
"minimum_cluster_count": 20,
"minimum_first_stage_fidelity_effect": 0.1,
"seed": 0,
"simultaneous_gate_count": 5
},
"decision": "represented_causal_execution_fidelity_value_supported",
"guardrails": [
"The Wald effect identifies a complier-average causal value only when enablement assignment was genuinely randomized, exclusion holds, assignment affects outcomes only through fidelity, monotonicity is plausible, clusters do not interfere and attrition is handled.",
"Whole-cluster arm bootstrap and first-stage/ratio gates expose weak instruments but do not prove exclusion or monotonicity. The prespecified negative control is a falsification check, not a guarantee.",
"Programs and clusters must be privacy-eligible aggregate units. The estimate separates strategy execution from strategy choice; it is not an employee score, proof of effort, or authority to reward, punish, fund or terminate a person."
],
"method": "cluster_randomized_encouragement_wald_execution_fidelity_value_v1",
"program_diagnostics": [
{
"absolute_baseline_imbalance": 0,
"absolute_baseline_imbalance_simultaneous_upper_bound": 0,
"absolute_negative_control_effect": 0,
"absolute_negative_control_effect_simultaneous_upper_bound": 0,
"causal_value_per_fidelity_unit": 10,
"causal_value_simultaneous_lower_bound": 10,
"cluster_count": 40,
"comparison_cluster_count": 20,
"decision": "causal_execution_fidelity_value_supported",
"enablement_cluster_count": 20,
"failed_gates": [],
"first_stage_fidelity_effect": 0.6,
"first_stage_simultaneous_lower_bound": 0.6,
"invalid_ratio_draw_fraction": 0,
"program_id": "implementation-enablement",
"reduced_form_outcome_value_effect": 6
}
],
"summary": {
"cluster_count": 40,
"program_count": 1,Truncated for display — the full payload is 51 lines.
How it works
Causal inference & experiment design — Separate what a change caused from what merely moved alongside it.
- 1 Randomize implementation enablement at a privacy-eligible aggregate cluster before execution, freeze outcomes, baseline and a negative control, and aggregate any within-cluster rows using prespecified weights.
- 2 Estimate the encouragement first stage on fidelity and reduced form on outcome change, then divide them for a Wald complier-average causal value per fidelity unit.
- 3 Bootstrap whole clusters separately within randomized arms and simultaneously gate cluster support, first-stage strength, ratio stability, baseline balance, negative-control exclusion and the causal-value lower bound.
Before you trust it
Every tool in the catalog ships with the conditions under which its answer is meaningful — and the conditions under which it should abstain instead of guessing.
Assumptions & guardrails
- Assignment or adoption timing, interference policy, overlap, attrition, and outcome availability match the estimand encoded by the method.
- Enablement was genuinely randomized; assignment affects the outcome only through fidelity; no cluster becomes less faithful because it was encouraged; clusters do not interfere; assignment and outcomes are not selectively missing; measurement and cluster boundaries were frozen.
- A causal label is warranted only when design and diagnostic requirements pass; otherwise treat the result as descriptive or abstain.
- The Wald estimate is a local effect for clusters whose fidelity responds to the randomized enablement, not the value of the underlying strategy, a universal productivity coefficient, or an employee-performance score.
Minimum evidence
- observations: at least 40 rows/items
How to validate it
Preserve the assignment/adoption design and validate overlap, pre-period or placebo diagnostics, attrition, interference policy, and cluster-level uncertainty; never tune on the estimated effect.
Calibrating it to your org
Same for everyone
The mathematical kernel, validation rules, method version, and JSON output semantics are organization-independent; no tenant-trained coefficients or company benchmark is embedded in the function.
Specific to you
- prospective cluster-randomized encouragement cohort preserving original assignment, zero and attrited outcomes, frozen fidelity measurement, outcome/baseline/negative-control definitions and every eligible cluster in both arms
- randomization protocol, cluster/interference boundary, enablement and fidelity contract, exclusion/monotonicity claim, attrition handling, finance-valued outcome, baseline and negative control, minimum clusters, first-stage/material-effect/balance/ratio gates, simultaneous confidence and experiment authority
Calibration workflow
- 1 Define the management decision, target outcome, aggregate unit, privacy boundary, cadence, and prediction/intervention horizon for this organization.
- 2 Build a tenant-scoped historical cohort using only information available before each prediction or decision; preserve zero periods, censoring, assignment probabilities, and unresolved outcomes when the method requires them.
- 3 Estimate statistical parameters on training history, but obtain costs, utilities, risk tolerance, practical-effect thresholds, capacity, and policy constraints from accountable decision owners.
- 4 Validate on later time windows or held-out aggregate units at the deployment grain, against a simple baseline and the function-specific validation strategy.
- 5 Deploy only if the returned decision clears evidence, overlap, calibration, robustness, and guardrail checks; warning, unsupported, schema-gap, and fallback decisions are abstentions.
- 6 Monitor realized outcomes, data drift, coverage, and decision regret; recalibrate at a governed cadence or after a detected regime/definition change, never merely because a stakeholder dislikes the result.
Call it from your AI
You don't wire up 388 tools in your MCP client. The GitRevio MCP server exposes 18 tools, three of which let an agent search the catalog, read a tool's schema, and run it — so the assistant finds this one on its own.
gitrevio_capabilities_search
{ "q": "estimate how much outcome value an" }
→ finds "estimate_causal_value_of_execution_fidelity"
gitrevio_capability_describe
{ "capability_id": "estimate_causal_value_of_execution_fidelity" }
→ returns the input schema and agent guidance shown on this page
gitrevio_capability_run
{ "capability_id": "estimate_causal_value_of_execution_fidelity", "arguments": { ... } }
→ returns the result shown aboveWorks in Claude Desktop, Claude Code, Cursor, Cline, Continue.dev, Goose and Aider. See the MCP server.
Related tools
Audit decision execution fidelity
Audit whether approved decisions actually became verified implementation at the promised aggregate-unit and component grain, with whole-unit bootstrap uncertainty and simultaneous gates for fidelity, overdue scope, unverifiable evidence, exceptions and critical gaps.
Calculate execution value leakage
Translate incomplete scope, delay-driven value decay, rework and approved-exception costs into coherent expected, reserve-quantile and tail-CVaR execution leakage, with exact decision-level reconciliation to net realized value.
Audit commercial technical commitment integrity
Audit signed commercial promises against explicitly allocated technical scope, dependency order, funded capacity, acceptance criteria and evidence; expose orphan scope, double allocation, cycles, late plans and maximum contractual penalty without interpreting legal rights from engineering activity.
Audit KPI threshold bunching
Detect a post-target excess concentration immediately above a governed KPI threshold: compare within-unit pre/post local mass and above-versus-below mirror asymmetry, bootstrap whole units, report density bins and a smoothed log-density jump, and explicitly refuse to equate bunching with individual gaming or intent.
Audit release risk prediction integrity
Audit a complete eligible-change release-risk cohort for point-in-time prediction lineage, exact change-to-deployment linkage, mature mutually exclusive outcomes, selective labels, score-triggered intervention contamination, calibration and false alarms before the score influences a release decision.
Audit sunk cost escalation
Audit whether cumulative sunk cost predicts aggregate project continuation after project fixed effects, checkpoint time, forward value, success probability, remaining cost, and future irreversibility, with project-cluster bootstrap uncertainty.