Audit multigroup metric measurement invariance
Audit whether a multi-indicator aggregate management metric measures a comparable one-factor construct across teams, products, repositories, periods, or companies: fit training-only pooled and group PCA loadings, test configural dominance, metric loading cosine, scalar intercept range, residual variance, and untouched-test reconstruction invariance before any group ranking is trusted.
What it's for
Protects every executive comparison Gitrevio makes: before calling one team, portfolio company, or period better, it tests whether the composite metric actually has the same measurement meaning across them.
What you give it
Inputs split into evidence read from your connected systems, calibration your team owns, and numerical controls that affect precision but never the result's meaning.
| Field | Type | Role | Required |
|---|---|---|---|
| indicator_names | array of string ≥ 3 items | Evidence | Yes |
| max_detail_rows | integer ≥ 1, ≤ 500 | Numerical control | Optional |
| maximum_normalized_intercept_range | number ≥ 0, ≤ 10 | Your calibration | Optional |
| maximum_residual_variance_ratio | number ≥ 1, ≤ 100 | Your calibration | Optional |
| maximum_test_reconstruction_ratio | number ≥ 1, ≤ 100 | Your calibration | Optional |
| minimum_factor_dominance | number ≥ 0, ≤ 1 | Your calibration | Optional |
| minimum_loading_cosine | number ≥ 0, ≤ 1 | Your calibration | Optional |
| observations | array of objects (4 fields) ≥ 360 items | Evidence | Yes |
Each observations
record
| Field | Type | Required |
|---|---|---|
| group_id | string (non-empty) | Yes |
| id | string (non-empty) | Yes |
| indicators | object | Yes |
| split | one of "train", "test" | Yes |
{
"indicator_names": [
"delivery",
"reliability",
"quality"
],
"maximum_normalized_intercept_range": 0.25,
"maximum_residual_variance_ratio": 5,
"maximum_test_reconstruction_ratio": 2,
"minimum_factor_dominance": 0.8,
"minimum_loading_cosine": 0.95,
"observations": [
{
"group_id": "group-0",
"id": "measurement-0-train-0",
"indicators": {
"delivery": -0.95,
"quality": -1.15,
"reliability": -0.8
},
"split": "train"
},
{
"group_id": "group-0",
"id": "measurement-0-train-1",
"indicators": {
"delivery": -0.7255809254520701,
"quality": -0.9095574736093486,
"reliability": -0.553898923325522
},
"split": "train"
},
{
"group_id": "group-0",
"id": "measurement-0-train-2",
"indicators": {
"delivery": -0.5413818032231541,
"quality": -0.6730292633039294,
"reliability": -0.3906845005255644
},
"split": "train"
},
{
"group_id": "group-0",Truncated for display — the full payload is 3614 lines.
What you get back
This is the actual output of running the example above — computed by the same function the platform calls, not an illustration.
{
"assumptions": [
"A one-factor reflective measurement model is substantively meaningful for these indicators and groups; indicator direction and definitions are identical.",
"Groups and train/test epochs are privacy-eligible, independently defined, sufficiently sampled, and not selected after observing invariance failures.",
"PCA loading similarity, intercept range, residual variance, and reconstruction gates are practical diagnostics rather than a universal chi-square equivalence test."
],
"decision": "aggregate_metric_measurement_invariance_supported",
"executive_summary": {
"failed_gates": [],
"maximum_normalized_intercept_range": 0.0013,
"maximum_residual_variance_ratio": 1.0545,
"minimum_factor_dominance": 0.9982,
"minimum_loading_cosine": 1,
"test_reconstruction_ratio": 1.0065
},
"group_diagnostics": [
{
"group_id": "group-0",
"test_factor_dominance": 0.9982,
"test_loading_cosine": 1,
"test_reconstruction_rmse": 0.0275,
"train_factor_dominance": 0.9982,
"train_loading_cosine": 1
},
{
"group_id": "group-1",
"test_factor_dominance": 0.9982,
"test_loading_cosine": 1,
"test_reconstruction_rmse": 0.0277,
"train_factor_dominance": 0.9982,
"train_loading_cosine": 1
},
{
"group_id": "group-2",
"test_factor_dominance": 0.9982,
"test_loading_cosine": 1,
"test_reconstruction_rmse": 0.0277,
"train_factor_dominance": 0.9982,
"train_loading_cosine": 1
}
],
"group_support": [
{
"group_id": "group-0",Truncated for display — the full payload is 85 lines.
How it works
Statistical audit & measurement — Check whether a number is fit to decide on: coverage, timing, reconciliation, and the gaps a dashboard hides.
- 1 Freeze the construct theory, indicator definitions and directions, privacy-eligible groups, and within-group train/test split; require exact indicator schemas and adequate support in both splits.
- 2 Standardize from pooled training data only, extract the pooled first principal direction, align each group's train and test loading sign, and measure factor dominance plus cosine agreement as practical configural and metric-invariance diagnostics.
- 3 Regress standardized indicators on the pooled factor inside each training group, compare factor-zero intercept ranges and residual-variance ratios, then project untouched test observations to compare reconstruction error across groups.
- 4 Fail group comparability when any prespecified configural, loading, intercept, residual, or held-out reconstruction gate fails; expose indicator and group diagnostics rather than publishing a misleading latent league table.
Before you trust it
Every tool in the catalog ships with the conditions under which its answer is meaningful — and the conditions under which it should abstain instead of guessing.
Assumptions & guardrails
- Metric definitions, weights, aggregate grain, sampling, missingness, dependence, and comparison windows correspond to the management claim being audited.
- A reflective one-factor measurement model is substantively defensible, indicators are directionally aligned and version-identical, and the latent construct is not known to be multidimensional or formative.
- Groups and holdout epochs are independently defined, privacy-eligible, sufficiently sampled, and not selected or merged after observing results.
- Practical PCA loading/intercept/residual/reconstruction tolerances match the management comparison; they are not a substitute for domain validation or every form of multi-group CFA.
- Association, instability, or measurement quality is not a causal effect and must not be converted directly into an individual employment decision.
- Passing supports comparability only for this indicator system and represented groups; it does not prove construct validity, fairness, causality, or that ranking groups is useful.
- Failure can reflect measurement translation, instrumentation, multidimensional work, or genuine structural differences—not poor team or individual performance.
- Never turn a non-invariant latent score into an employment, compensation, outsourcing, fraud, or security decision.
Minimum evidence
- observations: at least 360 rows/items
- indicator_names: at least 3 rows/items
How to validate it
Validate on future periods or held-out aggregate units, compare with a simple baseline, and require stability across plausible metric definitions and decision thresholds.
Calibrating it to your org
Same for everyone
The mathematical kernel, validation rules, method version, and JSON output semantics are organization-independent; no tenant-trained coefficients or company benchmark is embedded in the function.
Specific to you
- one fixed-grain complete indicator row with frozen within-group train/test membership
- directionally aligned indicator values under one zero/missingness and metric-version policy
- construct theory, indicator vocabulary and direction, group meaning, aggregate grain, historical epoch, split, missingness/weighting, factor dominance, loading cosine, intercept, residual, reconstruction, privacy, and publication gates
Calibration workflow
- 1 Define the management decision, target outcome, aggregate unit, privacy boundary, cadence, and prediction/intervention horizon for this organization.
- 2 Build a tenant-scoped historical cohort using only information available before each prediction or decision; preserve zero periods, censoring, assignment probabilities, and unresolved outcomes when the method requires them.
- 3 Estimate statistical parameters on training history, but obtain costs, utilities, risk tolerance, practical-effect thresholds, capacity, and policy constraints from accountable decision owners.
- 4 Validate on later time windows or held-out aggregate units at the deployment grain, against a simple baseline and the function-specific validation strategy.
- 5 Deploy only if the returned decision clears evidence, overlap, calibration, robustness, and guardrail checks; warning, unsupported, schema-gap, and fallback decisions are abstentions.
- 6 Monitor realized outcomes, data drift, coverage, and decision regret; recalibrate at a governed cadence or after a detected regime/definition change, never merely because a stakeholder dislikes the result.
Call it from your AI
You don't wire up 388 tools in your MCP client. The GitRevio MCP server exposes 18 tools, three of which let an agent search the catalog, read a tool's schema, and run it — so the assistant finds this one on its own.
gitrevio_capabilities_search
{ "q": "audit whether a multiindicator aggregate management" }
→ finds "audit_multigroup_metric_measurement_invariance"
gitrevio_capability_describe
{ "capability_id": "audit_multigroup_metric_measurement_invariance" }
→ returns the input schema and agent guidance shown on this page
gitrevio_capability_run
{ "capability_id": "audit_multigroup_metric_measurement_invariance", "arguments": { ... } }
→ returns the result shown aboveWorks in Claude Desktop, Claude Code, Cursor, Cline, Continue.dev, Goose and Aider. See the MCP server.
Related tools
Analyze delayed management feedback stability
Stress the dynamic stability of a delayed signed organizational feedback model: build a VAR companion matrix from interval-valued lagged influences, evaluate midpoint, interval corners, and sampled simultaneous coefficients, calculate spectral and transient amplification margins, and rank one-edge damping leverage without claiming an exhaustive robust-control certificate.
Forecast operational recovery half life
Forecast how quickly operational performance recovers after incidents, migrations, reorganizations, outages, or other shocks: estimate each resolved shock's exponential remaining-loss half-life, retain stalled trajectories at a governed cap, partially pool log half-lives by severity, and simulate current recovery confidence plus cumulative value loss.
Analyze coordination entropy
Quantify privacy-safe cross-team seam complexity, concentration, latency, and failure load.
Audit decision flow integrity
Audit management decision histories for unresolved work, state cycles, unowned dwell and excessive lead-time tails using immutable event sequences, whole-decision bootstrap uncertainty, simultaneous flow-level gates and state bottleneck diagnostics.
Audit executive technology reporting integrity
Audit a frozen executive technology pack for complete metric/risk scope, point-in-time source and definition lineage, numerical reconciliation, supported narrative direction, independent review and evidence coverage.
Audit organizational change simulation integrity
Audit whether an organizational or technology what-if simulation is fit for reliance by checking point-in-time model lineage, local history, factor support, second-order dependency structure, calibration, scenario reconciliation and individual-level safeguards.