How XNNs Support What-If and Counterfactual Analysis

From passive explanation to model exploration

Attribution explains the contributions behind an observed result. Scenario analysis asks how those contributions and the output change under an alternative input. Because XNN modules expose their partitions and local expressions, the comparison can show both the numerical difference and the rule transition that produced it.

What-if analysis

A what-if query changes one or more inputs while holding the remaining values constant. The modified query passes through the same preprocessing, modules, partition conditions and output function as the original.

The result should show:

• the original and modified values;
• the original and new prediction;
• modules and rules that changed;
• attribution differences; and
• any threshold or class transition.

This makes threshold effects visible. A small increase may remain within the same partition and change the local polynomial smoothly. A slightly larger increase may cross a boundary and activate a different rule.

Counterfactual analysis

A counterfactual begins with a goal: reach a target class, exceed a score or avoid an outcome. The system searches for a small or otherwise preferred set of input changes that meets the goal.

The search needs explicit constraints. Values must remain within valid ranges. Categorical changes must be valid. Immutable attributes should not be offered as actions. Policy may prohibit certain variables from being used in recommendations. Multiple counterfactuals may be appropriate when there is no single preferred route.

The output should distinguish:

• the target specified by the user;
• the changes proposed by the model search;
• the constraints applied;
• the new prediction and rule path; and
• whether the result is a model scenario rather than a guaranteed real-world outcome.

What-if-not analysis

What-if-not analysis asks whether a result would persist if a selected factor were absent, masked or substituted. It can reveal dependence on a sensitive feature or on an interaction that is not obvious in the feature rollup.

For example, a fairness review may replace a protected categorical value while holding other inputs constant. The comparison can show whether the final output, active partitions or attributions change. This is evidence about model sensitivity for that case, not by itself a complete fairness assessment.

Control swaps

A control swap is the operational mechanism behind many comparisons:

• Single-feature swap: change one value.
• Minimal-set swap: change a small set together.
• Structural swap: move the query to another meaningful group or partition condition.

The model output is recomputed rather than estimated from a static importance score. This retains the non-linear and interaction effects present in the XNN.

Using the partition hierarchy

The partition hierarchy provides a map of nearby behaviours. An adjacent case within the same partition may show gradual sensitivity. A proximate case in a neighbouring partition may reveal the nearest rule boundary. These comparisons are useful when searching for minimal changes or explaining why similar cases diverge.

Avoiding misleading recommendations

Counterfactuals are often described as actionable, but actionability is contextual. Raising income, changing a medical observation and altering a product configuration have different feasibility and ethical implications. An Explanation Structure Model (ESM) should apply domain constraints and label the scenario appropriately.

Predicted change is not proof of causal effect. If the model has an explicit, validated causal structure, that structure can further constrain intervention analysis. Otherwise, the result should be described as a change in the model's output under altered inputs.

Recording the comparison

For governed use, preserve the original query, modified query, model version, constraints and both sets of explanation data. A new Query Unique Verification Code (UVC) can be generated for each scenario so that the compared records have their own integrity references.

Why it matters

What-if and counterfactual tools make model behaviour testable in terms users can explore. They reveal thresholds, interactions and dependencies, while a traceable comparison shows exactly why the alternative result differs.