How XNN Attributions Explain Decisions

A contribution, not a generic importance score

Feature importance is often a global statistic summarising how strongly a model relies on a feature across many examples. Attribution is more specific. It describes the contribution of a module, feature or interaction for a particular query or defined group.

In an XNN, the active rule's THEN expression calculates the module attribution. That value is part of the model's output computation. It is therefore possible to connect the contribution to the exact conditions and local expression that generated it.

The additive prediction structure

At a high level, an XNN combines a background value with the active module attributions:

pre-activation output = background value + sum of module attributions

For regression, the identity function returns the resulting value. For binary classification, a sigmoid converts the combined score to a probability-like output. For multi-class classification, class-specific scores are passed through a softmax.

The sign of an attribution shows direction; its magnitude shows strength on the relevant output scale. For regression, raw attributions inherit the unit of the prediction. For classification, the contribution may be on a score or logit scale and should not be described as a direct probability change unless the output has been transformed accordingly.

Raw, absolute and normalised attributions

Raw attribution retains the signed contribution. It answers whether a component increased or decreased the output and by how much on the model's scale.

Absolute attribution removes the sign. It is useful when ranking components by magnitude, but it cannot show direction.

Normalised attribution expresses relative importance within a defined group. Normalised values can support charts and comparisons, but the normalisation scope must be stated: one prediction, one module, a population or the entire model. Normalising signed values requires care; a percentage chart should not obscure opposing contributions.

Interaction attributions

An interaction module represents combined behaviour such as Age × Education. Its total attribution belongs to the interaction, but a feature-level explanation may need to distribute that value between the original inputs.

XNN decomposition evaluates the local function to estimate each input's intermediate effect, then allocates the module output proportionally. The decomposition must conserve the total:

sum of decomposed feature attributions = interaction module attribution

If an Age × Marital Status module contributes -0.15461, a possible decomposition is Marital Status = -0.16348 and Age = +0.00887. The interaction is negative overall, but the decomposition shows that age slightly mitigated the larger negative component.

Feature rollup

The same feature may appear in several modules. Age may have a single-feature module and also participate in interactions with education, occupation or income. A feature rollup collects Age's decomposed contribution from each relevant module and sums them.

This produces an original-feature view that is easier to read while preserving reconciliation with the module-level explanation. The rollup should always allow a technical reviewer to drill back into the source interactions.

Activation and impact

Attribution is complemented by activation metrics:

• Activation frequency: how often a module, partition or rule is used.
• Impactful frequency: how often that activation materially affects the output.

A rule may activate frequently but contribute very little. Another may be rare but highly influential. Looking at both measures helps identify common logic, niche edge cases and components that deserve scrutiny.

Interpreting attribution responsibly

Attribution describes the model's computation. It does not automatically establish real-world causation, fairness or policy appropriateness. High attribution for a sensitive feature is a signal for investigation. Low direct attribution does not prove absence of indirect effects because the feature may be represented through interactions or correlated variables.

Attribution analysis becomes stronger when combined with rule inspection, control swaps, subgroup evaluation and external domain knowledge.

Why it matters

Attributions connect technical computation with a readable account of influence. They enable local explanations, module diagnostics, feature rollups and global importance analysis without introducing a different explanation model.