How to Interpret XNN Results

Step 1: identify the prediction task

First determine whether the model performs binary classification, multi-class classification or regression.

For binary classification, inspect the output and the decision threshold. A value of 0.62 may map to the positive class at a 0.5 threshold, but the operational meaning depends on how the threshold was selected.

For multi-class classification, review the score for every class, not only the winning label. Closely grouped probabilities communicate a different situation from one dominant class.

For regression, confirm the output unit. The same unit should apply to the background value and raw module attributions.

Step 2: reconcile the output

The pre-activation output is built from a background value plus module attributions. The background value is the model's baseline or intercept. Each attribution shows how an active module moves the query away from that starting point.

For a regression model:

prediction = background value + sum of module attributions

For classification, the combined score is transformed by the relevant activation function. The values should be interpreted on the correct side of that transformation.

Reconciliation is a useful quality check. If the displayed contributions do not reproduce the reported output within the stated rounding tolerance, the view requires investigation.

Step 3: rank direction and magnitude

Sort raw attributions to see which modules pushed the result up or down. Sort absolute attributions to see which had the greatest magnitude. Do not conflate the two.

Normalised values can support a quick visual summary, but always check the normalisation scope and retain access to raw values. A module contributing -0.4 and another contributing +0.4 have equal magnitude but opposite meaning.

Step 4: inspect interactions and feature rollups

Module-level results may include both individual features and interactions. A large Income × Employment Stability contribution should not be presented as if it belonged wholly to either feature.

Use decomposed attributions to see how the interaction contribution is distributed, then review the feature rollup to understand the net contribution of each original input across all modules.

Step 5: open the rule view

The rule view shows which partition activated in each module. Review:

• the IF condition and feature bins;
• the THEN expression used to compute the attribution;
• the raw input and transformed values;
• the rule or partition identifier; and
• the output-specific expression for multi-class models.

The rule view explains local behaviour. A module dependency view provides the broader pattern across neighbouring partitions and can reveal thresholds, monotonic trends or abrupt changes.

Step 6: examine activation context

Activation frequency indicates how common a rule or partition is across the comparison dataset. Impactful frequency indicates how often it materially affects results.

A rare activation is not automatically wrong, but it may represent an edge case with limited data support. An unfamiliar path can also be an indicator of out-of-distribution input or drift. Interpretation should therefore combine the local result with the module's usage profile.

Step 7: compare where useful

Use what-if analysis to change a specified input, a counterfactual to search for a target outcome, or a what-if-not test to isolate the effect of a factor. Compare the prediction, active rules and attributions, not just the final label.

The comparison should respect feasibility constraints and should not turn immutable or protected attributes into inappropriate recommendations.

Step 8: check lineage and integrity

For decisions requiring audit, retain the model identifier, version, query time, input reference, active components and explanation metadata. A Query Unique Verification Code (UVC) may provide a tamper-evident fingerprint over selected elements and incorporate the Model UVC.

A matching UVC supports the claim that the checked information is unchanged relative to the reference. It does not validate the substantive quality of the original decision.

Why it matters

An explanation can be technically correct and still be misread. A consistent interpretation sequence prevents common errors: treating logits as probabilities, ignoring the baseline, losing interactions in rollups, overlooking rare activation paths or assuming that integrity verification proves fairness.