How XNN Predictions are Built
Explainable Neural Network (XNN) predictions are additive before the final task-specific transformation. Each active module rule produces an attribution, those contributions are combined with a background value, and an identity, sigmoid or softmax function produces the model output.
The common structure
Although regression and classification return different forms of output, their explanations share a common structure. Each XNN module evaluates the query, selects one partition and executes its local rule. The resulting module attribution contributes to the model score.
At a high level:
z = B0 + Σ Mm(x)
ŷ = σ(z)
B0 is the background value, Mm(x) is the attribution produced by module m, and σ is the output function.
This structure makes the components additive on the pre-activation scale. It also provides a direct reconciliation route from the active modules to the output.
The background value
The background value is the common starting point or intercept. In a regression model it may resemble an average baseline. A positive attribution raises the prediction relative to the baseline; a negative attribution reduces it.
The background should not be described as an explanation for the individual case. It is the reference from which case-specific contributions are measured.
Regression
For regression, the output function is the identity: the combined value is returned directly.
If a property model has a background value of £334,000 and the active modules sum to £635,000, the prediction is £969,000. If the module sum is -£56,000, the prediction is £278,000.
Raw regression attributions inherit the unit of the target, making them readily interpretable. They still require context: a £20,000 contribution may be material in one valuation range and minor in another.
Binary classification
For binary classification, the combined score is passed through a sigmoid to produce a value between zero and one. A threshold then maps the value to a class. The default may be 0.5, but production thresholds should reflect the model evaluation and decision objective.
It is important to distinguish three objects:
- the additive pre-sigmoid score;
- the post-sigmoid output;
- the class selected by the threshold.
A module attribution is generally not a direct percentage-point change in the final probability. Its effect depends on the combined score and the non-linear sigmoid.
Multi-class classification
In multi-class classification, the model produces a score for each class. Softmax converts the vector into outputs that sum to one, and the class with the highest value is selected.
Each active rule can contain one expression per output class. A module may therefore support one class while opposing another. A useful explanation presents the full class vector and class-specific contributions where the distinction is material.
Rules behind the numbers
The module attribution is produced by the active partition's THEN expression. The IF condition defines the local region, for example a numeric range or categorical combination. The THEN expression may include linear, quadratic and interaction terms.
This local structure allows the same feature to behave differently in different regions while retaining explicit logic. A threshold in the partition hierarchy may create a visible change in the attribution curve.
Feature and interaction views
The model is additive by module, not necessarily by original feature. Interaction modules need decomposition before their contributions can be included in a feature rollup.
The module view is the most direct representation of the computation. The feature rollup is a derived but reconcilable presentation designed for readability. Both should remain available for technical review.
From prediction to operational decision
The model may return a score or class, but the operational system may apply a policy threshold, evidence requirement or manual review step. The product interface should record these separately. A changed business threshold does not alter the underlying XNN prediction; it changes how the organisation acts on it.
Why it matters
Understanding the output construction prevents attribution and probability from being confused. It also enables reconciliation, threshold governance and consistent explanation across regression, binary and multi-class applications.
