UMNAI
Collection 3

How Feature Rollups, Rule Views and Attribution Views Work

Explainable Neural Network (XNN) views answer different questions about the same execution. An attribution view shows contribution, a rule view shows the logic that activated, and a feature rollup brings direct and interaction effects back to the original inputs.

By Angelo Dalli8 min read
Feature rollupRule viewIncomeUtilisationTenureAge bandIF utilisation > 0.7AND income highAND income lowTHEN declineTHEN reviewSame query, complementary views
The same query rendered two ways: original-feature rollup on the left, active IF-THEN rule tree on the right.

Why one view is not enough

A concise explanation and a technical audit rarely need the same structure. Showing only original features can hide interactions. Showing only modules can overwhelm an operational user. Showing rules without their attribution values explains where the query went but not which components mattered most.

Hybrid Intelligence addresses this through complementary views with a common underlying record.

Attribution view: what contributed

The attribution view lists the signed output of each active module. It preserves the XNN's computational structure:

  • the background value;
  • single-feature module attributions;
  • interaction-module attributions;
  • the attribution sum; and
  • the final output or pre-activation score.

Raw values show direction and scale. Absolute values rank magnitude. Normalised values support relative charts. The view should state which form is displayed and the scope of normalisation.

An interaction attribution belongs to the interaction module. It should remain visible even when the user also receives a feature rollup, because it shows that the joint behaviour was modelled explicitly.

Rule view: which logic applied

The rule view opens the active partition for each module. It includes the IF condition and the THEN expression used to calculate the attribution.

The IF condition may use numeric ranges, encoded categorical values or symbolic bins. The THEN expression is generally linear or polynomial. For a multi-class model, it may include a separate expression for each output.

Rule identifiers and feature-bin links help technical reviewers compare behaviour across queries and versions. A decision-table presentation can show several partitions together, while a single-query view highlights only the active rule.

Feature rollup: which original inputs mattered

Original features can appear in several places. Age may have its own module and may also participate in Age × Education and Age × Employment Status.

To create the rollup:

  1. Retain each single-feature attribution.
  2. Decompose every relevant interaction attribution into feature components.
  3. Collect all components associated with the same original feature.
  4. Sum them to produce the net feature attribution.

The rollup can be sorted by signed or absolute contribution. The total should reconcile with the modules included, subject to the treatment of the background value and rounding.

Module dependency view: how behaviour changes

A local rule explains one point. A module dependency view shows how the attribution changes across feature values and partitions. It can reveal monotonic increases, decreasing regions, plateaux and discontinuities at rule boundaries.

For an interaction module, a surface or grouped plot can show how two inputs behave together. Visual complexity should remain controlled; a technical viewer can inspect the full structure while a business viewer receives a focused slice.

Choosing a view by question

  • Which inputs mattered most? Use the feature rollup.
  • Which learnt interaction drove the result? Use the module attribution view.
  • Which condition was met? Use the rule view.
  • How does the module behave across values? Use the dependency view.
  • Can I reconstruct the output? Use background plus raw module attributions.

The views should link to each other. Selecting a feature in the rollup can reveal the source modules; selecting a module can reveal the active rule; selecting a rule can show its neighbouring partitions and usage statistics.

Risks in presentation

A rollup can conceal offsetting interaction effects. Normalisation can hide negative direction. Rule text can look like policy even when it represents learnt statistical behaviour. Dependency plots can suggest causation when they only describe the model response.

Clear labels and linked drill-downs help maintain the distinction between model behaviour, domain interpretation and organisational policy.

Why it matters

Complementary views allow different users to work from the same model history. They make explanations concise where possible and technically inspectable where necessary, without substituting a separate approximation for the XNN computation.

Continue the journey

Continue with Model Lifecycle.