How Local, Module-Level and Global Interpretability Work

One model, three scopes

Interpretability breaks down when each audience receives unrelated metrics. XNNs use module attributions, partitions and rules at every scope, allowing analysis to move from a global pattern to a module and then to the individual cases that produced it.

Local interpretability

Local interpretability answers: "Why did the model produce this result for this input?"

A local explanation can include:

• the prediction and relevant threshold or unit;
• the background value;
• signed module attributions;
• decomposed interaction contributions;
• feature rollups;
• active rules and partition conditions;
• activation-path context; and
• query and model identifiers.

This view supports case review, user explanation, debugging and comparison. It is also the starting point for what-if and counterfactual analysis.

Module-level interpretability

Module-level interpretability asks: "How does this feature or interaction behave?"

The reviewer examines the module's partitions, rules, local expressions, attribution curve and activation profile. This can reveal:

• monotonic or non-monotonic behaviour;
• thresholds and discontinuities;
• rarely used partitions;
• high-impact regions;
• interactions that oppose their single-feature effects; and
• potential sensitivity involving protected attributes.

Module dependency plots and partition tables are useful here. For multi-feature modules, the analysis may require slices or surfaces that keep the relevant interaction understandable.

Global interpretability

Global interpretability asks: "What behaviour dominates across a dataset, population or period?"

Common measures include:

• mean absolute attribution by module or feature;
• normalised global attribution;
• activation frequency;
• impactful frequency;
• rule and partition coverage; and
• attribution direction across groups.

Global analysis can identify the modules on which the model relies most, detect underused components and show where behaviour differs between segments. It does not replace performance, fairness or robustness evaluation; it tells reviewers where to investigate.

The comparison dataset must be named and representative of the question. A global ranking over recent production queries may differ from one calculated over the original validation set, a particular region or a protected subgroup. Without that scope, a global statistic can look more universal than it is.

Moving between the levels

The three scopes form a diagnostic loop:

1. A global view identifies a high-impact interaction.
2. The module view shows that its attribution changes sharply at one boundary.
3. Local queries near that boundary reveal the active rules and affected cases.
4. A what-if comparison tests how stable the result is to small changes.
5. Aggregate monitoring determines whether the pattern is growing over time.

The same identifiers and attribution basis allow these steps to stay connected.

Interpreting activation metrics

Activation frequency measures usage. Impactful frequency adds a materiality criterion. Their relationship is informative:

• high activation, high impact: core model behaviour;
• high activation, low impact: frequently evaluated but often weak;
• low activation, high impact: rare behaviour deserving edge-case review;
• low activation, low impact: candidate for examination, simplification or retention as legitimate niche logic.

No component should be pruned from these measures alone. Data coverage and domain significance must be considered.

Monotonicity in context

A monotonic module changes consistently in one direction. This is often easier to validate and may align with domain expectations, such as distance increasing a predicted journey time.

Non-monotonic behaviour is not automatically undesirable. Age or physiological measures may have meaningful effects at both extremes. The important question is whether the shape is supported, stable and acceptable for the use case.

Population views and explanation stability

Global interpretability can be repeated across time windows or model versions. Changes in attribution rankings, activation paths or rule coverage may reveal drift even before a single headline metric deteriorates.

Stable explanations do not mean that every attribution must remain identical. They mean that changes can be measured, located and related to data, structure or parameter updates.

Why it matters

Connected interpretability enables disciplined investigation. It lets teams find an aggregate concern, understand the local model behaviour behind it and identify the individual decisions affected.

It also gives model owners a shared language for moving between executive oversight, technical validation and case-level challenge without changing the evidence base.