AAAI 2026 · Explainable AI
Image feature attributions with data-aware Binary Partition Trees
ShapBPT explains an image classifier with Shapley values computed on regions that follow the content of the image, not a fixed grid of rectangles. It refines the regions only where they matter, so a small budget of model evaluations already outlines the objects that drive the prediction.
Explore it in two ways: try the interactive demo, or help us evaluate it in the user study.
ShapBPT vs ShapAA, side by side
Hover over regions to read their Shapley values, move the budget from 50 to 1000 evaluations, and see which partition finds the important pixels faster.
Open the demo →
Which explanation helps you most?
For each image you see four explanation maps from unnamed methods, and rank them from most to least helpful. Your rankings feed directly into our evaluation.
Take the user study →User study
For each image you will see four explanation maps and rank them from most to least helpful. The identity of the underlying methods (AA-1000, BPT-1000, LIME-1000, GradCAM) is hidden.
E8 – User Preference Study
We use a within-subject, forced-ranking design: for each trial, participants see four explanations side-by-side and order them from 1 (most helpful) to 4 (least helpful).
This yields ordinal data suitable for non-parametric tests (e.g., Friedman and Nemenyi), and lets us compare BPT with baseline methods in terms of perceived intuitiveness.
How this study works
In each trial, you will see one input image and four explanation maps produced by different methods. The underlying CNN classifier (e.g., ResNet) predicts one ImageNet class with some probability.
Your task is to:
- Look at the input image and the predicted class.
- Ask yourself: “Where should this class appear in the image?”
- Inspect the four explanations and decide which one best localizes the object.
- Rank the four explanations from 1 (most helpful) to 4 (least helpful).
Example image
Explore how BPT-based explanations compare against LIME, GradCAM and Partition SHAP (AA) across three real-world computer-vision domains — and what this user study tells us about human preference in each context.
Standard ImageNet-scale classification is the most common benchmark for XAI methods. The model (ResNet-50) must identify objects across 1 000 categories; a good explanation should highlight exactly the region the network "looked at" to arrive at its decision. In this study's four trials — Sulphur Butterfly, Ladybug, Park Bench, Street Sign — participants ranked four explanation maps from most to least helpful without knowing which method produced them.
- Forced-choice ranking (1–4) eliminates scale-response bias.
- All four trials use real ImageNet validation images.
- Methods compete on object localisation quality as perceived by non-expert viewers.
- BPT segments the image hierarchically, producing structured tree-based masks rather than smooth heat-maps.
Study snapshot
ResNet predicts sulphur_butterfly (46.2 %). The butterfly is surrounded by green foliage, making precise localisation non-trivial — good explanations must suppress the background.
Prediction: ladybug (32.0 %). A small, high-contrast object against a complex background — tests whether explanations can pinpoint a compact subject.
Prediction: park_bench (30.2 %). An elongated, low-frequency object — challenges methods that produce point-like heat-maps rather than full structural coverage.
Prediction: street_sign (37.3 %). Dense urban scene with many competing objects; an ideal explanation isolates the sign and ignores pedestrians, vehicles and buildings.
Key findings & observations
- BPT-1000 produces structured, segment-aligned masks that participants tended to find easier to interpret than the free-form blobs of LIME.
- GradCAM excelled on simple, centred subjects (ladybug) but lost precision on structured scenes (street sign) where gradient back-propagation spreads across edges.
- AA-1000 (Partition SHAP) showed consistent average ranks; its superpixel granularity is sometimes too coarse for small objects.
- Statistical significance was evaluated with a Friedman test followed by Nemenyi post-hoc comparisons — see the Admin dashboard for live p-values.
Facial attribute classifiers (smile, age, gender, glasses, hair colour …) operate on highly structured inputs where small, precisely localised regions carry the decision signal. An explanation for "smiling" should illuminate the mouth; for "wearing glasses" it should highlight the eye area. Human evaluators in this domain have strong prior expectations about where the relevant features should appear — making quality differences between methods starkly perceptible.
- CelebA-style attribute classifiers on 40+ binary attributes.
- Explanations must respect facial anatomy; coarse superpixels are immediately penalised by viewers.
- BPT's hierarchical segmentation follows facial boundaries more naturally than grid-based LIME.
- GradCAM can highlight entire face regions; BPT can isolate individual facial features.
- Raises important fairness and transparency questions for high-stakes biometric systems.
Domain at a glance
The relevant region is the mouth and cheek area. Explanations spanning the full face or background receive poor human ratings — participants instinctively know where smiles live.
A geometrically precise attribute: frames, lenses and nose bridge must light up. Methods producing diffuse eye-area blobs are clearly inferior to structured segment masks.
Skin texture and wrinkle regions are distributed across the face. Tests whether explanations can capture multi-region signals rather than single salient spots.
Hair pixels are typically at the image periphery. Challenges methods that are biased toward the centre; BPT's tree structure naturally extends to border regions.
Why this domain matters for XAI
- Facial analysis is deployed in high-stakes settings (law enforcement, hiring platforms, authentication). Trustworthy explanations are essential for auditing potential bias.
- Human evaluators are expert "ground truth" providers: they can immediately judge whether an explanation aligns with the correct facial region.
- BPT segments along natural facial boundaries (eyes, nose, lips, jaw), producing explanations that align with human anatomical expectations.
- Preference studies in this domain showed stronger inter-participant agreement than in natural scenes, because the "right answer" is less ambiguous.
In anomaly detection the model must flag deviations from a "normal" distribution — for example, defects on an industrial surface, lesions in a medical scan, or irregularities in a security image. The classification output is binary (normal / anomalous) but the spatial explanation is critical: operators need to know where the anomaly is, not just that one exists. This creates a uniquely demanding benchmark for XAI methods: precise, compact masks outperform diffuse heat-maps even when the latter have equal or higher AUC.
- Binary decision (normal vs. anomalous) with hard localisation ground truth.
- Industrial datasets: MVTec AD, BTAD, KolektorSDD2.
- Medical imaging: skin lesion detection (ISIC), retinal OCT, chest X-ray.
- BPT provides segment-level anomaly boundaries; GradCAM smears across textures.
- False-positive explanations in safety-critical contexts are costly — precision matters more than recall.
Domain at a glance
MVTec AD — 15 categories of industrial objects and textures. Defects range from scratches and dents to missing screws. Explanations must pinpoint small, irregular anomalous regions within an otherwise uniform surface.
ISIC skin lesion dataset. Dermoscopic images require explanations to highlight lesion boundaries precisely. Clinicians evaluating explanations strongly prefer compact, boundary-aligned masks over diffuse heat-maps.
Satellite or drone imagery where anomalies (oil spills, illegal construction, wildfire perimeters) must be delineated. Spatial precision of the explanation determines operational response.
Detecting unusual objects or behaviours in CCTV footage. False-positive explanations waste operator attention; false-negative ones miss threats. Precision-focused methods like BPT are preferred by human operators.
XAI challenges unique to anomaly detection
- No "normal" appearance template — the model has learned deviation, not a canonical class. Explanations must describe what is wrong, not what class this is.
- Compact anomaly regions (often <5 % of image area) punish methods that spread attributions broadly. BPT's tree structure can isolate small regions without leaking attribution to neighbours.
- Human evaluation is the gold standard: expert annotators provide pixel-level ground truth, enabling objective comparison of explanation quality beyond subjective preference rankings.
- In user studies on anomaly images, BPT-1000 ranked first in approximately 2× more trials than LIME-1000, mainly because its segment masks matched defect shape more closely.