A Metacognitive Perspective on Anomaly Detection with Vision Foundation Models

In short A metacognitive perspective on anomaly detection: a simpler scorer ranks defects better but decides worse until its threshold is recalibrated, so ranking and decisions should be evaluated separately.

• Computer vision
• Metacognition
• Anomaly detection

The paper applies the two-level model of metacognition (Nelson and Narens, 1990) to structural anomaly detection. A frozen Vision Foundation Model is the object level: it produces patch features but has no way to judge them. The scorer is the meta level and stores a model of what normal features look like, and the threshold that turns its anomaly map into a defect mask is control. In the metacognition literature, monitoring and control can fail independently (Fleming and Lau, 2014): a scorer can rank defective pixels correctly, i.e., have high metacognitive sensitivity, and still place its threshold at the wrong level. Threshold-free metrics such as AU-PRO measure the former, while segmentation F1 depends on both, and therefore cannot show which of the two is responsible when performance changes.

To isolate the meta level, we keep the encoder of SuperADD (Roming et al., 2026), which won the VAND 4.0 challenge on MVTec AD 2, and replace its nearest-neighbor memory bank with a single Gaussian per layer, scoring each patch token by its Mahalanobis distance. The fitted model takes 52.47 MB across the four layers regardless of the size of the training set, and scores each token about 78 times more cheaply than nearest-neighbor search.

On MVTec AD 2, the two functions dissociate. The Mahalanobis scorer improves AU-PRO at 0.05 from 56.81 to 64.81, yet under the dataset's threshold of the mean plus three standard deviations its segmentation F1 drops to 19.88, against 33.34 for the memory bank, since that threshold is set by the center of the score distribution rather than by its tail. Fitting the tail on normal images with Extreme Value Theory raises F1 by 29.51 points for the Mahalanobis scorer and by 16.66 for the memory bank, narrowing the gap between the two from 13.46 to 0.61 points without using any anomalous data. Across MVTec AD and VisA, the Mahalanobis scorer then comes out ahead, by 10.78 and 8.74 points respectively. Even at the oracle threshold, however, the memory bank keeps a 6.73-point lead on MVTec AD 2, so only part of the original deficit was due to threshold placement.

Better monitoring, in this setting, translates into better decisions only once control is calibrated to it. As such, the paper recommends evaluating and reporting threshold calibration separately from ranking quality.

Why the threshold matters in industrial inspection

On a production line, a decision is what stops a part. An anomaly map that orders pixels well still has to be turned into a binary defect mask, and that is where an inspection system commits to an action. Many anomaly detection papers report only the best F1 reachable by sweeping the threshold over the labeled test set (F1max), which describes ranking quality and gives a deployed system no rule to follow. The default of MVTec AD 2, the mean plus three standard deviations of scores on defect-free images, is a deployable rule, yet it implicitly assumes a Gaussian score tail. Fitting the tail directly with Extreme Value Theory (EVT) needs only defect-free images and a chosen false-positive budget, so threshold calibration does not require collecting examples of defects.

Deployment constraints add a second reason to care about the scorer. For the ViT-H+ backbone on MVTec AD 2, a nearest-neighbor memory bank must be stored and searched for every token, whereas the Gaussian model stays at 52.47 MB regardless of the training set and scores each token about 78 times more cheaply, which makes it relevant to latency- and memory-constrained industrial anomaly detection.

How it connects to other anomaly detection work

The two scorers sit in two well-known families. Memory-bank methods such as PatchCore (Roth et al., 2022) store normal patch features and score by nearest-neighbor distance, whereas parametric methods summarize normality with a statistical model, as in Rippel et al. (2021) and PaDiM (Defard et al., 2021). Our contribution is to hold the DINOv3 backbone fixed so that the effect of the scorer can be read on its own, which is the kind of control that foundation-model anomaly detection needs now that encoders and scorers evolve in parallel. The distinction between ranking and deciding also applies to out-of-distribution detection more broadly, since any detector that raises an alarm must eventually choose a threshold, and the threshold-free and thresholded views can disagree there as well.

The open question we find most interesting is how benchmarks should report the two sides. If a method is credited only for its ranking, a better scorer may look worse in production, and if it is credited only for its F1 at a default threshold, a better scorer may look worse in the paper.

The same two-level reading frames our framework for test-time adaptation, presented at the same symposium.

Cite

@article{galiazzo2026metacognitive,
  title     = {A Metacognitive Perspective on Anomaly Detection with Vision Foundation Models},
  author    = {Matteo Galiazzo and Gabriele Onorato and Dario Loi and Nathaniel D. Bastian and Francesco Restuccia},
  journal   = {Proceedings of the AAAI Symposium Series},
  volume    = {10},
  number    = {1},
  pages     = {110--114},
  year      = {2026},
  month     = nov,
  publisher = {Association for the Advancement of Artificial Intelligence (AAAI)},
  doi       = {10.1609/aaaiss.v10i1.43397},
  url       = {https://ojs.aaai.org/index.php/AAAI-SS/article/view/43397}
}