Scalable Linearized Laplace Approximation via Surrogate Neural Kernel

L.A. Ortega, S. Rodríguez-Santana, D. Hernández-Lobato

34th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning - ESANN 2026, Brujas (Bélgica). 22-24 abril 2026

Resumen:

We introduce a scalable method to approximate the kernel of the Linearized Laplace Approximation (LLA). For this, we use a surrogate deep neural network (DNN) that learns a compact feature representation whose inner product replicates the Neural Tangent Kernel (NTK). This avoids the need to compute large Jacobians. Training relies solely on efficient Jacobian-vector products, allowing to compute predictive uncertainty on large-scale pre-trained DNNs. Experimental results show similar or improved uncertainty estimation and calibration compared to existing LLA approximations. Notwithstanding, biasing the learned kernel significantly enhances out-of-distribution detection. This remarks the benefits of the proposed method for finding better kernels than the NTK in the context of LLA to compute prediction uncertainty given a pre-trained DNN.

Palabras clave: Deep neural networks; Learning systems; Linearization; Scalability

DOI: https://doi.org/10.14428/esann/2026.ES2026-71

Publicado en: ESANN 2026: Proceedings 34th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pp: 709-714, ISBN: 9782875870957

Fecha de publicación: 24-mar-2026

Cita:
L.A. Ortega, S. Rodríguez-Santana, D. Hernández-Lobato, "Scalable Linearized Laplace Approximation via Surrogate Neural Kernel", presentado en 34th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning - ESANN 2026, Brujas, Bélgica, 22-24 abril 2026. En: ESANN 2026: Proceedings 34th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pp. 709-714, doi: 10.14428/esann/2026.ES2026-71

IIT-26-270C