OATML

Sergio is a DPhil student in the Random Systems CDT at the University of Oxford, co-supervised by Álvaro Cartea, Yarin Gal, and José Miguel Hernández-Lobato (University of Cambridge). His research interests lie broadly in Probabilistic Machine Learning. In particular, he studies the connections between Gaussian Processes and Deep Learning, and explores advances in Generative Modelling (particularly LLMs, diffusion models, and flow matching). His work has two main aims: to develop theoretically grounded and efficient techniques for quantifying model uncertainty, and to design methods that accelerate training and/or inference in state-of-the-art generative models.

Before arriving in Oxford, Sergio completed the MPhil in Machine Learning and Machine Intelligence at the University of Cambridge and earned a BSc in Theoretical Physics from QMUL. He has prior industry experience in both quantitative finance and big tech. As a Research Scientist Intern at Spotify, he worked at the intersection of Gaussian Processes and Deep Learning theory. Later, as a Quant Research Intern at a hedge fund, he developed an LLM-based systematic trading strategy. He is funded by the Man Group through the Oxford-Man Institute Scholarship.

Richer Bayesian Last Layers with Subsampled NTK Features

Bayesian Last Layers (BLLs) provide a convenient and computationally efficient way to estimate uncertainty in neural networks. However, they underestimate epistemic uncertainty because they apply a Bayesian treatment only to the final layer, ignoring uncertainty induced by earlier layers. We propose a method that improves BLLs by leveraging a projection of Neural Tangent Kernel (NTK) features onto the space spanned by the last-layer features. This enables posterior inference that accounts for variability of the full network while retaining the low computational cost of inference of a standard BLL. We show that our method yields posterior variances that are provably greater or equal to those of a standard BLL, correcting its tendency to underestimate epistemic uncertainty. To further reduce computational cost, we introduce a uniform subsampling scheme for estimating the projection matrix and for posterior inference. We derive approximation bounds for both types of subsampling. Empir... [full abstract]

---

Sergio Calvo Ordoñez, Jonathan Plenk, Richard Bergna, Álvaro Cartea, Yarin Gal, Jose Miguel Hernández-Lobato, Kamil Ciosek
arxiv
[paper]