Dear Cambridge AI & Machine Learning Enthusiast,
We’re thrilled to announce our next eagerly-anticipated AI & Pizza talk series! Mark your calendars for an insightful evening at 5:15 pm on Thursday, November 16th, 2023. It’s your chance to savour the latest advancements in AI and machine learning, right here in Cambridge.
Location: The Auditorium, 21 Station Rd
Agenda:
5:15 pm – 5:45 pm: Talks
5:45 pm – 7:00 pm: Networking, Pizza, and Refreshments
Cheers,
Wenbo Gong
Speaker:
17:15-17:30: Jihao Andreas Lin, University of Cambridge
Title: Stochastic Gradient Descent for Gaussian Processes Done Right
Abstract: Recently, stochastic gradient descent has gained traction as an alternative to solve the optimisation problem associated with Gaussian process regression. In this talk, we show that when done right — by which we mean using specific insights from the optimisation and kernel communities — this approach is highly effective. We introduce a particular stochastic dual gradient descent algorithm and explain our design decisions by illustrating their advantage against alternatives with ablation studies. Our evaluations on standard regression benchmarks and a Bayesian optimisation task set our approach apart from preconditioned conjugate gradients, variational Gaussian process approximations, and a previous version of stochastic gradient descent for Gaussian processes. On a molecular binding affinity prediction task, our method places Gaussian process regression on par in terms of performance with state-of-the-art graph neural networks.
Reference: [2310.20581] Stochastic Gradient Descent for Gaussian Processes Done Right (arxiv.org)
17:30-17:45
Speaker: Chao Ma, Microsoft Research Cambridge
Title: Towards Causal Foundation Model: on Duality between Causal Inference and Attention
Abstract: Foundation models have brought changes to the landscape of machine learning, demonstrating sparks of human-level intelligence across a diverse array of tasks. However, a gap persists in complex tasks such as causal inference, primarily due to challenges associated with intricate reasoning steps and high numerical precision requirements. In this work, we take a first step towards building causally-aware foundation models for complex tasks. We propose a novel, theoretically sound method called Causal Inference with Attention (CInA), which utilizes multiple unlabeled datasets to perform self-supervised causal learning, and subsequently enables zero-shot causal inference on unseen tasks with new data. This is based on our theoretical results that demonstrate the primal-dual connection between optimal covariate balancing and self-attention, facilitating zero-shot causal inference through the final layer of a trained transformer-type architecture. We demonstrate empirically that our approach CInA effectively generalizes to out-of-distribution datasets and various real-world datasets, matching or even surpassing traditional per-dataset causal inference methodologies.