UPenn Optimization Seminar: Aaron Roth

Title: Conditional Calibration for Task Specific Uncertainty Quantification

Abstract: For many tasks, optimal solutions would be simple exercises if only we had direct access to some “true distribution on outcomes”, conditional on all of our observed covariates. Unfortunately “true probabilities” are fundamentally impossible to come by, without making very strong modelling assumptions about the environment. On the other hand, there are various non-parametric methods for uncertainty quantification — such as calibration and conformal prediction — that can be used without making essentially any assumptions at all — but their guarantees are marginal, and it is unclear what kinds of tasks they are good for. A recent line of work has given algorithms that can produce calibration guarantees that hold conditionally on any bounded set of conditioning events. This turns out to form a useful design framework for making predictions that can be used as if they were probabilities for many downstream tasks, so long as one selects the conditioning events appropriately with the downstream task in mind. We’ll see three applications — giving a method to make predictions that can be usefully consumed by many downstream decision makers, a method to make predictions that can be used to form prediction sets that are conditionally valid subject to any collection of conditioning events, and a method of making forecasts that can be used to interact with another forecaster and quickly reach agreement, recovering tractable versions of Aumann’s agreement theorem.

Bio: Aaron Roth is the Henry Salvatori Professor of Computer and Cognitive Science, in the Computer and Information Sciences department at the University of Pennsylvania, with a secondary appointment in the Wharton statistics department. He is affiliated with the Warren Center for Network and Data Science, and co-director of the Networked and Social Systems Engineering (NETS) program.  He is also an Amazon Scholar at Amazon AWS. He is the recipient of the Hans Sigrist Prize, a Presidential Early Career Award for Scientists and Engineers (PECASE), an Alfred P. Sloan Research Fellowship, an NSF CAREER award, and research awards from Yahoo, Amazon, and Google.  His research focuses on the algorithmic foundations of data privacy, algorithmic fairness, game theory, learning theory, and machine learning.  Together with Cynthia Dwork, he is the author of the book “The Algorithmic Foundations of Differential Privacy.” Together with Michael Kearns, he is the author of “The Ethical Algorithm”.