Aaron Roth

About Me

I am the Henry Salvatori Professor of Computer and Cognitive Science at the University of Pennsylvania computer science department. I also hold a secondary appointment at the Department of Statistics and Data Science at the Wharton School, and I am associated with the theory group, PRiML (Penn Research in Machine Learning) the Warren Center for Network and Data Sciences, and am co-director of our program in Networked and Social Systems Engineering. I am also affiliated with the AMCS program (Applied Mathematics and Computational Science). I spent a year as a postdoc at Microsoft Research New England. Before that, I received my PhD from Carnegie Mellon University, where I was fortunate to have been advised by Avrim Blum. My main interests are in algorithms and machine learning, and specifically in the areas of private data analysis, fairness in machine learning, game theory and mechanism design, and learning theory. I am the recipient of the Hans Sigrist Prize, a Presidential Early Career Award for Scientists and Engineers (PECASE), an Alfred P. Sloan Research Fellowship, an NSF CAREER award, a Google Faculty Research Award, an Amazon Research Award, and a Yahoo Academic Career Enhancement award. I am also an Amazon Scholar at Amazon Web Services (AWS). Previously, I was involved in advisory and consulting work related to differential privacy, algorithmic fairness, and machine learning, including with Apple and Facebook. I was also a scientific advisor for Leapyear and Spectrum Labs.

For more information, see my CV, Research Statement, and those of my talks that appear on YouTube.

Photo of Aaron Roth

Contact Information

Office: 511 Amy Gutmann Hall

Email: aaroth@cis.upenn.edu

Teaching

In Spring 2026 I am teaching NETS 4120 Algorithmic Game Theory

Previous courses:

Students and Postdocs

I'm fortunate to be able to work with several excellent graduate students and postdocs.

Current:

Alumni:

Books and Notes

  1. Learning in Games and Games in Learning. (DRAFT -- WORK IN PROGRESS).
  2. Uncertain: Modern Topics in Uncertainty Estimation. (DRAFT -- WORK IN PROGRESS).
  3. The Ethical Algorithm. Joint with Michael Kearns. Oxford University Press. November 2019.
  4. The Algorithmic Foundations of Differential Privacy. Joint with Cynthia Dwork. Foundations and Trends in Theoretical Computer Science, NOW Publishers. 2014.

Recent and Selected Publications

(See here for all publications, or my Google Scholar profile)

Click for abstract/informal discussion of results

  1. Optimal Deterministic Multicalibration and Omniprediction. Joint work with Georgy Noarov. Manuscript.
  2. What Fits (Into Few Tokens) Doesn't Overfit: Compression and Generalization in ML Research Agents. Joint work with Martin Bertran and Steven Wu. Manuscript.
  3. Instance Adaptive Online Multicalibration. Joint work with Zhiming Huang, Jamie Morgenstern, and Claire Jie Zhang. Manuscript.
  4. The Sample Complexity of Multicalibration. Joint work with Natalie Collina, Jiuyao Lu, and Georgy Noarov. Manuscript.
  5. Personalization Aids Pluralistic Alignment Under Competition. Joint work with Natalie Collina, Surbhi Goel and Mirah Shi. Best Paper Award at the ICLR 2026 Workshop on AI for Mechanism Design and Strategic Decision Making (AIMS). Manuscript.
  6. The Value of Ambiguous Commitments in Multi-Follower Games. Joint work with Natalie Collina and Rabanus Derr. Manuscript.
  7. Optimal Lower Bounds for Online Multicalibration. Joint work with Natalie Collina, Jiuyao Lu, and Georgy Noarov. In the Proceedings of FOCS 2026.
  8. Prior-Agnostic Incentive-Compatible Exploration. Joint work with Ramya Ramalingam and Osbert Bastani. In the Proceedings of EC 2026.
  9. Model Agreement via Anchoring. Joint work with Eric Eaton, Surbhi Goel, Marcel Hussing, Michael Kearns, Sikata Sengupta, and Jessica Sorrell. In the Proceedings of COLT 2026.
  10. Online Omniprediction with Long-Term Constraints. Joint work with Yahav Bechavod and Jiuyao Lu. In the Proceedings of COLT 2026.
  11. Emergent Alignment via Competition. Joint work with Natalie Collina, Surbhi Goel, Emily Ryu, and Mirah Shi. In the Proceedings of ICML 2026.
  12. Robust Decision Making with Partially Calibrated Forecasts. Joint work with Shayan Kiyani, George Pappas, and Hamed Hassani. In the Proceedings of ICLR 2026.
  13. Networked Information Aggregation via Machine Learning. Joint work with Michael Kearns and Emily Ryu. In the Proceedings of SODA 2026.
  14. Collaborative Prediction: Tractable Information Aggregation via Agreement. Joint work with Natalie Collina, Ira Globus-Harris, Surbhi Goel, Varun Gupta, and Mirah Shi. In the Proceedings of SODA 2026.
  15. Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents . Joint work with Shayan Kiyani, George Pappas, and Hamed Hassani. In the Proceedings of ICML 2025.
  16. High-Dimensional Prediction for Sequential Decision Making. Joint work with Georgy Noarov, Ramya Ramalingam, and Stephan Xie. In the Proceedings of ICML 2025.
  17. Resolving the Reference Class Problem at Scale. Joint work with Alexander Tolbert. Philosophy of Science, 2025.
  18. Tractable Agreement Protocols. Joint work with Natalie Collina, Surbhi Goel, and Varun Gupta. In the Proceedings of STOC 2025.
  19. An Elementary Predictor Obtaining 2√T Distance to Calibration. Joint work with Eshwar Ram Arunachaleswaran, Natalie Collina, and Mirah Shi. In the Proceedings of SODA 2025.
  20. Forecasting for Swap Regret for All Downstream Agents. Joint work with Mirah Shi. In the Proceedings of EC 2024.
  21. Scalable Membership Inference Attacks via Quantile Regression. Joint work with Martin Bertran, Shuai Tang, Michael Kearns, Jamie Morgenstern, and Steven Wu. In the Proceedings of NeurIPS 2023.
  22. The Scope of Multicalibration: Characterizing Multicalibration via Property Elicitation. Joint work with Georgy Noarov. In the Proceedings of ICML 2023.
  23. Multicalibration as Boosting for Regression. Joint work with Ira Globus-Harris, Declan Harrison, Michael Kearns, and Jessica Sorrell. In the Proceedings of ICML 2023. (Also presented at FORC 2023). Selected as an oral presentation
  24. Batch Multivalid Conformal Prediction. Joint work with Christopher Jung, Georgy Noarov, and Ramya Ramalingam. In the Proceedings of ICLR 2023.
  25. Practical Adversarial Multivalid Conformal Prediction. Joint work with Osbert Bastani, Varun Gupta, Christopher Jung, Georgy Noarov, and Ramya Ramalingam. In the Proceedings of NeurIPS 2022. Selected as an oral presentation
  26. An Algorithmic Framework for Bias Bounties. Joint work with Ira Globus-Harris and Michael Kearns. In the Proceedings of FAccT 2022.
  27. Online Multivalid Learning: Means, Moments, and Prediction Intervals. Joint work with Varun Gupta, Christopher Jung, Georgy Noarov, and Mallesh Pai. In the Proceedings of ITCS 2022.
  28. Adaptive Machine Unlearning. Joint work with Varun Gupta, Christopher Jung, Seth Neel, Saeed Sharifi-Malvajerdi, and Chris Waites. In the Proceedings of NeurIPS 2021.
  29. Moment Multicalibration for Uncertainty Estimation. Joint work with Christopher Jung, Changhwa Lee, Mallesh Pai, and Rakesh Vohra. In the Proceedings of COLT 2021.
  30. Gaussian Differential Privacy. Joint work with Jinshuo Dong and Weijie Su. In the Journal of the Royal Statistical Society, Series B. 2020.
  31. A New Analysis of Differential Privacy's Generalization Guarantees. Joint work with Christopher Jung, Katrina Ligett, Seth Neel, Saeed Sharifi-Malvajerdi, and Moshe Shenfeld. In the Proceedings of ITCS 2020.
  32. Exponential Separations in Local Differential Privacy Through Communication Complexity. Joint work with Matthew Joseph and Jieming Mao. In the Proceedings of SODA 2020.
    Invited to ACM Transactions on Algorithms Special Issue.
  33. The Role of Interactivity in Local Differential Privacy. Joint work with Matthew Joseph, Jieming Mao, and Seth Neel. In the Proceedings of FOCS 2019.
  34. How to Use Heuristics for Differential Privacy. Joint with Seth Neel and Steven Wu. In the Proceedings of FOCS 2019.
  35. Online Learning with an Unknown Fairness Metric. Joint with Stephen Gillen, Christopher Jung, and Michael Kearns. In the proceedings of NeurIPS 2018.
  36. A Smoothed Analysis of the Greedy Algorithm for the Linear Contextual Bandit Problem. Joint with Sampath Kannan, Jamie Morgenstern, Bo Waggoner, and Steven Wu. In the proceedings of NeurIPS 2018.
  37. Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness. Joint with Michael Kearns, Seth Neel, and Steven Wu. In the proceedings of ICML 2018.
  38. Fairness in Learning: Classic and Contextual Bandits. Joint with Matthew Joseph, Jamie Morgenstern, and Michael Kearns. In the proceedings of NIPS 2016.
  39. Robust Mediators in Large Games. Joint with Michael Kearns, Mallesh Pai, Ryan Rogers, and Jon Ullman. Manuscript. (This paper subsumes both "Mechanism Design in Large Games: Incentives and Privacy" which appeared in ITCS 2014, and "Asymptotically Truthful Equilibrium Selection" which appeared in EC 2014).
  40. Max-Information, Differential Privacy, and Post-Selection Hypothesis Testing. Joint with Ryan Rogers, Adam Smith, and Om Thakkar. In the proceedings of FOCS 2016.
  41. Do Prices Coordinate Markets?. Joint work with Justin Hsu, Jamie Morgenstern, Ryan Rogers, and Rakesh Vohra. In the proceedings of STOC 2016.
  42. Watch and Learn: Optimizing from Revealed Preferences Feedback. Joint with Jon Ullman and Steven Wu. In the proceedings of STOC 2016.
  43. Jointly Private Convex Programming. Joint work with Justin Hsu, Zhiyi Huang, and Steven Wu. In the proceedings of SODA 2016.
  44. The Reusable Holdout: Preserving Validity in Adaptive Data Analysis. Joint work with Cynthia Dwork, Vitaly Feldman, Moritz Hardt, Toniann Pitassi, and Omer Reingold. In Science, August 7 2015.
  45. Preserving Statistical Validity in Adaptive Data Analysis. Joint work with Cynthia Dwork, Vitaly Feldman, Moritz Hardt, Toniann Pitassi, and Omer Reingold. In the proceedings of STOC 2015.
  46. Approximately Stable, School Optimal, and Student-Truthful Many-to-One Matchings (via Differential Privacy). Joint with Sampath Kannan, Jamie Morgenstern, and Steven Wu. In the proceedings of SODA 2015.
  47. Private Matchings and Allocations. Joint work with Justin Hsu, Zhiyi Huang, Tim Roughgarden, and Steven Wu. In the proceedings of STOC 2014.
  48. Differential Privacy for the Analyst via Private Equilibrium Computation. Joint with Justin Hsu and Jon Ullman. In the proceedings of STOC 2013.
  49. Privately Releasing Conjunctions and the Statistical Query Barrier. Joint with Anupam Gupta, Moritz Hardt, and Jonathan Ullman. In the proceedings of STOC 2011.
    Full version appears in SIAM Journal on Computing (SICOMP) 2013.
  50. Selling Privacy at Auction. Joint work with Arpita Ghosh. In the proceedings of EC 2011.
    Invited to a special issue of Games and Economic Behavior (GEB) 2013.
  51. Interactive Privacy via the Median Mechanism. Joint with Tim Roughgarden. In the proceedings of STOC 2010.
  52. A Learning Theory Approach to Non-Interactive Database Privacy. Joint with Avrim Blum and Katrina Ligett. In the proceedings of STOC 2008: The 40th ACM Symposium on the Theory of Computing.
    Full version appears in Journal of the ACM (JACM) 2013.
  53. Regret Minimization and the Price of Total Anarchy. Joint with Avrim Blum, MohammadTaghi Hajiaghayi, and Katrina Ligett. In the proceedings of STOC 2008: The 40th ACM Symposium on the Theory of Computing.
The Ethical Algorithm book cover

Michael Kearns and I have written a general-audience book about the science of designing algorithms that embed social values like privacy and fairness. You can read a review in Nature and an excerpt from the introduction in Penn Today. We've given a number of recorded talks about the book, including one on CSPAN's BookTV. We wrote a related policy brief for the Brookings Institution.

Over the course of a generation, algorithms have gone from mathematical abstractions to powerful mediators of daily life. Algorithms have made our lives more efficient, more entertaining, and, sometimes, better informed. At the same time, complex algorithms are increasingly violating the basic rights of individual citizens. Allegedly anonymized datasets routinely leak our most sensitive personal information; statistical models for everything from mortgages to college admissions reflect racial and gender bias. Meanwhile, users manipulate algorithms to "game" search engines, spam filters, online reviewing services, and navigation apps. Understanding and improving the science behind the algorithms that run our lives is rapidly becoming one of the most pressing issues of this century. Traditional fixes, such as laws, regulations and watchdog groups, have proven woefully inadequate. Reporting from the cutting edge of scientific research, The Ethical Algorithm offers a new approach: a set of principled solutions based on the emerging and exciting science of socially aware algorithm design. Michael Kearns and Aaron Roth explain how we can better embed human principles into machine code - without halting the advance of data-driven scientific exploration. Weaving together innovative research with stories of citizens, scientists, and activists on the front lines, The Ethical Algorithm offers a compelling vision for a future, one in which we can better protect humans from the unintended impacts of algorithms while continuing to inspire wondrous advances in technology.

Professional Activities

Workshops, Tutorials, Interviews, and Panels:

Program Committee Member For:

Presentations

(Slides Available Upon Request)

Podcast, Radio, TV, and Other Media Appearances

Web and Print Media:

Amazon Science, Quanta, Penn Engineering Blog, Amazon Science, Penn News Today, Communications of the ACM, The Guardian, Communications of the ACM, Simons Foundation, Wired, Bloomberg, Amazon Science, Amazon Science, ZDnet, Adexchanger, SINC, Nature, The New York Times, Scientific American, Knowledge@Wharton, Strategy+Business, The Regulatory Review, Marketplace, MIT Technology Review, Wired, Pacific Standard