Course Home Page for Ethical Algorithm Design (CIS 423/523) University of Pennsylvania, Spring 2022

Ethical Algorithm Design CIS 423/523
Spring 2022
Tuesdays and Thursdays 1:45-3:15PM ET
118 Fagin Hall

Instructor:

Prof. Michael Kearns
mkearns@cis.upenn.edu
Office hours (virtual): TBD

Teaching Assistants:

Ira Globus-Harris (they/them)
igh@seas.upenn.edu
Office hours: Virtual Wed 11-1 here or by appointment.

Elizabeth Margolin
ecmargo@seas.upenn.edu
Office hours: Virtual Thu 11-1 here or by appointment.

Course Description

This course is about the social and human problems that can arise from algorithms, AI and machine learning, and how we might design these technologies to be "better behaved" in the first place. It is first and foremost a science or engineering course, since we will be developing algorithm design principles. You can get a rough sense of course themes and topics by visiting the websites for the pilot versions of this course offered in 2021, 2020 and 2019

Here are the lecture videos from the pilot version from last year. Please note that they will not correspond exactly to this year's lectures, and should not be viewed as a substitute for mandatory lecture attendance.

Prerequisites: Familiarity with some machine learning, basic statistics and probability theory will be helpful. While this is not a theory class, you need to be comfortable with mathematical notation and formalism. There will be some simple coding and data analysis assignments, so some basic programming ability is needed.

Course content will include readings from the scientifc literature, the mainstream media and other articles and books.

Grades will be based on some TBD mixture of quizzes, coding assignments, possible a midterm and/or final, and perhaps a final project or book club.

Since at least the first couple of lectures will be virtual, here is the Zoom link for the live sessions.

CIS 423/523 fulfills the SEAS Engineering Ethics Requirement for these programs: ASCS, BE, CMPE, CSCI, DMD and NETS.

Lecture Dates

Topic

Slides, Readings, Assignments, Announcements

Thu Jan 13
Course Introduction and Overview
Lecture video: [Jan 13]
Lecture Notes
The first assignment of the semester is to complete this survey.
A general-audience introduction to some of the themes of the course is given in the (recommended but not required) book The Ethical Algorithm: The Science of Socially Aware Algorithm Design, by M. Kearns and A. Roth.
Also recommended but not required: Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, by C. O'Neil.

Tue Jan 18
Tue Jan 25
Foundations of Machine Learning Lecture videos: [Jan 18]
Lecture Notes
There is good and comprehensive set of videos and readings related to many of the topics we covered in these lectures in this Google machine learning course.

Thu Jan 27
Tue Feb 1
Discrimination in Machine Learning: COMPAS and ProPublica
Lecture Notes
The following readings are required; you should read the two ProPublica pieces before the Jan 25 lecture so we can discuss them then.
ProPublica article on COMPAS
ProPublica analysis
COMPAS Risk Assesment Survey (just skim)
Northpointe response to ProPublica
ProPublica github repository, including dataset (we'll look at the dataset a bit in lecture)(technical, just skim)

Thu Feb 3
Tue Feb 8
Thu Feb 10
Tue Feb 15
Thu Feb 17
Tue Feb 22
Thu Feb 24
Science of Fair ML: Models and Algorithms
Lecture Notes
Readings:
Inherent Trade-Offs in the Fair Determination of Risk Scores, J. Kleinberg, S. Mullainathan, M. Raghavan. (Read first 8 pages)
The Frontiers of Fairness in Machine Learning. Alexandra Chouldechova, Aaron Roth. (Read entire article)
Please play around with the following Google demo site on fairness and ML.
Equality of Opportunity in Supervised Learning. M. Hardt, E. Price, N. Srebro. (Optional; this is the post-processing/bolt-on method.)
A Reductions Approach to Fair Classification. A. Agarwal, A. Beygelzimer, M. Dudík, J. Langford, H. Wallach. (Optional; this is the in-processing/constrained optimization/game theory method.)
And here is a series of papers extending the game-theoretic approach to different/stronger notions of fairness (all optional):
An Empirical Study of Rich Subgroup Fairness for Machine Learning. MK, S. Neel, A. Roth, S. Wu.
Average Individual Fairness: Algorithms, Generalization and Experiments. MK, A. Roth, S. Sharifi-Malvajerdi.
An Algorithmic Framework for Fairness Elicitation. C. Jung, MK, S. Neel, A. Roth, L. Stapleton, S. Wu.
Minimax Group Fairness: Algorithms and Experiments Emily Diana, Wesley Gill, MK, Krishnaram Kenthapadi, Aaron Roth.

Tue Mar 1
Thu Mar 3
"Bias Bounty" Project Introduction Here is the algorithmic ``bias bounty'' framework; it is recommended that you take at least a high-level reading for the coding project:
Beyond the Frontier: Fairness Without Accuracy Loss. I. Globus-Harris, MK, A. Roth.

Tue Mar 15
Thu Mar 17
Tue Mar 20
Thu Mar 22
Algorithmic Privacy and Related Topics Lecture Notes

Lecture Dates	Topic	Slides, Readings, Assignments, Announcements
Thu Jan 13	Course Introduction and Overview	Lecture video: [Jan 13] Lecture Notes The first assignment of the semester is to complete this survey. A general-audience introduction to some of the themes of the course is given in the (recommended but not required) book The Ethical Algorithm: The Science of Socially Aware Algorithm Design, by M. Kearns and A. Roth. Also recommended but not required: Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, by C. O'Neil.
Tue Jan 18 Tue Jan 25	Foundations of Machine Learning	Lecture videos: [Jan 18] Lecture Notes There is good and comprehensive set of videos and readings related to many of the topics we covered in these lectures in this Google machine learning course.
Thu Jan 27 Tue Feb 1	Discrimination in Machine Learning: COMPAS and ProPublica	Lecture Notes The following readings are required; you should read the two ProPublica pieces before the Jan 25 lecture so we can discuss them then. ProPublica article on COMPAS ProPublica analysis COMPAS Risk Assesment Survey (just skim) Northpointe response to ProPublica ProPublica github repository, including dataset (we'll look at the dataset a bit in lecture)(technical, just skim)
Thu Feb 3 Tue Feb 8 Thu Feb 10 Tue Feb 15 Thu Feb 17 Tue Feb 22 Thu Feb 24	Science of Fair ML: Models and Algorithms	Lecture Notes Readings: Inherent Trade-Offs in the Fair Determination of Risk Scores, J. Kleinberg, S. Mullainathan, M. Raghavan. (Read first 8 pages) The Frontiers of Fairness in Machine Learning. Alexandra Chouldechova, Aaron Roth. (Read entire article) Please play around with the following Google demo site on fairness and ML. Equality of Opportunity in Supervised Learning. M. Hardt, E. Price, N. Srebro. (Optional; this is the post-processing/bolt-on method.) A Reductions Approach to Fair Classification. A. Agarwal, A. Beygelzimer, M. Dudík, J. Langford, H. Wallach. (Optional; this is the in-processing/constrained optimization/game theory method.) And here is a series of papers extending the game-theoretic approach to different/stronger notions of fairness (all optional): An Empirical Study of Rich Subgroup Fairness for Machine Learning. MK, S. Neel, A. Roth, S. Wu. Average Individual Fairness: Algorithms, Generalization and Experiments. MK, A. Roth, S. Sharifi-Malvajerdi. An Algorithmic Framework for Fairness Elicitation. C. Jung, MK, S. Neel, A. Roth, L. Stapleton, S. Wu. Minimax Group Fairness: Algorithms and Experiments Emily Diana, Wesley Gill, MK, Krishnaram Kenthapadi, Aaron Roth.
Tue Mar 1 Thu Mar 3	"Bias Bounty" Project Introduction	Here is the algorithmic ``bias bounty'' framework; it is recommended that you take at least a high-level reading for the coding project: Beyond the Frontier: Fairness Without Accuracy Loss. I. Globus-Harris, MK, A. Roth.
Tue Mar 15 Thu Mar 17 Tue Mar 20 Thu Mar 22	Algorithmic Privacy and Related Topics	Lecture Notes