CS 591 — FAIRNESS, ACCOUNTABILITY, AND TRANSPARENCY IN AI, Spring Semester

Tuesdays, 6:30–9:15 PM EST Online via Zoom (Lectures will be recorded)

Instructor: Mehrnoosh Sameki

Course Description: Enabling the responsible development of artificial intelligence technologies is one of the major challenges we face as the field moves from research to practice. Researchers and practitioners from different disciplines have highlighted the ethical and legal challenges posed by the use of machine learning in many current and future real-world applications. Now there are calls from across the industry (academia, government, and industry leaders) for technology creators to ensure that AI is used only in ways that benefit people and "to engineer responsibility into the very fabric of the technology." Overcoming these challenges and enabling responsible development is essential to ensure a future where AI and machine learning can be widely used across different domains. This course will pursue a cross-disciplinary investigation of several areas under the responsible AI umbrella (fairness, interpretability, and accountability). Students will learn about state-of-the-art research and best practices in the covered domains and use available open-source fairness and interpretability toolkits to apply their learnings to publicly available datasets from healthcare, finance, and other domains.

Course Format: This course is a weekly meeting in which participants discuss recent and important concepts and results in the area of responsible AI research/practice. For a typical meeting, I will spend the first ~75 minutes of the session presenting on the listed topic of that week. We will next discuss 1 or 2 relevant papers that are selected for discussion according to the session's focus topic. Students must read each week's selected papers prior to the class and submit a one page summary of main takeaways for each paper.

Objectives: To increase participants' familiarity with recent and important research results in responsible AI and in particular AI accountability, interpretability, and fairness; to improve participants' skills in presenting and discussing relevant topics.

Grading policy: 30% reading assignments summary, 30% mid-semester project, 40% final project
Final course scores represent the following grades (scores are rounded to the nearest integer):
A (94-100%)
A- (90-93%)
B+ (87-89%)
B (84-86%)
B- (80-83%)
C+ (77-79%)
C (74-76%)

Schedule

TO BE UPDATED FOR SPRING 2023

Week	Date	Topics	Slides	Papers
1	1/26	Class Intro General Introduction to Responsible AI, Part 1	Lecture Slides	No Paper Discussion
2	2/2	Class Intro General Introduction to Responsible AI, Part 2	Lecture Slides	No Paper Discussion
3	2/9	Overview of Machine Learning Interpretability	Lecture Slides Chapters 1 and 2	Paper 1: Machine Learning Interpretability: A Survey on Methods and Metrics Diogo V. Carvalho et al.
-	2/16	No class, Monday substitute	NA	NA
4	2/23	Training Interpretable (Glassbox) Models	Lecture Slides Chapter 4	Paper 1: Questioning the AI: Informing Design Practices for Explainable AI User Experiences Q. Vera Liao et al. Paper 2: Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs Sungsoo Ray Hong et al.
5	3/2	Explaining Black-Box Models	Lecture Slides Chapter 5	Paper 1 (Only submit your abstract for this one): Robustness in Machine Learning Explanations: Does It Matter? Leif Hancox-Li et al. Paper 2 (additional read, no abstract submission): Explainable Machine Learning in Deployment Umang Bhatt et al.
6	3/9	- Interpretability: Connections with Debugging - Evaluating Interpretability	Lecture Slides	Paper 1 (Only submit your abstract for this one): Manipulating and Measuring Model Interpretability Forough Poursabzi-Sangdeh et al. Paper 2: Towards A Rigorous Science of Interpretable Machine Learning Finale Doshi-Velez et al.
7	3/16	Overview of Machine Learning Fairness, Part 1	Lecture Slides Part 1 Lecture Slides Part 2	Paper 1: A Survey on Bias and Fairness in Machine Learning Ninareh Mehrabi et al.
8	3/23	Overview of Machine Learning Fairness, Part 2	Lecture Slides	Paper 1 (Only submit your abstract for this one): Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? Ken Holstein et al. Paper 2: Big Data's Disparate Impact Solon Barocas et al.
9	3/30	Fairness Assessment and Unfairness Mitigation	Lecture Slides Midterm Assigment	Paper 1: Fairness and Abstraction in Sociotechnical Systems Andrew Selbst et al.
10	4/6	Accountability in AI	Lecture Slides	Only submit your abstract for paper 1 and paper 2: Paper 1: Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches Kasper Sokol et al. Paper 2: Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI. Michael Madaio et al. Paper 3: Accountability of AI Under the Law: The Role of Explanation Finale Doshi-Velez et al.
11	4/13	Student Presentations (Midterm Projects)	No Lecture Slides
12	4/20	Ethics in Industry Guidelines for Human-AI Interaction	Lecture Slides	Paper 1: Guidelines for Human-AI Interaction Saleema Amershi et al. Paper 2: Owning Ethics: Corporate Logics, Silicon Valley, and the Institutionalization of Ethics Jacob Metcalf et al.
13	4/27	Course Takeaways	Slides TBA