H.B. Keller Colloquium
Sanmi (Oluwasanmi) Koyejo an Assistant Professor in the Department of Computer Science at the University of Illinois at Urbana-Champaign. Koyejo's research interests are in developing the principles and practice of adaptive and robust machine learning. Additionally, Koyejo focuses on applications to neuroscience and biomedical imaging. Koyejo completed his Ph.D. in Electrical Engineering at the University of Texas at Austin advised by Joydeep Ghosh and completed postdoctoral research at Stanford University with a focus on developing machine learning techniques for neuroimaging data. His postdoctoral research was primarily with Russell A. Poldrack and Pradeep Ravikumar. Koyejo has been the recipient of several awards including a best paper award from the conference on uncertainty in artificial intelligence (UAI), a Kavli Fellowship, an IJCAI early career spotlight, and a trainee award from the Organization for Human Brain Mapping (OHBM). Koyejo serves on the board of the Black in AI organization, which he co-founded.
Given a prediction problem with real-world tradeoffs, which cost function should the machine learning model be trained to optimize? Unfortunately, typical default metrics in machine learning, such as accuracy applied to binary classifiers, may not capture tradeoffs relevant to the problem at hand. This talk proposes metric elicitation as a formal strategy to address the metric selection problem, specifically by automatically discovering implicit preferences from an expert or an expert panel via interactive feedback. I will primarily focus on algorithms for eliciting classification metrics, showing that simple algorithms are efficient for metric elicitation under broad assumptions. Finally, I will briefly outline early work on metric selection for measuring group fairness in classification problems with sensitive groups.