Sherry's Profile Photo
Sherry Wu
Assistant Professor
Carnegie Mellon University
Diyi's Profile Photo
Diyi Yang
Assistant Professor
Stanford University
Sebastin's Profile Photo
Sebastin Santy
PhD Student
University of Washington


The rapid advancement of natural language processing (NLP) research has led to various applications spanning a wide range of domains that require models to interact with humans — e.g., chatbots responding to human inquiries, machine translation systems assisting human translators, designers prompting Large Language Models for co-creation or prototyping AI-infused applications, etc. In these cases, humans interaction is key to the success of NLP applications; any potential misconceptions or differences might lead to error cascades at the subsequent stages. Such interaction involves a lot of design choices around models, e.g. the sensitivity of interfaces, the impact of design choice and evaluation questions, etc.

This tutorial aims to provide a systematic and up-to-date overview of key considerations and effective approaches for studying human-NLP model interactions. Our tutorial will focus specifically on the scenario where end users – lay people and domain experts who have access to NLP models but are less familiar with NLP techniques — use or collaborate with deployed models.

Throughout the tutorial, we will use five case studies (on classifier-assisted decision making, machine-aided translation, dialog systems, and prompting) to cover three major themes: (1) how to conduct human-in-the-loop usability evaluations to ensure that models are capable of interacting with humans; (2) how to design user interfaces (UIs) and interaction mechanisms that provide end users with easy access to NLP models; (3) how to learn and improve NLP models through the human interactions. We will use best practices from HCI to ground our discussion, and will highlight current challenges and future directions.


Dec 6th 2023, 9am-12:30pm
Leo 3 & 4, Resorts World Sentosa

Design 09:15-10:05 (40 mins lecture + 10 mins Q&A)
Evaluate 10:05-10:30 (25 mins lecture)
Coffee break 10:30-10:50
Evaluate (cont') 10:50-11:15 (15 mins lecture + 10 mins Q&A)
Learn from 11:15-12:05 (40 mins lecture + 10 mins Q&A)
Conclusion 12:05-12:15
Extended Q&A 12:15-12:30