Generative Interaction Models
Abstract
Generative models of human–computer interaction offer a powerful paradigm for designing, optimizing, and evaluating user interfaces. By simulating realistic human behavior, these models enable scalable, controllable, and reproducible experimentation beyond traditional user studies. In this talk, I will present three classes of generative interaction models developed in our lab. First, I will introduce generative adversarial networks (GANs) that model fine-grained typing behavior on soft keyboards, capturing both tap typing and word-gesture dynamics. Second, I will present a reinforcement learning–based model that simulates how blind users navigate and perform menu selections with screen readers, modeling sequential decision-making in non-visual interaction. Third, I will describe Bayesian hierarchical models of human motor behavior, including probabilistic extensions of Fitts’ law. Together, these approaches demonstrate how generative AI and probabilistic modeling can serve as computational proxies for human behavior, moving interaction design toward a simulation-driven science.