Generative and Language Models for Mobile Text Interaction
Abstract
Recent advances in generative AI and large language models (LLMs) are transforming how we model user behavior and design interactive systems. In this talk, I will present our lab’s work on leveraging these models to improve mobile text interaction. First, I will introduce generative models of typing behavior, including GAN-based models that capture fine-grained spatial–temporal patterns in tap and gesture typing. These models enable scalable, simulation-based evaluation of keyboard designs and decoding algorithms without relying solely on user studies. Second, I will present how LLMs can enhance text input pipelines, replacing or augmenting traditional decoders to better handle noisy touch input, and supporting multimodal text editing by integrating voice and touch. This reframes text correction as a high-level language understanding problem rather than low-level cursor manipulation. Together, this work highlights a unified direction: using generative and language models to simulate, interpret, and improve human–computer interaction, enabling more robust and intelligent text input systems.