This Tuesday, I will talk about some in-progress work on understanding how people feel about AI. Along with Sara Court, Emily Sagasser and Galey Modan, I am analyzing the online discourse around the recent study “Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task” (https://arxiv.org/abs/2506.08872) through analysis of Youtube and Reddit reactions to the study. We find that people online have a variety of pre-existing attitudes towards AI which shape their understanding of new information. Some people really hate AI, some are really excited about it, and many are in between, with various ways of discussing what makes for “good” or “bad” usage.

This talk will not involve much in the way of computational methods, but it may still be helpful for understanding how people out there in the world react to the research we do.

Contextual entropy is a psycholinguistic measure capturing the anticipated difficulty of processing a word just before it is encountered. Recent studies have tested for entropy-related effects as a potential complement to well-known effects from surprisal. For convenience, entropy is typically estimated based on a language model’s probability distribution over a word’s first subword token. However, this approximation results in underestimation and potential distortion of true word entropy. To address this, we generate Monte Carlo (MC) estimates of word entropy that allow words to span a variable number of tokens. Regression experiments on reading times show divergent results between first-token and MC word entropy, with evidence that the latter provides better predictions of human sentence processing difficulty. These results suggest a need for caution in using first-token approximations of contextual entropy.

Responsible Applications of LLMs for Low-Resource Machine Translation

This dissertation lays the groundwork for a machine learning (ML) framework intended to facilitate community-driven development of language technologies and NLP applications. Inspired by morphological theories of the mental lexicon, language learning pedagogy, and human translation methods, the framework implements an LLM-based text generation pipeline that combines Retrieval Augmented Generation (RAG) with multistep agentic loops for quality estimation and refinement. The dissertation will also describe a set of minimal specifications for the construction and maintenance of a digital database used to steer generation performance. Maltese-English machine translation (MT) serves as a case study in order to empirically assess a selection of post-training methods for adapting an LLM for use in a particular language or domain for which its parametric knowledge is insufficient. The proposed framework is designed with language agnostic principles in mind, and the dissertation will specify how to configure the framework with respect to a specific community’s self-defined language and usage conventions. Ablation studies will analyze the relative contributions of the pipeline’s component parts and their effects on downstream generation performance. Human analysis of model outputs will also clarify and document the risks and limitations associated with the proposed methods. Ultimately, this dissertation aims to support infrastructure for collaborative and “bottom-up” methods of developing modern language technologies. As the use of LLM-based tools becomes increasingly normalized throughout many of our daily lives, such initiatives are critical to ensure that participation in the data-driven future may be accessible to speakers of all of the world’s languages — not just those of a select few.