Month: October 2023

Clippers 10/31: Jingyi Chen on “Aligning Text-to-Image Models using Human Feedback”

by

On Halloween in Clippers, Jingyi Chen will present the paper Aligning Text-to-Image Models using Human Feedback (https://arxiv.org/abs/2302.12192).

Abstract: Deep generative models have shown impressive results in text-to-image synthesis. However, current text-to-image models often generate images that are inadequately aligned with text prompts. We propose a fine-tuning method for aligning such models using human feedback, comprising three stages. First, we collect human feedback assessing model output alignment from a set of diverse text prompts. We then use the human-labeled image-text dataset to train a reward function that predicts human feedback. Lastly, the text-to-image model is fine-tuned by maximizing reward-weighted likelihood to improve image-text alignment. Our method generates objects with specified colors, counts and backgrounds more accurately than the pre-trained model. We also analyze several design choices and find that careful investigations on such design choices are important in balancing the alignment-fidelity tradeoffs. Our results demonstrate the potential for learning from human feedback to significantly improve text-to-image models.

Clippers 10/24: Sara Court on software tools for low-resource morphological analysis and annotation

by

Micha and I will present our ongoing work developing software tools for low-resource morphological analysis and annotation. This is part of a larger project we presented last summer at ACL’s ComputEL workshop in collaboration with Maria Copot, Stephanie Antetomaso, and Noah Diewald.

We combine unsupervised methods for morphological paradigm discovery with a browser-based interface and supervised learner implemented in tensorflow.js. We’re currently experimenting with various model designs and active learning selection heuristics and look forward to your feedback as we continue our work!

Clippers 10/17: Sam Stevens on mixture-of-experts (MoE) language models

by

In Clippers next week I will present some early-stage planning for a mixture-of-experts (MoE) language model project I hope to pursue. It will consist of:

  1. A literature review of neural MoE models in NLP
  2. How MoE models changed my thinking around model parallelism, FLOPs and compute efficiency
  3. What this implies about GPT-4 (which is rumored to be a MoE model)
  4. Soft MoE: a recent paper that aims to solve many problems with MoE models, but only applies it to vision
  5. Ideas I have on how to apply soft MoE to language modeling

I hope that #1 and #2 will be valuable to everyone, because I think MoE models are very under-utilized in research, despite supposedly powering the best language model in the world (GPT-4).

Clippers 10/3: Alex Petrov on intelligence in LLMs

by

Is GPT-4 Intelligent? What Does This Mean and How Can We Tell?

Artificial intelligence (AI) capabilities are improving at an unprecedented and alarming rate. Existing Large language models (LLMs) such as GPT-4 already demonstrate “sparks” of artificial general intelligence (AGI). That is, they do according to a controversial paper by Bubeck et al. that many ML researchers consider to be a disgrace to the profession, whereas other scientists (myself included) consider to be insightful and of pivotal importance.

These polarized opinions point to a methodological problem. The scientific community does not know how to evaluate opaque models with trillions of parameters. In my talk, I will try to shed some light on this question, drawing from philosophy, psychology, machine learning, theoretical computer science, hardware design, and linguistics. It is a remarkable fact that all these disparate disciplines provide valuable pieces of the puzzle.