Spring ’23, TTh 3:55–5:15, Oxley Hall 122
Instructor: Michael White
Controllability and Explainability in Neural Language Generation
Large, pretrained neural language models have been receiving just a wee bit of hype of late. For example, the New York Times has recently reported that despite the tech slump, OpenAI, the lab that created ChatGPT, may soon complete a deal valuing it at $29 billion (give or take a few bucks), based on the promise of this type of AI to “reinvent everything.”
At the same time, such models have been criticized as dangerous and on the wrong path. For example, Emily Bender, the ACL’s next president, told her local TV station that “OpenAI’s guard rails are not that strong” and that model is prone to producing lots of racist and sexist language. Moreover, in an interview with Ezra Klein, cognitive scientist Gary Marcus noted that ChatGPT is the “king of pastiche,” i.e., it really just does “a kind of glorified cut and paste.” He then goes on to explain why he considers its development to be on the wrong path: “They’re just trying to autocomplete our sentences. And that’s not the depth that we need to actually get to what people call A.G.I., or artificial general intelligence.”
How should we think of these conflicting views?
This course will dig into the technical details of these models, emphasizing useful tasks such as response generation in dialogue systems, question answering, summarization, simplification, data augmentation and explanation generation. The advent of large, pretrained language models has clearly yielded amazing progress on these tasks, though the black box nature of these models make them hard to understand and control, posing ethical and safety obstacles for deploying them in real world settings. As such, we will focus on readings that examine ways of controlling these models and making them reliably generate explanations, using techniques such as decomposition, chain-of-thought and data-augmented training.
Students are encouraged to pursue related research interests for their term project. Topics will be finalized based on the interests of the participants.
Students will be expected to actively participate in the discussion and research carried out in the seminar. As detailed below, students will be required to facilitate discussions and post questions on the readings in advance, as well as locate relevant background/tutorial materials. Additionally, students taking the course for 3 credits will be required to carry out a class project on a topic related to the seminar; alternatively, for students already working on a related topic, integrating their focus into the seminar will be an option.
Ling 5802 or equivalent, or permission of the instructor.
We’ll use Carmen to schedule discussion facilitators and post advance questions on the readings, as well as links to background/tutorial materials. We’ll also use it for submitting project documents.
Class participation (25%)
We are aiming for a dynamic discussion of papers, not death by powerpoint. Thus, we plan on taking a page from Eric Fosler-Lussier’s playbook and requiring everyone (this includes you!) to post at least one question to the discussion list on Carmen by 8 p.m. the day before each reading will be discussed. Participants should also feel free to share their (initial) thoughts and views of the papers in their posts. In particular, questions of the type “What did they mean by X” or “Why did they do X instead of Y” are encouraged. Remember that most of the papers are targeted to people who are already expert in the area, so you shouldn’t expect to alway understand everything. Airing such questions can help everyone gain a better understanding of the paper — even those who thought they understood it!
Facilitating discussions (25%)
Each meeting where we discuss a paper will have a discussion facilitator. For the main readings, the facilitator should look over the posted questions and choose a subset for discussion. In class, the facilitator should start the session with a brief, five to ten minute summary of the paper, including the highlights and lowlights. Following the opening summary, the facilitator is responsible for managing the discussion, and ensuring that as many viewpoints are heard as possible. Finally, the facilitator is also tasked with keeping track of the potentially most useful background papers for the reading.
Students will be required to facilitate at least one session during the course. If the discussion does not take up the entire class period, the remaining time may be used to (informally) discuss class projects.
We will also dedicate various sessions to reading background papers that we expect to be useful to better understand the main papers of interest. The sessions and papers dedicated to background readings will be determined collaboratively.
Term project (50%)
As noted above, students taking the course for 3 credits will be required to carry out a term project, either alone or in a team setting. A project sketch will be required to be presented informally in class for brainstorming by the fourth week; followed by a project proposal by the eighth week; followed by a presentation during the last week of class; and (finally) a final report by the day the final exam would be held (if there were one).
For students taking the course for 1 credit, no project will be required, with the other requirements scaled accordingly.
The topics and readings we expect to cover are listed below; these will be refined as the course progresses.
Pretrained Language Models
- Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, Ryan Lowe. 2022. Training language models to follow instructions with human feedback. arXiv:2203.02155. Also: ChatGPT blog post.
- Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug Fritz, Jaume Sanchez Elias, Richard Green, Soňa Mokrá, Nicholas Fernando, Boxi Wu, Rachel Foley, Susannah Young, Iason Gabriel, William Isaac, John Mellor, Demis Hassabis, Koray Kavukcuoglu, Lisa Anne Hendricks, Geoffrey Irving. 2022. Improving alignment of dialogue agents via targeted human judgements. arXiv:2209.14375. Also: DeepMind Sparrow blog post.
- William Merrill, Yoav Goldberg, Roy Schwartz, Noah A. Smith. 2021. Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand? Transactions of the Association for Computational Linguistics (2021) 9: 1047–1060.
Ethical and Safety Concerns
- Emily Dinan, Gavin Abercrombie, A. Bergman, Shannon Spruit, Dirk Hovy, Y-Lan Boureau, Verena Rieser. 2022. SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4113–4133, Dublin, Ireland. Association for Computational Linguistics.
- Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, Zac Kenton, Sasha Brown, Will Hawkins, Tom Stepleton, Courtney Biles, Abeba Birhane, Julia Haas, Laura Rimell, Lisa Anne Hendricks, William Isaac, Sean Legassick, Geoffrey Irving, Iason Gabriel. 2021. Ethical and social risks of harm from Language Models. arXiv:2112.04359.
- Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Jackson Kernion, Kamal Ndousse, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Jared Kaplan. 2021. A General Language Assistant as a Laboratory for Alignment. arXiv:2112.00861.
- Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamilė Lukošiūtė, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, Dawn Drain, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jackson Kernion, Jamie Kerr, Jared Mueller, Joshua Landau, Kamal Ndousse, Karina Nguyen, Liane Lovitt, Michael Sellitto, Nelson Elhage, Noemi Mercado, Nova DasSarma, Oliver Rausch, Robert Lasenby, Robin Larson, Sam Ringer, Sandipan Kundu, Saurav Kadavath, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, Christopher Olah, Jack Clark, Samuel R. Bowman, Jared Kaplan. 2023. The Capacity for Moral Self-Correction in Large Language Models. arXiv:2302.07459.
- Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, Maarten Sap. 2022. ProsocialDialog: A Prosocial Backbone for Conversational Agents. In Proc. EMNLP-22.
- Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, Paul Christiano. 2020. Learning to summarize from human feedback. arXiv:2009.01325.
- Hannah Rashkin, David Reitter, Gaurav Singh Tomar, and Dipanjan Das. 2021. Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 704–718, Online. Association for Computational Linguistics.
- Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, David Reitter. 2022. Measuring Attribution in Natural Language Generation Models. arXiv:2112.12870.
- Xintong Li, Symon Stevens-Guille, Aleksandre Maskharashvili, and Michael White. 2021. Self-Training for Compositional Neural NLG in Task-Oriented Dialogue. In Proceedings of the 14th International Conference on Natural Language Generation, pages 87–102, Aberdeen, Scotland, UK. Association for Computational Linguistics.
- Soumya Batra, Shashank Jain, Peyman Heidari, Ankit Arun, Catharine Youngs, Xintong Li, Pinar Donmez, Shawn Mei, Shiunzu Kuo, Vikas Bhardwaj, Anuj Kumar, and Michael White. 2021. Building Adaptive Acceptability Classifiers for Neural NLG. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 682–697, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Zhiyu Chen, Wenhu Chen, Hanwen Zha, Xiyou Zhou, Yunkai Zhang, Sairam Sundaresan, and William Yang Wang. 2020. Logic2Text: High-Fidelity Natural Language Generation from Logical Forms. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2096–2111, Online. Association for Computational Linguistics.
- Hyunwoo Kim, Jack Hessel, Liwei Jiang, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap, Yejin Choi. 2022. SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization. arXiv:2212.10465.
- Braden Hancock, Antoine Bordes, Pierre-Emmanuel Mazare, and Jason Weston. 2019. Learning from Dialogue after Deployment: Feed Yourself, Chatbot!. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3667–3684, Florence, Italy. Association for Computational Linguistics.
- Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, Athul Paul Jacob, Mojtaba Komeili, Karthik Konath, Minae Kwon, Adam Lerer, Mike Lewis, Alexander H. Miller, Sasha Mitts, Adithya Renduchintala, Stephen Roller, Dirk Rowe, Weiyan Shi, Joe Spisak, Alexander Wei, David Wu, Hugh Zhang, and Markus Zijlstra. 2002. Human-level play in the game of Diplomacy by combining language models with strategic reasoning. Science (2022) 378:6624, 1067–1074.
- Kushal Arora, Timothy J. O’Donnell, Doina Precup, Jason Weston, Jackie C.K.Cheung. 2023. The Stable Entropy Hypothesis and Entropy-Aware Decoding: An Analysis and Algorithm for Robust Natural Language Generation. arXiv:2302.06784.
- Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, Peter Clark. 2023. Self-Refine: Iterative Refinement with Self-Feedback. arXiv:2303.17651.
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903. Also: CoT blog post.
- Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, Denny Zhou. 2022. Self-Consistency Improves Chain of Thought Reasoning in Language Models. arXiv:2203.11171.
- Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, and Yejin Choi. 2022. Reframing Human-AI Collaboration for Generating Free-Text Explanations. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 632–658, Seattle, United States. Association for Computational Linguistics.
- Ruibo Liu, Jason Wei, Shixiang Shane Gu, Te-Yen Wu, Soroush Vosoughi, Claire Cui, Denny Zhou, Andrew M. Dai. 2022. Mind’s Eye: Grounded Language Model Reasoning through Simulation. arXiv:2210.05359
- Abulhair Saparov, He He. 2022. Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought. arXiv:2210.01240.
- Amrith Krishna, Sebastian Riedel, Andreas Vlachos. 2022. ProoFVer: Natural Logic Theorem Proving for Fact Verification. Transactions of the Association for Computational Linguistics (2022) 10: 1013–1030.
- Vivek Gupta, Shuo Zhang, Alakananda Vempala, Yujie He, Temma Choji, Vivek Srikumar. 2022. Right for the Right Reason: Evidence Extraction for Trustworthy Tabular Reasoning. In Proc. ACL-22.
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. arXiv:1707.06347.
- John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, Pieter Abbeel. 2015. High-Dimensional Continuous Control Using Generalized Advantage Estimation. arXiv:1506.02438.
- Hugging Face DeepRL course. Also: Bias-Variance Trade-off in RL blog post.
Policy on Academic Misconduct
It is the responsibility of the Committee on Academic Misconduct to investigate or establish procedures for the investigation of all reported cases of student academic misconduct. The term “academic misconduct” includes all forms of student academic misconduct wherever committed; illustrated by, but not limited to, cases of plagiarism and dishonest practices in connection with examinations. Instructors shall report all instances of alleged academic misconduct to the committee (Faculty Rule 3335-5-487). For additional information, see the Code of Student Conduct.
Students with Disabilities
The University strives to make all learning experiences as accessible as possible. In light of the current pandemic, students seeking to request COVID-related accommodations may do so through the university’s request process, managed by Student Life Disability Services. If you anticipate or experience academic barriers based on your disability (including mental health, chronic or temporary medical conditions), please let me know immediately so that we can privately discuss options. To establish reasonable accommodations, I may request that you register with Student Life Disability Services. After registration, make arrangements with me as soon as possible to discuss your accommodations so that they may be implemented in a timely fashion. SLDS contact information: firstname.lastname@example.org; 614-292-3307; slds.osu.edu; 098 Baker Hall, 113 W. 12th Avenue.
Our inclusive environment allows for religious expression. Students requesting accommodations based on faith, religious or a spiritual belief system in regard to examinations, other academic requirements or absences, are required to provide the instructor with written notice of specific dates for which the student requests alternative accommodations at the earliest possible date. For more information about religious accommodations at Ohio State, visit odi.osu.edu/religious-accommodations.
Weather or Other Short-Term Closing
Should in-person classes be canceled, we will meet virtually via CarmenZoom during our regularly scheduled time.
This syllabus is subject to change. All important changes will be made in writing (email), with ample time for adjustment.