Spring ’22, TTh 2:20–3:40, PAES Bldg A103
Instructor: Michael White
This course will explore both traditional, linguistically inspired methods, which offer high precision and transparency, as well as recent methods using neural language models, which are much easier to engineer and generally more robust. In the first part of the course, students will learn how to use Prolog to construct semantic representations for fragments of natural language and perform inference with these representations. In the second part of the course, students will learn about methods for using large language models and the PyTorch deep learning toolkit to map from sentences to meaning representations and back, concluding with a group project. At the end of the course, students should be in a good position to appreciate ongoing developments in computational semantics.
Students in the course will have an opportunity to:
- Become familiar with the principal terminology, concepts and techniques of computational semantics
- Learn how to automatically construct and reason with semantic representations of natural language
- Discover the properties of natural language most relevant to logical reasoning
- Explore continuation-based approaches to semantic construction
- Investigate recent approaches to using large language models for semantic parsing and question answering
- Examine how deep learning models can be used with logical reasoning, including natural logic
- See the messy side of a real research project
- Develop an appreciation of the awesome complexity and richness of
Ling 5801 and 5802 or equivalent, or permission of the instructor. The course is open to advanced undergraduate and graduate students.
Topics are expected to include:
- Programming in Prolog
- First-order logic as a tool for computational semantics
- Using the lambda calculus to construct semantic representations
- Handling scope ambiguities with continuations
- Constructing Discourse Representation Theory’s DRSs (Discourse
Representation Structures) and translating them to first-order
logic; resolving pronouns to accessible antecedents; implementing van der Sandt’s algorithm for presupposition projection and accommodation
- Using distributional semantics together with compositional
- Learning to interpret natural language queries
- Topics TBA, depending on your project proposals
We’ll use Carmen for the detailed schedule, for homework and project assignments, and for distributing slides and links to background/tutorial materials. There will also be discussion forums for posting advance questions on the readings.
It is up to you to make sure you have access to the appropriate hardware and software for doing the assignments. As some assignments in the second part of the course may have runtimes on the order of hours, you will have access to an Ohio Supercomputer Center (OSC) classroom account. We will go over how to use OSC resources in class.
Letter grades will be assigned using the standard OSU scale based on
class participation, homework assignments and the group project.
Class participation (10%)
You will be expected to keep up with the readings and actively participate in class discussions and activities, which naturally requires regular attendance.
In addition, as we will be reading some papers from the primary literature, we will tackle those papers as in advanced seminars. Accordingly, we will take a page from Eric Fosler-Lussier’s playbook and require everyone (this includes you!) to post at least one question to the discussion forum on Carmen by 8 p.m. the evening before the reading will be discussed, so that the discussion facilitator can organize the class session around key questions. Participants should also feel free to share their (initial) thoughts and views of the papers in their posts. In particular, questions of the type What did they mean by X? or Why did they do X instead of Y? are encouraged. Remember that most of the papers are targeted to people who are already expert in the area, so you shouldn’t expect to alway understand everything. Airing such questions can help everyone gain a better understanding of the paper — even those who thought they understood it!
Homework assignments (60%)
Homework assignments play a central role in achieving the course’s learning objectives. There will be four regular homework assignments, with the lowest score dropped in calculating the grade, and possibly a bonus homework. Homework assignments are due by the beginning of class, in the Carmen dropbox. No late homeworks will be accepted.
Each assignment has a programming component and an analytical writeup. In general, I will not grade your code directly; you are expected to provide evidence in your writeup that your code works (or explain how and why it does not work). If your code does not work, I will give feedback if I can.
Collaborative discussion of the homeworks is encouraged but each student should turn in their own programs and write-ups. Copying of notes or code snippets is strictly disallowed, and you must fully understand what you turn in.
Group project (30%)
You will be expected to investigate an interesting NLP problem as part of a group of 2–3 students. Where feasible, you are encouraged to pursue a project related to your own research interests.
Note that the group project is not required to be novel research. Instead, it is expected to require roughly the same level of effort as one of the homework assignments. However, as the project requires both design and presentation activities, it carries 1.5 times the weight of one of the homework assignments. A typical project might involve attempting to replicate the results reported in the textbook or the research literature on an NLP task
using your own implementation.
You should declare your group on Carmen by the end of week 4 of the semester. Preliminary proposals for projects are due in the 8th week of the semester, and should include the choice of a related research paper that the class will read. (Revisions to the project proposal may be made based on feedback.) Groups will lead the discussion of the selected research paper towards the end of the course, and present the project results during the last week of the course. Taking into account feedback from the presentations, project write-ups will be due by the day the final exam would be held (if there were one).
When facilitating the discussion of the selected reading, groups should look over the questions posted on Carmen and choose a subset for discussion. In class, facilitators should start the session with a brief, five to ten minute summary of the paper, including the highlights and lowlights. Following the opening summary, facilitators are responsible for managing the discussion, and ensuring that as many viewpoints are heard as possible.
Points for the group project will be distributed as follows (when submitted on time):
- group declaration (2%)
- project proposal (3%)
- selected paper discussion (5%)
- project presentation (10%)
- project write-up (10%)
Additionally, a best project bonus (3%) will be given to the project voted to be the best following the project presentations.
The first part of the course will be based in part on two classic textbooks by Patrick Blackburn & Johan Bos, Representation and Inference for Natural Language: A First Course in Computational Semantics and Working with Discourse Representation Theory: An Advanced Course in Computational Semantics, as well as additional classic and more recent primary readings listed below. The first Blackburn & Bos book is out in paperback and available in the library. We will rely on the publicly available slides rather than requiring the book to be purchased. Their second book is only available in a draft form that is somewhat out-of-date with respect to the accompanying slides. The implementation of van der Sandt’s approach to anaphora resolution and presupposition projection is also described in Bos’s (2003) article in Computational Linguistics.
In the second part of the course, we will make occasional use of the nascent third edition of Jurafsky and Martin’s textbook, Speech and Language Processing, as well as other sources.
Alternative approaches to semantic construction
- Chris Barker. 2002. Continuations and the nature of quantification. Natural Language Semantics, 10:211–242.
- Chris Barker and Chung-chieh Shan. 2015. Continuations and Natural Language. Oxford University Press.
- Simon Charlow. 2014. On the semantics of exceptional scope. Doctoral dissertation, New York University.
- Chris Barker. 2021. Rethinking Scope Islands. Linguistic Inquiry.
- Judith Tonhauser, David Beaver, Craige Roberts and Mandy Simons. 2013. Toward a taxonomy of projective content. Language, 89:66–109.
Analyzing Neural Models
- Daniel Keysers, Nathanael Schärli, Nathan Scales, Hylke Buisman, Daniel Furrer, Sergii Kashubin, Nikola Momchev, Danila Sinopalnikov, Lukasz Stafiniak, Tibor Tihon, Dmitry Tsarkov, Xiao Wang, Marc van Zee and Olivier Bousquet. 2020. Measuring Compositional Generalization: A Comprehensive Method on Realistic Data. In Proc. ICLR.
- Yanai Elazar, Hongming Zhang, Yoav Goldberg and Dan Roth. 2021. Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema. In Proc. EMNLP.
- Inbar Oren, Jonathan Herzig and Jonathan Berant. 2021. Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalization. In Proc. EMNLP.
- Gabor Angeli and Christopher D. Manning. 2014. NaturalLI: Natural Logic Inference for Common Sense Reasoning. In Proc. EMNLP.
- Leon Weber, Pasquale Minervini, Jannes Münchmeyer, Ulf Leser and Tim Rocktäschel. 2019. NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language. In Proc. ACL.
- Jihao Shi, Xiao Ding, Li Du, Ting Liu and Bing Qin. 2021. Neural Natural Logic Inference for Interpretable Question Answering. In Proc. EMNLP.
Policy on Academic Misconduct
As with any class at this university, students are required to follow the Ohio State Code of Student Conduct. In particular, note that students are not allowed to, among other things, submit plagiarized (copied but unacknowledged) work for credit. If any violation occurs, the instructor is required to report the violation to the Council on Academic Misconduct.
Students with Disabilities
Students who need an accommodation based on the impact of a disability should contact me to arrange an appointment as soon as possible to discuss the course format, to anticipate needs, and to explore potential accommodations. I rely on the Office of Disability Services for assistance in verifying the need for accommodations and developing accommodation strategies. Students who have not previously contacted the Office for Disability Services are encouraged to do so (292-3307; http://www.ods.ohio-state.edu).
This syllabus is subject to change. All important changes will be made in writing (email), with ample time for adjustment.