Ling 5802 sample syllabus

Computational linguistics II: Statistical Natural language processing

This course covers the fundamentals of designing, building and evaluating data-driven programs for natural language processing. The course will proceed step-by-step to walk you through a series of weekly activities intended to teach you some of the hands-on skills needed to do practical machine learning. It will also aim to teach you to read papers in the field of natural language processing, and to understand some of the reasoning behind the design of popular models.

The course is designed for graduate students and advanced undergraduates. It will involve a significant amount of programming (Python recommended), and also a reasonable amount of probability. Computational Linguistics 1 provides these prerequisites. If CL1 was your first programming experience, I hope to make CL2 useful and accessible to you, but be warned that it will be a substantial amount of work.

There are various texts and supplemental readings available on Carmen. Most of these are drawn from “Speech and Language Processing” by Jurafsky and Martin, which is available on the web. We will also collaboratively read some scientific papers and discuss them in class.

Some of the topics and due dates listed may change as the semester goes on. The course is made up of a number of small “modules” which take a variable amount of time to cover. We may cover a unit faster than expected and go on to the next one within a single class meeting— or we may need to spend extra time. Lecture notes, assignments and materials uploaded to Carmen may change before the dates on which they are scheduled to be presented.

The course modality is in-person. Although I am willing to livestream or record any lecture on Zoom by request, be aware that virtual attendance is a second-best accommodation. I will not make accommodations for online attendees beyond simply streaming the in-person meeting. However, if you are sick, please stay home. I am glad to go over course materials with you during office hours and to extend deadlines so that you do not need to work when you are sick.

Grading: Grades will be assigned on the following basis:

  • Weekly assignments: 75%
  • Participation and paper responses: 25%
Day Topic Reading Assigned
M 1/9 Basics of estimation Jurafsky ch. 3 Assignment 1
W 1/11 Regularization / class activities Assignment 2
M 1/16 MLK day
W 1/18 Metrics Assignment 3
M 1/23 Kettunen 2014 / class activities  
W 1/25 Naive Bayes and maximum entropy Jurafsky ch. 4/5 Assignment 4
M 1/30 Goldsmith 2000 / class activities    
W 2/1 Perceptrons, gradient estimation Jurafsky ch. 5 Assignment 5
M 2/6 Hayes and Wilson 2008 / class activities    
W 2/8 Tensorflow Assignment 6
M 2/13 Wiemerslage et al 2022 / class activities  
W 2/15 Feature selection and dimensionality reduction Assignment 7
M 2/20 Mager et al 2018 / class activities  
W 2/22 Feedforward NNs Jurafsky ch. 7 Assignment 8
M 2/27 Elhage et al 2022 / class activities    
W 3/1 Static word embeddings Jurafsky ch. 6 / Riedel tutorial Assignment 9
M 3/6 Hamilton et al 2016 / class activities    
W 3/8 Sequential and contextual problems Assignment 10
3/13 Spring break!
3/15 Spring break!
M 3/20 Moeller 2019 / class activities
W 3/22 Recurrent NNs and LSTMs Jurafsky ch. 9 (to 9.6) / Chris Olah tutorial Assignment 11
M 3/27 Kann et al 2018, Roest et al 2020 / class activities    
W 3/29 Alignment and attention Jurafsky ch. 10 Assignment 12
M 4/3 Dankers 2021 / class activities    
W 4/5 Large pretrained models Jurafsky ch. 9 (9.7) Assignment 13
M 4/10 Huang et al 2021 / class activities    
W 4/12 BERT-ology Assignment 14
M 4/17 Li et al 2022 / class activities
W 4/19 TBA
M 4/24 TBA

Collaboration policy

You are allowed to discuss assignments with classmates, but the code and written assignments you turn in must be entirely your own work; you may not incorporate notes or code snippets written by others in the class, nor should you allow your classmates to use your written notes or view your computer code.

You may use all standard Python libraries as well as their accompanying documentation. Some assignments specify particular parts of the Tensorflow or Keras APIs which are required; otherwise you may use any part of the APIs you would like. You may not use arbitrary code from Github, library example code or other code which you did not write yourself to complete homework assignments, although you may use code written by others to complete the final research project as long as it is appropriately cited. Contact the instructor if you have any questions.

It is the responsibility of the Committee on Academic Misconduct to investigate or establish procedures for the investigation of all reported cases of student academic misconduct. The term “academic misconduct” includes forms of student academic misconduct wherever committed; illustrated by, but not limited to, cases of plagiarism and dishonest practices in connection with examinations. Instructors shall report all instances of alleged academic misconduct to the committee (Faculty Rule 3335-5-487). For additional information, see the Code of Student Conduct (hppt://studentaffairs.osu.edu/info_for_students/csc.asp).

Assignments and grades

Grading will be based primarily on the assignments, each of which has a programming component and an analytical writeup. In general, I will not grade your code directly; you are expected to provide evidence in your writeup that your code works (or explain how and why it does not work). If your code does not work, I will give feedback if I can. In general, the feedback you get will be designed to help you change the code so that it does work, and I expect you to do so.

It is up to you to make sure you have access to the appropriate hardware and software for doing the assignments. Some assignments may have runtimes on the order of hours. All students in arts and sciences may gain access to the Unity cluster using http://go.osu.edu/unitysubmit

Links to an external site..

Projects must be turned in on time. However, if you submit the project on time, but you made some effort to do the project, you may turn in further versions of the project after the due date, which will be regraded for up to full credit. Turn in some attempt to do the project, by the due date. If you don’t turn in anything by the due date, you won’t get credit.

The course contains paper discussions nearly every week (on Wednesday). For these classes, you are expected to post a comment or question related to the reading. Your participation can take the form of a new discussion question/comment, or response to an existing question/comment. Your goal should be to identify important and interesting issues for in-class discussion (including, potentially, things you find confusing). You are encouraged to ask purely comprehension questions on the discussion boards as well, but this should be in addition to a question/comment that is discussion-oriented.

Administrative material

Disability Accommodation

The university strives to make all learning experiences as accessible as possible. In light of the current pandemic, students seeking to request COVID-related accommodations may do so through the university’s request process, managed by Student Life Disability Services.  If you anticipate or experience academic barriers based on your disability (including mental health, chronic, or temporary medical conditions), please let me know immediately so that we can privately discuss options.  To establish reasonable accommodations, I may request that you register with Student Life Disability Services.  After registration, make arrangements with me as soon as possible to discuss your accommodations so that they may be implemented in a timely fashion. SLDS contact information: slds@osu.edu; 614-292-3307; slds.osu.edu; 098 Baker Hall, 113 W. 12th Avenue.

Health and safety requirements

All students, faculty and staff are required to comply with and stay up to date on all university safety and health guidance (https://safeandhealthy.osu.edu), which includes following university mask policies and maintaining a safe physical distance at all times. Non-compliance will be warned first and disciplinary actions will be taken for repeated offenses.

A reminder that other helpful information can be found on the Safe and Healthy Teaching website: https://safeandhealthy.osu.edu/information/faculty-and-staff/teaching.

Diversity As a Core Value:

The Ohio State University affirms the importance and value of diversity in the student body. Our programs and curricula reflect our multicultural society and global economy and seek to provide opportunities for students to learn more about persons who are different from them. We are committed to maintaining a community that recognizes and values the inherent worth and dignity of every person; fosters sensitivity, understanding, and mutual respect among each member of our community; and encourages each individual to strive to reach his or her own potential. Discrimination against any individual based upon protected status, which is defined as age, color, disability, gender identity or expression, national origin, race, religion, sex, sexual orientation, or veteran status, is prohibited.

Information about Counseling and Consultation Services:

As a student you may experience a range of issues that can cause barriers to learning, such as strained relationships, increased anxiety, alcohol/drug problems, feeling down, difficulty concentrating and/or lack of motivation. These mental health concerns or stressful events may lead to diminished academic performance or reduce your ability to participate in daily activities. The Ohio State University offers services to assist you with addressing these and other concerns you may be experiencing. If you or someone you know are suffering from any of these conditions, you can learn more about the broad range of confidential mental health services available on campus via the Office of Student Life Counseling and Consultation Services (CCS) by visiting ccs.osu.edu or calling (614) 292- 5766. CCS is located on the 4th Floor of the Younkin Success Center and 10th Floor of Lincoln Tower. You can reach an on-call counselor when CCS is closed at (614) 292-5766 and 24-hour emergency help is also available through the 24/7 National Prevention Hotline at 1-(800)-273-TALK or at suicidepreventionlifeline.org.

Religious accommodations: Our inclusive environment allows for religious expression. Students requesting accommodations based on faith, religious or a spiritual belief system in regard to examinations, other academic requirements or absences, are required to provide the instructor with written notice of specific dates for which the student requests alternative accommodations at the earliest possible date. For more information about religious accommodations at Ohio State, visit odi.osu.edu/religious-accommodations.

Class cancelation: Should in-person classes be canceled, I will notify you as to which alternative methods of teaching will be offered to ensure continuity of instruction for this class. Communication will be via CarmenCanvas.