Iowa State University
Department of Computer Science
Com S 579: Natural Language Processing (NLP)
Spring 2024
This syllabus is subject to change.
INSTRUCTOR: Dr. Forrest Sheng Bao, email: fsb at iastate dot edu
Teaching Assistant(TA): Zefu Hu, zefuh at iastate dot edu Please CC the TA for all communication with the instructor
Slides and reading materials URL: https://github.com/forrestbao/nlp-class
Canvas: https://canvas.iastate.edu/courses/108437 Canvas will be used to post announcements, grades, and lecture videos. Also, feel free to make discussion on Canvas.
OFFICE HOURS: 4:30 to 5:30 PM Tuesdays and Thursdays on Zoom, or by appointment
TEXTBOOK: Suggested
- Speech and Language Processing, by Daniel Jurafsky and James Martin, 3rd Edition, 2020. Free online at https://web.stanford.edu/~jurafsky/slp3/.
- Introduction to Natural Language Processing, by Jacob Eisenstein, MIT Press, 2019 https://mitpress.mit.edu/books/introduction-natural-language-processing
Introduction to concepts and techniques for automatically processing and understanding natural languages with computers; tokenziation; language models; machine learning approaches to natural language processing; neural language models; common tasks in NLP: information extraction, question answering, summarization, machine translation, and Retrieval Augmented Generation (RAG); Transformers and Large Language Models (LLMs).
The course involves the instructor's lectures and student presentations. The instructor will cover the basics of NLP. Students are required to complete a group or individual research project. An oral presentation of the term project is expected at the end of the semester. You are encouraged to pick a topic that would apply NLP in your research area.
- Presenting a recent milestone paper in the area of NLP: 50%
- Project: 50%
FINAL GRADES: Final grades will be given as follows, although the instructor retains the right to lower the grade thresholds at his discretion.
A 90%-100% B 80%-89% C70%-79% D 60%-69% F below 60%
TENTATIVE SCHEDULE Weekly:
| Week | Topic | Notes |
|---|---|---|
| 1 | Introduction, Background of AI and NLP | |
| 2 | Python programming and Preprocessing (syntactical analysis, data crawling, etc.) | |
| 3 | POS tagging and HMM | |
| 4 | N-gram language models, TF-IDF and BM25 | |
| 5 | Machine Learning basics, Linear classifiers, feature engineering in NLP | |
| 6 | Neural Networks and Deep Learning | |
| 7 | Neural Language models, word and sentence embeddings, word2vec | |
| 8 | Transformers and BERT | |
| 9 | The LLM era: PEFT/LoRA, RAG, Agent, and multimodality | |
| 10 | Text reasoning and knowledge extraction | |
| 11 | Text generation: machine translation, summarization | |
| 12 | Special topics: text generation metrics and hallucination | |
| 13 | Student presentation | |
| 14 | Student presentation | |
| 15 | Student presentation |
Iowa State University supports and upholds the First Amendment protection of freedom of speech and the principle of academic freedom in order to foster a learning environment where open inquiry and the vigorous debate of a diversity of ideas are encouraged. Students will not be penalized for the content or viewpoints of their speech as long as student expression in a class context is germane to the subject matter of the class and conveyed in an appropriate manner.