Natural Language Processing

This is the repository for the Natural Language Processing course at the Asian Institute of Technology.

Google slide lectures can be found: https://drive.google.com/drive/folders/14x9_-Y_aWysPIZFLaVrZE2ngy2h_beJj?usp=sharing

YT playlist: https://www.youtube.com/playlist?list=PLqL-7eLmqd9V3faivSAST76YQClS44dSz

Credits

I would also like to give huge credits to several GitHub / web resources that I have revised to create this:

• https://web.stanford.edu/class/cs224n/
• http://spacy.pythonhumanities.com/
• http://courses.spacy.com
• https://github.com/kushalj001/pytorch-question-answering
• https://github.com/DSKSD/DeepNLP-models-Pytorch
• https://github.com/bentrevett
• https://github.com/graykode/nlp-tutorial
• https://huggingface.co/course
• https://kikaben.com
• https://github.com/kipgparker/soft-prompt-tuning
• https://github.com/moein-shariatnia/OpenAI-CLIP

Useful GitHub

• https://github.com/keon/awesome-nlp (collection of all NLP learning resources)
• https://github.com/sebastianruder/NLP-progress (omg...this is like a mini Wikipedia for NLP!)
• https://github.com/mhagiwara/100-nlp-papers (listed first 100 influential NLP papers)
• https://captum.ai/tutorials/ (good for working on explainable AI)

I would also like to thank students who have contributed:

• Amanda Raj Shrestha; Email: st122245@ait.ac.th
• Pranisaa Charnparttarvanit; Email: st121720@ait.ac.th
• Chanapa Pananookooln; Email: st121395@ait.ac.th
• Todsavad Tangtortan; Email: st123012@ait.ac.th

Outline

1. Fundamentals

   • Word Vectors - Word2Vec (Naive)
   • Word Vectors - Word2Vec (Negative sampling)
   • Word Vectors - GloVe
   • Window-Based Name Entity Recognition
   • Dependency Parsing
   • Information Retrieval / Salient Spans

2. DL

   • Classification
   • Sequence-to-Sequence Models
     • LSTM, biGRU, CNN, Transformer
   • Langauge Models
     • LSTM, Transformer
   • Masked Language Models
     • BERT
     • Much more...
   • Case studies
     • QA
     • Summarization
     • Pruning
     • distilBERT
     • SentenceBERT
     • SimCSE
     • Much more...

3. SpaCy

4. Huggingface

   • Case studies

5. Retrieval Augmented Generation

   • Prompt, Chain, Tools, Agent

6. Multimodal Language Model

   • ViT, BEIT, CLIP, SimVLM, Flamingo, BLIP-2, CoCa

7. Reinforcement Learning with Human Feedback

   • SFT, PPO, DPO, RRHF