CS 781: Large Language Models (LLMs)

Units: 3-0-0-0 (9) (L-T-P-C)

Pre-requisites: Proficiency in Linear Algebra, Probability and Statistics, Proficiency in Python Programming.

 

Course Description

In recent times, Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP). However, the application of LLMs has not just remain limited to NLP but has also advanced other areas like Biology, Chemistry, Economics, etc. This calls for in-depth understanding of LLMs. This course will introduce the fundamentals of LLMs and go in-depth into various techniques to develop LLMs, model architecture, scaling laws, training, alignment (RLHF), etc. It will teach how to fine-tune LLMs using parameter efficient techniques, how LLMs could be used in conjunction with external knowledge sources such as vector databases (RAG), how to do inference over LLMs efficiently. We will look at applications of LLMs such as agentic architectures, etc. We will also look into inner workings of the LLM via mechanistic interpretability techniques. The course covers the required mathematical theory and delves into on the practical aspects with the focus on hands-on development. CS781 is a practically oriented course and the main goal of this course is to develop good hands-on experience in developing and using LLMs for various applications. There will be several assignments and a capstone project where students will be building a practical working system. The outcome of this course is that you will have expertise to build your own LLMs (and associated systems) and have the knowledge to conduct advance research in LLMs or get a job in company working on developing LLMs and applications.

 

Course Contents

1. Module 1: Language Modeling Task
   1. Introduction to Language Modeling
   2. Classical Language Modeling (CLM): n-grams, smoothing, class-based, brown clustering, etc
   3. Neural Language Modeling (NLM): Word Embeddings, Word2Vec, FeedForward Neural LM, Contextualization, Sub-tokenization and Subword information, etc.
   4. Brief History of Language Modeling
2. Module 2: Background and Neural Sequence Models
   1. Recurrent Neural Networks (RNNs)
   2. Attention Mechanism
   3. Transformer Architecture
   4. Attention Variants (MHA, MQA, GQA, Flash) and Mixture of Experts
   5. Position Embeddings: Standard PE, RoPE
   6. Transformers for Language Modeling: Encoder Models, Encoder-Decoder Model, Decoder Models, Pre-trained LMs (PLMs), objective functions for training, etc.
   7. Transformer Alternatives: xLSTM, State based model, etc.
3. Module 3: Building Large Language Models (LLMs)
   1. Introduction to Large Language Models (LLMs): PLMs vs LLMs, LLM families
   2. Scaling Laws: Kaplan’s law, Chinchilla Law
   3. Training LLMs from Scratch: Corpus scrapping and construction (e.g., Pile dataset construction), cleaning and pre-processing, architecture, deciding objective function, modern optimizer (e.g., muon), using scaling laws, training curriculum, etc.
4. Module 4: Using LLMs
   1. Context Length
   2. Temperature, Decoding Strategies: Greedy, Beam Search, Top-K, Top-p Nucleus Sampling, Speculative Decoding
   3. In Context Learning (ICL): Prompting techniques (zero shot, few shot, etc.), Chain of Thought, Tree of Thought, X of Thought, etc.
   4. Instructing Tuning
   5. Parameter Efficient Fine-Tuning (PEFT): Transfer Learning, Soft-Prompting, Adaptors, LoRA (and variants)
   6. Quantization
   7. Knowledge Distillation
   8. Efficient LLM Inference: KV Caching and related techniques.
   9. Self-consistency and Test Time Scaling
5. Module 5: LLM Alignment, Reasoning and Interpretability
   1. Alignment Tuning: Providing Human Feedback, RLHF, DPO, GRPO, ORPO, etc.
   2. LLM Reasoning
   3. Grokking and Double Descent
   4. Understanding LLM inner workings: Mechanistic Interpretability
6. Module 6: LLM Applications and Advances
   1. Using LLMs with Vector Databases: Retrieval Augmented Generation and related techniques.
   2. LLM Evaluation: Benchmarks, LLM-as-a-judge (pros and cons)
   3. Agentic Systems
   4. Multimodal LLMs: Vision Language Models

 

References:

Since this is new and emerging area, there are no specific references, this course gleans information from a variety of sources like research papers, tutorials, blogs, etc. Relevant references would be suggested in the lectures