Programming for Performance
CS 610, Semester 2024-25-I, IIT Kanpur
Class hours: MF 8-9 AM Tu 9-10 AM in KD 103
Instructor Information
TA Information
Name |
Email (AT cse.iitk.ac.in) |
Srinjoy Sarkar |
srinjoy |
Course Description
To obtain good performance, one needs to write correct but scalable
parallel programs using programming language abstractions like threads. In
addition, the developer needs to be aware of and utilize many
architecture-specific features like vectorization to extract the full
performance potential. This course will discuss programming language
abstractions with architecture-aware development to learn to write
scalable parallel programs. This is not a “programming tips and tricks”
course.
We will have 4-6 assignments to use the concepts learned in class
and appreciate the challenges in extracting performance.
Prerequisites
|
-
Exposure to the following courses (or equivalent) is desirable:
CS220 (Computer Organization), and CS330 (Operating Systems).
-
Programming maturity with popular programming languages like C,
C++, and Java.
|
Syllabus
The course will focus on a subset of the following topics.
-
Challenges in parallel programming, correctness, and
performance errors, understanding performance, and performance models
-
Exploiting spatial and temporal locality with caches, analytical cache
miss analysis
-
Performance bottleneck analysis: PAPI counters, performance analysis
tools
-
Dependence Testing, Loop Transformations
- Shared-memory programming and Pthreads
-
Compiler vectorization: vector ISA, auto-vectorizing compiler, vector
intrinsics
- Core OpenMP, Advanced OpenMP, Heterogeneous programming with
OpenMP
-
Parallel Programming Models and Patterns
- GPU Architecture and CUDA Programming
- Shared-memory Synchronization
- Concurrent Data Structures
We may add new, drop existing, or reorder topics depending on progress and
class feedback. The course may also involve reading and critiquing related
research papers.
Policies
- Please be on time for class.
-
Please try to avoid using laptops and/or mobile devices in class, they
are distracting for both the instructor and other students who want to
concentrate.
-
You are allowed up to two late days to submit your
assignment, with a 25% penalty for each day.
Feedback
I am open to feedback about the course content and
presentation. Feel free to provide suggestions for improvements.
-
You may discuss concepts with classmates, but
all submissions must be your own or your team's work when teamwork is
permitted.
-
You should not search online for existing solutions
related to the assignments, even as a reference.
-
Students caught cheating or plagiarizing will receive penalties,
automatically fail the course, and will be reported to the institute.
Assignments |
40% |
Midsem |
30% |
Endsem |
30% |
-
This is a tentative allocation and might change slightly depending on the strength of the class
- Grading will be relative
-
Register on
Canvas
for course management
-
Register on
Piazza
for discussions
Date |
Topic |
Resources |
Recommended Reading |
|
|
First Course
Handout
|
|
30/07, 02/08 |
Compiler Challenges for Parallel Architectures
|
Slides
|
AK 1.1-1.6
|
05/08, 06/08, 09/08 |
Cache Memory |
Slides
|
HP APP B.1-B.4, 2.1--2.3
CSAPP 6.2-6.4
|
12/08, 13/08 |
Write Cache-Friendly Code |
Slides
|
CSAPP 6.5-6.6
DRAG 11.1-11.2
|
16/08 |
PAPI Library |
Slides
|
|
19/08, 20/08, 23/08, 27/08, 30/08 |
Cache Coherence and False Sharing |
Slides
|
MCM Chapters 2, 6, 8 (IITK has subscribed to the ebook)
|
|
Dependence Testing |
Slides
|
AK Chap 2, 3
DRAG 11.6
|
|
Loop Transformations |
|
AK 5.2-5.4, 5.7.2, 5.9, 6.2.1, 6.2.2, 6.2.5, 6.3.1-6.3.4
AP 4.1, 4.2, 4.5, 5.1-5.6
HP 4.5
Compiler Transformations for High-Performance Computing
|
|
Vectorization |
|
HP 4.3
Intel oneAPI DPC++/C++ Compiler
Developer Guide and Reference
Intel C++ Compiler Classic Developer
Guide and Reference
Guide for Intel Compilers
|
|
OpenMP |
|
OpenMP Application Programming Interface
v5.2
OpenMP Application Programming Interface
Examples v5.2.1
PP Chapter 5 (IITK has subscribed to the ebook)
LLNL OpenMP Tutorial
|
|
GPU Programming with CUDA
|
|
NVIDIA CUDA C Programming Guide
NVIDIA CUDA C Best Practices Guide
KH Chapters 1-5 (IITK has subscribed to the ebook)
HP 4.3
|
|
Concurrent Data Structures |
|
MP Chapters 9, 10, 13 (IITK has subscribed to the
ebook)
|
I have listed (NOT in any particular order) a few popular references.
-
[CSAPP] Computer Systems: A Programmer's Perspective, 3rd edition
- R. Bryant and D. O'Hallaron
-
[DRAG] Compilers: Principles, Techniques, and Tools - A. Aho, M.
Lam, R. Sethi, and J. Ullman
-
[HP] Computer Architecture: A Quantitative Approach, 6th edition
- J. Hennessy and D. Patterson
-
[AK] Optimizing Compilers for Modern Architectures - R. Allen and
K. Kennedy
-
[AP] Automatic Parallelization: An Overview of Fundamental
Compiler Techniques - Samuel P. Midkiff
-
[PP] An Introduction to Parallel Programming - Peter S. Pacheco
-
[KH] Programming Massively Parallel Processors: A Hands-on
Approach, 3rd edition - David B. Kirk and Wen-mei W. Hwu
- [MCM] A Primer on Memory Consistency and Cache Coherence, 2nd edition - Vijay
Nagarajan, Daniel J. Sorin, Mark D. Hill and David A. Wood
- [MP] The Art of Multiprocessor Programming, 1st edition - Maurice Herlihy and Nir
Shavit
We may read and discuss related materials and research papers, which we will
announce in class.