Lectures
1. Introduction & Motivation
2. Probability Theory I
- Date: August, 27 (Wednesday)
- Lecture Notes
- Reference: Gardiner Chapter 2
3. Probability Theory II
- Date: September 03, 2025 (Wednesday)
- Lecture Notes
- Reference: Gardiner Ch 2
- Random walk demonstration
4. Entropy
- Date: September 08, 2025 (Monday)
- Lecture Notes
- Reference: Cover & Thomas Ch 2
- Shannon’s original paper
5. Applications of Entropy I (Fano’s inequality, Statistical Mechanics)
- Date: September 10, 2025 (Wednesday)
- Lecture Notes
- Reference: Cover & Thomas Ch 2 & Parrondo/Horowitz/Sagawa Paper
6. Applications of Entropy II (Landauer’s principle, AEP)
- Date: September 15, 2025 (Monday)
- Lecture Notes
- Reference: Parrondo/Horowitz/Sagawa Paper & Cover & Thomas Ch 3
7. Applications of Entropy III (AEP, Coding & Compression)
- Date: September 17, 2025 (Wednesday)
- Lecture Notes
- Reference: Cover & Thomas Ch 3, 5
8. Applications of Entropy IV (Optimal coding & Phase Transitions)
- Date: September 22, 2025 (Monday)
- Lecture Notes
- Reference: Cover & Thomas Ch 5, Vogel/Saravia/Cortez paper
- Compressing a phase transition code
9. Relative Entropy
- Date: September 24, 2025 (Wednesday)
- Lecture Notes
- Reference: Cover & Thomas Ch 2
10. Applications of relative entropy
- Date: September 29, 2025 (Monday)
- Lecture Notes
- Reference: Parrondo/Horowitz/Sagawa Paper & Cover & Thomas Ch 5
- Other references: Thermodynamics of information book chapter
11. Mutual information, Data Processing Inequality, and Sufficient Statistics
- Date: October 1, 2025 (Wednesday)
- Lecture Notes
- Reference: Cover & Thomas Ch 2
12. Channel Coding I
- Date: October 6, 2025 (Monday)
- Lecture Notes
- Reference: Cover & Thomas Ch 7
13. Channel Coding II
- Date: October 8, 2025 (Monday)
- Lecture Notes
- Reference: Cover & Thomas Ch 7