ECE1502F: Information Theory
Course Schedule
| Lectures | Topics | Text Reference | Homework | Solutions |
| 9/9, 9/12 | Introduction. Entropy. | Ch. 1.1, 2.1 | -- | -- |
| 9/16, 9/19 | Joint Entropy, Mutual Information. Jensen's inequality. Data Processing inequality. Entropy Rate of a Stochastic Process. | Ch. 2.2-2.6, 2.8 Ch. 4.1-4.2 |
-- |
-- |
| 9/23, 9/26 | Asymptotic Equipartition Property (AEP) | Ch. 3 | -- |
-- |
| 9/30, 10/3 | Gambling on horse race. | Ch. 6.1-6.3 | -- | -- |
| 10/7, 10/10 | Data Compression: Kraft Inequality. Huffman Codes | Ch. 5.1-5.7 | -- |
-- |
| 10/14 | Huffman Codes, Arithmetic Codes. (optional) Lempel-Ziv codes (optional) |
Ch. 5.8, 5.10 Ch. 12.10 |
-- |
-- |
| 10/21, 10/24 | Channel Capacity. Joint Typicality | Ch. 8.1-8.6 | Midterm Exam: Oct. 21 11am-1pm |
-- |
| 10/28, 10/31 |
Achievability and Converse for the Channel Capacity Theorem. Fano's Inequality. Channel with Feedback | Ch. 8.7, 8.9. Ch 2.11, 8.12 |
-- |
-- |
| 11/4, 11/7 | Source-Channel Separation. Differential Entropy, Gaussian Channels |
Ch 8.13, 9.1, 9.4-9.6 Ch. 10.1-10.4 |
-- | -- |
| 11/11, 11/14 | Maximum Entropy Distribution. Gaussain Vector Channels. Waterfilling. Band-limited Gaussian Channel. | Ch. 11.1 Ch. 10.3-10.5 |
-- |
-- |
| 11/18, 11/21 | Multiple Access Channel | Ch. 14.1-14.3 | -- | -- |
| 11/25, 11/28 | Broadcast Channel, Interference Channel | Ch. 14.6 | -- |
-- |
| 12/2 |
Rate-distortion Theory (optional) | Ch. 13.1-13.5 | -- | -- |
| Final Exam | -- | -- |
|