ECE367H1: Matrix Algebra and Optimization (Fall 2026)

 

Instructor:

·      Prof. Wei Yu < weiyu@ece.utoronto.ca >  

·      Office hour: Available after Tuesday class at 5pm, or by Appointment.

 

Teaching Assistants:

·      Faeze Moradi Kalarde < faeze.moradi@mail.utoronto.ca >

·      Nicholas Kwan < nick.kwan@mail.utoronto.ca >

·      Anthony Ho < anth.ho@mail.utoronto.ca >

 

Lectures: (Starting Sept 8)

·      Tuesday 15:10-17:00 GB248

·      Thursday 13:10-14:00 GB244

 

Tutorials: (Starting Sept 15)

·      Tuesday 9:10-11:00 (GB-303 & BA-B024)

 

Important Dates:

·      First day of lecture: Sept 8. (First tutorial: Sept 15)

·      No lectures/tutorials during the study break: Oct 26-30.

·      Midterm: Nov 10, 15:00-17:00.

·      Last day for dropping the course without academic penalty: Nov 17.

·      Last day of the lecture: Dec 8.

·      Final Exam Period: Dec 10-22.

 

Calendar Description:

This course will provide students with a grounding in optimization methods and the matrix algebra upon which they are based. The first part of the course focuses on fundamental building blocks in linear algebra and their geometric interpretation: matrices, their use to represent data and as linear operators, and the matrix decompositions (such as eigen-, spectral-, and singular-vector decompositions) that reveal structural and geometric insight. The second part of the course focuses on optimization, both unconstrained and constrained, linear and non-linear, as well as convex and nonconvex; conditions for local and global optimality, as well as basic classes of optimization problems are discussed. Applications from machine learning, signal processing, and engineering are used to illustrate the techniques developed.

 

Textbooks:

[1]  Giuseppe Calafiore and Laurent El Ghaoui, Optimization Models, Cambridge University Press, 2014. (Main textbook)

[2]  Stephen Boyd and Lieven Vandenberghe, Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares, Cambridge University Press, 2018. (PDF available at authors’ website. Some homework problems are taken from this textbook.)

 

Course Schedule:

 

Week	Topics	Text References	Assessment	Tutorial
Sept 8	Introduction	Ch. 1
Sept 15	Vectors, Norms, Inner Products, Orthogonal Decomposition	Ch. 2.1-2.2	Homework #1: Due Sept 29, 11:59pm (Word Vector, Fourier Series)	Homework #1: Theory
Sept 22	Projection onto Subspaces, Fourier Series. Gram-Schmidt and QR decomposition. Hyperplanes and Half-Spaces. Non-Euclidean Projection.	Ch. 2.3		Homework #1: Applications
Sept 29	Projection onto Affine Sets. Functions, Gradients and Hessians.	Ch. 2.3-2.4	Homework #2: Due Oct 13, 11:59pm (Function Approximation, PageRank)	Homework #2: Theory
Oct 6	Matrices, Range, Null Space, Eigenvalues and Eigenvectors Matrices Diagonalization.	Ch. 3.1-3.5		Homework #2: Application
Oct 13	PageRank Algorithm, Symmetric matrices. Function Approximation.	Ch. 4.1-4.4	Homework #3: Due Nov 3, 11:59pm (Latent Semantic Indexing, EigenFace)	Homework #3: Theory
Oct 20	Orthogonal Matrices. Spectral Decomposition. Positive Semidefinite Matrices. Ellipsoids.	Ch. 5.1, 5.3.2		Homework #3: Applications
Oct 27	Study Break
Nov 3	Singular Value Decomposition. Principal Component Analysis.  Interpretation of SVD. Low-Rank Approximation.	Ch. 5.2-5.3.1		Midterm review
Nov 10	Midterm. Least Squares		Midterm Nov 10 Tuesday 15:00-17:00	Midterm office hour
Nov 17	Overdetermined and Underdetermined Linear Equations.	Ch. 6.1-6.4	Homework #4: Due Nov 24, 11:59pm (Optimal Control, CAT Scan)	Homework #4: Theory
Nov 24	Regularized Least-Squares. Convex Sets and Convex Functions.	Ch. 6.7.3 Ch. 8.1-8.4		Homework #4: Applications
Dec 1	Lagrangian Method for Constrained Optimization. Linear Programming and Quadratic Programming.	Ch. 8.5 Ch. 9.1-9.6	Homework #5: Due Dec 8, 11:59pm (Portfolio Design, Sparse Coding of Image)	Homework #5: Theory
Dec 8	Numerical Algorithms for Unconstrained and Constrained Optimization	Ch. 12.1-12.3		Homework #5: Applications
			Final Exam: TBD

 

Grades:

·      Homework: 15% (Graded for completeness only. 3% per homework x 5. Penalty for late submission: 1% per 24hrs)

·      Midterm: 30% (Type C3, one aid-sheet, non-programmable calculator allowed)

·      Final Exam: 55% (Type C3, one aid-sheet, non-programmable calculator allowed.)

 

Faculty Policy Regarding the Final Exam:

·      This course includes a mandatory final examination. All students are required to complete the final exam; there is no provision for an assessed grade in place of the examination. Students who are unable to write the exam for valid reasons must submit a petition, and if approved will be granted a deferred exam.

·      To earn a passing grade in this course, students must achieve a minimum grade of 40% on the final examination, regardless of their performance on other course components. Students who do not achieve at least 40% on the final exam will not pass the course, and a grade of 49% will be assigned, regardless of their overall course average.