Stanford EE364A Convex Optimization I Stephen Boyd I 2023 I Lecture 13

()

Linear Algebra

Linear algebra is the foundation of the course, and everyone should be familiar with its concepts.
Direct methods for solving sets of equations, such as Gaussian elimination, factor the coefficient matrix into easily invertible matrices.
Sparse matrix factorization deals with efficient methods for factorizing and solving sparse matrices.
The Cholesky factorization is used to solve positive definite systems of equations efficiently.
Sparse Cholesky factorization is used to solve positive definite systems of equations efficiently.
Permuting the matrix before factorization can significantly affect the sparsity of the resulting L factor.
Finding a good permutation is crucial for achieving efficient factorization and solving times.
Cholesky factorization fails if the matrix is not positive definite.
LDL transpose factorization is a method for solving non-singular symmetric matrices.
The sure complement method is used to solve matrix equations efficiently in certain cases.
Sparse matrices with a banded structure and dense rows and columns can be solved efficiently using block elimination.
The sparsity pattern of a matrix can indicate the complexity of solving the associated system of equations.
The Matrix Inversion Lemma provides a way to efficiently invert a matrix that is a perturbation of a diagonal matrix.
Un-elimination can be an effective technique for solving certain types of linear systems.
Sparse solvers can efficiently solve large systems of linear equations, making many traditional methods obsolete.

Unconstrained Minimization

Unconstrained minimization involves finding the minimum value of a smooth, twice continuously differentiable function without any constraints.
Iterative methods are used to solve unconstrained minimization problems since analytical solutions are generally not available.
Stopping criteria are used to determine when the iterative process should stop.
Descent methods are widely used for convex optimization and involve finding a descent direction and choosing a step length.
Gradient descent is the most intuitive iterative method for unconstrained minimization.
Gradient descent exhibits linear convergence, meaning that each iteration reduces the error by a constant factor.