(2024/2025) Numerical Linear Algebra

These notes are an unofficial resource and shouldn’t replace the course material or any other book on numerical linear algebra. It is not made for commercial purposes. I’ve made the following notes to help me improve my knowledge and maybe it can be helpful for everyone.

As I have highlighted, a student should choose the teacher’s material or a book on the topic. These notes can only be a helpful material.

The notes are taken from the books required for the course:

• Course slides.

You can view/download the PDF here. In the notes folder, you can also see the source code.

In the CHANGELOG file you can see the changes made to each version of the PDF file. The versioning can be helpful if you want to understand if there are any new features/fixes in the file.

For any issue, use the appropriate section.

Course Syllabus

According to the official course syllabus:

Lectures will cover the following topics:

• Iterative methods for large linear systems:
  • Sparse matrix storage formats;
  • Krylov subspace iterative methods (CG, GMRES, BiCG, BiCGstab, …);
  • HPC programming and implementation of iterative solvers;
  • Domain decomposition preconditioners: non-overlapping and overlapping techniques;
  • (Algebraic) multigrid/multilevel methods.
• Direct methods for sparse linear systems:
  • Graph reordering and fill-in;
  • Graph partitioning and parallelization;
  • The multifrontal method.
• Numerical algorithms for Machine Learning:
  • Least square approximations;
  • Numerical techniques for QR factorization;
  • (L)-BFGS (hints).
• Numerical approximation of eigenvalues:
  • Lanczos methods;
  • Numerical algorithms based on approximate factorizations.

Computer Labs. The Computer Lab sessions are based the parallel software library LIS (Library of Iterative Solvers for linear systems). A quick overview of HPC linear algebra libraries such as Eigen, MUMPS, Lapack will also be covered.