An Introduction to Matrix Computations in Python: Just the basics

Recall that the action of a linear transformation from \(\mathbb{R}^n \rightarrow \mathbb{R}^n\) can be completely determined by a set of \(n\) basis vectors. These basis vectors create a coordinate grid. For example, in \(\mathbb{R}^2\) any set of two non-parallel vectors form a basis and a corresponding coordinate grid of parallelograms. The coordinate grid determines a basic area, the area of the parallelogram between the two basis vectors. Since the transformation is linear, the transformed basis vectors either create a transformed parallelogram, or they collapse to a line or the origin. In particular, the areas of the parallelograms are scaled by the transformation and, since we started with a basis, all areas in the plane are scaled by that same amount. How the area is scaled by a transformation on \(\mathbb{R}^2\) is the value of the determinant.

In \(\mathbb{R}^3\) the coordinate grid determines a basic volume. Here how the volume is scaled by a transformation on \(\mathbb{R}^3\) is the value of the determinant. Finally in \(\mathbb{R}^n\) the coordinate grid determines a basic hypervolume and the value by which the volume is scaled by a transformation on \(\mathbb{R}^n\) is the value of the determinant.

The determinant can be positive or negative depending on whether or not the orientation of the basis vectors changes or not. For example, a reflection in \(\mathbb{R}^2\) will switch the orientation and change the sign of the determinant.

We can compute the determinant of square matrices both in NumPy and in SymPy as follows.

The determinant gives us a quick way to tell whether or not a system of linear equations has a unique solution or not. Recall that every system of linear equations corresponds to a matrix-vector equation.

If the corresponding matrix transformation collapses down a dimension, more than one vector gets mapped to the same place, preventing an inverse transformation (since one output would have to map back to several inputs). In this case, the determinant of the transformation is 0. This corresponds exactly to linear systems of equations with either infinitely many solutions or no solution.

When the determinant is non-zero, however, the matrix transformation is invertible. This corresponds exactly to when a system of linear equations has a unique solution, since you can solve for it using \(A^{-1}.\)

In Section 2.1, we learned how to solve systems of linear equations using rref() in SymPy. More efficient algorithms

⁷

e.g. using \(LU\)-decomposition of our matrix

can be implemented when we know we have a unique solution. NumPy’s linalg.solve(A,b) and SymPy’s A.solve(b) uses such algorithms to solve \(A\vec{x}=\vec{b}\text{.}\)

In practice, we often check the determinant to determine whether or not an algorithm such as linalg.solve() will work before solving the system numerically.

⁸

Many stackoverflow questions have been written asking about the “singular” error when attempting to solve a large system numerically. The response always involves someone in the know checking the determinant.

Recall in Example 2.18 we explored solutions to a system of equations where the corresponding matrix was ill-conditioned. How does ill-conditioning relate to the determinant and the matrix transformation?

If we visualize the linear system of equations as a matrix transformation of \(\mathbb{R}^2\) we can graphically see the problem.