Linear algebra (Osnabrück 2024-2025)/Part I/Lecture 12

Invertible matrices

Definition

Let ${}K$ be a field, and let ${}M$ denote an ${}n\times n$-matrix over ${}K$. Then ${}M$ is called invertible, if there exists a matrix ${}A\in \operatorname {Mat} _{n}(K)$ such that

${}A\circ M=E_{n}=M\circ A\,$

holds.

Definition

Let ${}K$ denote a field. For an invertible matrix ${}M\in \operatorname {Mat} _{n}(K)$, the matrix ${}A\in \operatorname {Mat} _{n}(K)$ fulfilling

${}A\circ M=E_{n}=M\circ A\,,$

is called the inverse matrix of ${}M$. It is denoted by

$M^{-1}.$

The product of invertible matrices is again invertible. Due to Lemma 9.5 , the matrix describing a base change is invertible, and the matrix of the reversed base change is its inverse matrix.

Definition

For a field ${}K$ and ${}n\in \mathbb {N} _{+}$, the set of all invertible ${}n\times n$-matrices with entries in ${}K$ is called the general linear group

over ${}K$. It is denoted by ${}\operatorname {GL} _{n}\!{\left(K\right)}$.

Definition

Two square matrices ${}M,N\in \operatorname {Mat} _{n}(K)$ are called similar, if there exists an invertible matrix ${}B$ with

${}M=BNB^{-1}$.

For a linear mapping ${}\varphi \colon V\rightarrow V$, the describing matrices with respect to two bases are similar to each other, due to Corollary 11.12 .

Properties of linear mappings

Lemma

Let ${}K$ be a field, and let ${}V$ and ${}W$ be vector spaces over ${}K$ of dimensions ${}n$ and ${}m$. Let

$\varphi \colon V\longrightarrow W$

be a linear map, described by the matrix ${}M\in \operatorname {Mat} _{m\times n}(K)$

with respect to two bases. Then the following properties hold.

1. ${}\varphi$ is injective if and only if the columns of the matrix are linearly independent.
2. ${}\varphi$ is surjective if and only if the columns of the matrix form a generating system of ${}K^{m}$.
3. Let ${}m=n$. Then ${}\varphi$ is bijective if and only if the columns of the matrix form a basis of ${}K^{m}$, and this holds if and only if ${}M$ is invertible.

Proof

Let ${}{\mathfrak {v}}=v_{1},\ldots ,v_{n}$ and ${}{\mathfrak {w}}=w_{1},\ldots ,w_{m}$ denote the bases of ${}V$ and ${}W$ respectively, and let ${}s_{1},\ldots ,s_{n}$ denote the column vectors of ${}M$. (1). The mapping ${}\varphi$ has the property

${}\varphi (v_{j})=\sum _{i=1}^{m}s_{ij}w_{i}\,,$

where ${}s_{ij}$ is the ${}i$-th entry of the ${}j$-th column vector ${}s_{j}$. Therefore,

${}\varphi {\left(\sum _{j=1}^{n}a_{j}v_{j}\right)}=\sum _{j=1}^{n}a_{j}{\left(\sum _{i=1}^{m}s_{ij}w_{i}\right)}=\sum _{i=1}^{m}{\left(\sum _{j=1}^{n}a_{j}s_{ij}\right)}w_{i}\,.$

This is ${}0$ if and only if ${}\sum _{j=1}^{n}a_{j}s_{ij}=0$ for all ${}i$, and this is equivalent with

${}\sum _{j=1}^{n}a_{j}s_{j}=0\,.$

For this vector equation, there exists a nontrivial tuple ${}{\left(a_{1},\ldots ,a_{n}\right)}$, if and only if the columns are linearly dependent, and this holds if and only if the kernel of ${}\varphi$ is not trivial. Due to Lemma 11.4 , this is equivalent with ${}\varphi$ not being injective.
(2). See Exercise 12.7 .
(3). Let ${}n=m$. The first equivalence follows from (1) and (2). If ${}\varphi$ is bijective, then there exists a (linear) inverse mapping ${}\varphi ^{-1}$ with

$\varphi \circ \varphi ^{-1}=\operatorname {Id} _{W}{\text{ and }}\varphi ^{-1}\circ \varphi =\operatorname {Id} _{V}.$

Let ${}M$ denote the matrix for ${}\varphi$, and ${}N$ the matrix for ${}\varphi ^{-1}$. The matrix for the identity is the identity matrix. Because of Lemma 11.10 , we have

${}M\circ N=E_{n}=N\circ M\,$

and therefore ${}M$ is invertible. The reverse implication is proved similarly.

$\Box$

Elementary matrices

We want develop effective methods to determine the rank of a matrix, and to decide whether a matrix is invertible. For this, elementary matrices are helpful.

Definition

Let ${}K$ be a field, and let ${}M$ be an ${}m\times n$-matrix over ${}K$. Then the following manipulations on ${}M$ are called elementary row operations.

1. Transpositions of two rows.
2. Multiplication of a row with a scalar ${}s\neq 0$.
3. Addition of ${}a$ times a row to another row.

Definition

Let ${}K$ be a field. We denote by ${}B_{ij}$ the ${}n\times n$-matrix with entry ${}1$ at the position ${}(i,j)$, and entry ${}0$ everywhere else. Then the following matrices are called elementary matrices.

1. ${}V_{ij}:=E_{n}-B_{ii}-B_{jj}+B_{ij}+B_{ji}$.
2. ${}S_{k}(s):=E_{n}+(s-1)B_{kk}{\text{ for }}s\neq 0$.
3. ${}A_{ij}(a):=E_{n}+aB_{ij}{\text{ for }}i\neq j{\text{ and }}a\in K$.

In detail, these elementary matrices look as follows.

${}V_{ij}={\begin{pmatrix}1&0&\cdots &\cdots &\cdots &\cdots &0\\\vdots &\ddots &\cdots &\cdots &\cdots &\cdots &\vdots \\0&\cdots &0&\cdots &1&\cdots &0\\\vdots &\cdots &\cdots &1&\cdots &\cdots &\vdots \\0&\cdots &1&\cdots &0&\cdots &0\\\vdots &\cdots &\cdots &\cdots &\cdots &\ddots &\vdots \\0&\cdots &\cdots &\cdots &\cdots &0&1\end{pmatrix}}\,.$

${}S_{k}(s)={\begin{pmatrix}1&0&\cdots &\cdots &\cdots &\cdots &0\\\vdots &\ddots &\ddots &\vdots &\vdots &\vdots &\vdots \\0&\cdots &1&0&\cdots &\cdots &0\\0&\cdots &0&s&0&\cdots &0\\0&\cdots &\cdots &0&1&\cdots &0\\\vdots &\vdots &\vdots &\vdots &\ddots &\ddots &\vdots \\0&\cdots &\cdots &\cdots &\cdots &0&1\end{pmatrix}}\,.$

${}A_{ij}(a)={\begin{pmatrix}1&0&\cdots &\cdots &\cdots &\cdots &0\\\vdots &\ddots &\cdots &\cdots &\cdots &\cdots &\vdots \\0&\cdots &1&\cdots &a&\cdots &0\\\vdots &\cdots &\cdots &\ddots &\cdots &\cdots &\vdots \\0&\cdots &\cdots &\cdots &1&\cdots &0\\\vdots &\cdots &\cdots &\cdots &\cdots &\ddots &\vdots \\0&\cdots &\cdots &\cdots &\cdots &0&1\end{pmatrix}}\,.$

Elementary matrices are invertible, see Exercise 12.1 .

Lemma

Let ${}K$ be a field and ${}M$ a ${}n\times n$-matrix with entries in ${}K$. Then the multiplication by elementary matrices from the left with ${}M$ has the following effects.

1. ${}V_{ij}\circ M=$ exchange of the ${}i$-th and the ${}j$-th row of ${}M$.
2. ${}(S_{k}(s))\circ M=$ multiplication of the ${}k$-th row of ${}M$ by ${}s$.
3. ${}(A_{ij}(a))\circ M=$ addition of ${}a$-times the ${}j$-th row of ${}M$ to the ${}i$-th row (${}i\neq j$).

Proof

See Exercise 12.8 .

$\Box$

Elementary row operations do not change the solution space of a homogeneous linear system, as shown in Lemma 5.3 .

Theorem

Let ${}K$ be a field, and let ${}M$ denote an ${}m\times n$-matrix over ${}K$. Then there exist elementary row operations, and a (new) numbering of the columns

$j_{1},\,j_{2},\ldots ,j_{n},$

and an ${}r\leq n$ such that, in the new matrix, the columns have the form

$s_{j_{k}}={\begin{pmatrix}b_{1,j_{k}}\\\vdots \\b_{k,j_{k}}\\0\\\vdots \\0\end{pmatrix}}{\text{ with }}b_{k,j_{k}}\neq 0{\text{ for }}k\leq r$

and

$s_{j_{k}}={\begin{pmatrix}b_{1,j_{k}}\\\vdots \\b_{r,j_{k}}\\0\\\vdots \\0\end{pmatrix}}{\text{ for }}k>r.$

By further elementary row operations, and by swapping of columns, the matrix can be brought to the form

${\begin{pmatrix}d_{11}&*&\cdots &*&*&\cdots &*\\0&d_{22}&\cdots &*&*&\cdots &*\\\vdots &\vdots &\ddots &\vdots &\vdots &\ddots &\vdots \\0&0&\cdots &d_{rr}&*&\cdots &*\\0&0&\cdots &0&0&\cdots &0\\\vdots &\vdots &\ddots &\vdots &\vdots &\ddots &\vdots \\0&0&\cdots &0&0&\cdots &0\end{pmatrix}}$

with

${}d_{ii}\neq 0$.

Proof

This rests on the corresponding manipulations as in the elimination procedure, see Lecture 5.

$\Box$

Corollary

Let ${}K$ be a field, and let ${}M$ denote an invertible ${}n\times n$-matrix over ${}K$. Then there exist elementary row operations such that, after these manipulations, a matrix of the form

${\begin{pmatrix}d_{1}&\ast &\cdots &\cdots &\ast \\0&d_{2}&\ast &\cdots &\ast \\\vdots &\ddots &\ddots &\ddots &\vdots \\0&\cdots &0&d_{n-1}&\ast \\0&\cdots &\cdots &0&d_{n}\end{pmatrix}}$

with ${}d_{i}\neq 0$ arises. By further elementary row operation, one can also obtain the

identity matrix.

Proof

This rests on the manipulations of the elimination procedure, and on the fact that elementary row manipulations are achieved, due to Lemma 12.18 , by multiplications with elementary matrices from the left. In doing this, it can not happen that a zero-column or a zero-row arises, because the elementary matrices are invertible, and, in each step, invertibility is preserved. If we have an invertible upper triangular matrix, then the diagonal entries are not ${}0$ by Exercise 12.6 , and, by multiplication with a scalar, we can normalize them to ${}1$. With this, we can further achieve, in every column, that all entries above the diagonal entry are ${}0$.

$\Box$

In particular, for an invertible matrix ${}M$, there are elementary matrices ${}E_{1},\ldots ,E_{k}$ such that

$E_{k}\circ \cdots \circ E_{1}\circ M$

is the identity matrix.

Finding the inverse matrix

Method

Let ${}M$ denote a square matrix. How can we decide whether the matrix is invertible, and how can we find the inverse matrix ${}M^{-1}$?

For this we write down a table, on the left-hand side we write down the matrix ${}M$, and on the right-hand side we write down the identity matrix (of the right size). Now we apply on both sides step by step the same elementary row manipulations. The goal is to produce in the left-hand column, starting with the matrix, in the end the identity matrix. This is possible if and only if the matrix is invertible. We claim that we produce, by this method, in the right column the matrix ${}M^{-1}$ in the end. This rests on the following invariance principle. Every elementary row manipulation can be realized, according to Lemma 12.18 , as a matrix multiplication with some elementary matrix ${}E$ from the left. If in the table we have somewhere the pair

$(M_{1},M_{2}),$

after the next step (in the next line) we have

$(EM_{1},EM_{2}).$

If we multiply the inverse of the second matrix (which we do not know yet; however, we do know its existence, in case the matrix is invertible) with the first matrix, then we get

${}(EM_{1})^{-1}EM_{2}=M_{1}^{-1}E^{-1}EM_{2}=M_{1}^{-1}M_{2}\,.$

This means that this expression is not changed in each single step. In the beginning, this expression equals ${}M^{-1}E_{n}$, hence in the end, the pair ${}(E_{n},N)$ must fulfil

${}N=E_{n}^{-1}N=M^{-1}E_{n}=M^{-1}\,.$

Example

We want to find for the matrix ${}{\begin{pmatrix}1&3&1\\4&1&2\\0&1&1\end{pmatrix}}$ its inverse matrix ${}M^{-1}$, following Method 12.11 .

${}{\begin{pmatrix}1&3&1\\4&1&2\\0&1&1\end{pmatrix}}$	${}{\begin{pmatrix}1&0&0\\0&1&0\\0&0&1\end{pmatrix}}$
${}{\begin{pmatrix}1&3&1\\0&-11&-2\\0&1&1\end{pmatrix}}$	${}{\begin{pmatrix}1&0&0\\-4&1&0\\0&0&1\end{pmatrix}}$
${}{\begin{pmatrix}1&3&1\\0&1&1\\0&-11&-2\end{pmatrix}}$	${}{\begin{pmatrix}1&0&0\\0&0&1\\-4&1&0\end{pmatrix}}$
${}{\begin{pmatrix}1&3&1\\0&1&1\\0&0&9\end{pmatrix}}$	${}{\begin{pmatrix}1&0&0\\0&0&1\\-4&1&11\end{pmatrix}}$
${}{\begin{pmatrix}1&3&1\\0&1&1\\0&0&1\end{pmatrix}}$	${}{\begin{pmatrix}1&0&0\\0&0&1\\{\frac {-4}{9}}&{\frac {1}{9}}&{\frac {11}{9}}\end{pmatrix}}$
${}{\begin{pmatrix}1&0&-2\\0&1&1\\0&0&1\end{pmatrix}}$	${}{\begin{pmatrix}1&0&-3\\0&0&1\\{\frac {-4}{9}}&{\frac {1}{9}}&{\frac {11}{9}}\end{pmatrix}}$
${}{\begin{pmatrix}1&0&0\\0&1&0\\0&0&1\end{pmatrix}}$	${}{\begin{pmatrix}{\frac {1}{9}}&{\frac {2}{9}}&{\frac {-5}{9}}\\{\frac {4}{9}}&{\frac {-1}{9}}&{\frac {-2}{9}}\\{\frac {-4}{9}}&{\frac {1}{9}}&{\frac {11}{9}}\end{pmatrix}}$

Rank of matrices

Definition

Let ${}K$ be a field, and let ${}M$ denote an ${}m\times n$-matrix over ${}K$. Then the dimension of the linear subspace of ${}K^{m}$, generated by the columns, is called the column rank of the matrix, written

$\operatorname {rk} \,M.$

Lemma

Let ${}K$ denote a field, and let ${}V$ and ${}W$ denote ${}K$-vector spaces of dimensions ${}n$ and ${}m$. Let

$\varphi \colon V\longrightarrow W$

be a linear mapping, which is described by the matrix ${}M\in \operatorname {Mat} _{m\times n}(K)$, with respect to bases of the spaces. Then

${}\operatorname {rk} \,\varphi =\operatorname {rk} \,M\,$

holds.

Proof

See Exercise 12.30 .

$\Box$

To formulate the next statement, we introduce row rank of an ${}m\times n$-matrix to be the dimension of the linear subspace of ${}K^{n}$ generated by the rows.

Lemma

Let ${}K$ be a field, and let ${}M$ denote an ${}m\times n$-matrix over ${}K$. Then the column rank coincides with the row rank. The rank equals the number ${}r$ from

Theorem 12.9 .

Proof

In an elementary row manipulation, the linear subspace generated by the rows is not changed, therefore the row rank is not changed. The row rank of ${}M$ equals the row rank of the matrix in echelon form obtained in Theorem 12.9 . This matrix has row rank ${}r$, since the first ${}r$ rows are linearly independent, and, apart from this, this, there are only zero rows. It has also column rank ${}r$, since the ${}r$ columns, where there is a new step, are linearly independent, and the other columns are linear combinations of these ${}r$ columns. By Exercise 12.20 , the column rank is preserved by elementary row manipulations.

$\Box$

Both ranks coincide, so we only talk about the rank of a matrix.

Corollary

Let ${}K$ be a field, and let ${}M$ denote an ${}n\times n$-matrix

over ${}K$. Then the following statements are equivalent.

1. ${}M$ is invertible.
2. The rank of ${}M$ is ${}n$.
3. The rows of ${}M$ are linearly independent.
4. The columns of ${}M$ are linearly independent.

Proof

This follows from Lemma 12.5 and from Lemma 12.15 .

$\Box$

<< | Linear algebra (Osnabrück 2024-2025)/Part I | >> PDF-version of this lecture Exercise sheet for this lecture (PDF)