Eigenvalues and eigenvectors

The intent of this article is to get physics and engineering students interested in tensors and the eigenvalue problem in a way that does not require them to remember mathematical details. Such intuitive insights are less likely to be forgotten in the years between the informal introduction, and when the topic is rigorously encountered.

Fig. 1(ab): Alignment and misalignment of the springs with the coordinate system.

Here we apply concepts introduced in a first-year physics course to the the spring mechanism shown in figures 1(ab). Our solution will not be elegant, but instead designed to be understood by students at the lowest possible mathematical level. For that reason, we introduce the matrix, not as an array of real numbers, but as a handy way to write a set of linear equations. This lesson does require an understanding of the force on a particle when the potential energy can be expressed as a function of $x$ and $y$ :

$1\mid \quad {\begin{aligned}F_{x}&=-{\frac {\partial \Phi (x,y)}{\partial x}}=-\alpha x\\[.9ex]F_{y}&=-{\frac {\partial \Phi (x,y)}{\partial y}}=-\beta y\end{aligned}},$

where contours of potential energy, $\Phi (x,y)={\tfrac {1}{2}}(\alpha x^{2}+\beta y^{2}),$ are shown at the top of figure 2, where the blue arrows indicate the direction of the force. Equation 1 is a generalization of Hooke's law:

$2\mid \quad F=-k\,x,$ where $k$ is the spring constant. In later calculations, it will emerge that the sign of $k$ is not yet known. If $k<0,$ the equilibrium point is unstable. If both springs are unstable, the contours are ellipses, but now the contours resemble the "top of a hill"; a ball placed there is likely to roll down. Another possibility is the saddle shape shown at the bottom of figure 2.

Two items worth noting:

The images of spring mechanisms in figures1(ab) are misleading because we assume that no change in the springs' orientation will occur as the object is displaced. One remedy is to assume very small displacements (or very long springs.) Or, one could assume that the springs are attached to the outer box on wheels that maintain the each spring's original orientation. The reader might have noticed that the images show small blocks with wheels at the ends of the spring opposite to the connections to the red spherical object. Exploration second order complications caused by large amplitude motion with springs of finite length would make for a good student project.^[1]
The focus of this lesson is exclusively on symmetric tensors. It can be shown that representing a linear relationship between force and displacement with an asymmetric tensor is incompatible with a force that is the gradient of a scalar potential.

For instructors ... and students

The ideas presented here can have educational value in four different ways:

Advanced students who know linear algebra will this algebra too easyin and too tedious. But they might benefit from verifying these calculations using a Computer Algebra System.^[2] Also, advanced students who skim this article can see an example how a problem can become far simpler after a few theorems are established. Much of this article is devoted tedious algebra verifying that that eigenvalues are real and eigenvectors are orthogonal for any 2x2 symmetric matrix.
The tensor is fundamental to advanced physics, and there value in an early exposure to an advanced concept. This early exposure must be free of details that a student is apt to forget. Here we begin with a Hooke's law, which is a simple linear relationship between two vectors (force and displacement.) Generalization of this idea to an object constrained by two different springs introduces a working definition a rank-2 tensor as a linear relationship between two vectors.
Physics is taught using visual representations:

Fig. 3: Red shade used to indicate "negative" spring constants.
The an arrow is used to define the vector as an entity that magnitude and direction. Vector addition is introduced as subsequent "steps" on a two-dimensional plane, and dividing the difference between two displacements defines (average) velocity. With physics students, we take the ladder up the steps of complexity slowly and gently. The most straightforward way to visually depict the spring system of qwer figure with the sides aligned with the springs, using the side's length to represent the spring constant.^[3] But this shape has a major flaw: The rectangle's diagonals will distract the eye and have no physical meaning. In the analysis of the two dimensional strain tensor, the rhombus , uses diagonal lengths to represent eigenvalues. Both methods are insufficient if we allow for negative eigenvalues (i.e., "spring constants".) Here, one might use the ellipse and hyperbola ) (, with red shading to distinguish between positive and negative eigenvalues, as shown in figure 3.
Rotational transformations. Perhaps the most powerful insight a novice to linear algebra can gain from this discussion involves the rotational transformations of vectors and tensors. The following exercise is virtually free of any mathematics: A rotation of a vector by 180^o will convert a vector to its additive inverse: $\,{\underline {v}}'=-{\underline {v}}.$ But neither of the three shapes used to depict a symmetric tensor exhibit this behavior under a 180^o rotation. Instead the rotation restores the original shape. The same is true of the actual mass/spring system. Each spring is moved to the other side of the object, but this has no impact on the relationship between force and displacement. This topic could inspire a number of student investigations.^[4] One curious insight involves the fact that while two or three directional vector has direction and magnitude, a two dimensional tensor has two magnitudes and one property that might be called "orientation". Here we will see that the magnitudes involve eigenvalues, and the orientations involve two orthogonal lines that meet at the origin. The orientation of these lines is associated with eigenvectors.
Matching the number of parameters to the information conveyed by vectors and tensors. If a vector is an entity with direction and magnitude, then the direction in two dimensions must be conveyed with one parameter. This is because a vector is contains two components: Magnitude requires one parameter, and angle above the x-axis requires one variable (in 2 dimensions.) The ellipse and other shapes used to describe a symmetric 2x2 matrix requires three parameters (major, minor radius, and orientation.) This matches the number of parameters required to define a 2x2 symmetric matrix. An antisymmetric 2x2 matrix has only one free parameter. It can be shown that the force field of a "spring" described by an antisymmetric tensor in two dimensions obeys ${\vec {F}}=k{\underline {r}}\times {\hat {z}},$ a force law that requires only one parameter to define. It is also a force that cannot be defined as the gradient of potential energy.
In three dimensions, direction requires two variables (see Wikipedia:Geographic_coordinate_system#Latitude_and_longitude). A good question to ask introductory physics students is how many parameters are required to orient the xyz axes of a cartesian coordinate system. The answer is three parameters, as follows:

Begin with the first direction, which like a direction in the sky, is defined by two parameters. Draw a plane with a normal aligned with that direction and center a unit circle around the arrow pointing in the direction of the first eigenvector.
The location (angle) on that circle requires a third parameter, and that defines the orientation of the second eigenvector. This brings the number of real numbers to define the orientation of the eigenvectors to three.
The third eigenvector can be deduced without introducing any more real numbers, since it must be perpendicular to the first to directions.

Given that the six parameters required to define a 3x3 symmetric matrix have been exhausted, it should be no surprise that the eigenvectors must be orthogonal (assuming that the eigenvalues are not degenerate.)^[5] For example, a symmetric matrix offer 6 free parameters. If three parameters define the eigenvalues, that leaves us with three parameters to define the eigenvectors. Since the magnitude of an eigenvector is arbitrary, we need two parameters to define the direction of a 3D eigenvector. This would require a total of 6 parameters to define the direction of all three eigenvectors, a feat that is not possible for a matrix that is defined with only six parameters if it also obligated to provide 3 eigenvalues. It must be understood that this is non-rigorous (intuitive) logic. Nevertheless, intuition can be useful because facts often begin as guesses.

For students:

This section assumes skills often associated with a first-year calculus based physics course.^[6] All you need to know about linear algebra is that matrices can be used as shorthand for writing systems of linear equations. In other words, you need to recognize that: These three equations convey the same information: $3\mid \quad -\underbrace {\begin{bmatrix}F_{x}\\F_{y}\end{bmatrix}} _{\underline {F}}=\underbrace {\begin{bmatrix}\alpha &\gamma \\\gamma &\beta \end{bmatrix}} _{\underline {\underline {\kappa }}}\,\underbrace {\begin{bmatrix}x\\y\end{bmatrix}} _{\underline {r}}\iff {\begin{aligned}-F_{x}&=\alpha x+\gamma y\\-F_{y}&=\gamma x+\beta y\end{aligned}}\iff -{\underline {F}}={\underline {\underline {\kappa }}}\cdot {\underline {r}}.$ The double arrow $(\iff )$ indicates that these are three equivalent ways to say the same thing.

Here we have introduced:

${\underline {\underline {\kappa }}}$ = "kappa", defined using the three parameters $(\alpha ,\beta ,\gamma )$ is analogous to the spring constant, but valid for two perpendicular springs.
${\underline {r}}=x{\hat {x}}+y{\hat {y}}$ is the displacement vector, or the location of the object with respect to the equilibrium point (where the net force by the springs vanishes.)
${\underline {F}}=F_{x}{\hat {x}}+F_{y}{\hat {y}}$ is the force,
${\hat {x}}\,{\text{ and }}\,{\hat {y}}$ are two unit vectors.
$k$ is "spring constant", defined for one dimensional motion of an object with one spring attached by the formula ${\underline {F}}=-k{\underline {r}}.$

If we view k and ${\underline {\underline {\kappa }}}$ as constants, equation (3) is a set of two homogeneous linear equations in the two variables $x$ and $y.$ It is linear because no variable is raised to a power greater than 1. Here, "linear" excludes terms like $xy$ (despite the fact that both terms in $x^{1}y^{1}$ are raised to the first power.) The term "linear" probably refers to the fact that graphs of linear functions are straight lines in two dimensions. In three dimensions, linear functions can also appear as flat planes (such as $z=0,$ or $x+y+z=1.)$

Warning: If you don't call $\kappa$ "kappa", you will confuse yourself when you try to do algebra with $\,{\underline {\underline {\kappa }}}\,$ and $k.$

Our notation and its flaws

Greek letters are faster to write when doing calculations by hand: $(\alpha ,\beta ,\gamma ,\kappa )\leftrightarrow$ ("alpha", "beta", "gamma", "kappa".) The Greek letters also allow us to separate the use of the first Greek letter $\alpha$ in ${\underline {\underline {\kappa }}}$ and the Latin script's use in the quadratic equation, $ak^{2}+bk+c=0.$
A single underline denote a vector and a double underline to denote tensors (matrices). A major flaw in this convention would emerge if we ever encountered a reason to have three underlines (denoting a rank-3 tensor.)^[7]
In a major violation of what is right and sensible, we waste the two valuable symbols $y,z$ we follow the convention used in introductory physics books that position is denoted by (x,y,z), instead of (x¹, x², x³)^[8]

Depending on the context, entities need to be expressed in a variety of ways. For example, among the ways to express position, we have: $4\mid \quad (x,y)\iff {\underline {r}}\iff {\begin{bmatrix}x\\y\end{bmatrix}}\iff x{\hat {x}}+y{\hat {y}}.$

Eigenvalue and eigenvector defined

Consider the following equations, where ${\underline {\underline {\kappa }}}$ is the "matrix", $k$ is "eigenvalue", and $(x,y)$ is the "eigenvector". $5\mid \quad {\begin{bmatrix}\alpha &\gamma \\\gamma &\beta \end{bmatrix}}{\begin{bmatrix}x\\y\end{bmatrix}}=k{\begin{bmatrix}x\\y\end{bmatrix}}$

The "eigenvalue problem" is to find numerical values for the eigenvalue $k$ and eigenvector, $(x,y)$ . The eigenvector remains an eigenvector if it multiplied by any nonzero constant.^[9] It is typical for an nxn matrix to have n eigenvalues and n eigenvectors, though exceptions exist. To understand these definitions, you need to look at two examples:

Example 1: Springs aligned to coordinate system (gamma=0)

The matrix ${\underline {\underline {\kappa }}}$ is diagonal if $\kappa _{ij}=0$ if $i\neq j,$ or equivalently, if $\gamma =0.$ The following equation establishes that ${\underline {\underline {\kappa }}}$ is diagonal if both springs are aligned parallel to the $x$ and $y$ axes:

$6\mid \quad {\begin{bmatrix}\alpha &0\\0&\beta \end{bmatrix}}{\begin{bmatrix}x\\y\end{bmatrix}}=k{\begin{bmatrix}x\\y\end{bmatrix}}\iff {\begin{aligned}\alpha x&=kx\\\beta y&=ky\end{aligned}}$

Let us assume that $\alpha \neq \beta$ . The last version of this statement establishes that both equations cannot be true for all values of $x$ and $y.$ But both equations can be true if either $x$ or $y$ vanishes. This condition defines the eigenvector and allows $k$ to be called the eigenvalue. If the motion is in the x direction, then $k=\alpha$ is the eigenvalue, and the eigenvector is any non-zero vector with a y-component that vanishes. An obvious choice is $(1,0).$ Likewise, the other eigenvalue/eigenvector pair is $k=\beta$ and $(0,1)$ for the eigenvector.

Expressing the force using normalized eigenvectors

This is difficult to explain in Example 1 because our eigenvectors are aligned with our unit vectors $({\hat {x}},{\hat {y}}.)$ In other examples we will need a parallel set of unit vectors. $7\mid \quad {\begin{aligned}k&=\alpha \;\Rightarrow \;{\hat {u}}_{X}={\hat {x}}\;\Rightarrow \;F_{X}&=-\alpha X\\[.9ex]k&=\beta \;\Rightarrow \;{\hat {u}}_{X}={\hat {x}}\;\Rightarrow \;F_{Y}&=-\beta Y\\\end{aligned}}$

Here the $(X,Y)$ coordinates are aligned with the $(x,y)$ coordinates.

Example 2: Springs not aligned to coordinates (gamma ≠ 0)

It is not always desirable to orient the spring mechanism with the coordinate system. It can be shown that the symmetric tensor, ${\underline {\underline {\kappa }}}$ can model any spring mechanism with springs not aligned to the coordinate system. While there are procedures for matching a tensor to a given spring mechanism, it is instructive to start with a simple matrix that can be solved intuitively: $8\mid \quad {\begin{bmatrix}1&1\\1&1\end{bmatrix}}{\begin{bmatrix}x\\y\end{bmatrix}}=k{\begin{bmatrix}x\\y\end{bmatrix}}\;\iff \;{\begin{aligned}x+y&=kx\\[.9ex]x+y&=ky\end{aligned}}$

Making two equations equivalent $x+y=kx\iff x+y=ky$
equation	Eigenvalue	Eigenvector
$x+y=0$	$k=0$	${\frac {{\hat {x}}-{\hat {y}}}{\sqrt {2}}}$
$x=y$	$k=2$	${\frac {{\hat {x}}+{\hat {y}}}{\sqrt {2}}}$

Fig. 4:

{\underline {\underline {\kappa }}}={\begin{bmatrix}1&1\\1&1\end{bmatrix}}

Original coordinates are

(x,y),

while

(X,Y)

are aligned with the springs

Equation 8 states the same pair of equations using two different formats. We begin our discussion with the second format, which consists of writing two equations: $x+y=kx$ and $x+y=ky.$ For reasons that will be explained in the next example, we seek values of $k$ that cause both equations at G to be be equivalent.

Case 1: An obvious case occurs if $k=0.$ This implies that all displacements must obey, $x+y=0.$ There are no restrictions on the magnitude of $x$ or $y$ , but it is convenient to focus a numerical example. So we take $x=1,$ which forces, $y=-1.$ This gives us a displacement equal to ${\hat {x}}-{\hat {y}}.$ Although it is not strictly necessary, we shall divide this vector by ${\sqrt {2}}$ in order to create a vector with magnitude equal to $1.$ This gives us $({\hat {x}}-{\hat {y}})/{\sqrt {2}}$ as our unit vector for all motion for which the the spring constant is effectively $k=0,$ as shown in the table.

Case 2: A slightly less obvious way to render both equations in equation&nsp;8 to be true is to set $k=2,$ which the reader can verify will stipulate that $k=2.$ Setting $x=1,$ leads to $y=1,$ which leads to ${\hat {x}}+{\hat {y}}$ as our eigenvector. For convenience we also divide this eigenvector by ${\sqrt {2}}$ to obtain $({\hat {x}}+{\hat {y}})/{\sqrt {2}}$ , which is also a unit vector because it also has magnitude equal to 1.

Figure 4^[10] shows the lines of equipotential energy for this example. The original xy coordinate system is shown at the center of the diagram. But for the purposes of understanding and describing the motion of the mass in the spring mechanism, the two unit eigenvectors $k=0$ and $k=2$ are more convenient. They are:

$9\mid \quad {\begin{aligned}k=0:\quad {\hat {u}}_{0}&=\;{\frac {{\hat {x}}-{\hat {y}}}{\sqrt {2}}}\;\equiv \;{\widehat {X}}\quad \scriptstyle {\text{points down and to the right}}\\[.9ex]k=2:\quad {\hat {u}}_{2}&=\;{\frac {{\hat {x}}+{\hat {y}}}{\sqrt {2}}}\;\equiv \;{\widehat {Y}}\quad \scriptstyle {\text{points up and to the right}}\end{aligned}}$

Here we have introduced capital letters $(X,Y)$ to denote the spatial variables in this rotated coordinate system. First-year physics students model two-dimensional free fall under a uniform gravitational field as a pair of equations (one for horizontal motion and the other for vertical.) Eigenvectors permit us to do the same with a new coordinate system that uses capital letters:

$10\mid \quad m{\frac {d^{2}X}{dt^{2}}}=0\quad \quad \quad m{\frac {d^{2}Y}{dt^{2}}}=-2Y$
equations to be the same is for $k=2,$ which leads both equations to become equivalent to the single equation, $x=y.$ This constraining forces the particle to lie on the red dotted line passing through the origin in figure 4. A good question to ask students is what the spring constant is for motion along motion along that line.^[11]Earlier it was claimed that any stable 2-spring mechanism is associated with a plot of potential energy in which the contours are ellipses (or circles). That is only true if we permit the major axis of an ellipse to be infinite. As shown in figure^[12](?), the object can move freely (without external forces) along the line $y=-x.$

Fig. 5: Where 2 lines meet

Example 3: General case

Here we solve the problem symbolically, using $(\alpha ,\beta ,\gamma )$ to define any symmetric 2x2 matrix.

$11\mid \quad {\begin{bmatrix}\alpha &\gamma \\\gamma &\beta \end{bmatrix}}{\begin{bmatrix}x\\y\end{bmatrix}}=k{\begin{bmatrix}x\\y\end{bmatrix}}\;\iff \;{\begin{aligned}\alpha x+\gamma y&=kx\\\gamma x+\beta y&=ky\end{aligned}}$ Previously we were able to look at the two equations and "guess" an appropriate value of $k,$ but in the case of equation 9, it is not so obvious what to do. Equation 9 is homogeneous in the two variables x and y, which means that every term contains x or y to the first power (no constants are present). The implies that both lines pass through the origin. Fig. 5 reminds that two lines can meet at 0, 1 or an infinity of places (in the latter case they overlap.).

Write both parts of equation 1 as an expression of the line's slope, and demand that the two slopes be equal: $12\mid \quad {\begin{aligned}y&={\frac {k-\alpha }{\gamma }}\,x\\[0.9ex]y&={\frac {\gamma }{k-\beta }}\,x\end{aligned}}\Longrightarrow {\begin{aligned}{\frac {k-\alpha }{\gamma }}={\frac {\gamma }{k-\beta }}\end{aligned}}$

Since this is the well-known quadratic equation, we can solve:

$13\mid \quad k_{\pm }={\frac {\alpha +\beta \pm {\sqrt {(\alpha -\beta )^{2}+4\gamma ^{2}}}}{2}}$

This always yields two real solutions, unless there is only one real solution (which happens when $\alpha +\beta =2\gamma .)$ We find the eigenvectors using the same procedure as before, this time labeling the unnormalized vectors with plus or minus signs, ${\underline {v}}_{\pm },$ and normalizing them to ${\underline {u}}_{\pm }=v_{\pm }/||v_{\pm }||.$ Earlier, we found the unnormalized eigenvector by setting its x component equal to unity (i.e., 1.) Here we do it both ways by also setting the y component equal to unity.^[13]

$14\mid \quad {\begin{aligned}{\underline {v}}_{\pm }&={\hat {x}}+{\frac {k_{\pm }-\alpha }{\gamma }}{\hat {y}}\;\Rightarrow \;||{\underline {v}}_{\pm }||={\sqrt {1+\left({\frac {k_{\pm }-\alpha }{\gamma }}\right)^{2}}}\;\Rightarrow \quad {\hat {u}}_{\pm }={\frac {{\underline {v}}_{\pm }}{||{\underline {v}}_{\pm }||}}\\{\underline {w}}_{\pm }&={\frac {k_{\pm }-\beta }{\gamma }}{\hat {x}}+{\hat {y}}\;\Rightarrow \;||{\underline {w}}_{\pm }||={\sqrt {\left({\frac {k_{\pm }-\beta }{\gamma }}\right)^{2}+1}}\;\Rightarrow \quad {\hat {u}}_{\pm }^{\prime }={\frac {{\underline {w}}_{\pm }}{||{\underline {w}}_{\pm }||}}?\end{aligned}}$ The question mark indicates that it is not known whether the two calculations yield the same unit vectors, since a unit vector is only uniquely defined up to a minus sign. In other words, ${\hat {x}}$ and $-{\hat {x}}$ are different unit vectors suitable for the x-axis.