Eigenvalues and eigenvectors

The intent of this article is to get physics and engineering students interested in tensors and the eigenvalue problem in a way that does not require them to remember mathematical details. Such intuitive insights are less likely to be forgotten in the years between the informal introduction, and when the topic is rigorously encountered.

Fig. 1(ab): Alignment and misalignment of the springs with the coordinate system.
Fig. 2(ab): Contour plots of constant potential energy for (top) two positive spring constants, and (bottom) one spring constant is negative.

Here we apply concepts introduced in a first-year physics course to the the spring mechanism shown in figures 1(ab). Our solution will not be elegant, but instead designed to be understood by students at the lowest possible mathematical level. For that reason, we introduce the matrix, not as an array of real numbers, but as a handy way to write a set of linear equations. This lesson does require an understanding of the force on a particle when the potential energy can be expressed as a function of and :

where contours of potential energy, are shown at the top of figure 2, where the blue arrows indicate the direction of the force. Equation 1 is a generalization of Hooke's law:

where is the spring constant. In later calculations, it will emerge that the sign of is not yet known. If the equilibrium point is unstable. If both springs are unstable, the contours are ellipses, but now the contours resemble the "top of a hill"; a ball placed there is likely to roll down. Another possibility is the saddle shape shown at the bottom of figure 2.

Two items worth noting:

  1. The images of spring mechanisms in figures1(ab) are misleading because we assume that no change in the springs' orientation will occur as the object is displaced. One remedy is to assume very small displacements (or very long springs.) Or, one could assume that the springs are attached to the outer box on wheels that maintain the each spring's original orientation. The reader might have noticed that the images show small blocks with wheels at the ends of the spring opposite to the connections to the red spherical object. Exploration second order complications caused by large amplitude motion with springs of finite length would make for a good student project.[1]
  2. The focus of this lesson is exclusively on symmetric tensors. It can be shown that representing a linear relationship between force and displacement with an asymmetric tensor is incompatible with a force that is the gradient of a scalar potential.

For instructors ... and students

edit

The ideas presented here can have educational value in four different ways:

  1. Advanced students who know linear algebra will this algebra too easyin and too tedious. But they might benefit from verifying these calculations using a Computer Algebra System.[2] Also, advanced students who skim this article can see an example how a problem can become far simpler after a few theorems are established. Much of this article is devoted tedious algebra verifying that that eigenvalues are real and eigenvectors are orthogonal for any 2x2 symmetric matrix.
  2. The tensor is fundamental to advanced physics, and there value in an early exposure to an advanced concept. This early exposure must be free of details that a student is apt to forget. Here we begin with a Hooke's law, which is a simple linear relationship between two vectors (force and displacement.) Generalization of this idea to an object constrained by two different springs introduces a working definition a rank-2 tensor as a linear relationship between two vectors.
  3. Physics is taught using visual representations:
     
    Fig. 3: Red shade used to indicate "negative" spring constants.
    The an arrow is used to define the vector as an entity that magnitude and direction. Vector addition is introduced as subsequent "steps" on a two-dimensional plane, and dividing the difference between two displacements defines (average) velocity. With physics students, we take the ladder up the steps of complexity slowly and gently. The most straightforward way to visually depict the spring system of qwer figure   with the sides aligned with the springs, using the side's length to represent the spring constant.[3] But this shape has a major flaw: The rectangle's diagonals will distract the eye and have no physical meaning. In the analysis of the two dimensional strain tensor, the rhombus  , uses diagonal lengths to represent eigenvalues. Both methods are insufficient if we allow for negative eigenvalues (i.e., "spring constants".) Here, one might use the ellipse   and hyperbola ) (, with red shading to distinguish between positive and negative eigenvalues, as shown in figure 3.
  4. Rotational transformations. Perhaps the most powerful insight a novice to linear algebra can gain from this discussion involves the rotational transformations of vectors and tensors. The following exercise is virtually free of any mathematics: A rotation of a vector by 180o will convert a vector to its additive inverse:   But neither of the three shapes used to depict a symmetric tensor exhibit this behavior under a 180o rotation. Instead the rotation restores the original shape. The same is true of the actual mass/spring system. Each spring is moved to the other side of the object, but this has no impact on the relationship between force and displacement. This topic could inspire a number of student investigations.[4] One curious insight involves the fact that while two or three directional vector has direction and magnitude, a two dimensional tensor has two magnitudes and one property that might be called "orientation". Here we will see that the magnitudes involve eigenvalues, and the orientations involve two orthogonal lines that meet at the origin. The orientation of these lines is associated with eigenvectors.
  5. Matching the number of parameters to the information conveyed by vectors and tensors. If a vector is an entity with direction and magnitude, then the direction in two dimensions must be conveyed with one parameter. This is because a vector is contains two components: Magnitude requires one parameter, and angle above the x-axis requires one variable (in 2 dimensions.) The ellipse and other shapes used to describe a symmetric 2x2 matrix requires three parameters (major, minor radius, and orientation.) This matches the number of parameters required to define a 2x2 symmetric matrix. An antisymmetric 2x2 matrix has only one free parameter. It can be shown that the force field of a "spring" described by an antisymmetric tensor in two dimensions obeys   a force law that requires only one parameter to define. It is also a force that cannot be defined as the gradient of potential energy.
         In three dimensions, direction requires two variables (see Wikipedia:Geographic_coordinate_system#Latitude_and_longitude). A good question to ask introductory physics students is how many parameters are required to orient the xyz axes of a cartesian coordinate system. The answer is three parameters, as follows:
  1. Begin with the first direction, which like a direction in the sky, is defined by two parameters. Draw a plane with a normal aligned with that direction and center a unit circle around the arrow pointing in the direction of the first eigenvector.
  2. The location (angle) on that circle requires a third parameter, and that defines the orientation of the second eigenvector. This brings the number of real numbers to define the orientation of the eigenvectors to three.
  3. The third eigenvector can be deduced without introducing any more real numbers, since it must be perpendicular to the first to directions.

Given that the six parameters required to define a 3x3 symmetric matrix have been exhausted, it should be no surprise that the eigenvectors must be orthogonal (assuming that the eigenvalues are not degenerate.)[5] For example, a symmetric matrix offer 6 free parameters. If three parameters define the eigenvalues, that leaves us with three parameters to define the eigenvectors. Since the magnitude of an eigenvector is arbitrary, we need two parameters to define the direction of a 3D eigenvector. This would require a total of 6 parameters to define the direction of all three eigenvectors, a feat that is not possible for a matrix that is defined with only six parameters if it also obligated to provide 3 eigenvalues. It must be understood that this is non-rigorous (intuitive) logic. Nevertheless, intuition can be useful because facts often begin as guesses.

For students:

edit

This section assumes skills often associated with a first-year calculus based physics course.[6]  All you need to know about linear algebra is that matrices can be used as shorthand for writing systems of linear equations. In other words, you need to recognize that: These three equations convey the same information:  The double arrow   indicates that these are three equivalent ways to say the same thing.

Here we have introduced:

  •   = "kappa", defined using the three parameters   is analogous to the spring constant, but valid for two perpendicular springs.
  •   is the displacement vector, or the location of the object with respect to the equilibrium point (where the net force by the springs vanishes.)
  •   is the force,
  •   are two unit vectors.
  •   is "spring constant", defined for one dimensional motion of an object with one spring attached by the formula  

If we view k and   as constants, equation (3) is a set of two homogeneous linear equations in the two variables   and   It is linear because no variable is raised to a power greater than 1. Here, "linear" excludes terms like   (despite the fact that both terms in  are raised to the first power.) The term "linear" probably refers to the fact that graphs of linear functions are straight lines in two dimensions. In three dimensions, linear functions can also appear as flat planes (such as   or  


  Warning: If you don't call   "kappa", you will confuse yourself when you try to do algebra with   and  

Our notation and its flaws

edit
  • Greek letters are faster to write when doing calculations by hand:   ("alpha", "beta", "gamma", "kappa".) The Greek letters also allow us to separate the use of the first Greek letter   in   and the Latin script's use in the quadratic equation,  
  • A single underline denote a vector and a double underline to denote tensors (matrices). A major flaw in this convention would emerge if we ever encountered a reason to have three underlines (denoting a rank-3 tensor.)[7]
  • In a major violation of what is right and sensible, we waste the two valuable symbols   we follow the convention used in introductory physics books that position is denoted by (x,y,z), instead of (x1, x2, x3)[8]

Depending on the context, entities need to be expressed in a variety of ways. For example, among the ways to express position, we have:  

Eigenvalue and eigenvector defined

edit

Consider the following equations, where   is the "matrix",   is "eigenvalue", and   is the "eigenvector".  

The "eigenvalue problem" is to find numerical values for the eigenvalue   and eigenvector,  . The eigenvector remains an eigenvector if it multiplied by any nonzero constant.[9] It is typical for an nxn matrix to have n eigenvalues and n eigenvectors, though exceptions exist. To understand these definitions, you need to look at two examples:

Example 1: Springs aligned to coordinate system (gamma=0)

edit

The matrix   is diagonal if   if   or equivalently, if   The following equation establishes that   is diagonal if both springs are aligned parallel to the   and   axes:

 

Let us assume that  . The last version of this statement establishes that both equations cannot be true for all values of   and   But both equations can be true if either   or   vanishes. This condition defines the eigenvector and allows   to be called the eigenvalue. If the motion is in the x direction, then   is the eigenvalue, and the eigenvector is any non-zero vector with a y-component that vanishes. An obvious choice is   Likewise, the other eigenvalue/eigenvector pair is   and   for the eigenvector.

Expressing the force using normalized eigenvectors

edit

This is difficult to explain in Example 1 because our eigenvectors are aligned with our unit vectors   In other examples we will need a parallel set of unit vectors.  

Here the   coordinates are aligned with the   coordinates.

Example 2: Springs not aligned to coordinates (gamma ≠ 0)

edit

It is not always desirable to orient the spring mechanism with the coordinate system. It can be shown that the symmetric tensor,   can model any spring mechanism with springs not aligned to the coordinate system. While there are procedures for matching a tensor to a given spring mechanism, it is instructive to start with a simple matrix that can be solved intuitively:  

Making two equations equivalent 
equation Eigenvalue Eigenvector
     
     
 
Fig. 4:  
Original coordinates are   while   are aligned with the springs

Equation 8 states the same pair of equations using two different formats. We begin our discussion with the second format, which consists of writing two equations:   and   For reasons that will be explained in the next example, we seek values of   that cause both equations at G to be be equivalent.

Case 1: An obvious case occurs if   This implies that all displacements must obey,   There are no restrictions on the magnitude of   or  , but it is convenient to focus a numerical example. So we take   which forces,   This gives us a displacement equal to   Although it is not strictly necessary, we shall divide this vector by   in order to create a vector with magnitude equal to   This gives us   as our unit vector for all motion for which the the spring constant is effectively   as shown in the table.

Case 2: A slightly less obvious way to render both equations in equation&nsp;8 to be true is to set   which the reader can verify will stipulate that   Setting   leads to   which leads to   as our eigenvector. For convenience we also divide this eigenvector by   to obtain  , which is also a unit vector because it also has magnitude equal to 1.

Figure 4[10] shows the lines of equipotential energy for this example. The original xy coordinate system is shown at the center of the diagram. But for the purposes of understanding and describing the motion of the mass in the spring mechanism, the two unit eigenvectors   and   are more convenient. They are:

 

Here we have introduced capital letters   to denote the spatial variables in this rotated coordinate system. First-year physics students model two-dimensional free fall under a uniform gravitational field as a pair of equations (one for horizontal motion and the other for vertical.) Eigenvectors permit us to do the same with a new coordinate system that uses capital letters:

 
equations to be the same is for   which leads both equations to become equivalent to the single equation,   This constraining forces the particle to lie on the red dotted line passing through the origin in figure 4. A good question to ask students is what the spring constant is for motion along motion along that line.[11]Earlier it was claimed that any stable 2-spring mechanism is associated with a plot of potential energy in which the contours are ellipses (or circles). That is only true if we permit the major axis of an ellipse to be infinite. As shown in figure[12](?), the object can move freely (without external forces) along the line  

Fig. 5: Where 2 lines meet

Example 3: General case

edit

Here we solve the problem symbolically, using   to define any symmetric 2x2 matrix.

 Previously we were able to look at the two equations and "guess" an appropriate value of   but in the case of equation 9, it is not so obvious what to do. Equation 9 is homogeneous in the two variables x and y, which means that every term contains x or y to the first power (no constants are present). The implies that both lines pass through the origin. Fig. 5 reminds that two lines can meet at 0, 1 or an infinity of places (in the latter case they overlap.).

Write both parts of equation 1 as an expression of the line's slope, and demand that the two slopes be equal:  

Since this is the well-known quadratic equation, we can solve:

 

This always yields two real solutions, unless there is only one real solution (which happens when   We find the eigenvectors using the same procedure as before, this time labeling the unnormalized vectors with plus or minus signs,   and normalizing them to   Earlier, we found the unnormalized eigenvector by setting its x component equal to unity (i.e., 1.) Here we do it both ways by also setting the y component equal to unity.[13]

 The question mark indicates that it is not known whether the two calculations yield the same unit vectors, since a unit vector is only uniquely defined up to a minus sign. In other words,   and   are different unit vectors suitable for the x-axis.

See also

edit


  1. After calculating the exact energy for firmly attached springs, students could either (a) use two-dimensional Taylor expansions to calculate correction terms to first or second order. A more ambitious project would be to create a contour plots of potential energy.
  2. See Chatbot BARD's review of free online CASs
  3. The side's length might be proportional or inversely proportional to the spring constant.
  4. To keep things simple, focus on rotations that are multiples of 90o, or even 45o. If deriving the procedures rotating tensors and vectors is deemed too time consuming, a useful exercise would involve stating the procedures and asking multiple choice questions aimed at determined what physical rotation corresponds to what rotation matrix.
  5. In other words, if the eigenvectors are not constrained in some way, it would take 9 parameters to define three eigenvalues and three different directions (since each direction requires two parameters.)
  6. Though only briefly covered, and sometimes given as a Challenge Problem, topics in OpenStax University Physics Volume 1 include rotations, partial derivatives, and gradient.
  7. The best system would use Einstein notation, where every entity (except spatial variables) has a covariant(Fj) and contravariant (Fj).
  8. Subscripts (x1, x2, x3) are more common and seem more natural, but the true notation connoisseur uses einstein's notation where the geometry forbids writing a coordinate in covariant form because there is no such thing as a displacement vector from the origin.
  9. For example, if   is shown to be an eigenvector, then   is also an eigenvector
  10. 2D harmonic well with a zero spring constant.svg
  11. Note that   and   both equal   if   Adding the perpendicular forces as vectors yields   while the object   from the origin. Hence the spring constant is the ratio  . It's an easy answer to guess, so demand they show their work or explain!
  12. File:2D_harmonic_well_with_a_zero_spring_constant.svg
  13. See page 5 of https://www.math.wustl.edu/~wick/teaching/Math2605Notes/chap3.pdf