M251 – Elementary Linear Algebra, UTK Fall 2003, MWF 1:25-2:15 Introductory Notes by Jochen Denzler

Read these notes first, and then reread them whenever we have accomplished another chapter in the textbook. And whenever in later courses they want you to remember some stuff from this course, you may want to have this overview over the course available, so you know how that stuff in the new course fits in the greater context.

A friend of mine heard the following message at the Courant Institute: You never know enough linear algebra. When I was a student, linear algebra was the most boring course I attended. Both observations are NOT in contradiction. The punchline is that the linear algebra from the introductory course needs to be related to experiences outside this course. If it isn’t, forget it: you’re being fed the bones, and the meat goes to the dog. The importance of the course lies in its interconnections. You’ll explore some of them later, and I can only hint at them now. The purpose of these notes is to help you with the big picture, and with these interconnections, whenever they come up, now or later. Savour and cherish them now, or be bored.

Not everything in these notes will be required exam knowledge. I’ll clarify later how much is fair game for exams. Everything is fair game for introduction, illustration and back reference.

A Bird’s Eye View of M251

Your first encounter with Linear Algebra in this course will be an organized way of solving systems of linear equations, like, e.g., 5x + 3y − z = 13, −2x + y + z = −2, Ch. 1 4x + 5z = −6. You have probably done this task already at high school: adding equations to eliminate variables, or solving for one variable and plugging it back into the remaining equations. This is in essence what we are going to do here as well; the difference is that we are doing it in an organized way, not haphazardly. Nowadays, many problems require the solution of hundreds, thousands or millions of linear equations in just as many unknowns, and this will be done by computer. In doing this job, the computer will crunch the coefficients, but will not care about the names of the variables. The calculation is the same, whether you call the unknowns x, y, z or u, v, w. So the essential information is contained in a rectangular array of the numbers showing up in these equations, namely in our example:

[ \begin{matrix}5 & 3 & −1 & 13 \\ −2 & 1 & 1 & −2 \\ 4 & 0 & 5 & −6 \end{matrix} ]  a 3 × 4 matrix

Any rectangular array (of numbers) is called a matrix. (The vertical line is not part of the matrix, but serves only for our visual convenience in interpreting where these numbers belong.) Chapter 1 of the book deals with matrices and solution procedures for linear equations.

Matrices were invented as a concise notation already before the invention of computers, but for more or less the same purpose as outlined above. It turns out that (subject to certain limitations) matrices can be added, subtracted, multiplied almost like real numbers. There are however also some important differences, which you will need to learn thoroughly. The most important among them is that for the product AB of two matrices A and B, which we will define (under certain assumptions on the size of these matrices), it is NOT true in general that AB = BA. This may sound weird, if you judge on the basis of your experience with the arithmetic of numbers, but after all, multiplication of matrices is something really new, so it is entitled to some new properties!

To learn the arithmetic of matrices is not particularly difficult on its own, but it may leave you with a “so what?” feeling. I do hope it does, because linear algebra would be pretty useless, if there weren’t a practical down-to-earth meaning hidden behind these straightforward calculations. Basically, this information is postponed Ch. 8 until Chapter 8 in the textbook, but I will give a brief hint now. For instance, a 3 × 3 matrix can be viewed to represent a linear mapping applied on objects in 3dimensional space. If you find this vocabulary daunting, take a chair in your hands and do the following experiment. (I won’t draw pictures, because that’s an awful lot of work, and if you do the experiment you don’t need pictures.)

Let’s assume you are holding the chair in such a position that somebody sitting on it would face you. Now turn the chair 90 degrees about a vertical axis. If someone were sitting on the chair he would now look sideways (lets assume you turned the chair so the person would look to the right). There is a certain 3 × 3 matrix that describes the turning of the chair, and we’ll call it A. Next you turn the chair by 90 degrees, counterclockwise, about a horizontal axis that goes straight away from you in forward direction. The imagined person on the chair is now lying with his back on the back of the chair, facing upwards. This operation is also represented by a matrix, and we call it B. Doing these two operations one after the other is decribed by the product of both matrices, BA. ^{1} Now start over with the chair again facing you and do the same two operations in reverse order: first B, which will make the person sitting on the chair fall down to your left.^{2} Then A, which turns the chair about a vertical axis, and makes the back of the chair hit your left side. The final position of the chair is different from the first experiment. This is why, in this example, AB ≠ BA.

(This was just one example. Other operations than rotations, like, e.g., stretching in some direction, can be represented by matrices as well.)

Chapter 2 of the book deals with determinants. Every square matrix (ie., the Ch. 2 rectangluar array has as many columns as rows) gets assigned a certain number, called its determinant. Determinants absolutely don’t belong at this place in the course, but they wouldn’t yield their place in the curriculum but to brutal violence, and so we better deal with them peacefully. The most important thing is of course postponed until later^{3}: Determinants represent volumes (or volume ratios). So the matrices A and B in the chair example just rotate the chair, but don’t change its volume; their determinant is 1. For the moment, you’ll have to trust me on this volume interpretation and just deal with the fancy calculation of these miraculous determinants that have come totally out of the blue. If I can concoct a way to move the determinants at a later time in the semester without screwing up other parts of the course, I’ll do so.

1In case you think it should be AB, because you first do A and then B: no, it shouldn’t; it’s like in calculus, where cos ln x means that you first take the logarithm, and then the cosine.

2Remember that it’s a hypothetical person. If you put a real one on the chair and then have him sue me, I’m gonna turn you into chop suey, or worse, give you a failing grade;-)

3They don’t want you to understand everything right away, because they have many more textbooks they want to sell you later;-)

I have insisted above to relate, in the style of a sneak preview, the calculational

algebra vs. geometry

objects (matrices, determinants) to a geometric meaning. The next chapters will put substance into these claims. You will learn about vectors in the plane and in space. The word vector is of Latin origin and its original meaning is something like “driver”: not primarily the guy in the driver’s seat, but the tool that transports you. Geometrically, vectors are represented by arrows, beginning at one point P and pointing to another point Q. The arrow “drives” you from point P to point Q, hence the name vector. In Chapter 3, you will learn about adding and subtracting Ch. 3 vectors, about multiplying them with numbers, and about a scalar product or dot product between two vectors, which will be (miraculously?) a number. This product is useful in physics: in the formula “work = force times distance”, force is actually a vector, and the ‘distance’ is also a vector, comprising not only the distance (length) but the full transport information, including the direction. And the multiplication refers to the scalar product. Another type of product is the cross product, or vector product, which, at the level of M251, only makes sense for vectors in space, not for any of the more general vectors we are about to encounter soon. It is also very important in physics, for instance in electrodynamics (the laws that govern radiowaves, voltage transformers, power generators). If we manage to cover the vector product (it’s optional material), you’ll see a hue of a connection with these obscure determinants from Chapter 2.

In mathematics, we have two ways of viewing vectors: (1) geometrically, as outlined above. From this point of view, the scalar product encodes angle and length measurement. This was the view of Chapter 3. — (2) The abstract view. In a Ch. 5 vast generalization, any kind of objects that permit calculations and rules like the geometric vectors studied in Chapter 3 will also be called vectors. We just reason: If it walks like a duck and quacks like a duck, then (no, it may not be a duck, but…) let’s call it a duck! This is the point of view in Chapter 5.^{4} A vector space will be the collection of all vectors of a certain kind. Now if you think that’s pretty abstract, think again: it will seem even more abstract when I give you a concrete example (what a strange world that is)! Functions can be viewed as vectors (in the duck sense). And if you have already taken M231 (Introduction to Differential Equations), you will find that the same fancy vocabulary of linear dependence vs. linear independence that was used there for solutions of linear ordinary differential equations shows up in our Chapter 5 again. If in contrast, you will take M231 later, you are in for the same deja-vu from the other side. In either case, I urge you to either dig out your former M231 notes when we are in Chapter 5, or else return to our course material from Chapter 5, when you get to linear (in)dependence of solutions of linear differential equations in M231 later.

4If you still think it’s strange to call functions “vectors”, only because both functions and geometrical vectors can be added and multiplied similarly, let me remind you that the word ‘bulb’ existed in the English language already before Edison. It was only the rough shape similarity that made the (light-)bulb borrow its name from the bulbs that become tulips: Abstraction is NOT an exotic thing!

Chapter 6 will generalize the scalar product to this abstract setting. In some of Ch. 6 your future endeavors, you may learn about Fourier series. A typical course for this subject is M435 (Introduction to partial differential equations). When they tell you about “orthogonality relations for trigonometric functions” and you wonder what this geometric concept of orthogonality has to do with trig functions, you will want to reread this very paragraph here, and the Chapter 6 from this course. — If you are headed towards electrical engeneering and will learn about signal processing, you are certain to learn about Fourier series. — If you are a math major and will later focus towards algebra rather than towards the calculus-based subjects, these abstract vector spaces will be your first encounter with pure algebraic structures, while you may not have business with M435. Then this chapter, all for its own sake, is your initiation to the more advanced algebraic courses.

Chapter 4 is meant to bridge the gap between the geometric vectors from before Ch. 4 and the abstract (duck definition) vectors that are to follow. It is about “geometric” vectors in spaces that may have more than three dimensions. You cannot imagine (let alone draw) pictures and arrows any more, but the analogy of the calculations with the ones in Chapter 3 is sufficiently conspicuous. Think of a trained chess player, who knows that the king may move from square e1 to square f1, because these squares can be seen to be adjacent from their coordinates, and that the queen may move from c3 to c8, if the squares in between (namely c4, c5, c6, and c7) are free. Actually drawing a chessboard is not necessary for this reasoning. This is the way how we think “geometrically” of, say, four or five dimensional space.

It is in this context that vectors turn out to be equivalent to n × 1 matrices, that matrices can be multiplied with vectors (as a special case of multiplying matrices), and that you can see the first time why the operation of whirling a chair around can be represented by a (3 × 3) matrix.

The concept of eigenvectors and eigenvalues is the culmination of the entire Ch. 7 course. You will already have encountered a bit of the subject in Chapter 4. The word “eigen” is one of the few German words that entered into the English technical vocabulary. It means “own” or “proper”. (The French and Russians actually translate the word in their technical language.) To a square matrix, there are associated its eigenvalues (certain numbers), and each eigenvalue comes with a bunch of eigenvectors. When the matrix represents a transformation (like turning the chair), the eigenvectors represent directions that are not changed under the transformation.^{5} In the case of a rotation of a chair, eigenvectors of the matrix representing this rotation would point in the direction of the axis of rotation, because this is the very direction that remains the same under the rotation.

In our textbook, eigenvalues will be real numbers. When you learn how to calculate them, you will see, e.g., that to find the eigenvalues of a 2 × 2 matrix, you need to solve a quadratic equation. You know that quadratic equations may not have real solutions, but that we can always assign them complex numbers as solutions. If you are merely into calculations, you can accept complex numbers as eigenvalues just as easily as real numbers (provided you aren’t afraid of complex numbers). But the geometric interpretation for the corresponding eigenvectors breaks down, when you admit complex eigenvalues. This is why, with regard to time limitations, the book deals with real eigenvalues only. Be aware that this restriction is not warranted by the subject itself, only by practical purposes.

5Do not read this footnote until later. I am slightly misusing the word ‘direction’ here: north and east are of course different directions, but in this context, north and south are considered as the ‘same’ direction. What is really the issue are that the lines along the vectors are parallel. When we come to the detailed discussion, this definition will be natural. I can’t avoid that it sounds strange in this sneak preview.

This course cannot cover the applications of eigenvalues and eigenvectors: Let me just mention some for you, with the request that you accept them in good faith without the details needed for explanations.

• In mechanics, they assign to each solid body (like, e.g., a wheel), a 3 × 3 matrix called its tensor of inertia. The eigenvectors of this matrix are the directions of those axes around which the body can rotate without wobbling.

• The mapping that assigns to a body in space its shadow on the floor can also be viewed as a matrix. The eigenvectors of this matrix are those vectors that are parallel to the floor (their shadow points in the very same direction as the vectors itself), and vectors pointing in the direction of the sun rays. Their shadow collapses to a point, and by a slight abuse of language, a point looks in any direction you please; in particular the one into which the original vector pointed. Other vectors pointing upwards, away from the floor, but not right towards the sun, are NOT eigenvectors, because they do not point in the same direction as their shadow.

When you view functions as abstract vectors in the duck sense of Chapter 6, many further applications arise; I won’t go into details about who is going to play the role of the matrices in this setting (that’s beyond M251), but let it be enough to say that the eigenvectors will be functions and the eigenvalues will still be numbers. In this setting, many more applications arise:

• In quantum physics, the energy levels of atoms are eigenvalues of some “generalized abstract matrix”: when quantum physics was a subject in its early beginnings, understood by only a few people worldwide, Heisenberg, one of the founders of the theory, explained his calculations to mathematicians. An anecdote (whose authenticity I do not know, but I learned it from my physics professor) tells that the mathematicians told Heisenberg that what he was doing here was actually calculating with infinite dimensional matrices! Which left Heisenberg very impressed, because he said he hadn’t even known how to calculate with finite dimensional matrices! (Strange, because he would certainly have known the tensor of inertia.) The eigenvector, actually a function, tells you where the electron is most likely to be encountered. — Quantum physics was a big boost for eigenvalue questions in mathematics.

• When you let a string on a guitar or violin vibrate, its frequency is an eigenvalue of “a generalized abstract matrix”. I have lied a little bit here: The eigenvalue is minus the square of the frequency. — Same thing with a vibrating membrane (like the head of a drum). The eigenvectors (actually eigenfunctions) then tell the shape of the vibrating membrane. The places where the eigenfunction takes the value zero are those where the membrane is at rest. Those where the eigenfunction has the largest values are the ones where the membrane oscillates with the largest amplitude.