Linear algebra: best fit line: method of least squares by hand

The tutor shows a nice trick to obtain a least squares line using matrix algebra.

In my Nov 25 post I mentioned how, with Excel or LibreOffice Calc, to get a best-fit line for

x y
0 3
2 5
6 10
8 12

The same line can be gotten by hand, with the following ideas:

  1. Imagining the best fit line to be of form y=mx+b, the table above can be related as the matrix equation
  2. We multiply each side by the transpose matrix:
  3. Performing the multiplication on each side yields
  4. which means

    Solving, we get m=1.15, b=2.9: our best-fit line is y=1.15x + 2.9, same with the result from November 25 using a spreadsheet.

    Neat, eh?

    Source:

    Johnson/Riess/Arnold. Introduction to Linear Algebra, Second Ed. Don Mills: Addison-Wesley, 1989.

    Jack of Oracle Tutoring by Jack and Diane, Campbell River, BC.

Linear algebra: matrix multiplication: no cancellation rule

The tutor gives an example to show that with matrices, AB=AC doesn’t mean B=C.

Consider the matrices A, B, C, and D:

By matrix multiplication, AB=AC. (In fact, both products equal D.) Yet, obviously, B≠C. Matrix multiplication lacks the cancellation law that you see with the real numbers.

HTH:)

Source:

Johnson/Riess/Arnold. Introduction to Linear Algebra. Don Mills: Addison-Wesley, 1989.

Linear algebra: projecting a vector along another one, matrix method.

The tutor shows an example of a projection from one dimension into another.

Let’s imagine we have the vector v

3
2
0

which we want to project in the direction of w

3
-1
0

The projection matrix, Pw, is given by wwT/wTw:

Now, the projection of v onto w, Pwv, is

Note that P2wv = Pwv:

Note, also, that Pwv has the same proportions as w.

Source:

mit.edu

Jack of Oracle Tutoring by Jack and Diane, Cambpell River, BC.

Linear algebra: cofactors, signed minors

The tutor defines cofactor (aka signed minor) with an example.

In my post from Nov 24, 2014 I tell how to evaluate a determinant by hand.

Example: Evaluate the determinant of matrix A:

1 0 -7
0 2 1
3 1 0

If the determinant is calculated from the top row, as follows

1[(-1)1+1(2×0-1×1)]+0[(-1)1+2(0x0-3×1)]+-7[(-1)1+3(0x1-3×2)]

we get

1[-1] + 0[3] -7[-6]

The numbers in red are called the cofactors or signed minors.

Cofactor Cmn is calculated as follows:

  1. Remove row m and column n.
  2. Calculate the determinant of the resultant matrix.
  3. Multiply that number by (-1)m+n

For every term in the matrix there is a signed minor.

I’ll be talking more about this:)

Source:

Johnson/Reiss/Arnold. Introduction to Linear Algebra. Don Mills: Addison-Wesley, 1989.

Jack of Oracle Tutoring by Jack and Diane, Campbell River, BC.

Spreadsheets: finding the determinant of a matrix using Excel

The tutor tells how to have Excel evaluate a determinant for you.

Here’s how:

  1. Key in your matrix values. Let’s imagine you use the range a1:d4.
  2. Select a cell away from the range, let’s say f6. Key in =mdeterm(a1:d4) then press Enter.

HTH:)

Jack of Oracle Tutoring by Jack and Diane, Campbell River, BC.

Linear Algebra: solving a three-variable linear system with Excel or LibreOffice Calc

The tutor shows an example of how to solve a system of three equations using a spreadsheet.

Example: solve the system

  1. -3x + 2y + z = 16
  2. x – y – 3z = -19
  3. 2x + y + 7z = 30

Solution:

From linear algebra, this system becomes Aq = r, where A is the matrix

-3 2 1
1 -1 -3
2 1 7

while q is the column vector

x
y
z

and r is the column vector

16
-19
30

From

Aq=r

follows

A-1Aq=A-1r

Furthermore, since

A-1Aq=Iq=q

then

q=A-1r

So we need to perform the calculation A-1r on the spreadsheet.

My post from yesterday shows how to get A-1 (the inverse of matrix A). It’s short, but detailed.

Assuming you’ve produced A-1, it’s time to enter vector r:

16
-19
30

Let’s imagine, on your spreadsheet, that A-1 resides in the range e1:g3, and that r resides in h1:h3. To multiply A-1 by r,

  1. Select a cell away from both ranges: k1, for example. Drag down to extend the selection from k1 to k3.
  2. Click to the formula bar and type =mmult(e1:g3,h1:h3). Don’t press enter.
  3. While holding down Ctrl and Shift, press Enter.
  4. (Note): With matrices, unlike numbers, the order of the multiplication matters: in Step 2, it must be =mmult(e1:g3,h1:h3), rather than h1:h3,e1:g3.

Hopefully you see the result

-3
1
5

Therefore, x=-3, y=1, and z=5.

Source:

educ.jmu.edu

Johnson, Lee et al. Introduction to Linear Algebra. Don Mills: Addison-Wesley, 1989.

Jack of Oracle Tutoring by Jack and Diane, Campbell River, BC.

Spreadsheets: Matrix inverse on Excel or LibreOffice Calc

The MINVERSE() function can be tricky to use; the tutor tells how.

Example: find the inverse of matrix A:

-5 0 1
3 8 9
0 1 0

Solution:

  1. Enter the matrix in Excel or LibreOffice Calc. Let’s imagine, for this example, you use the 3×3 range a1:c3.
  2. Select a cell to the right, away from the cells you’ve used. Drag across and down so that you’ve captured a 3×3 area.
  3. Click into the formula box and type =minverse(a1:c3) but don’t press enter!
  4. Hold down both Ctrl and Shift, then press Enter.

Hopefully you see the result

-0.1875 0.020833 -0.166671
0 0 1
0.0625 0.104167 -0.83333

Source:

www.excelfunctions.net

Jack of Oracle Tutoring by Jack and Diane, Campbell River, BC.

Linear algebra: Matrix multiplication

The tutor demonstrates the technique of multiplying matrices.

To go further with Markov chains (introduced in my previous post), the reader needs to understand matrix multiplication. To many, the method is surprising at first.

In preparation, the reader needs to know that matrix entries are commonly referred to by (row,column) like so:

    \[\left[\begin{array}{c c c}1,1 & 1,2 & 1,3 \\ 2,1 & 2,2 & 2,3 \\ 3,1 & 3,2 & 3,3 \end{array}\right]\]

Example: Consider the matrices A and B:

    \[A=\left[\begin{array}{c c c}-1 & 5 & 9 \\2 & 0 & 3 \end{array}\right] \  \ B=\left[\begin{array}{c c}9 & 0 \\ 2 & 1 \\0 & -7  \end{array}\right]\]

Find the matrix product A \times B:

    \[\left[\begin{array}{c c c}-1 & 5 & 9 \\2 & 0 & 3 \end{array}\right]\times \left[\begin{array}{c c}9 & 0 \\ 2 & 1 \\0 & -7  \end{array}\right]\]

Solution:

We start by mulitplying row one of A by column one of B, as follows:

-1(9)+5(2)+9(0)=1

This result is entry (1,1) of the solution matrix S:

    \[S = \left[\begin{array}{c c}1 & \ \\ \ & \ \end{array}\right]\]

Next, we multiply the first row of A by the second column of B to get entry (1,2) of S:

-1(0)+5(1)+9(-7)=-58

    \[S = \left[\begin{array}{c c} 1 & -58 \\ \ & \ \end{array}\right]\]

We move on to the second row of A, multiplying it by the first column of B. The result will be (2,1) of S:

2(9)+0(2)+3(0)=18

    \[S = \left[\begin{array}{c c} 1 & -58 \\ 18 & \ \end{array}\right]\]

Finally, we multiply the second row of A by the second column of B, yielding (2,2) in S:

2(0)+0(1)+3(-7)=-21

    \[S = \left[\begin{array}{c c}1 & -58 \\ 18 & -21\end{array}\right]\]

Matrix S is the solution to A\times B

I’ll continue about this topic in coming posts.

HTH:)

Sources:

www.holoborodko.com/pavel/quicklatex

www.ccs.tulane.edu/~jchrispe

Johnson/Riess/Arnold. Introduction to Linear Algebra. Don Mills: Addison-Wesley   Publishing Company, 1989.

Jack of Oracle Tutoring by Jack and Diane, Campbell River, BC.

Math: Solving systems of equations: Cramer’s Rule

Tutoring high school math, I don’t see this used.  However, I did see it at university.  The tutor introduces Cramer’s Rule.

Consider the following problem, common in high school math:

Solve the system.

    \[\begin{array}{ccccc} 2x&-4y&=&20 \\ 3x&+5y&=&-3 \\ \end{array}\]

If you know determinants (having read my articles here and here), then you have the option to use Cramer’s Rule.

Cramer’s Rule sees the problem as follows:

The original system, in matrix form, is called the augmented matrix – so-called because it contains the constant numbers in the third column:

    \[\left[ \begin{array}{cr|c} 2&-4&20 \\ 3&5&-3 \\ \end{array} \right] \]

The matrix with just the x and y columns is called “the 2×2 matrix”:

    \[\left[ \begin{array}{cr} 2&-4 \\ 3&5 \\ \end{array} \right] \]

With Cramer’s Rule, we evaluate determinants of various matrices derived from the augmented matrix. The solution to each variable begins in fraction form. Its numerator is a 2×2 determinant in which the third column from the augmented matrix is substituted for the column of the variable being found. The denominator is the determinant of the 2×2 matrix. Let’s proceed:

Solving the above system for x, Cramer’s Rule gives

    \[x=\frac{\left| \begin{array}{cr} 20&-4 \\ -3&5 \right| \end{array}\right|}{\left | \begin{array}{cr} 2&-4 \\ 3&5 \\ \end{array}\right|}\]

Notice that, in the numerator, the third column from the augmented matrix is substituted for the x column, while the y column stays the same.

Then, Cramer’s Rule gives, for y:

    \[y=\frac{\left| \begin{array}{cr} 2&20 \\ 3&-3 \end{array} \right |}{\left | \begin{array}{cr} 2&-4 \\ 3&5 \end{array}\right|}\]

We evaluate the determinants in each case, then simplify. First for x:

    \[x=\frac{\left| \begin{array}{cr} 20&-4 \\ -3&5 \right| \end{array}\right|}{\left | \begin{array}{cr} 2&-4 \\ 3&5 \\ \end{array}\right|} =\frac{20(5)-(-3)(-4)}{2(5)-3(-4)}=\frac{88}{22}=4\]

We do the same for y:

    \[y=\frac{\left| \begin{array}{cr} 2&20 \\ 3&-3 \end{array} \right |}{\left | \begin{array}{cr} 2&-4 \\ 3&5 \end{array}\right|}=\frac{2(-3)-3(20)}{2(5)-3(-4)}=\frac{-66}{22}=-3\]

Apparently, x=4,y=-3. Let’s sub the values back into our original equations just to make sure:

    \[\begin{array}{cccc} 2(4)&-4(-3)&=20 \\ 3(4)&+5(-3)&=-3 \end{array}\]

Simplifying, we see that indeed

    \[\begin{array}{crcc} 8&+12&=&20 \\ 12&-15&=&-3 \end{array}\]

Since, in each of our original equations, the left side equals the right side, we have confirmation that our solution x=4,y=-3 is correct.

Cramer’s Rule can be very handy, especially for 2-variable systems.

I’ll be saying more about Cramer’s Rule and systems of equations in future posts:)

Source:

Johnson/Riess/Arnold. Introduction to Linear Algebra, 2nd edition. Don Mills:
   Addison-Wesley, 1989.

Jack of Oracle Tutoring by Jack and Diane, Campbell River, BC.

Linear algebra: more on determinants

Following up on yesterday’s post, the tutor continues about determinants. Tutoring university math or natural sciences, they come up often.

Yesterday’s post covered some basics about determinants including a 2×2 and a 3×3 example.  Although it revealed the necessities for many practical situations, it left out most of the theory.  However, determinants are a playground for mathematicians; covering all the theory about them in a dozen posts would still be impossible.

We face the following question:  are there little bits of extra theory that could really help a student with determinants?  Are there little observations that could ease a student’s uptake of the topic?

Today:  one observation and one bit of theory:

Observation 1:  When a matrix is shown in vertical brackets rather than square ones, it usually means the determinant of the matrix.

That is, if you have matrix A:

    \[A=\left[\begin{array}{rr}7&-4\\-3&5\end{array}\right]\]

then

    \[det\ A=\left|\begin{array}{rr}7&-4\\-3&5\end{array}\right|=35-12=23\]

Theoretical point 1:

You can expand the determinant along any row or column. In yesterday’s example, I showed how to evaluate

    \[det\ B=\left|\begin{array}{rrr}3&11&1\\4&2&-7\\-1&0&5\end{array}\right|\]

from the top row. However, you could evaluate det B from the middle column instead. For the negative flip-flopping, remember to multiply each step by (-1)^{r+c}, where r is the row, and c is the column.

I’ll now evaluate det B from the middle column (using the procedure from my previous post):

The middle column starts at 11, which is in row 1, column 2. Therefore, the “flip-flop” factor will be (-1)^{1+2}=-1. Imagining the matrix without the first row and second column, we proceed:

    \[-1\times11\times((4\times5)-(-1\times-7))=-11\times13=-143\]

We move to the next number in the second column: the 2. Its flip-flop factor is (-1)^{2+2}=1:

    \[1\times2\times((3\times5)-(-1\times1))=2\times16=32\]

Now, we arrive at the third member of the middle column, which is a 0. Here we get a break: 0 times anything else is 0.

Finally, we add our results:

    \[det\ B=-143+32+0=-111\]

So, det B comes to -111, just as it did from yesterday’s expansion along the top row.

Having the freedom to choose the row or column to expand from is definitely an advantage when evaluating the determinant, since you can make convenient use of zeros in a matrix.

I hope this helps all you college/university students, for whom first term exams draw near:)

Source: Johnson|Riess|Arnold. Introduction to Linear Algebra, 2nd Edition.
   Don Mills, Ontario: 1989.

Jack of Oracle Tutoring by Jack and Diane, Campbell River, BC.