linear algebra lecture 14 & 15 08/21/2017

lecture 14:
正交：
row space and null space 正交 orthogonal 因为A*x = 0
column space and null space of At 正交
Xt*y=0

subspace S is orthogonal to subspace T means that every vector in S is orthogonal to every vector in T.

Ax = b 可能没有solution
=> 变成这个更好 AtAx = Atb (best solution in this situation)

AtA不一定永远可逆

N(AtA) = N(A)
rank (AtA) = R(A)

AtA is invertible exactly if A has independent columns.


lecture 15：projections and least square
求b在a上面的投影p: (a,b是个向量)
正交 aT(b-xa) = 0
x = aTb/aTa
projection p = ax

p = a*aT/(aT*a) 是projection matrix
性质1：rank是1 因为是一行*一列得到的
性质2：Pt = P
性质3： P2 = P

Why project?
Because Ax = b may not have solution
solve Ax = p instead. p is the projection of b onto the column space. (等式右边是column space 的话肯定会有解)

AT(b-Ax_hat) = 0
=>ATAx = ATb

x = (ATA)-1ATb
p = A(ATA)-1ATb

projection matrix P =  A(ATA)-1AT (不能再简化了，因为A不一定可逆，可能吧不是square matrix)

least square fitting by a line: