Linear algebra is a powerful tool for data manipulation.
It is easier to conceptualize using geometry. This article uses a two dimensional plane because they are easier to visualize.
A 2-D plane means a data set with only two entries. But same ideas extend to n-dimensional plane or data sets with n entries.
Linear algebra concepts π
Linear algebra involves writing numbers in columns () called vectors, and boxes () called matrices.
Vector shows a location π
A vector, single column of numbers (3, 5) represents a point on a cartesian 2-D plane, or any n-D space for that matter.
We need two things to make this work-
- Define a origin (0, 0) from where the vector points to the location of the point.
- Define the units of direction in x and y axes.
The basis vectors π
The unit vectors are called basis vectors. They inform how far to move from the origin, and in which direction. The basis vectors along the x-axis, y-axis and z-axis are called i, j and k respectively.
Linear combination of vectors and span π
Any point on the 2-D plane can be reached with sum of two or more vectors.
It’s impossible when two vectors are parallel. In that case, all possible linear combinations are on a straight line.
There is also a third possibility, when the two vectors are zeros. In that case, no linear combination will get us anywhere from zero.
The span of vectors v and w is the set of all their linear combinations.
In a 3-D space, a span of 2 vectors will most likely be a 2-D plane, or a line if they are parallel. A span of 3 vectors in the same plane will cover the entire 3-D space, or a plane, or a line, or, in the most extreme case, just the origin.
Linear transformation of the plane π
If we stretch, skew, rotate or invert the vector plane keeping-
- The same origin same, and
- Grid lines parallel to one another,
Then it is called a linear transformation of the plane.
The 2-D plane doesn’t have to be with all square grids. We can define a plane where the axes are not perpendicular to one another.
Linear transformation lets us define a vector on one plane using basis vectors of another plane.
Matrix π
A matrix is just a row of vectors that defines a linear transformation. The first column tells where i lands, the second column defines where j lands, and so forth.
Vector dot products π
When you multiple two vectors using dot product, you get the area of the parallelogram enclosed by the vectors. For a 3-D plane, this will be the volume.
Determinant π
When you transform a plane using a matrix, for example, skew it. The area enclosed by vectors in that plane changes. Determinant tells you by how much the area changes.
When talking about a 3-D plane, the determinant shows the change of volume.
When the output of a transformation reduces the 2-D plane into a line, the determinant is zero.
Rank π
The number of dimensions of the output of transformation is called rank.
When we apply a matrix on a 2-D plane and the output is a line, we call it has a rank of 1.
Inverse matrices π
An inverse matrix completes the transformation in reverse.
The inverse cannot be found if-
- The rank of transformation < initial dimensions in the space (for example, you squish a 2-D plane into a line)
- The determinant is zero
Change of basis π
Matrices transform vector space. Vector spaces are arbitrary. The point defined by vector A, in one vector space F, will be defined by another vector B in second vector space, S.
How can we explain vector A in terms of the other vector space?
If matrix A transforms basis of space F to space S. A-1 does the opposite. It informs where any vector in space F lands in space S.
Eigenvectors and eigenvalues π
When a matrix is applied on a plane, the vectors change direction and are scaled. The vectors that remain on their own span, are called eigenvectors. The value by which these special vectors are scaled is called a eigenvalues.
For example, if the transformation is stretching 2x along the horizontal axis, the x-axis is an eigenvector with eigenvalue of 2.
Additionally, for a 3-D plane, the eigenvectors with eigenvalues of 1 are rotational axis.
Extending to abstract world of data π
These ideas are explained using a 2-D, and sometimes, a 3-D plane. But we can (somewhat) easily extend it to n-D environment. Even abstract functions such as file compression or dimension reduction, when thought in light of geometric manipulation of space, starts making sense!
Most of the heavy work in linear algebra can be handled using computers. But these fundamental ideas are key to understanding what is happening under the hood.
Resources π
YouTube playlist The Essence of Linear Algebra from 3Blue1Brown.
In this playlist, Grant Sanderson explains the ideas using simple geometric operations using lines and dots on paper, i.e., a two dimensional plane.