Linear Algebra

Linear Algebra was developed to simplify linear equations. It provides a simple way of representing and formalizing the solution of linear equations.

Consider the two equations below

2x + y = 10
2x - y = 2

One can trivially solve these two equations of two variables. That is high school algebra. We don't need any code to solve it. But what would we do if we were given 100,000 equations of 100,000 variables? Linear Algebra helps us here. Let us see how

Of course, we do not have enough space here to write down the 100,000 equations. But we can use the above two equations to understand the concept. These two equations can be written in matrix form as

Essentially, we have represented the set of equations in the form

    Ax = b       # Where A is a matrix; x and b are vectors.

This is the short hand way of representing a set of linear equations - in form of matrices.


Reducing the representational size is not the only advantage of this. We will see below how it helps in computation. Before that, let us look into the notifications.

  • A ∈ Rm x n => A is a matrix of m rows and n columns, and all its elements are real numbers.
  • x ∈ Rn => x is a vector with n entries and all its elements are real numbers.
  • Aij => Denotes the element of A at the ith row and jth column
  • xi => Denotes the element of ith element of the vector x
  • Ai,: => ith row in the matrix A
  • A:,j => jth column in the matrix A

Intuitively, we can think of a vector as a point in n-dimensional space. And a matrix A as an operation that can map a vector V1 in n dimensional space into another vector V2 in m dimensional space.

Important Concepts

Linear Algebra is a vast domain. In order to use it in Machine Learning, it is necessary to understand some basic concepts.