Linear Algebra for AI: Vectors, Matrices and Examples

Quick answer: Linear algebra is the main math of AI because AI represents everything as vectors and matrices. A word, an image or a sound becomes a vector of numbers; a neural network layer is a matrix that transforms one vector into another, y = Wx + b; and training adjusts the numbers in those matrices. Computers run AI on chips built to multiply matrices quickly. The key ideas for students are vectors, dot products, matrix multiplication and, later, eigenvectors.

If you ask what math AI really runs on, the honest answer is mostly linear algebra. Every image, word and sound an AI handles becomes a list of numbers, and almost every step it takes multiplies those lists by grids of numbers. Here are the core ideas, each with an example you can do by hand.

Linear Algebra in AI at a Glance

IdeaWhat it isWhere AI uses it
VectorAn ordered list of numbersWords, images and data points
Dot productMultiply matching entries and addSimilarity, attention scores
MatrixA grid of numbersThe weights of a network layer
Matrix multiplicationCombining rows with columnsRunning every layer
EigenvectorsDirections a matrix only stretchesCompressing data (PCA)

Vectors: Everything Becomes Numbers

A vector is an ordered list of numbers, like \((3, 4)\). Geometrically, it is an arrow from the origin to the point \((3, 4)\), with length \(\sqrt{3^2 + 4^2} = 5\). AI uses vectors with thousands of entries:

  • A 28×28 grayscale image of a handwritten digit has \(28 \times 28 = 784\) pixel values, which can be flattened into one vector; a color (RGB) image of the same size has 2,352 values.
  • A word in a chatbot becomes a vector called an embedding, where similar words get similar vectors.
  • A customer might be a vector of age, purchases and visits.

The Dot Product: Measuring Similarity

The dot product multiplies matching entries and adds:

\[ (1, 2, 3) \cdot (4, 0, -1) = 1(4) + 2(0) + 3(-1) = 1. \]

When vectors point in similar directions, the dot product is positive; when they are perpendicular, it is 0. (Raw dot products depend on vector lengths too.) AI uses this to measure how similar two items are, and transformers use it in attention (see the math behind ChatGPT).

Matrices: Transforming Vectors

A matrix is a grid of numbers. Multiplying a matrix by a vector produces a new vector: each entry is a dot product of a row with the vector.

\[ \begin{pmatrix} 2 & 0 \\ 1 & 1 \end{pmatrix} \begin{pmatrix} 3 \\ 4 \end{pmatrix} = \begin{pmatrix} 2(3) + 0(4) \\ 1(3) + 1(4) \end{pmatrix} = \begin{pmatrix} 6 \\ 7 \end{pmatrix}. \]

Geometrically, a matrix stretches, rotates or shears space. Multiplying by another matrix transforms each of its columns separately:

\[ \begin{pmatrix} 2 & 0 \\ 1 & 1 \end{pmatrix} \begin{pmatrix} 3 & 1 \\ 4 & 2 \end{pmatrix} = \begin{pmatrix} 6 & 2 \\ 7 & 3 \end{pmatrix}. \]

The shape rule: an \(m \times n\) matrix times an \(n \times p\) matrix gives an \(m \times p\) matrix; the inner sizes must match. Matrix multiplication is not the same as multiplying entry by entry. See matrix multiplication.

A Neural Network Layer Is a Matrix

One layer of a neural network computes

\[ y = \text{ReLU}(Wx + b), \]

where \(x\) is the input vector, \(W\) a matrix of weights, \(b\) a bias vector, and ReLU replaces negative numbers with 0. Using the matrix above with \(b = (-7, 0)\):

\[ Wx + b = (6, 7) + (-7, 0) = (-1, 7) \quad\Rightarrow\quad \text{ReLU} = (0, 7). \]

\(Wx + b\) is an affine transformation; ReLU, applied to each entry, adds the nonlinearity that lets stacked layers represent more than a single affine map. A deep network stacks many such layers, and training changes the numbers in each \(W\) and \(b\) to reduce a loss (see gradient descent explained). In Python with NumPy, the whole layer is one line:

import numpy as np
W = np.array([[2, 0], [1, 1]])
x = np.array([3, 4])
b = np.array([-7, 0])
print(np.maximum(W @ x + b, 0))   # [0 7]

Here @ is matrix multiplication, while * would multiply entry by entry.

Why AI Needs Special Chips

Large models do trillions of these multiply-and-add operations. AI accelerators, including Google’s TPUs and many modern GPUs, speed up matrix operations by doing many in parallel. Even small savings interest researchers: in 2025, Google DeepMind’s AlphaEvolve found an algorithm that multiplies two 4×4 complex matrices with 48 scalar multiplications instead of 49 (a count of multiplications, not a guaranteed speedup) (see AI-discovered algorithms).

Eigenvectors: Finding What Matters

An eigenvector of a matrix is a nonzero vector that the matrix only stretches (or flips), by a factor called the eigenvalue: \(Av = \lambda v\) with \(v \ne 0\). In data science, principal component analysis (PCA) finds the directions of greatest variance in centered data, using eigenvectors of the covariance matrix, so hundreds of features can be summarized by a handful. Greatest variance is not always the same as most useful for prediction.

How to Learn Linear Algebra for AI

  1. Master vectors, coordinates and the Pythagorean theorem.
  2. Practice dot products and matrix multiplication by hand.
  3. Learn what matrices do geometrically (rotation, scaling).
  4. Study eigenvalues and eigenvectors.
  5. Try it in code, for example with Python’s NumPy.

Common Misconceptions

“AI is mostly calculus.” Calculus trains models, but linear algebra is what runs them, step after step.

“Linear algebra is only about solving equations.” It is also the language of vectors, transformations and data.

“You need advanced math to start.” The core operations are multiplication and addition, organized cleverly.

Watch: Linear Algebra, Visualized

If you learn better by watching, these videos are worth your time.

Vectors | Chapter 1, Essence of linear algebra (3Blue1Brown). A beautiful visual start to linear algebra.
Linear transformations and matrices (3Blue1Brown). What matrices do to space, the key idea behind neural network layers.

Check Your Understanding

Five quick questions to see how much of this has stuck. Pick an answer and press Submit; you will see a short explanation either way.

Answer each question and press Submit to see your result right away.

1. What is (1, 2, 3) · (4, 0, −1)?
Found a bug? Let us know!
2. Multiply the matrix with rows (2, 0) and (1, 1) by the vector (3, 4).
Found a bug? Let us know!
3. What does ReLU do to (−1, 7)?
Found a bug? Let us know!
4. A 3×2 matrix times a 2×4 matrix gives what shape?
Found a bug? Let us know!
5. Why are GPUs used for AI?
Found a bug? Let us know!

Frequently Asked Questions

Why is linear algebra important for AI?

AI represents data as vectors and each neural network layer as a matrix that transforms vectors. Running and training models is mostly matrix multiplication, so linear algebra is the main math of AI.

What linear algebra do I need for machine learning?

Vectors, dot products, matrix multiplication, transposes, and later eigenvalues and eigenvectors and matrix decompositions such as SVD.

What is a neural network layer in math?

A function y = ReLU(Wx + b): multiply the input vector by a weight matrix, add a bias vector, then apply a simple nonlinear function.

Is linear algebra harder than calculus?

Many students find early linear algebra easier because its core operations are multiplication and addition; the abstract parts come later.

Can I learn linear algebra in high school?

Yes. Vectors and matrices appear in many high school courses, and the basics need only algebra and coordinates.

Keep Exploring

Sources and Further Reading

Related to This Article

What people say about "Linear Algebra for AI: Vectors, Matrices and Examples - Effortless Math"?

No one replied yet.

Leave a Reply