Module 0 · Foundations

Summation notation
Vectors and the dot product
1. 01What a vector is
2. 02Adding vectors
3. 03Multiplying by a number
4. 04The length of a vector
5. 05The dot product
6. 06What the sign tells you
7. 07Cosine similarity
Matrices and the matrix product
Derivatives
The chain rule
Partial derivatives and gradients
Probability
Statistics and confidence intervals
The shell, git and a debugger

Module 1 · How machines learn

Linear regression
Overfitting and validation
03Regularisation
04Logistic regression
05Precision, recall and other metrics
06k-nearest neighbours and naive Bayes
07Decision trees
08Random forests and gradient boosting
09Support vector machines and kernels
10k-means and Gaussian mixtures
11Eigenvectors, the SVD and PCA

Vectors and the dot product

A vector is a list of numbers that can be drawn as an arrow. Learn to add and scale vectors, measure their length, and use the dot product to tell how alike two directions are: the idea behind neural networks and search by meaning.

Lesson 7 stepsExercise Python, 5 functionsQuiz 4 questions

Step 1 What a vector is

01/07

A vector is a list of numbers, written in brackets: v = (3, 4). Each number is a component, named the way list items were named in Summation notation: v₁ = 3 and v₂ = 4.

v = (v_{1}, v_{2})

Symbol	Say	Means	In this lesson
$v$	v	A vector: a list of numbers, here two of them.	(3, 4)
$v_{i}$	v i	Component i of v: the number at position i.	v₁ = 3, v₂ = 4

A vector with two components can be drawn as an arrow on a grid. Start at the origin, the point (0, 0). Go v₁ across and v₂ up, and draw the arrow to where you land: 3 across and 4 up. The arrow has a direction and a length, and both carry meaning.

Vectors in AI usually have far more components than two. A word or a sentence can be turned into a vector of hundreds or thousands of numbers, called an embedding. It can’t be drawn, but every rule in this lesson works the same way for any number of components.

Try it yourself

Drag the tips of the two arrows. Their sum, their lengths, their dot product and the angle between them are worked out below as you move.

v w v + w

v = (3, 4), w = (2, −1)

v + w = (3 + 2, 4 + (−1)) = (5, 3)

||v|| = √(3² + 4²) = √25 = 5

||w|| = √(2² + (−1)²) = √5 = 2.2361

v · w = 3 × 2 + 4 × (−1) = 2

cos θ = 2 ÷ (5 × 2.2361) = 0.1789

The dot product is positive: the arrows are less than a right angle apart, about 79.7°.

Lengths and the cosine are rounded to 4 decimal places.

Programming exercise

Write the vector operations in plain Python, with lists for vectors. Save vectors.py and test_vectors.py in the same folder, fill in each function in vectors.py, and run the tests:

python test_vectors.py

"""Vectors and the dot product: programming exercise. Write each vector operation in plain Python, with a list for a vector, thenrun the tests from this folder:     python test_vectors.py Every function works for vectors of any length. Use a loop over thepositions; math.sqrt gives a square root.""" import math  def add(u, v):    """Return u + v: the components in the same position added, as a new list."""    raise NotImplementedError  def scale(c, v):    """Return c × v: every component of v multiplied by the number c, as a new list."""    raise NotImplementedError  def length(v):    """Return ‖v‖: the square root of the sum of the squared components."""    raise NotImplementedError  def dot(u, v):    """Return u · v: the components in the same position multiplied, then added up."""    raise NotImplementedError  def cosine_similarity(u, v):    """Return cos θ: the dot product divided by both lengths multiplied together.     1 means the same direction, 0 a right angle, -1 opposite directions.    """    raise NotImplementedError

Stuck? Each function is one loop over the positions, like the sums in Summation notation. math.sqrt gives a square root. The Solution tab has one way to write it.

In practice: vectors in AI

Search by meaning. An embedding model turns a piece of text into a vector, so that texts with similar meanings point in similar directions. To find the documents closest to a question, a search system turns the question into a vector too and ranks every document by cosine similarity. This is the retrieval step of the AI applications built in Module 7.

One neuron is a dot product. A neuron in a neural network multiplies its inputs by its weights, adds them up and adds one more number, the bias: w · x + b. It is the line y = m·x + b from Linear regression, with vectors in place of single numbers. Module 2 builds a network out of them.

Check the shapes. A dot product of two vectors with different numbers of components has no meaning, and libraries refuse it: NumPy stops with “shapes (3,) and (4,) not aligned”. When you see that error, print the length of each vector first.

Test your knowledge

01u = (1, 2) and v = (3, 4). What are u + v and 2u?Show answer
u + v = (1 + 3, 2 + 4) = (4, 6). 2u = (2 × 1, 2 × 2) = (2, 4).
02What is the length of (6, 8)?Show answer
√(6² + 8²) = √(36 + 64) = √100 = 10. It is (3, 4) multiplied by 2, so it is twice as long: 2 × 5 = 10.
03What is (2, 3) · (3, −2), and what does the answer tell you?Show answer
2 × 3 + 3 × (−2) = 6 − 6 = 0. A dot product of 0 means the two arrows are at a right angle.
04Why do search systems compare embeddings with cosine similarity instead of the plain dot product?Show answer
The dot product grows when either vector is longer, so a long vector can score high against everything. Cosine similarity divides by both lengths, so only the direction counts. If every vector is first scaled to length 1, the dot product and the cosine similarity are the same number, which is why many systems store vectors that way.