Vectors and the dot product
A vector is a list of numbers that can be drawn as an arrow. Learn to add and scale vectors, measure their length, and use the dot product to tell how alike two directions are: the idea behind neural networks and search by meaning.
Warm-up
One question before the lesson. Choose an answer and check it.
An arrow goes 3 squares across and 4 squares up. How long is it?
Show the answer
B: 5, by Pythagoras: √(3² + 4²) = √25 = 5. The arrow is the longest side of a right-angled triangle whose other sides are 3 and 4. By Pythagoras its length is √(9 + 16) = √25 = 5. This lesson calls that the length of a vector.
Step 1 What a vector is
A vector is a list of numbers, written in brackets: v = (3, 4). Each number is a component, named the way list items were named in Summation notation: v₁ = 3 and v₂ = 4.
| Symbol | Say | Means | In this lesson |
|---|---|---|---|
| v | A vector: a list of numbers, here two of them. | (3, 4) | |
| v i | Component i of v: the number at position i. | v₁ = 3, v₂ = 4 |
A vector with two components can be drawn as an arrow on a grid. Start at the origin, the point (0, 0). Go v₁ across and v₂ up, and draw the arrow to where you land: 3 across and 4 up. The arrow has a direction and a length, and both carry meaning.
Vectors in AI usually have far more components than two. A word or a sentence can be turned into a vector of hundreds or thousands of numbers, called an embedding. It can’t be drawn, but every rule in this lesson works the same way for any number of components.
Try it yourself
Drag the tips of the two arrows. Their sum, their lengths, their dot product and the angle between them are worked out below as you move.
v w v + w
v = (3, 4), w = (2, −1)
v + w = (3 + 2, 4 + (−1)) = (5, 3)
||v|| = √(3² + 4²) = √25 = 5
||w|| = √(2² + (−1)²) = √5 = 2.2361
v · w = 3 × 2 + 4 × (−1) = 2
cos θ = 2 ÷ (5 × 2.2361) = 0.1789
The dot product is positive: the arrows are less than a right angle apart, about 79.7°.
Lengths and the cosine are rounded to 4 decimal places.
Practice problems
Work each problem out on paper, then type your answer and press Check. Every problem has hints and a full solution.
Score: 0 of 10 points
Problem 1
1 pointu = (2, −1) and v = (4, 3). What is the dot product u · v?
Hint 1
Multiply the components in the same position, then add the products.
Solution
u · v = 2 × 4 + (−1) × 3
= 8 − 3 = 5
Problem 2
2 pointsWhat is the length of (5, 12)?
Hint 1
Square each component, add the squares, then take the square root.
Solution
||(5, 12)|| = √(5² + 12²)
= √(25 + 144) = √169 = 13
Problem 3
2 pointsWhat is the length of u = (2, −1)? Give it to 2 decimal places.
Hint 1
Square each component, add the squares, then take the square root.
Hint 2
A negative number squared is positive.
Solution
||u|| = √(2² + (−1)²)
= √(4 + 1) = √5 = 2.2361
Problem 4
2 pointsWhat is the cosine similarity of u = (2, −1) and v = (4, 3)? Give it to 2 decimal places.
Hint 1
cos θ = (u · v) ÷ (||u|| × ||v||).
Hint 2
u · v is Problem 1’s answer and ||u|| is Problem 3’s. ||v|| = √(4² + 3²) = 5.
Solution
cos θ = 5 ÷ (2.2361 × 5)
= 5 ÷ 11.1803 = 0.4472
Problem 5
3 pointsFor which number k is (2, k) at a right angle to (3, 4)?
Hint 1
Two vectors are at a right angle when their dot product is 0.
Hint 2
(2, k) · (3, 4) = 2 × 3 + k × 4. Set it equal to 0.
Solution
2 × 3 + k × 4 = 0
6 + 4k = 0
4k = −6
k = −6 ÷ 4 = −1.5
Programming exercise
Write the vector operations in plain Python, with lists for vectors. Save vectors.py and test_vectors.py in the same folder, fill in each function in vectors.py, and run the tests:
python test_vectors.py
"""Vectors and the dot product: programming exercise. Write each vector operation in plain Python, with a list for a vector, thenrun the tests from this folder: python test_vectors.py Every function works for vectors of any length. Use a loop over thepositions; math.sqrt gives a square root.""" import math def add(u, v): """Return u + v: the components in the same position added, as a new list.""" raise NotImplementedError def scale(c, v): """Return c × v: every component of v multiplied by the number c, as a new list.""" raise NotImplementedError def length(v): """Return ‖v‖: the square root of the sum of the squared components.""" raise NotImplementedError def dot(u, v): """Return u · v: the components in the same position multiplied, then added up.""" raise NotImplementedError def cosine_similarity(u, v): """Return cos θ: the dot product divided by both lengths multiplied together. 1 means the same direction, 0 a right angle, -1 opposite directions. """ raise NotImplementedError Stuck? Each function is one loop over the positions, like the sums in Summation notation. math.sqrt gives a square root. The Solution tab has one way to write it.
In practice: vectors in AI
Search by meaning. An embedding model turns a piece of text into a vector, so that texts with similar meanings point in similar directions. To find the documents closest to a question, a search system turns the question into a vector too and ranks every document by cosine similarity. This is the retrieval step of the AI applications built in Module 12.
One neuron is a dot product. A neuron in a neural network multiplies its inputs by its weights, adds them up and adds one more number, the bias: w · x + b. It is the line y = m·x + b from Linear regression, with vectors in place of single numbers. Module 7 builds a network out of them.
Check the shapes. A dot product of two vectors with different numbers of components has no meaning, and libraries refuse it: NumPy stops with “shapes (3,) and (4,) not aligned”. When you see that error, print the length of each vector first.
Test your knowledge
01u = (1, 2) and v = (3, 4). What are u + v and 2u?Show answer
u + v = (1 + 3, 2 + 4) = (4, 6). 2u = (2 × 1, 2 × 2) = (2, 4).
02What is the length of (6, 8)?Show answer
√(6² + 8²) = √(36 + 64) = √100 = 10. It is (3, 4) multiplied by 2, so it is twice as long: 2 × 5 = 10.
03What is (2, 3) · (3, −2), and what does the answer tell you?Show answer
2 × 3 + 3 × (−2) = 6 − 6 = 0. A dot product of 0 means the two arrows are at a right angle.
04Why do search systems compare embeddings with cosine similarity instead of the plain dot product?Show answer
The dot product grows when either vector is longer, so a long vector can score high against everything. Cosine similarity divides by both lengths, so only the direction counts. If every vector is first scaled to length 1, the dot product and the cosine similarity are the same number, which is why many systems store vectors that way.
Exit ticket
One last question on the main idea of the lesson.
The dot product of two vectors is negative. How do they point?
Show the answer
B: More than a right angle apart: roughly opposite ways. A negative dot product means the angle between them is more than 90°. A dot product of 0 means exactly a right angle, and a positive one means less than a right angle apart.