Summation notation
Learn to read the Σ sign and work out sums, means and dot products, on paper and in Python. Most formulas in machine learning are written this way, including the mean squared error.
Warm-up
One question before the lesson. Choose an answer and check it.
x is the list 1, 2 and 3. What does Σ xᵢ² mean?
Show the answer
C: Square each number, then add the squares: 1² + 2² + 3² = 14. Σ means “add up”, and what follows it, xᵢ², is worked out for each number first. So square each number, then add: 1 + 4 + 9 = 14. This lesson shows how to read Σ and the small letters around it.
Step 1 A list of numbers
A list is a set of numbers in order. This lesson uses one list, called x, with 5 numbers: 3, 5, 2, 6 and 4.
Each number has a position. The first number is at position 1, the second at position 2, and so on. To name one number, write the list’s name with its position small, below and to the right: x₁ (say “x one”) is the first number and x₅ the last.
| Position i | xᵢ |
|---|---|
| 1 | 3 |
| 2 | 5 |
| 3 | 2 |
| 4 | 6 |
| 5 | 4 |
Formulas use a letter instead of a fixed position: xᵢ means “the number at position i”. When i is 1, xᵢ is x₁. When i is 4, xᵢ is x₄. The letter n stands for how many numbers the list has, here 5.
| Symbol | Say | Means | In this lesson |
|---|---|---|---|
| n | How many numbers the list has. | 5 | |
| i | A position in the list: 1 for the first number, 2 for the second, up to n. | 1 to 5 | |
| x i | The number at position i. | x₃ = 2 |
Try it yourself
Change the numbers and pick what to add up. The sum is written out term by term, so you can check each one.
Σ = 3² + 5² + 2² + 6² + 4²
= 9 + 25 + 4 + 36 + 16
= 90
Each number is squared first, then the squares are added. Squaring the total instead would give 20² = 400.
Practice problems
Work each problem out on paper, then type your answer and press Check. Every problem has hints and a full solution.
Score: 0 of 10 points
Problem 1
1 pointx is the list 4, 1, 7 and 3. What is Σ xᵢ, for i from 1 to 4?
Hint 1
Σ xᵢ means: add up every number in the list.
Solution
Σ xᵢ = 4 + 1 + 7 + 3 = 15
Problem 2
2 pointsFor the same list, what is Σ xᵢ²?
Hint 1
Work out xᵢ² for each number first, then add.
Hint 2
Square each number: multiply it by itself.
Solution
Σ xᵢ² = 4² + 1² + 7² + 3²
= 16 + 1 + 49 + 9 = 75
Problem 3
2 pointsWhat is the mean of the same list, x̄?
Hint 1
x̄ = (1 ÷ n) Σ xᵢ: add the numbers up, then divide by how many there are.
Solution
x̄ = (4 + 1 + 7 + 3) ÷ 4
= 15 ÷ 4 = 3.75
Problem 4
2 pointsy is the list 2, 0, 1 and 5. What is Σ xᵢyᵢ, with x the same list as before?
Hint 1
At each position, multiply the number in x by the number in y.
Hint 2
Then add up the products.
Solution
Σ xᵢyᵢ = 4 × 2 + 1 × 0 + 7 × 1 + 3 × 5
= 8 + 0 + 7 + 15 = 30
Problem 5
3 pointsA model predicted 5, 1, 6 and 2 where the real values are the list x, 4, 1, 7 and 3. What is its mean squared error, (1 ÷ n) Σ (yᵢ − ŷᵢ)²?
Hint 1
Work from the inside out: the error yᵢ − ŷᵢ at each position first.
Hint 2
Square each error, add the squares, then divide by n.
Solution
errors: 4 − 5 = −1, 1 − 1 = 0, 7 − 6 = 1, 3 − 2 = 1
squares: 1, 0, 1, 1
MSE = (1 + 0 + 1 + 1) ÷ 4 = 3 ÷ 4 = 0.75
Programming exercise
Turn each formula into Python with a for loop. Save sums.py and test_sums.py in the same folder, fill in each function in sums.py, and run the tests:
python test_sums.py
"""Summation notation: programming exercise. Turn each formula into Python with a for loop, then run the tests from thisfolder: python test_sums.py Write each sum yourself: start a total at 0 and add to it in a loop. Don't usePython's built-in sum() here.""" def total(xs): """Return Σ xᵢ: every number in xs, added up.""" raise NotImplementedError def mean(xs): """Return x̄: the total of xs divided by how many numbers it has.""" raise NotImplementedError def sum_of_squares(xs): """Return Σ xᵢ²: each number squared, then added up.""" raise NotImplementedError def dot(xs, ys): """Return Σ xᵢyᵢ: the two numbers at each position multiplied, then added up. xs and ys have the same length. """ raise NotImplementedError def mse(ys, predictions): """Return the mean squared error: (1/n) Σ (yᵢ − ŷᵢ)². ys are the real values and predictions are the ŷ values, the same length. """ raise NotImplementedError Stuck? Every Σ is a loop: start a total at 0, go through the positions, and add the expression for each one. The Solution tab has one way to write it.
In practice: Σ in code
A Σ is a for loop. Σ xᵢ² for i from 1 to n becomes: set a total to 0, then for each x in the list, add x × x to the total. The programming exercise above is exactly this.
Watch the positions. Maths counts from 1 and Python counts from 0, so x₁ is xs[0] and xₙ is xs[n − 1]. Mixing the two up skips the first number or runs past the end of the list, a classic bug when turning a formula into code.
In real projects these loops come ready made in NumPy, the library most Python machine learning code is built on: np.sum(x ** 2) is Σ xᵢ², np.mean(x) is x̄, and np.dot(x, y) is Σ xᵢyᵢ. Later modules use them.
Test your knowledge
01The list x is 2, 4 and 6. What is Σ xᵢ, and what is the mean x̄?Show answer
Σ xᵢ = 2 + 4 + 6 = 12. There are 3 numbers, so x̄ = 12 ÷ 3 = 4.
02Is Σ xᵢ² the same as (Σ xᵢ)²? Try it with x = 1, 2.Show answer
No. Σ xᵢ² squares each number first, then adds: 1² + 2² = 1 + 4 = 5. (Σ xᵢ)² adds first, then squares the total: (1 + 2)² = 3² = 9. Where the brackets and the square sit decides the order. The exact solution in the Linear regression lesson uses both.
03x is 1, 2, 3 and y is 4, 5, 6. What is the dot product Σ xᵢyᵢ?Show answer
1 × 4 + 2 × 5 + 3 × 6 = 4 + 10 + 18 = 32.
04In Python, xs = [3, 5, 2]. Which element is x₁, and which is xₙ?Show answer
x₁ is xs[0], which is 3. There are n = 3 numbers, so xₙ is x₃, which is xs[2], which is 2. Maths counts positions from 1 and Python from 0, so xᵢ is xs[i − 1].
Exit ticket
One last question on the main idea of the lesson.
For a list of n numbers, what does (1 ÷ n) Σ xᵢ work out?
Show the answer
A: The mean of the list. Σ xᵢ adds up the n numbers, and dividing by n turns the sum into their mean, x̄. The mean squared error is built the same way: a Σ of squared errors, then ÷ n.