Summation notation
Learn to read the Σ sign and work out sums, means and dot products, on paper and in Python. Most formulas in machine learning are written this way, including the mean squared error.
Step 1 A list of numbers
A list is a set of numbers in order. This lesson uses one list, called x, with 5 numbers: 3, 5, 2, 6 and 4.
Each number has a position. The first number is at position 1, the second at position 2, and so on. To name one number, write the list’s name with its position small, below and to the right: x₁ (say “x one”) is the first number and x₅ the last.
| Position i | xᵢ |
|---|---|
| 1 | 3 |
| 2 | 5 |
| 3 | 2 |
| 4 | 6 |
| 5 | 4 |
Formulas use a letter instead of a fixed position: xᵢ means “the number at position i”. When i is 1, xᵢ is x₁. When i is 4, xᵢ is x₄. The letter n stands for how many numbers the list has, here 5.
| Symbol | Say | Means | In this lesson |
|---|---|---|---|
| n | How many numbers the list has. | 5 | |
| i | A position in the list: 1 for the first number, 2 for the second, up to n. | 1 to 5 | |
| x i | The number at position i. | x₃ = 2 |
Try it yourself
Change the numbers and pick what to add up. The sum is written out term by term, so you can check each one.
Σ = 3² + 5² + 2² + 6² + 4²
= 9 + 25 + 4 + 36 + 16
= 90
Each number is squared first, then the squares are added. Squaring the total instead would give 20² = 400.
Programming exercise
Turn each formula into Python with a for loop. Save sums.py and test_sums.py in the same folder, fill in each function in sums.py, and run the tests:
python test_sums.py
"""Summation notation: programming exercise. Turn each formula into Python with a for loop, then run the tests from thisfolder: python test_sums.py Write each sum yourself: start a total at 0 and add to it in a loop. Don't usePython's built-in sum() here.""" def total(xs): """Return Σ xᵢ: every number in xs, added up.""" raise NotImplementedError def mean(xs): """Return x̄: the total of xs divided by how many numbers it has.""" raise NotImplementedError def sum_of_squares(xs): """Return Σ xᵢ²: each number squared, then added up.""" raise NotImplementedError def dot(xs, ys): """Return Σ xᵢyᵢ: the two numbers at each position multiplied, then added up. xs and ys have the same length. """ raise NotImplementedError def mse(ys, predictions): """Return the mean squared error: (1/n) Σ (yᵢ − ŷᵢ)². ys are the real values and predictions are the ŷ values, the same length. """ raise NotImplementedError Stuck? Every Σ is a loop: start a total at 0, go through the positions, and add the expression for each one. The Solution tab has one way to write it.
In practice: Σ in code
A Σ is a for loop. Σ xᵢ² for i from 1 to n becomes: set a total to 0, then for each x in the list, add x × x to the total. The programming exercise above is exactly this.
Watch the positions. Maths counts from 1 and Python counts from 0, so x₁ is xs[0] and xₙ is xs[n − 1]. Mixing the two up skips the first number or runs past the end of the list, a classic bug when turning a formula into code.
In real projects these loops come ready made in NumPy, the library most Python machine learning code is built on: np.sum(x ** 2) is Σ xᵢ², np.mean(x) is x̄, and np.dot(x, y) is Σ xᵢyᵢ. Later modules use them.
Test your knowledge
01The list x is 2, 4 and 6. What is Σ xᵢ, and what is the mean x̄?Show answer
Σ xᵢ = 2 + 4 + 6 = 12. There are 3 numbers, so x̄ = 12 ÷ 3 = 4.
02Is Σ xᵢ² the same as (Σ xᵢ)²? Try it with x = 1, 2.Show answer
No. Σ xᵢ² squares each number first, then adds: 1² + 2² = 1 + 4 = 5. (Σ xᵢ)² adds first, then squares the total: (1 + 2)² = 3² = 9. Where the brackets and the square sit decides the order. The exact solution in the Linear regression lesson uses both.
03x is 1, 2, 3 and y is 4, 5, 6. What is the dot product Σ xᵢyᵢ?Show answer
1 × 4 + 2 × 5 + 3 × 6 = 4 + 10 + 18 = 32.
04In Python, xs = [3, 5, 2]. Which element is x₁, and which is xₙ?Show answer
x₁ is xs[0], which is 3. There are n = 3 numbers, so xₙ is x₃, which is xs[2], which is 2. Maths counts positions from 1 and Python from 0, so xᵢ is xs[i − 1].