The shell, git and a debugger
The tools engineers use every day: the shell to run programs and move between folders, git to save every version of your work, a debugger to stop a program and look inside it, and a profiler to find what is slow. Every lesson from here on uses them.
Step 1 The shell
A shell is a program that runs other programs. You type a command, press Enter, and it prints the result. Engineers use it to run code, install tools, look at files and work on servers. In this course every command shown after a $ is typed into a shell. The $ is the prompt, which the shell prints, so you don’t type it.
On macOS and Linux, open the Terminal app. On Windows, install WSL (Windows Subsystem for Linux), which gives you a real Linux shell: open PowerShell as administrator, run wsl --install, and restart. The course’s commands are written for that shell.
Here is a first session. Each line after a $ is a command, and the lines under it are what it printed:
$ pwd
/home/you
$ ls
course
$ cd course
$ ls
data line.py
$ mkdir notes
$ ls
data line.py notes
| Command | What it does | What it printed |
|---|---|---|
| pwd | Print the folder you are in. | /home/you |
| ls | List what is in a folder. | course |
| cd course | Change to another folder. | nothing |
| mkdir notes | Make a new folder. | nothing |
Many commands print nothing when they work, as cd and mkdir did. The shell speaks up when there is something to show, or when something went wrong.
Try it yourself
A practice shell with a small set of folders. Type the commands from the lesson and work through the tasks. Nothing you type here touches your computer.
- Print the folder you are in:
pwd - Go into the course folder:
cd course - List what is in it:
ls - Make a folder called notes inside course:
mkdir notes - Make an empty file called todo.txt inside notes:
touch notes/todo.txt - Go back to your home folder:
cd ~
0 of 6 done. Type help to see every command this shell knows.
Programming exercise
metrics.py has three functions, and each has one bug. Run the tests to see which fail, then stop inside each failing function with breakpoint(), look at the values, and fix it. Save metrics.py and test_metrics.py in the same folder and run:
python test_metrics.py
"""The shell, git and a debugger: programming exercise. Each function below has one bug. Run the tests from this folder: python test_metrics.py For each test that fails, put breakpoint() at the top of that function, runthe tests again, and use the debugger to find the line that goes wrong:n runs the next line, p followed by a name prints it, c carries on.Fix the bug, delete the breakpoint() line, and run the tests again.""" def mean(xs): """Return the mean of xs: their sum divided by how many there are.""" total = sum(xs) return total / (len(xs) - 1) def mse(ys, preds): """Return the mean squared error: the average of (pred - y) squared, over every pair.""" total = 0.0 for i in range(1, len(ys)): total += (preds[i] - ys[i]) ** 2 return total / len(ys) def accuracy(labels, preds): """Return the share of predictions that match their labels.""" right = sum(1 for y, p in zip(labels, preds) if y != p) return right / len(labels) Stuck? Put breakpoint() on the first line of a function, run the tests, then type n to go one line at a time and p followed by a name to print it. The Solution tab shows each fix.
In practice: tools at work
Commit small and often, with messages that say why. A commit that does one thing is easy to review and easy to undo with git revert, which adds a new commit that reverses it. Before every commit, read git diff to check exactly what is going in.
Never commit secrets. An API key or password committed to a repository stays in its history even after you delete the file. Keep secrets in environment variables or in a .env file listed in .gitignore, the file that tells git what to leave out.
Use the debugger before adding print statements. Editors such as VS Code have one built in: click in the margin to set a breakpoint, run, then step through and hover over any variable to see its value. Before a long training run, stop inside the first step this way and check the shape and first few values of every array.
Test your knowledge
01You are in /home/you/course. You type cd data, then cd .. . Where are you now?Show answer
Back in /home/you/course. cd data moves into the data folder inside it, and .. always means one folder up.
02What is the difference between git add and git commit?Show answer
git add chooses which changes go into the next commit, which is called staging them. git commit saves the staged changes as a new version in the repository’s history, with a message and an id.
03Your program has stopped at breakpoint(). How do you see the value of a variable called loss, run one more line, and then let the program carry on?Show answer
Type p loss to print it, n to run the next line, and c to continue until the program ends or reaches another breakpoint.
04A training script is slow. How do you find which function takes the time?Show answer
Run it under the profiler: python -m cProfile -s cumulative train.py. It lists every function with how many times it was called and how long it took, the slowest first.