What is on the exam
Format: 12 multiple-choice questions, plus 2 or 3 short or long problems solved by hand.
| Topic on the midterm list | Where you stand | Study it here |
|---|---|---|
| NumPy: broadcasting, universal functions, vectorization | Quiz 1, done well | Revision sheet |
| Measurement scales: nominal, ordinal, interval, ratio | Quiz 1, done well | Revision sheet |
| Introduction to neural networks | Quiz 1, done well | Revision sheet |
| Single-layer neural network | Quiz 1, done well | Revision sheet |
| Multi-layer perceptrons and back-propagation | Quiz 2, done well | Problem type 1 |
| Multi-layer neural network introduction | Shapes question needed a second look | Problem type 2 |
| Deep neural network: CNN | Not studied yet | Learn CNN |
Best guess at the problems
The review sheet had three problems: a computational graph with backprop, weight-matrix shapes, and softmax with cross-entropy. The first two were also on Quiz 2. Expect those three with new numbers. With seven topics and 12 MCQs, expect one to three CNN questions, most likely at the level of “what is pooling” or “what are CNNs used for”.
The plan
Tick things off as you go. The boxes are saved in this browser.
Tonight · about 75 minutes, then sleep
- 45 min
- 10 min
- 10 min
- 5 min
- 7 h
Tomorrow morning · about 2 hours
- 40 min
- 20 min
- 25 min
- 35 min
- 10 min
Last 15 minutes before the exam
- 8 min
- 5 min
If you only have one hour in total
CNN lesson sections “Start here”, “Convolution in 2-D”, “Pooling” and the cheat sheet (25 min). Then mock exam Part B (25 min). Then the last-look sheet (10 min).
In the exam room
- Show every value. The Quiz 2 key deducted marks when values were missing and gave partial credit for a correct method with a wrong number. Write each forward value and each gradient on the graph.
- State your convention on the shapes question: say what the rows and columns mean and show one dimension check, such as X(5×8)·W₁(8×9) = 5×9.
- Do the MCQs first, about a minute each. They are quick marks and they warm you up. Circle any you are unsure of and come back.
- Check gradients with algebra. For a min or max problem, simplify the active branch and differentiate directly. It takes 30 seconds and catches sign errors.
- A gradient of 0 can be correct. It happens when two branches cancel or when a max, min or ReLU blocks the path.
- Read whether numbers are scores or probabilities. Scores need softmax first. Probabilities go straight into the loss.
Last-look sheet
Backprop on a graph
- Gradient on an input = upstream × local.
- Add: passes it on. Multiply: swap the inputs.
- Max / min / ReLU: all to the winner, 0 to the rest.
- \(a/b\): \(1/b\) and \(-a/b^2\). \(\sigma' = \sigma(1-\sigma)\).
- A variable used twice: add its gradients.
Weight shapes
- net = XW, bias in first row: (inputs + 1) × neurons.
- net = Wx + b: neurons × inputs.
- Next layer’s inputs = this layer’s neurons.
- Batch output: samples × last-layer neurons.
- Activations never change a shape.
Softmax and loss
- \(\hat{y}_k = e^{z_k} / \sum_j e^{z_j}\)
- \(L = -\sum_k y_k \ln \hat{y}_k\); one-hot → \(-\ln \hat{y}_{\text{true}}\)
- \(e = 2.718\), \(e^2 = 7.389\), \(e^3 = 20.09\)
- Scores [1, 2] → [0.269, 0.731]
- MSE → regression. Cross-entropy → sigmoid or softmax.
Activations
- Sigmoid (0, 1): vanishing gradients, binary output layer.
- Tanh (−1, 1): zero-centred, still vanishes.
- ReLU [0, ∞): reduces vanishing gradients, can die.
- Leaky ReLU: small slope for negatives, no dying.
- No activation → the network is one linear layer.
Single layer
- Bias shifts the boundary. No bias → through the origin.
- 2 inputs → line. 3 inputs → plane.
- Perceptron: step, not differentiable, update only on mistakes, needs linear separability. No XOR.
- Loss going up → lower the learning rate.
Scales and NumPy
- Nominal → ordinal → interval → ratio adds identity, order, equal intervals, true zero.
- °C is interval. Height, weight, age are ratio.
*is element-wise;np.dotis the matrix product.- Broadcasting: prepend 1s, stretch size-1 axes.
CNN
- Convolution = slide a filter, local dot product. Output size N − F + 1.
- One filter → one activation map. # filters = # output channels.
- Pooling: max or average, no weights, shrinks size, slight translation invariance.
- Conv → activation → pool → fully connected.
- LeNet: digits, 10. AlexNet: ImageNet, 1000.
ML basics
- Four components: data, model, objective function, optimization algorithm.
- Supervised = labeled. Unsupervised = unlabeled. Reinforcement = rewards.
- Dendrites, synapses, soma, axon hillock, axon ↔ inputs, weights, sum, activation, output.
- Backprop = chain rule, output to input.
You already did the hard part by getting the quizzes right. Tonight is one new topic and some rehearsal. Good luck tomorrow.