NumPy: Array Operations for ML

NumPy is the lingua franca of scientific Python. Every ML library (PyTorch, TensorFlow, scikit-learn) borrows or interoperates with its array semantics.

This piece: enough NumPy to read and write ML code.

Why NumPy

Python lists are flexible but slow. NumPy arrays are:

• Fast (C under the hood, vectorized operations)
• Memory-efficient (contiguous, typed)
• Multi-dimensional (1D vectors, 2D matrices, ND tensors)

import numpy as np

# 100M element addition
a = list(range(100_000_000))
b = list(range(100_000_000))

# Python loops: ~30 seconds
result = [x + y for x, y in zip(a, b)]

# NumPy: ~0.3 seconds → 100× faster
arr_a = np.array(a)
arr_b = np.array(b)
result = arr_a + arr_b

The 100× speedup is why ML uses NumPy (and PyTorch tensors are basically NumPy with autograd).

Creating Arrays

import numpy as np

# From lists
a = np.array([1, 2, 3])
b = np.array([[1, 2], [3, 4]])     # 2D matrix

# From shape
zeros = np.zeros((3, 4))            # 3×4 zeros
ones = np.ones((2, 2))
empty = np.empty((100,))            # uninitialized (fast)
identity = np.eye(3)                # 3×3 identity matrix

# Sequences
arange = np.arange(0, 10, 2)        # [0, 2, 4, 6, 8]
linspace = np.linspace(0, 1, 5)     # [0, 0.25, 0.5, 0.75, 1]

# Random
np.random.seed(42)
rand = np.random.rand(3, 4)         # uniform [0, 1]
randn = np.random.randn(3, 4)       # standard normal
randint = np.random.randint(0, 10, size=(3, 4))

Shapes and Indexing

a = np.array([[1, 2, 3], [4, 5, 6]])

a.shape       # (2, 3)
a.ndim        # 2 (number of dimensions)
a.size        # 6 (total elements)
a.dtype       # int64

# Indexing
a[0, 0]       # 1
a[1, 2]       # 6
a[:, 0]       # [1, 4] (column 0)
a[0, :]       # [1, 2, 3] (row 0)
a[0:2, 1:3]   # [[2, 3], [5, 6]] (sub-matrix)

# Boolean mask
mask = a > 3
a[mask]       # [4, 5, 6]

# Fancy indexing
indices = [0, 2]
a[:, indices] # columns 0 and 2

Math Operations

NumPy operations are element-wise by default:

a = np.array([1, 2, 3])
b = np.array([10, 20, 30])

a + b         # [11, 22, 33]
a * b         # [10, 40, 90] (element-wise multiply)
a ** 2        # [1, 4, 9]
np.exp(a)     # [e, e², e³]
np.log(a)     # [0, 0.693, 1.098]
np.sqrt(a)    # [1, 1.414, 1.732]

For true matrix multiplication:

A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])

A @ B         # matrix multiply: [[19, 22], [43, 50]]
A.dot(B)      # same
A * B         # element-wise (different!): [[5, 12], [21, 32]]

@ is matrix multiply, * is element-wise. Confusing them is a top NumPy bug.

Aggregations

a = np.array([[1, 2, 3], [4, 5, 6]])

a.sum()        # 21 (all elements)
a.sum(axis=0)  # [5, 7, 9] (sum each column)
a.sum(axis=1)  # [6, 15] (sum each row)

a.mean()       # 3.5
a.std()        # 1.707
a.min(), a.max()
a.argmin(), a.argmax()  # indices of min/max

axis=0 reduces rows, axis=1 reduces columns. Remember by: axis is the one that disappears.

Reshaping

a = np.arange(12)         # [0, 1, ..., 11]

a.reshape(3, 4)            # 3×4 matrix
a.reshape(2, 3, 2)         # 2×3×2 cube
a.reshape(-1, 4)           # let NumPy infer first dim → 3×4

# Common: flatten
a.ravel()                  # 1D view
a.flatten()                # 1D copy

# Add / remove axis
a = np.array([1, 2, 3])
a[:, np.newaxis]           # [[1], [2], [3]] — 1D → column vector
a[np.newaxis, :]           # [[1, 2, 3]] — 1D → row vector

# Transpose
A = np.array([[1, 2], [3, 4]])
A.T                        # transpose

reshape(-1, ...) is essential — auto-compute one dimension.

Broadcasting

The killer NumPy feature. Apply operations between different-shaped arrays without manual loops:

a = np.array([[1, 2, 3],
              [4, 5, 6]])  # shape (2, 3)
b = np.array([10, 20, 30])  # shape (3,)

a + b
# [[11, 22, 33],
#  [14, 25, 36]]
# b is "broadcast" to each row

Rule: shapes are compatible if matched from the right, each dim is either equal or 1.

(2, 3) and (3,)    → broadcast b to (1, 3) then to (2, 3)
(2, 3) and (2, 1)  → broadcast last dim
(2, 3) and (4,)    → ERROR (3 ≠ 4)

Broadcasting avoids explicit loops = fast + clean.

Common ML Operations

Normalize data

X = np.random.randn(1000, 50)   # 1000 samples, 50 features

mean = X.mean(axis=0)             # mean per feature
std = X.std(axis=0)
X_normalized = (X - mean) / std    # broadcasting!

One-hot encoding

labels = np.array([0, 1, 2, 1, 0])
n_classes = 3

one_hot = np.eye(n_classes)[labels]
# [[1, 0, 0],
#  [0, 1, 0],
#  [0, 0, 1],
#  [0, 1, 0],
#  [1, 0, 0]]

Train/test split (simple)

n = X.shape[0]
indices = np.random.permutation(n)
split = int(n * 0.8)

train_idx = indices[:split]
test_idx = indices[split:]

X_train = X[train_idx]
X_test = X[test_idx]

Matrix multiplication (linear layer)

batch_size = 32
in_dim = 100
out_dim = 10

X = np.random.randn(batch_size, in_dim)
W = np.random.randn(in_dim, out_dim) * 0.01   # init small
b = np.zeros(out_dim)

Y = X @ W + b                # (32, 10)

This is literally what a linear layer does.

Common Gotchas

1. Integer overflow

a = np.array([100, 200, 300], dtype=np.int8)   # max 127
a * 2   # OVERFLOWS — wraps around

For ML, use float32 or float64 mostly. Be careful with int types.

2. View vs Copy

a = np.arange(10)
b = a[3:7]        # view — modifying b changes a!
b[0] = 999
a   # [0, 1, 2, 999, 4, 5, 6, 7, 8, 9]

c = a[3:7].copy()  # explicit copy

3. NaN and Inf

np.log(0)        # -inf
np.log(-1)       # nan + warning

np.isnan(arr).any()    # check
np.isfinite(arr).all()
np.nan_to_num(arr)     # replace nan with 0

ML training often produces NaN — usually too-large learning rate or bad data.

Convert to / from PyTorch

import torch

# NumPy → PyTorch (shares memory if possible)
t = torch.from_numpy(arr)

# PyTorch → NumPy
arr = t.numpy()    # only if on CPU and no grad
arr = t.detach().cpu().numpy()  # safest

You’ll see this a lot in real code.

Resources

• NumPy docs — numpy.org/doc — comprehensive
• From Python to NumPy by Nicolas Rougier — free book
• 100 numpy exercises GitHub repo — practice problems

Next recommended: L1-09 Pandas or L1-10 PyTorch.