← Back to the portfolio RSS

· 3 min read

Keras: my first neural network

Step 4: dense networks, learning curves, convolutions and transfer learning.

Neural networks take over where classic machine learning plateaus: images, text, sound, anything that doesn’t naturally fit into a table of columns. Keras is the ideal way in: a short, readable API that shows the ideas without drowning you in details. Since Keras 3, the same code runs on TensorFlow, PyTorch or JAX, your choice.

What a neural network really is

A “dense” layer computes exactly what we saw with NumPy, followed by a non-linearity:

h = r e l u ( X W + b ) h = \mathrm{relu}(XW + b)

Stacking several layers makes it possible to learn non-linear relationships. Training adjusts W W and b b by gradient descent, the same idea as in my article on linear regression, generalised to millions of parameters thanks to backpropagation.

A first network: recognising handwritten digits

The MNIST dataset: 70,000 images of 28 × 28-pixel digits.

import keras
from keras import layers

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.reshape(-1, 784).astype("float32") / 255   # flatten, scale to [0, 1]
x_test = x_test.reshape(-1, 784).astype("float32") / 255

model = keras.Sequential([
    keras.Input(shape=(784,)),
    layers.Dense(128, activation="relu"),
    layers.Dropout(0.2),
    layers.Dense(10, activation="softmax"),   # one probability per digit
])

model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"],
)

history = model.fit(x_train, y_train, epochs=10, batch_size=64, validation_split=0.1)
model.evaluate(x_test, y_test)   # around 98% accuracy

Every element deserves to be understood, not just copied:

  • Dense(10, activation="softmax"): ten outputs that sum to 1, one probability per class.
  • sparse_categorical_crossentropy: the right loss when labels are integers (0 to 9).
  • adam: a gradient descent that adapts its step size for each parameter.
  • Dropout(0.2): randomly switches off 20% of neurons during training, to fight overfitting.
  • validation_split: keeps 10% of the training data aside to monitor generalisation at each epoch.

Reading learning curves

import matplotlib.pyplot as plt

plt.plot(history.history["loss"], label="training")
plt.plot(history.history["val_loss"], label="validation")
plt.legend(); plt.show()
  • Both losses go down together: the model is learning.
  • The training loss goes down but the validation loss goes back up: overfitting. Stop earlier, regularise, or add data.

Keras can stop at the right time on its own:

early_stop = keras.callbacks.EarlyStopping(patience=3, restore_best_weights=True)
model.fit(x_train, y_train, epochs=50, validation_split=0.1, callbacks=[early_stop])

Moving to images: convolutions

Flattening an image throws away its spatial structure. A convolutional layer learns filters (edges, textures, shapes) that slide across the image:

x_train_img = x_train.reshape(-1, 28, 28, 1)

cnn = keras.Sequential([
    keras.Input(shape=(28, 28, 1)),
    layers.Conv2D(32, 3, activation="relu"),
    layers.MaxPooling2D(),
    layers.Conv2D(64, 3, activation="relu"),
    layers.MaxPooling2D(),
    layers.Flatten(),
    layers.Dense(10, activation="softmax"),
])

Same compilation, same training: we get past 99% on MNIST, with fewer parameters than the dense network.

Reusing an already-trained model

In practice, you almost never train a vision network from scratch. You start from a model pre-trained on millions of images and only adjust the end: that’s transfer learning.

base = keras.applications.MobileNetV2(include_top=False, pooling="avg", input_shape=(160, 160, 3))
base.trainable = False   # keep what it already knows

model = keras.Sequential([base, layers.Dense(1, activation="sigmoid")])

With a few hundred images per class, you get a serious classifier.

Project to validate the step

  1. MNIST with a dense network, then a convolutional network: compare accuracy and number of parameters.
  2. Plot and interpret the learning curves, deliberately cause overfitting, then fix it.
  3. Build an image classifier from your own photos (two or three classes) with transfer learning.
  4. Save the model (model.save("model.keras")) and serve it behind a small API, in a Docker container. This is where my job as a backend and infra developer meets AI.

Resources