Evaluation

Evaluating the Model

It is good that our neural network has learned from the training data, but this is no guarantee that it would work on real data.

To check it works we will evaluate it on our holdout testing subset of the data. This is data for which we know the ground truth, but that the model hasn’t been trained on. We will plot a “confusion matrix” that compares the neural network’s predictions to the real labels.

Note that this still isn’t a guarantee that the model will work when given some real data - e.g. the real data might have a different distribution, which is also something you might need to monitor!

# Evaluate on test set
model.eval()  # Set model to evaluation mode
with torch.no_grad():  # Disable gradient computation for inference
    test_outputs = model(X_test_t)
    predictions = test_outputs.argmax(dim=1).numpy()

# Calculate accuracy
accuracy = (predictions == y_test).mean()
print(f"\nTest Accuracy: {accuracy:.2%}")

# Confusion Matrix
plt.subplot(1, 2, 2)
cm = confusion_matrix(y_test, predictions)
disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=iris.target_names)
disp.plot(ax=plt.gca(), cmap="Blues", colorbar=False)
plt.title("Confusion Matrix")
plt.tight_layout()

Test Accuracy: 100.00%

We can see that the model is successful in predicting the correct class most (or all) of the time.