Day11: XOR Solver

XOR Solver — Your First Neural Network from Scratch

The sklearn.fit() Trap

Here is how a beginner might "solve" XOR in 6 lines:

It works. It prints 1.0. And it teaches you absolutely nothing.

What MLPClassifier hides from you: how the weights are initialized and why it matters, what the forward pass is actually computing at each layer, where the gradients come from and how they flow backward, and why this exact problem is impossible for a single linear layer to solve. The accuracy score is the end of the story when it should be the beginning. You have a black box that outputs a number. That number will not help you when your real model trains to 48% accuracy and you have no idea why.

This lesson is the antidote. You will build every moving part by hand, see exactly what breaks and why, and develop the intuition that no library wrapper can give you.

The Failure Mode: What a Naive Implementation Produces

Before showing the correct implementation, let's look at the two exact ways a from-scratch attempt goes wrong.

Failure 1: Zero Weight Initialization (Symmetry Breaking)

Run this and watch the loss: