Day 15: Seq2Seq Translator Encoder-Decoder Architecture
Lesson 15 — Seq2Seq Translator: Encoder-Decoder Architecture From First Principles
The Trainer.fit() Trap
The fast path: a pretrained MarianMTModel from a high-level library, .generate() called on some input text, done. This is the trap, and for encoder-decoder models specifically it hides a decision with a genuinely surprising consequence: what the decoder is allowed to see about its own past while it's being trained.
A translation decoder generates one token at a time, and each new token depends on the ones already generated. During training, you have the ground-truth target sequence sitting right there — so the obvious thing to do is feed the decoder the correct previous token at every step, regardless of what it just predicted. This is called teacher forcing, and a high-level .fit() call typically does it unconditionally, at a ratio of 100%, without ever surfacing that as a choice you made.
Here's what that silently decides for you:
Whether the decoder ever practices recovering from its own mistakes. At inference time, there is no ground truth to feed back in — the decoder must condition on its own previous predictions, errors included. If it never once saw its own errors during training, it has no experience handling them.
Whether your training loss is honest about deployment performance. Teacher-forced next-token accuracy measures something meaningfully different from full-sequence generation accuracy. A model can look excellent on the first and mediocre on the second, and a single scalar loss curve won't tell you which one you're looking at.
How the encoder's final state gets handed to the decoder at all. If your encoder and decoder use different hidden sizes — entirely reasonable, since they're solving different sub-problems — something has to bridge the shapes. A library picks a convention for you; build it yourself and you have to decide.
None of this crashes during training. The first two produce a working model with a training loss that looks completely normal.
The Failure Mode: A Model That Only Works With the Answer Key
Train a decoder with teacher_forcing_ratio=1.0 — always feed the true previous token, never the model's own prediction — and here's what happens. Training loss drops smoothly. Teacher-forced validation accuracy (still feeding ground truth at every step, just not updating weights) looks great, often 95%+ per-token. Everything about the numbers says the model works.
Keep going with the full lesson
The rest of this lesson — plus its implementation guide, runnable scripts and source code — is included with the course or an All Access plan. Create a free account to start the free lessons in every course.
Already a member? Sign in