Lesson 1: Docker Basics

Container Forge — Platform Architecture

Internal engineering reference for the Container Forge log processing platform: container packaging, image supply chain, network isolation, multi-service orchestration, and connectivity diagnostics.

System Scope

• Multi-stage container images with non-root execution, dumb-init signal handling, and layer caching that minimizes rebuild time on code-only changes

• Image supply chain variants comparing unoptimized, wheel-optimized, and production multi-stage builds with security hardening

• Segmented Docker networks with named volumes, Docker secrets, and healthcheck-gated startup across six services

• Connectivity diagnostics sandbox with deliberate networking defects mirroring production DNS and dependency failures

Operational Context

Every orchestrated workload runs inside containers. Teams that treat packaging as an afterthought routinely pay for it in incident volume: slow rollouts, oversized images, exposed data tiers, and race conditions at startup. Layer caching strategy directly affects CI throughput at scale. Non-root execution and minimal base images are baseline security controls before any image reaches a shared registry. Network isolation failures often surface as "orchestrator bugs" when the root cause is container DNS or bridge attachment misconfiguration.

Container Forge encodes production patterns locally before any cluster scheduler is involved. Embedded DNS at 127.0.0.11, service aliases, and health-gated depends_on behave the same way orchestrators expect readiness and service discovery to behave in production.

Container Packaging

Multi-Stage Builds and Layer Caching

Docker images are ordered layer stacks with content-addressable caching. Placing COPY requirements.txt before application source keeps dependency layers valid across iterative code changes. A single COPY . . ahead of dependency installation invalidates the expensive layers on every commit.

The log-api production Dockerfile defines base, dependencies, development, and production targets. The anti-pattern variant installs compiler tooling and runs as root in a single stage — increasing attack surface, pull time, and vulnerability scan noise.

Trade-off: Multi-stage Dockerfiles add authoring cost. The return is smaller images, faster deploys, and fewer packages in the runtime layer.

Image Supply Chain

Image size correlates with security posture: fewer packages mean fewer CVEs and faster scans. Three build paths exist under services/log-api/:

The wheel-builder pattern (pip wheel in a builder stage, pip install /wheels/* in runtime) excludes compilers from the final layer. .dockerignore keeps tests and local tooling out of the build context.

Anti-pattern: python:3.11 full image for convenience — the slim-bookworm base removes hundreds of megabytes of unused system libraries.

Network and Storage Topology

Container Forge uses three networks:

Named volumes (pgdata, redis_data, backup_data) persist across container recreation. Bind mounts supply read-only initialization SQL. Database credentials mount via Docker secrets as files, not plain environment variables visible in docker inspect.

Design constraint: data_net is internal — the data tier never binds host ports. Application containers reach postgres and redis only through shared network membership.

Multi-Service Orchestration

docker-compose.yml encodes production compose patterns:

• YAML anchors for shared restart and healthcheck blocks

• depends_on with condition: service_healthy

• Optional profiles: compare (image variants), backup (sidecar agent)

• nginx gateway upstream blocks resolving log-api and log-processor via embedded DNS

Failure mode: depends_on without a health condition allows the API to accept traffic before postgres or the processor is ready. Always gate on service_healthy.

Connectivity Diagnostics

Production container debugging follows: status → networks → DNS → dependencies → logs.

The debug/ sandbox injects three defects found in real incidents:

scripts/diagnose.sh automates the hierarchy. The same checks belong in CI smoke tests: if getent hosts processor fails inside the API container, the build should not promote.

Runtime Contracts

Production Controls

• Resource limits: docker-compose.prod.yml applies CPU and memory caps per service

• Signal handling: dumb-init forwards SIGTERM to uvicorn for graceful shutdown

• Probe separation: liveness must not depend on downstream services; readiness must

• Backup sidecar: profile-activated backup-agent writes timestamped snapshots to backup_data

Architectural Principle

Compose service DNS is not a simplified substitute for production service discovery — it is the same resolution model with different syntax. Master network attachment, alias registration, and health-gated startup locally; orchestrator-level failures become diagnosable instead of opaque.