Lesson 1: Twitter MVP Foundation

Realtime Feed Reliability — Integrated Social Media Platform

Scope

This document describes how the integrated Twitter-style MVP wires authentication, timeline reads, event-backed writes, cache layers, realtime fanout, and operational visibility into one coherent production surface. The React operations console and CLI verification scripts exercise each domain independently and together.

Architectural placement

The platform treats PostgreSQL as source of truth, Redis for cache and pub/sub, and a Node/Express API with a React dashboard. Observability flows through Prometheus, Grafana, and Jaeger. The integration layer binds schema design, tweet persistence, hybrid timeline generation, WebSocket delivery, L1/L2 caching, JWT auth, versioned HTTP APIs, full-text search, media ingestion, health probes, load smoke checks, and deployment readiness signals.

Request path

• Browser console invokes domain-specific verification actions.

• API enforces JWT auth and token-bucket rate limits.

• Read traffic prefers Redis; misses hydrate from PostgreSQL.

• Writes commit to Postgres, append to the event store, publish Redis notifications, and push WebSocket payloads.

• Health and info endpoints return checkedAt / fetchedAt for freshness proof.

Core mechanisms

Cache hit vs miss on reads
Timeline and search responses must be correct on miss and fast on hit. Both paths require explicit test coverage.

Event append on writes
Tweet creation updates durable state and emits an auditable event so downstream consumers and realtime clients stay aligned.

Operational timestamps
checkedAt on /health and fetchedAt on /api/info let operators and QA confirm live responses without inferring staleness.

Deterministic verification
Each console verification action must change or refresh at least one visible metric: counts, rate-limit headers, latency, object keys, or JSON payloads.

Data and control flow

Timeline read (hybrid fanout)

• Load follower graph metrics for the requesting user.

• Select pull, push, or hybrid materialization strategy.

• Resolve cache key timeline:{userId}:{cursor}:{limit}.

• On miss, query timeline_entries and supporting tables, then populate cache.

Tweet write

• Insert row into tweets.

• Append tweet.created (or related) to events.

• Fan out timeline_entries to author and followers.

• Publish notifications:{userId} and tweet_events on Redis.

• Invalidate timeline and related cache keys.

Request lifecycle

Naming these states explicitly supports SLO design, on-call runbooks, and trace correlation.

Implementation workstreams

Timeline cache
Adopt key pattern timeline:{userId}:{cursor}:{limit}. Honor cache bust via _t query parameter on operational reads.

Write pipeline
Tweet service must persist, version, fan out, emit events, and call cache invalidation in one transactional boundary where possible.

Freshness metadata
Health and info handlers attach ISO timestamps on every response.

Console metrics by domain

Local runtime

From the application repository root:

cd backend && npm install && npm run build
cd ../frontend && npm install && npm run build
cd ..
docker compose up -d --build backend frontend

Or run processes directly:

cd backend && npm run dev
cd frontend && npm run dev

Expected: API on port 4000, dashboard on port 5173.

Containerized runtime

chmod +x start.sh test.sh demo.sh stop.sh
./start.sh

Expected: compose health checks pass; dashboard serves at http://localhost:5173.

Verification

./test.sh
./demo.sh

Acceptance:

• Unit and integration tests exit zero.

• demo.sh reports all domain checks passed.

• Manual probes return changing timestamps:

  bashCopy

  curl -s http://localhost:4000/health | jq .checkedAt
  curl -s http://localhost:4000/api/info | jq .fetchedAt
  

Demo credentials: [email protected] / demo123!

Gemini integration

Load the Gemini API key from environment only. Never commit secrets to source, scripts, or documentation.

import os
api_key = os.getenv("GEMINI_API_KEY")
if not api_key:
    raise RuntimeError("Missing GEMINI_API_KEY")

Extension: idempotent media upload

Add Idempotency-Key header support on POST /api/media/upload. Duplicate keys for the same user must return the original object metadata without reprocessing. Surface idempotency status on the media console panel. Persist (user_id, idempotency_key) → object_key and add regression tests for initial upload and replay.

Acceptance criteria

• Repeated console verification runs update visible metrics across auth, API, search, media, monitoring, load, and deployment domains.

• demo.sh completes with zero failures.

• Health and info timestamps differ between consecutive probes.

• Cache warm cycles show measurable hit-rate and key-count movement.

Operational summary

Production readiness here means observable, repeatable behavior across read and write paths—not a single successful manual call. Treat cache keys, event append, fanout channels, and freshness fields as first-class contracts the same way schema migrations and API versions are.