Lesson 10: Security and Compliance — Protect logs before they become liability
What we're building today
When Datadog ingests payment logs, access is not a free-for-all—RBAC gates who sees what. Stripe encrypts PAN fields at ingest; Splunk redacts before analysts search. GDPR erasure in Shopify removes a buyer's trail across shards. This lesson wires seven security layers into one platform.
By the end you'll have:
JWT authentication with RBAC on log namespaces
AES-256-GCM field encryption for PII
Regex redaction before display
Hash-chained immutable audit trails
Automated retention policies
GDPR right-to-erasure workflow
SOX / HIPAA / GDPR compliance report export
Why this matters
AWS CloudTrail records who touched which resource—your audit chain mirrors that immutability model.
HashiCorp Vault encrypts secrets at the field level—same idea as encrypting
emailandssnin log JSON.Elastic Security redacts PAN before indexing—your redaction engine runs on the read path.
SOC 2 auditors ask for retention proof—your policy engine deletes or archives on schedule.
Meta GDPR tooling erases user data across stores—your erasure coordinator tracks locations and verifies completion.
Security is not a checkbox—it is a pipeline every log byte passes through.
Architecture
Single FastAPI service on port 8094. All security modules run in-process with SQLite persistence.
How data moves through the system
Ingest path: Authenticated user posts log JSON → RBAC checks logs:write:<resource> → sensitive fields encrypted → stored in SQLite → audit WRITE.
Read path: RBAC checks logs:read:<resource> → load records → admin may see decrypted; analyst gets redacted view → audit READ with hash chain link.
Compliance path: Retention applies policies → GDPR erasure removes by user_id → report endpoint aggregates audit stats + retention + erasure into exportable JSON.
Project layout
Clone or open week10-security-compliance-platform/:
Design highlights
SQLite + lock: API and background log generator share one database safely.
Encryption keys in memory: After a restart, older ciphertext may not decrypt—production systems use a key vault; here you see why rotation matters.
RBAC wildcards: Policies like
logs:read:application.*teach IAM-style matching without a cloud account.Dashboard via CDN: React loads from CDN—no
node_modulesor npm build step.Secrets: Set
JWT_SECRET_KEYin your shell before starting. Never commit real signing keys. Demo passwords are for local practice only.
Keep going with the full lesson
The rest of this lesson — plus its implementation guide, runnable scripts and source code — is included with the course or an All Access plan. Create a free account to start the free lessons in every course.
Already a member? Sign in