Day 3: The Read Path – Timeline Reconstruction and Feed Logic

Day 3: The Read Path – Timeline Reconstruction and Feed Logic

Welcome back, fellow architects and engineers! Today, we peel back another layer of the distributed systems onion, focusing on a critical yet often underestimated component: the Read Path. If our previous sessions laid the groundwork for how data gets into our system (the Write Path), today we explore the art and science of bringing that data back to life, specifically for a user's feed or timeline. This isn't just about fetching data; it's about reconstructing a coherent, timely narrative from potentially billions of data points, all while serving 100 million requests per second with razor-sharp latency.

The Read Path: A Different Beast

Think about your social media feed. It looks simple, right? A stream of posts, neatly ordered. But behind that simplicity lies a symphony of distributed services working in concert. Unlike the Write Path, which often prioritizes durability and eventual consistency, the Read Path for a user-facing feed demands low latency and high availability. Users expect their feed to load instantly, reflecting the latest updates from their network. This fundamental difference shapes our architectural choices.

Core Challenge: Timeline Reconstruction

Timeline Reconstruction is the process of assembling a personalized stream of content for a user. Imagine you follow 1,000 people. When you open your feed, the system needs to:

1. Identify everyone you follow.

2. Fetch the latest, say, 10-20 posts from each of those 1,000 people.

3. Aggregate all these posts.

4. Sort them by relevance or recency.

5. Return the top N posts.

At 100M RPS, if each request fans out to 1,000 other services, we're talking about trillions of internal requests per second. This "fan-out" problem is the Everest of read path scaling.

Feed Logic: The Art of Assembly

For our hands-on lesson, we'll start with a straightforward Feed Logic: reverse chronological order. This means the newest posts from the people you follow appear first. While real-world feeds use complex ranking algorithms (driven by machine learning, engagement, etc.), understanding the foundational chronological aggregation is paramount. It's the bedrock upon which all sophisticated feed algorithms are built.

Component Architecture: The Feed Service Ensemble

Our read path will revolve around a central Feed Service. This service doesn't store posts itself; it orchestrates the fetching and aggregation. It interacts with:

• User Service: To get details about the requesting user.

• Follower Service: To identify who the requesting user follows.

• Post Service: To fetch actual post content from the followed users.

• Cache: A critical component to absorb the fan-out storm.

How does this fit into the overall system? The Feed Service is the gateway to personalized content. It consumes data written by the Write Path (e.g., a user publishing a post via a "Post Write Service") and transforms it into a consumable stream for the end-user.

Data Flow & Control Flow:

1. User's device sends a GET /feed request to the Load Balancer.

2. Load Balancer routes it to an instance of the Feed Service.

3. Feed Service receives the request:

• It queries the Follower Service to get a list of followedUserIds for the requesting userId.

• For each followedUserId, it queries the Post Service to get their recent posts. This is where the fan-out happens.

• It aggregates all collected posts.

• Sorts them chronologically (newest first).

• Applies any basic filtering (e.g., remove duplicates, limit to N posts).

• Returns the compiled feed to the user.

Unleashing Concurrency with Project Loom (Virtual Threads)

The fan-out problem is fundamentally an I/O-bound challenge. We're waiting for network calls to the Follower Service and many Post Service instances. This is where Java's Project Loom, with its Virtual Threads, shines like a beacon. Traditional platform threads are expensive, limiting the number of concurrent I/O operations. Virtual Threads, being lightweight, allow us to launch thousands or even millions of concurrent tasks without significant overhead, making the fan-out on read pattern far more manageable and efficient.

Instead of complex CompletableFuture chains or thread pools, Virtual Threads allow us to write sequential-looking code that executes concurrently.

import java.time.Instant;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.stream.Collectors;

// --- Mock Services for demonstration ---
record Post(String postId, String userId, String content, Instant timestamp) {}
record User(String userId, String username) {}

class MockPostService {
    List<Post> getPostsByUserId(String userId, int limit) {
        // Simulate network latency and data retrieval
        try { Thread.sleep(50); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }
        return switch (userId) {
            case "userA" -> List.of(
                new Post("p1", "userA", "Hello world!", Instant.parse("2023-01-01T10:00:00Z")),
                new Post("p2", "userA", "Another day, another post.", Instant.parse("2023-01-01T11:00:00Z"))
            );
            case "userB" -> List.of(
                new Post("p3", "userB", "Learning distributed systems.", Instant.parse("2023-01-01T10:30:00Z")),
                new Post("p4", "userB", "Virtual Threads are awesome!", Instant.parse("2023-01-01T11:15:00Z"))
            );
            case "userC" -> List.of(
                new Post("p5", "userC", "Just had coffee.", Instant.parse("2023-01-01T09:45:00Z"))
            );
            default -> List.of();
        };
    }
}

class MockFollowerService {
    List<String> getFollowedUsers(String userId) {
        try { Thread.sleep(30); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }
        return switch (userId) {
            case "viewer1" -> List.of("userA", "userB", "userC");
            case "viewer2" -> List.of("userA");
            default -> List.of();
        };
    }
}
// --- End Mock Services ---

public class FeedService {
    private final MockFollowerService followerService;
    private final MockPostService postService;
    private final ExecutorService virtualThreadExecutor; // Executor for Virtual Threads

    public FeedService(MockFollowerService followerService, MockPostService postService) {
        this.followerService = followerService;
        this.postService = postService;
        // Project Loom: Create an ExecutorService backed by Virtual Threads
        this.virtualThreadExecutor = Executors.newVirtualThreadPerTaskExecutor();
    }

    public List<Post> getFeed(String viewerId, int limit) {
        long startTime = System.currentTimeMillis();
        System.out.println(STR."[{viewerId}] Fetching feed...");

        // 1. Get followed users (sequential for simplicity here, could be cached)
        List<String> followedUsers = followerService.getFollowedUsers(viewerId);
        System.out.println(STR."[{viewerId}] Followed: {followedUsers}");

        // 2. Concurrently fetch posts for each followed user using Virtual Threads
        List<Future<List<Post>>> futures = new ArrayList<>();
        for (String followedUserId : followedUsers) {
            futures.add(virtualThreadExecutor.submit(() -> {
                System.out.println(STR."[{viewerId}] Fetching posts for {followedUserId} on {Thread.currentThread().getName()}");
                return postService.getPostsByUserId(followedUserId, 10); // Get top 10 posts
            }));
        }

        // 3. Aggregate results
        List<Post> allPosts = new ArrayList<>();
        for (Future<List<Post>> future : futures) {
            try {
                allPosts.addAll(future.get()); // .get() blocks until result is ready
            } catch (Exception e) {
                System.err.println(STR."Error fetching posts for a user: {e.getMessage()}");
                // Handle partial failures gracefully (e.g., log, return what we have)
            }
        }

        // 4. Sort and limit
        List<Post> sortedFeed = allPosts.stream()
                .sorted(Comparator.comparing(Post::timestamp).reversed()) // Newest first
                .limit(limit)
                .collect(Collectors.toList());

        long endTime = System.currentTimeMillis();
        System.out.println(STR."[{viewerId}] Feed generated in {endTime - startTime} ms. Total posts: {sortedFeed.size()}");
        return sortedFeed;
    }

    public void shutdown() {
        virtualThreadExecutor.shutdown();
    }

    public static void main(String[] args) {
        FeedService feedService = new FeedService(new MockFollowerService(), new MockPostService());
        System.out.println("n--- Viewer 1 Feed ---");
        List<Post> viewer1Feed = feedService.getFeed("viewer1", 5);
        viewer1Feed.forEach(System.out::println);

        System.out.println("n--- Viewer 2 Feed ---");
        List<Post> viewer2Feed = feedService.getFeed("viewer2", 5);
        viewer2Feed.forEach(System.out::println);

        feedService.shutdown();
    }
}

Notice how Executors.newVirtualThreadPerTaskExecutor() allows us to treat each I/O bound task (fetching posts for a single user) as its own lightweight "thread," dramatically simplifying the concurrency logic compared to managing a fixed-size thread pool or callback hell. This is the high-concurrency simplicity Java 25/26 promises.

The Critical Role of Caching

While Virtual Threads make fan-out possible, they don't solve the problem of overwhelming downstream services. Imagine 100M RPS hitting your Feed Service, which then fans out to a Post Service. The Post Service would collapse. This is where caching becomes non-negotiable.

For timeline reconstruction:

• Followed Users List: This list changes infrequently. It's a prime candidate for caching at the Feed Service level (e.g., followedUserCache.get(userId)).

• Recent Posts per User: The most critical cache. Instead of hitting the Post Service for every followed user every time, the Feed Service could query a local or distributed cache (like Redis or Memcached) that stores recently published posts for each user. This cache would be updated by the Write Path.

The insight here is that you're not just caching results; you're caching intermediate data to reduce the fan-out amplification. At 100M RPS, even a 99% cache hit rate for followed users and 90% for recent posts can reduce downstream load by orders of magnitude, making your system viable. The trade-off is data freshness. If a post is written, how long until it appears in the cache and then in the feed? This becomes a product decision.

Real-World Implications & Nuances

• Latency Budgets: For 100M RPS, your end-to-end latency might be budgeted for 100-200ms. Each internal service call (like followerService.getFollowedUsers) must complete in milliseconds. Virtual Threads help keep the orchestration fast, but the underlying service calls must be optimized.

• Partial Failures: What if one of the 1,000 followed users' Post Services is down? You can't let the entire feed request fail. Implement robust error handling (e.g., try-catch around future.get()), collect successful results, and log failures. Return a partially constructed feed; it's better than no feed.

• Thundering Herd: When a cache entry expires, many concurrent requests might try to re-populate it simultaneously. Implement cache-aside patterns with single-flight requests (only one request goes to the database/downstream service, others wait for its result).

This read path architecture, leveraging efficient concurrency and strategic caching, is the blueprint for handling massive scale. It moves beyond theoretical discussions to practical, implementable solutions.

Assignment: Enhance Your Feed Service

Your task is to extend the FeedService to incorporate a basic caching mechanism for followedUserIds and demonstrate its impact.

Steps:

1. Introduce a Map based cache: In FeedService, add a Map<String, List<String>> followedUsersCache to store viewerId -> List<followedUserIds>.

2. Implement Cache-Aside Logic:

• Before calling followerService.getFollowedUsers(viewerId), check followedUsersCache.

• If found, return from cache.

• If not found, call the followerService, then store the result in followedUsersCache before returning it.

1. Simulate Cache Refresh: Add a simple mechanism (e.g., a method clearFollowedUsersCache()) to invalidate the cache. Observe how the first request after invalidation hits the followerService, and subsequent requests hit the cache.

2. Measure Performance Impact: Add System.out.println statements to show when the followerService is actually called versus when the cache is hit, and observe the time savings.

Solution Hints:

• Cache Implementation: A ConcurrentHashMap is a good choice for followedUsersCache if you want thread-safe access. For this simple exercise, a non-concurrent HashMap is fine if accessed within a single thread context or with external synchronization.

• Cache-Aside: Your getFollowedUsers logic within FeedService should look like:

List<String> followedUsers = followedUsersCache.get(viewerId);
if (followedUsers == null) {
    System.out.println(STR."[{viewerId}] Cache miss for followed users. Fetching from FollowerService.");
    followedUsers = followerService.getFollowedUsers(viewerId);
    followedUsersCache.put(viewerId, followedUsers); // Put in cache
} else {
    System.out.println(STR."[{viewerId}] Cache hit for followed users.");
}
return followedUsers;

• Testing: Run getFeed multiple times for the same viewerId to see cache hits. Then call clearFollowedUsersCache() and run again to simulate a cache miss.

By completing this assignment, you'll gain a tangible understanding of how caching dramatically reduces load and improves performance in a fan-out scenario, a crucial insight for any high-scale distributed system.