System Design and Scalability

Why this lesson matters

You already covered Data Structure and Algorithm Fundamentals and turned those tools into interview tactics in Problem Solving Techniques.

Now we zoom out: senior interviews at Google, Microsoft, and Amazon often ask, “Can you design something that survives real traffic, real failures, and real trade-offs?”

System design is not separate from DSA. It is DSA thinking applied to distributed products:

• Hash maps become caches and indexes.
• Queues become async pipelines.
• Graphs become dependency maps.
• Big-O becomes latency, throughput, storage, and cost.

By the end, you should be able to lead a design conversation with structure, numbers, and trade-offs—not just draw boxes.

Senior system design loop

Use a repeatable loop, similar to the interview loop from Problem Solving Techniques, but at architecture scale.

1. Clarify requirements

Ask what matters:

• Users and scale?
• Read-heavy or write-heavy?
• Latency target?
• Consistency requirement?
• Availability expectation?
• Core APIs?

2. Define the core objects

For example, in a URL shortener:

• User
• ShortUrl
• LongUrl
• ClickEvent

3. Estimate capacity

Rough numbers beat vague claims.

4. Propose simple design first

Start with one service and one database. Then scale the bottlenecks.

5. Discuss trade-offs

Senior candidates stand out by saying what they are not optimizing for.

You are designing a payment authorization service. The interviewer says users must never see a successful payment unless it is durably recorded.

Which system property should you prioritize first?

Back-of-the-envelope estimates

Great system design answers include numbers. They do not need to be perfect; they need to expose bottlenecks.

Suppose you design a feed service:

• 10 million daily active users
• Each user opens the app 20 times/day
• Each feed request returns 50 posts
• Average post metadata is 1 KB

Questions you should estimate:

• Requests per second
• Peak requests per second
• Data served per day
• Cache memory needed for hot content

This is the system-design version of Big-O: you are reasoning about growth before picking machinery.

```java
public class Main {
    public static void main(String[] args) {
        long dailyActiveUsers = 10_000_000L;
        int opensPerUserPerDay = 20;
        int postsPerFeed = 50;
        int kbPerPost = 1;

        long requestsPerDay = dailyActiveUsers * opensPerUserPerDay;
        double avgRps = requestsPerDay / 86_400.0;
        double peakRps = avgRps * 5; // common rough multiplier

        long kbPerRequest = postsPerFeed * kbPerPost;
        double gbPerDay = (requestsPerDay * kbPerRequest) / (1024.0 * 1024.0);

        System.out.printf("Requests/day: %,d%n", requestsPerDay);
        System.out.printf("Average RPS: %.0f%n", avgRps);
        System.out.printf("Peak RPS estimate: %.0f%n", peakRps);
        System.out.printf("Data served/day: %.1f GB%n", gbPerDay);
    }
}
```

The common scaling moves

When the simple design fails, scale the bottleneck deliberately.

Read bottleneck

Use:

• CDN for static assets
• Cache for hot objects
• Read replicas for database reads

Write bottleneck

Use:

• Partitioning/sharding
• Append-only logs
• Message queues
• Batch processing

CPU bottleneck

Use:

• Horizontal service replicas
• Load balancers
• Async workers

Latency bottleneck

Use:

• Regional deployment
• Precomputation
• Denormalized read models

The key senior move: do not add every component at once. Add the smallest component that directly addresses the measured bottleneck.

True or false: Adding a cache always improves a system because it reduces database load without introducing meaningful trade-offs.

Sharding and consistent hashing

When one database node cannot handle storage or write throughput, partition the data.

A basic strategy is:

shard = hash(key) % numberOfShards

This is simple, but adding a shard can move many keys.

Consistent hashing reduces key movement by placing shards and keys on a ring. Each key goes to the next shard clockwise.

Use it when:

• You have many cache/database nodes.
• Nodes may be added or removed.
• You want fewer remapped keys during scaling.

This connects directly to the hash map intuition from Data Structure and Algorithm Fundamentals, but now the buckets are machines.

```java
import java.util.*;

public class Main {
    static class ConsistentHashRing {
        private final TreeMap<Integer, String> ring = new TreeMap<>();

        void addShard(String shard) {
            ring.put(hash(shard), shard);
        }

        String getShard(String key) {
            if (ring.isEmpty()) throw new IllegalStateException("No shards available");
            int h = hash(key);
            Map.Entry<Integer, String> entry = ring.ceilingEntry(h);
            return entry != null ? entry.getValue() : ring.firstEntry().getValue();
        }

        private int hash(String value) {
            return Math.abs(value.hashCode() % 1000);
        }
    }

    public static void main(String[] args) {
        ConsistentHashRing ring = new ConsistentHashRing();
        ring.addShard("db-a");
        ring.addShard("db-b");
        ring.addShard("db-c");

        List<String> userIds = Arrays.asList("user-101", "user-202", "user-303", "user-404");
        System.out.println("Initial placement:");
        for (String userId : userIds) {
            System.out.println(userId + " -> " + ring.getShard(userId));
        }

        ring.addShard("db-d");
        System.out.println("\nAfter adding db-d:");
        for (String userId : userIds) {
            System.out.println(userId + " -> " + ring.getShard(userId));
        }
    }
}
```

Fill in the blank:

When a system allows replicas to temporarily disagree but guarantees they will converge if no new writes occur, it is called ______ consistency.

Cache design questions

A cache is not just a Map. In interviews, discuss policy.

Ask:

• What is the key?
• What is the value size?
• What is the eviction policy?
• What is the TTL?
• What happens on cache miss?
• Can one hot key overload one node?
• How do we invalidate stale data?

A common local policy is LRU: evict the least recently used item. This uses the same map/list intuition you already know, but packaged as a production behavior.

```java
import java.util.*;

public class Main {
    static class LruCache<K, V> extends LinkedHashMap<K, V> {
        private final int capacity;

        LruCache(int capacity) {
            super(capacity, 0.75f, true); // accessOrder = true
            this.capacity = capacity;
        }

        @Override
        protected boolean removeEldestEntry(Map.Entry<K, V> eldest) {
            return size() > capacity;
        }
    }

    public static void main(String[] args) {
        LruCache<String, String> cache = new LruCache<>(3);
        cache.put("u1", "Alice");
        cache.put("u2", "Bob");
        cache.put("u3", "Carla");

        cache.get("u1"); // u1 becomes recently used
        cache.put("u4", "Dev"); // evicts u2

        System.out.println("Cache contents in LRU order:");
        for (Map.Entry<String, String> entry : cache.entrySet()) {
            System.out.println(entry.getKey() + " = " + entry.getValue());
        }
    }
}
```

Your complete senior-prep arc

You have now moved through the full course path:

1. Data Structure and Algorithm Fundamentals gave you the primitives.
2. Problem Solving Techniques gave you tactical interview execution.
3. System Design and Scalability gave you architecture-level reasoning.

A strong system design answer sounds like this

Let me clarify the requirements.
For scale, I estimate roughly X requests/sec and Y storage/day.
I will start with a simple design, then identify bottlenecks.
The read path can be optimized with caching/read replicas.
The write path may need queues or sharding.
The trade-off is consistency vs latency/availability.
Here is how I would monitor and evolve it.

Final checklist

• Clarify before designing.
• Estimate before scaling.
• Start simple before adding distributed components.
• Name trade-offs explicitly.
• Tie choices to user impact.
• Discuss failure modes and observability.

That is the mindset interviewers expect from a senior software engineer: not just solving the local problem, but designing systems that keep working as the problem grows.