System Design & Architecture Interview Question 10: Scale Out Stateless Tier
System Design & Architecture Interview Questions · Test 1 · question 10 of 10
A web application runs on one server that is already the largest instance size the provider offers. At peak it runs at 90% CPU, and traffic is expected to triple within a year. User session state is currently kept in the server's memory. Profiling shows the database has plenty of headroom. The team wants autoscaling to add and remove web servers at any time without logging users out.
What should the team do?
Show answer and explanation
Answer:
The server is at the vertical-scaling ceiling, so the only way to handle three times the load is to scale out horizontally. Horizontal scaling works best when any instance can serve any request, which means the web tier must be stateless. Moving sessions to a shared store (a distributed cache or a database) lets the load balancer send each request anywhere. Autoscaling can then add or remove servers freely, because removing one no longer destroys anyone's session.
Option A is vertical scaling. The stem says the server is already the largest size, and waiting for a hypothetical larger machine is not a plan for a known threefold increase.
Option B does scale out, but sticky sessions keep the state inside individual servers. When autoscaling removes a server, its users are logged out, which the stem forbids. Stickiness also unbalances load, because a heavy user cannot be spread across instances.
Option D fixes a bottleneck that does not exist. The database has headroom, and adding replicas does not add web-tier CPU.