We’ve hired a lot of cloud engineers. And we’ve noticed a pattern: the question that filters most cleanly isn’t about specific services, certifications, or even past experience. It’s this:
“Walk me through how you’d design a system that processes 10,000 events per second, where some events are time-sensitive and others can be batched. You have a $500/month budget.”
The question isn’t designed to have a correct answer. It’s designed to reveal how someone thinks under constraint.
Look at what is packed into two sentences. A throughput figure that sounds large but is deliberately vague. Two classes of traffic with different needs. A budget that is tight enough to rule out the lazy answer but not so tight that it is impossible. And a verb — “processes” — that could mean almost anything. Every one of those is a decision the candidate has to make, and the interesting part is whether they notice.
What the strong candidates do
They ask clarifying questions first. “What’s the acceptable latency for time-sensitive events?” “What does ‘processed’ mean — stored, transformed, forwarded?” “Is $500 inclusive of storage and egress?”
That last question is the one that tells us most. Egress and storage are where real cloud bills go wrong, and an engineer who asks about them before drawing a single box has clearly paid one. The latency question matters just as much: “time-sensitive” could mean 200 milliseconds or 30 seconds, and those are different systems.
Then they reason out loud, making their trade-offs visible. “I’d use SQS with two queues — a standard queue for batch work and a FIFO queue for the time-sensitive path. Lambda for the processors, keeping an eye on cold start latency for the critical path. DynamoDB for state. At this volume we’re probably looking at $180–220/month for the compute and messaging, which leaves headroom for storage.”
They put numbers on it
Notice the estimate. It may not be right — we don’t mark anyone down for a first estimate that is off — but the act of producing one changes the conversation. Our follow-up is always “show me the arithmetic”, and the strong candidates enjoy it. Ten thousand events per second is roughly 864 million a day and around 26 billion a month. Any service priced per request is going to hurt at that volume unless you batch, so they start talking about pulling messages in batches, aggregating records at the producer, or moving the batchable path onto a stream where you pay for throughput rather than per call. The design gets better as the numbers get checked, which is exactly what happens on a real project.
They name what they are giving up
Two queues and two processors means two things to monitor, two failure modes, and two places a message can get stuck. A single pipeline would be simpler. The strong candidate says so, and then explains why the latency requirement is worth the extra complexity. They mention that Lambda cold starts on the critical path could be a problem and offer provisioned concurrency as a fix, then immediately note that it eats budget. They talk about what happens when the batch path falls behind — does it drop events, buffer them, or alert? Every choice comes with the thing it costs.
What the weak candidates do
They jump straight to a solution. Often it’s the architecture they know best, applied without interrogating the constraints. The budget mention barely registers. The distinction between event types gets flattened into one processing pipeline.
The most common shape is a Kubernetes cluster. Three nodes, a message broker running as a StatefulSet, a deployment of consumers, an ingress. It is a perfectly good architecture for a great many problems. It is not a $500-a-month architecture, and the candidate rarely notices, because they never priced it. When we ask what the control plane and three nodes cost, there is a pause.
The second most common shape is the opposite: a single managed service that does everything, described with real confidence and no numbers at all. Ask about the time-sensitive events and the answer is that the service is “fast”. Ask what fast means in milliseconds and the answer is that it depends.
None of this means the candidate is a poor engineer. Many of them are strong operators who can run a cluster well. What the question exposes is that they have learned architectures as fixed answers, and nobody has ever asked them to defend one against a constraint.
What this tells you
Cloud architecture is 20% knowing what services exist and 80% knowing which constraints matter most in a given situation. The engineers who compound over time are the ones who’ve internalised that trade-offs are the job — not service selection.
Service knowledge decays. The managed offering you learned two years ago has been renamed, repriced, or superseded. But the habit of asking “what are we optimising for, and what are we willing to give up to get it?” transfers to every platform and every decade. It is the same habit that makes someone good at incident reviews, capacity planning, and cost reduction — all of which are the same skill wearing different clothes.
It is also why we weight this one question more heavily than a certification. A certification shows you can recall what a service does. This shows you can decide whether to use it.
How to practise it
If you’re preparing for cloud interviews: practice thinking out loud with constraints. The architecture is almost always secondary to the reasoning.
- Take any system you know and cut its budget in half. Work out what you would drop first, what you would keep at any cost, and why. Then halve it again.
- Price your designs before you finish them. Open the provider’s pricing page and estimate compute, messaging, storage and egress separately. You will be wrong at first. The point is to become less wrong quickly.
- Write down three clarifying questions before you answer anything. Latency, definition of done, and what the budget includes are a good default set. Ask them even when you think you know the answer.
- Say the trade-off out loud. “I’m choosing X, which means we lose Y.” If you can’t name a Y, you haven’t made a choice; you have picked a default.
- Practise with a listener. Explaining a design to another person, in real time, is a different skill from drawing it alone. The interview tests the former.
The question has no right answer. But it has a very clear wrong one: the answer that arrives before the questions do.
