Anatomy of a Real-Time Ad Bidder: What Happens in the 50 Milliseconds After You Open a Webpage

Every time you open a webpage or an app with an ad slot on it, an auction happens. Not a metaphorical one — an actual auction, with multiple bidders, a clearing price, and a winner — and the entire thing completes before the page finishes rendering. A human blink takes about 300 milliseconds. Three complete auctions fit inside one blink.

I spent some time building one of the machines that participates in these auctions: a real-time bidder, written in Rust, designed for the workload real ad-tech companies deal with — tens of thousands of ad campaigns, a hundred million users, and a hard latency ceiling of 50 milliseconds at tens of thousands of requests per second. This series is a deep walkthrough of how such a system works, the engineering decisions involved, what the industry does at scale, and the measured results of my implementation — including the mistakes, which turned out to be the most educational part.

This first post is written to stand alone. Start from zero knowledge of advertising and you should finish it understanding the whole machine end to end — who the players are, what travels between them, what the bidder decides and how, where the money and the milliseconds go, and what happens after the auction closes. The four parts that follow magnify one subsystem each; none of them is required to make sense of this one.

The ecosystem: who is bidding on your attention#

The programmatic advertising world has a small cast of characters, and once you know them the rest of the machinery makes sense.

The publisher owns the webpage or app you’re looking at — a news site, a weather app, a blog. They have empty rectangles (ad slots) they want to sell.

The advertiser has a product and a budget. They want their ad shown to the right people. Advertisers organize their spending into campaigns: “show this sneaker ad to sports-interested users in the US, on mobile, at most 3 times per day per person, spending at most $5,000 today.”

The SSP (supply-side platform) works for publishers. When your browser loads the page, the SSP takes the ad slot and puts it up for auction.

The ad exchange runs the auction. It broadcasts a description of the opportunity — “banner slot, 300×250, on this site, seen by user X, on an iPhone, in San Francisco” — to many potential buyers simultaneously.

The DSP (demand-side platform) works for advertisers, and this is the machine this series is about. A DSP’s bidder receives the exchange’s broadcast, decides in a few milliseconds whether any of its campaigns wants this impression and at what price, and answers. Highest bid wins, their ad renders, money changes hands.

Those are roles, not companies — the distinction that trips up almost everyone new to this. One company routinely plays several roles, and some of the biggest names in advertising aren’t in this diagram at all.

Three things that table clears up:

SSP and exchange are usually the same company. They were separate businesses a decade ago: the SSP managed a publisher’s yield across many demand sources, the exchange was the auction venue. They consolidated. Google’s Ad Exchange is a feature of Google Ad Manager; Magnite, PubMatic and Index Exchange each run both halves. Read the two boxes in the diagram below as two functions that usually live in one vendor, not two companies you’d invoice separately.

Google Ads is not an SSP — this one confuses everybody, because Google sits on both sides. Google Ads is a buying tool: an advertiser types a budget into it and buys Google’s own inventory. The publisher-side products are Ad Manager and AdSense. And Google’s DSP for bidding on the open web — the box this series is about — is Display & Video 360. Same company, three different products, two opposing sides of the auction.

Meta isn’t in this flow at all. Meta, TikTok, and Amazon’s own storefront are walled gardens: they own the supply, the demand tools, the auction, and the user identity, so there is nothing to broadcast to outside bidders. You buy Meta ads inside Meta’s own interface and Meta’s servers run the auction against Meta’s own inventory. RTB is what the open web does instead — news sites, apps, connected TV — where no single company owns both ends.

So why tolerate a chain this long, when every hop costs milliseconds? Two reasons that aren’t going away. First, the two sides have opposed interests: an SSP’s job is to raise the publisher’s price and a DSP’s job is to lower the advertiser’s, so neither can be trusted to run the auction alone. Second, arithmetic: millions of publishers times hundreds of buyers is an impossible number of direct integrations, and an exchange in the middle turns n × m into n + m. Your instinct — that an advertiser should just deal with the publisher directly — is exactly how direct deals work, and they still exist for premium inventory sold in advance. RTB is what fills the rest, one impression at a time, for buyers who want to reach a specific person rather than a specific website.

The whole loop — page load, auction broadcast, bids, winner selection, ad delivery — fits inside roughly 100 ms, because any longer and the user would see the page stutter. This isn’t a soft convention: exchanges stamp a hard deadline into every bid request (the tmax field). Google’s exchange, for example, publishes response deadlines of 80–1000 ms and expects nearly all of your responses to beat them. In practice, a bidder engineers its internal budget to something like 50 ms at the 99th percentile — p99, the number 99 out of 100 requests come in under. Averages are useless here: the slow 1% is exactly the part that loses auctions.

Watch that rule play out. In the auction below, DSP C usually offers the highest price — and still loses, because its answer arrives after the timeout. Then drag your own bidder’s latency past the line and watch the same thing happen to you:

This model is called RTB — real-time bidding — and the wire protocol nearly everyone speaks is OpenRTB, an IAB Tech Lab standard (JSON or protobuf) that defines what a bid request and bid response look like.

What actually goes over the wire#

All the abstraction above collapses into two small JSON documents. This is a bid request, trimmed to the fields that matter (the full one is a few kilobytes):

{
  "id": "1234567893",
  "at": 2,
  "tmax": 120,
  "imp": [
    { "id": "2", "bidfloor": 0.03, "banner": { "w": 300, "h": 250, "pos": 0 } }
  ],
  "site": {
    "domain": "siteabcd.com",
    "cat": ["IAB2-1"],
    "publisher": { "id": "pub12345" }
  },
  "device": {
    "devicetype": 2,
    "os": "OS X",
    "geo": { "country": "USA", "region": "NY", "city": "New York" }
  },
  "user": { "id": "42", "buyeruid": "545678765467876567898765678987654" }
}

Read it out loud and it’s just a sentence: _a 300×250 banner, on siteabcd.com, for user 42 in New York on a Mac, floor price $0.03 CPM, second-price auction (at: 2), answer within 120 ms._ Everything the bidder decides comes from those fields plus what it already knows about user 42.

The answer is smaller still:

{
  "id": "1234567893",
  "cur": "USD",
  "seatbid": [{
    "bid": [{
      "id": "1234567893-2",
      "impid": "2",
      "price": 2.1,
      "cid": "871",
      "crid": "100",
      "adid": "100",
      "nurl": "https://bidder.example.com/rtb/win?request_id=1234567893&imp_id=2&campaign_id=871&creative_id=100&clearing_price_micros=2100000&user_id=42&exchange_id=generic&token=9F3C…",
      "ext": { "bidder_pctr": 0.0273, "bidder_score": 0.0273 }
    }]
  }]
}

A price, which creative to show, and a URL to call if we win. (crid references the creative rather than inlining its markup in adm; the token on the win URL is an HMAC the bidder signs — more on why in stage 9.) Everything in the rest of this post exists to produce those fields before tmax expires.

Both documents are the real shapes, not illustrations: the request is trimmed from the repo’s golden fixture (tests/fixtures/golden-bid-request.json, itself derived from the OpenRTB spec’s examples), and the response is what bidder-core/src/pipeline/stages/response_build.rs serializes on a win.

That ID is also the shakiest foundation in the whole industry. Third-party cookies and stable device IDs are being dismantled, and the replacement designs — Google’s Privacy Sandbox Bidding & Auction services, for one — move parts of the auction into trusted execution environments where the bidder never sees a raw identifier at all. This post assumes the classic ID-based model; the latency engineering it describes outlives that assumption, the identity model may not.

After the gavel: how the ad actually reaches the page#

The bidder’s job ends when it answers. The impression’s journey doesn’t, and the rest of it explains several things that would otherwise look like strange design choices.

1. The exchange compares the bids it received in time and picks a winner. Everyone else’s answer is discarded — including bids that were merely slow.
2. The winning creative reference travels back through the SSP to the ad tag on the page, and the browser fetches the actual image or video from the winner’s ad server or CDN. The bidder is not in this path; it handed over a pointer, not pixels.
3. The ad renders. Total elapsed time since the page started loading: usually under a few hundred milliseconds, most of it not the auction.
4. The exchange calls the win notice — the nurl from the bid response — telling the bidder it won and at what price. Some integrations also fire a separate billing notice (burl) only when the ad actually rendered.

Step 4 is the one that surprises people. A bidder does not know it won until it’s told, over a separate HTTP call that may arrive late, arrive twice, or never arrive at all — a browser that closes early, an ad that never scrolls into view, a dropped packet. Every spend number, every frequency counter, every budget decision downstream is built on that callback.

That’s why the bidder signs its own win URLs, deduplicates them, and treats its own budget accounting as an estimate that reconciles rather than a ledger that’s always exact. And it’s why the money and the machinery are separated in the design: the auction path must answer in milliseconds, while the accounting path is allowed to be slower, retried, and eventually consistent.

That is also why a bid response carries more than one callback URL. OpenRTB defines three, and the difference between them is the difference between winning, being billed, and learning why you lost:

This bidder implements nurl only; burl and lurl are in the response struct but unused, which is a fair description of many real integrations too.

Which brings up the part newcomers find genuinely surprising: entering an auction is free. Losing costs nothing. Winning and never rendering costs nothing. An advertiser is charged only for impressions that are won and billable — so a bidder can answer fifty thousand requests a second all day and, if it wins none of them, the advertiser’s money is untouched.

The DSP’s own income works the same way, one level up: it typically earns a take-rate, a percentage of the advertiser spend it successfully places. No wins, no revenue. That cuts both ways, and it explains the shape of everything in the pipeline: bidding on everything burns the advertiser’s budget on impressions worth less than they cost, and bidding on nothing earns the DSP nothing. The machine isn’t optimized to be fast, or eager, or cheap. It’s optimized to be right, within 50 milliseconds.

Worth naming plainly, too: every hop in that chain — exchange, SSP, verification vendors, the DSP itself — takes a percentage of what the advertiser spends. The engineering pressure to shave milliseconds and the commercial pressure to shorten the chain are the same pressure.

Who fills the machine with campaigns#

One more piece of the picture, because the bidder is useless without it. Nobody writes code to launch an ad campaign. Someone on the advertiser’s team fills in a form: who to target, what to pay, how much to spend per day, how often one person may see it, which creatives to rotate. That produces rows in a database.

Everything this post calls “the catalog” is that form’s output — tens of thousands of campaigns’ worth of targeting rules, budgets, and caps, which the bidder loads into memory and re-reads every 60 seconds. The auction machinery below is, in the end, a very fast way of answering “which of the things our customers typed into a form applies to the human who just opened a webpage?”

The vocabulary you need for the rest of the series#

The scale that shapes everything#

Public numbers from real players set the design targets — and they matter, because the architecture that survives 1,000 campaigns and the architecture that survives 100,000 are different machines. A claim like “30K requests/sec” is meaningless without stating the catalog and audience size it was measured against.

The problem, reduced to one sentence#

Strip away the ecosystem and the bidder’s job per request is:

Parse the request, figure out who the user is, find which of ~100,000 campaigns are eligible for this user and slot, predict which eligible campaign is most valuable, check it hasn’t been shown too often and still has budget, and answer with a price — in single-digit milliseconds, tens of thousands of times per second.

Each clause of that sentence became a pipeline stage in my implementation. The rest of this post walks through them.

The architecture: six layers, and a nine-stage pipeline at the core#

The system decomposes into six layers, each with one responsibility and a clean boundary to the next. This layering is doing real work: every layer is swappable or testable in isolation, and when a latency graph spikes, the layer structure tells you where to look first.

A request enters at L1, gets decoded at L2 into an internal representation, flows through the L3 stages — which call into L4 services, the only things allowed to touch L5 — and every layer reports into L6.

Two structural rules keep the layering honest:

• The core library has zero I/O dependencies. No HTTP framework, no Redis client, no Kafka client in bidder-core/; everything network-bound lives in the binary crate (bidder-server/). The entire domain logic tests without infrastructure.
• Services hide behind traits (Scorer, FrequencyCapper, BudgetPacer, ExchangeAdapter), so swapping a Redis-backed frequency capper for an in-process one, or a linear scorer for an ONNX model, is configuration, not surgery. Both swaps actually happened during the project — that’s how the abstraction earned its keep.

The heart of the machine is L3: a linear pipeline of stages. Each stage reads and mutates a per-request context and can short-circuit the whole run with a “no bid” outcome. Two enforcement mechanisms wrap the pipeline: every stage has a declared latency budget in configuration (exceeding it fires a metric, so drift is visible in monitoring before it becomes an incident), and the pipeline has a hard deadline (40 ms in my configuration) checked before every stage — cross it and the request immediately returns “no bid” rather than gambling on finishing.

The HTTP layer enforces a 50 ms timeout and a concurrency cap. When the server is saturated it rejects excess requests immediately with an error rather than queueing them — a queued bid request is a request that will miss its deadline anyway, plus it drags down every request behind it. This “shed load early, loudly” posture repeats throughout the design.

Decoding turns wire bytes into an internal request struct. OpenRTB JSON is parsed with simd-json, a SIMD-accelerated parser, because at a few kilobytes per request and tens of thousands of requests per second, JSON parsing is one of the top CPU consumers in the whole system — more on the “bottleneck hierarchy” below. An adapter trait isolates wire formats, so a protobuf-speaking exchange (Google’s, for instance) plugs in as a second implementation without touching the pipeline (bidder-core/src/exchange/).

1. Request validation checks structural invariants and rejects malformed requests before they cost anything.

2. User enrichment answers “who is this user?” by fetching the segment list for the request’s user.id. The lookup order is an in-process cache first (sub-microsecond), then Redis on a miss (a network hop — the single most expensive thing on the path). The segments were computed offline by the behavioral pipeline; the bidder only ever reads. If the fetch fails, the request continues with an empty segment list and a counter increments — a bid with worse targeting beats no bid at all.

3. Candidate retrieval answers “which campaigns are eligible?” — and is the stage where naive implementations die. Checking 100,000 campaigns’ targeting rules linearly at ~10 µs each would cost $100{,}000 \times 10\,\mu\text{s} = 1\,\text{s}$ per request — twenty times the entire SLA. Instead, the catalog is held in memory as inverted indices, the same trick as the index at the back of a book: you don’t read every page hunting for “sneakers”, you look up “sneakers” and get the exact page list. Here, every segment, geography, device type, and ad format maps to a compressed bitmap (RoaringBitmap) of the campaign IDs targeting it, and eligibility becomes a handful of bitmap unions and intersections — a few hundred microseconds. The catalog rebuilds from Postgres every 60 seconds on a background task and is swapped in atomically; the database is never queried during a request. (bidder-core/src/catalog/)

4. Candidate limit truncates to the top ~20 candidates by price, so the expensive stages that follow have bounded cost regardless of how broad the request’s eligibility was.

5. Scoring ranks the survivors by predicted value. Behind a trait sit several implementations: a linear feature-weighted formula, an ONNX neural model executed in-process (no network hop to a model server — it wouldn’t fit the budget), a cascade (cheap model ranks everything, expensive model re-scores the top slice), and an A/B splitter that routes a percentage of traffic to a challenger model. Part 3 of this series is entirely about this stage, because serving ML under a hard latency budget is its own discipline. (bidder-core/src/scoring/)

6. Frequency capping drops candidates the user has already seen too many times this hour or day. It sounds trivial — read a counter, compare — but at 50K requests/sec with dozens of candidates each, it is potentially millions of counter reads per second, and it became the defining performance story of this project — the next section shows exactly how badly. (bidder-core/src/frequency/)

7. Budget pacing filters out candidates whose campaign has exhausted its spending allowance. Each campaign has an in-memory budget counter, decremented atomically; a candidate reserves its bid amount, and — a subtle but important detail — losing candidates get their reservation released after ranking, otherwise budgets drain on bids that never happened. (bidder-core/src/pacing/)

8. Ranking picks the highest-scoring survivor per ad slot, tie-breaking by price. Worth being explicit, because it surprises people: the bidder returns one bid per slot, not a shortlist. Thousands of campaigns were narrowed to twenty, scored, filtered, and collapsed into a single number and a single creative. The exchange wants your best answer, not your reasoning.

9. Response build assembles the OpenRTB response: price, creative reference, and the win-notice URL — which the bidder signs with HMAC, because that URL, when called, increments spend counters. An unauthenticated win endpoint would let anyone fabricate wins, poison your spend accounting, and suppress competitors’ ads by inflating their frequency counters. The win endpoint verifies the signature in constant time and deduplicates replays via Redis.

After the response is on the wire, side effects happen fire-and-forget: impression counters flow to Redis through a write-behind queue, and business events (bids, wins) flow to Kafka from a bounded queue drained by a dedicated thread. Nothing downstream of the response — not Kafka being slow, not a metrics scrape — can add a millisecond to bid latency, by construction.

The loop closes#

Those fire-and-forget writes are not bookkeeping. They are what makes the next auction smarter.

The win notice increments the spend counters that stage 7 reads. The impression increments the frequency counters that stage 6 reads. And the bid/win/click stream landing in Kafka is the training data for the pCTR model that stage 5 uses to rank — every impression the bidder serves is simultaneously an ad delivery and a labelled training example (shown to this user, in this context, clicked / didn’t click).

So the honest picture isn’t a pipeline, it’s a cycle: the hot path spends microseconds consuming what the slow path computed, and emits the raw material the slow path will consume next. This post is about the fast half. The slow half — offline segment builders, model training, attribution — is a comparable amount of engineering that simply doesn’t happen in 50 ms.

Where does the price actually come from?#

One thing the nine stages above quietly skip: the number. Scoring decides which campaign deserves the impression, but what determines that it’s worth $2.10?

In production DSPs, the bid is derived, not configured. An advertiser tells you what an outcome is worth to them — say $4.00 per click — and the model tells you how likely this particular impression is to produce one. Multiply, convert to CPM, and you have a bid:

A 0.2% predicted click rate on a $4.00 click is worth $8.00 per thousand impressions. Halve the predicted rate and the correct bid halves with it. This is why pCTR accuracy is money and not a metric: it is the price. Everything else — pacing, capping, ranking — modulates a number the model produced.

Where the milliseconds actually go#

Every description of a bidder you’ll read — including everything above — is a story about what the stages do. None of it tells you what they cost, and the ranking is never what you’d guess. In a system with a deadline, that ranking is the only thing that decides whether you make it.

There’s no clever way to find out. You time each stage separately, record it as a histogram, and read the numbers. Here is that reading for this bidder — a 10K RPS run on a dev machine, back when frequency capping still went to Redis on every request — with each stage’s declared budget beside what it actually cost:

Three things fall out of that table that no diagram would have told me — and none of them are specific to advertising.

The typical request is cheap and the tail is not. At p50 the nine stages total about 200 µs. At p99, frequency capping alone is 6.3 ms — a 175× spread inside a single stage. The two halves aren’t even doing the same work: the requests that reached Redis paid for an MGET of ~50 keys, while 37% of requests in that run skipped the stage entirely because the circuit breaker in front of Redis was open. The cheap p50 was partly the system already degrading — visible only because the skip increments its own counter.

Being inside budget is not the same as being healthy. Frequency capping cleared its budget at p99 and still consumed roughly three-quarters of average pipeline time. What gave it away was the budget-overrun counter: 14,402 violations in that run, against 5,733 for candidate retrieval and a few hundred for all seven remaining stages put together. Percentiles said “fine”, the contract said “not fine”, and the contract was right.

Fixing a bottleneck doesn’t remove the bottleneck — it moves it. Caching frequency counters at the right granularity took that stage from 1,034 µs average to 9.2 µs, about 112×, and dropped it from 75.7% of pipeline time to 2.8%. Candidate retrieval then became ~90% of pipeline time without getting one microsecond slower; it had merely been standing in the shadow of something larger. Part 4 tells that whole story — the diagnosis used nothing but curl, grep, and division against the bidder’s own Prometheus histograms.

The supporting cast: one store per access pattern#

A bidder needs several kinds of persistence, and a useful design rule is that each store does exactly one kind of job — the way a kitchen separates the pantry, the countertop, the receipts drawer, and the oven thermometer:

Why not query Postgres per request? At 50K RPS even a 1 ms query would dominate the budget — so we trade up to 60 seconds of catalog staleness for predictable latency. Why not send metrics through Kafka? A metric increment costs ~10 nanoseconds; a Kafka publish costs microseconds and ~100 bytes — multiplied across every counter of every request, that’s gigabytes per second of traffic to convey what Prometheus captures in a scrape. Events and metrics are different things; conflating them is a common early mistake.

The interesting engineering in this layer is Redis key design — hash-tagging keys so a user’s data co-locates on one cluster shard, packing segment lists as raw binary instead of JSON (about 2.5x smaller, decodes at memcpy speed), and the sobering arithmetic that 100M users of segments and counters simply does not fit one Redis instance. Part 1 covers this in detail.

The design principles that did the most work#

Four ideas shaped almost every decision, and I’d carry all of them to any latency-sensitive system:

Know your bottleneck hierarchy before writing code. For this workload, expected cost ranks: (1) Redis network round-trips, (2) JSON parsing, (3) syscalls, (4) memory allocation, (5) actual compute. Rust’s compute speed is the least important factor — and if profiling ever shows compute dominating before network does, that’s a symptom of a missing index, not something to celebrate. This ordering told me where design effort belonged (caching, key design, parse efficiency) and what to ignore (micro-optimizing arithmetic).

A latency budget is a contract, not a comment. Every stage declares its budget in configuration; violations increment a per-stage metric, so regressions name their guilty stage in monitoring before anyone opens a profiler. And when every counter is green but the pipeline still feels heavy, the same per-stage histograms answer the deeper question — where does the time actually go? — which is exactly how the project’s biggest bottleneck was found (Part 4).

Never miss the SLA silently — drop, degrade, or shed, but always fast and observable. If Redis is slow, skip frequency capping for that request and count it, rather than blocking. If the ML model fails, fall back to the simple scorer, rather than not bidding. If the server is saturated, reject instantly, rather than queueing. Bid quality may dip during degradation; the deadline never does. And every one of those degradations increments a named counter — nothing absorbs load silently. Part 4 opens on this machinery: circuit breakers, hedged requests, bounded queues.

Profile before optimizing, and document negative results. Several planned optimizations (an arena allocator, an io_uring runtime) were dropped because measurements showed they’d address costs that weren’t there. Writing down “we tested this assumption and it was wrong” turned out to be as valuable as any speedup.

What it achieved, and what’s ahead in the series#

On a single developer machine (Apple Silicon laptop, with the load generator and databases sharing the same hardware), the final system sustained 50,000–75,000 requests per second with a p99 latency of 3.4–4.3 ms — more than tenfold inside the 50 ms SLA — and zero HTTP errors across 12.4 million measured requests over four tiers. Getting there involved finding one bottleneck that consumed three-quarters of the pipeline (found with nothing more than curl, grep, and division), a cache that never learned from its own misses, costing 25 needless Redis trips per request, and a load-test harness bug that manifested as a single request appearing to take 17 minutes.

Two numbers that headline figures usually leave out, because they’re the interesting ones:

The p99.9 is where the pipeline’s hard deadline stops being a design document and starts being load-bearing: those requests were one scheduling hiccup away from returning a no-bid instead of a bid, which is exactly what it’s there for. The multi-second max is a single sample — the load generator, the databases, and the bidder were all competing for the same laptop, and TCP retransmits and page reclaims land there. On this rig it’s noise; on a real fleet it would be a paging alert.

The 71% bid rate is worth sitting with too. A bidder that bids on everything isn’t working — it’s not capping, not pacing, or not targeting. Nearly a third of the traffic being answered “no thanks, and here’s the reason code” is the machine doing its job.

Equally important is what those numbers don’t say, and each post in this series keeps its limitations section honest: it’s one machine, a synthetic corpus, and — the conclusion I find most useful — the largest win came from caching architecture that would work in any language, not from Rust itself.

References & further reading#

• The bidder itself — github.com/HumbleBee14/rust-rtb-bidder: every file path cited above, the golden fixtures, the load-test harness, and the raw results
• OpenRTB specification — IAB Tech Lab
• Introduction to real-time bidding and callout quotas — Google Authorized Buyers
• Challenges in building a scalable DSP — Moloco R&D
• Building a real-time bidder on AWS — AWS reference architecture
• Programmatic advertising data flow — the RTB lifecycle with per-stage latency budgets
• The Trade Desk’s user store — Aerospike case study
• First-price auctions for Google Ad Manager — the announcement that moved the industry off second-price
• Bidding and Auction services architecture — Google Privacy Sandbox
• Roaring Bitmaps — the bitmap format behind the retrieval indices

Disclaimer: I don’t work in ad tech. This series documents a personal project built to learn high-performance systems engineering, using real-time bidding as the problem domain because its constraints (hard latency SLAs, large working sets, high request rates) are unusually honest teachers. Industry figures cited are from public sources; my own numbers are from synthetic workloads on developer hardware and are reported with their caveats. Corrections from people who do build these systems for a living are very welcome.