Building a Real-Time Ad Platform Under a Millisecond Budget

Priya opens Wanderlust Weekly’s trekking article. Urban Hiker’s DSP has an approved boot advertisement, a campaign with room to spend, and a model that can estimate the value of the opportunity. The exchange allows 120 milliseconds for a response.

Then the feature service slows down.

Should the bidder wait for a better prediction, use an older feature, switch to a simpler model, or decline the auction? The answer depends on what is missing. An optional contextual score and required permission to use an identifier cannot share the same fallback rule.

The OpenRTB article explained the messages and the ML article explained the prediction. This article connects them to a running system: one that can make a useful, permitted decision before the deadline, account for its consequences, and recover when its dependencies fail.

All services, identifiers, timings, prices, and failure scenarios below are constructed teaching examples. We design only the DSP bidder, as one possible implementation rather than an OpenRTB requirement.

---

1. Put Preparation Outside the Auction Path

A campaign editor can spend seconds validating an update. A training job can spend hours producing a model. An auction handler cannot synchronously repeat either process for every request.

We separate the system into three responsibilities:

• Preparation: validate campaign configuration, approve creatives, train models, and publish versioned artifacts.
• Auction serving: admit the request, enforce policy, identify eligible candidates, retrieve permitted features, score, reserve spending authority, and respond.
• Feedback: ingest outcomes, reconcile charges, update operational state, and build reporting and training data.

These responsibilities need not become dozens of microservices. The deadline may favor putting the model and frequently used configuration in the bidder process. A network boundary should earn its cost through a concrete operational benefit.

Figure 1. Prepare outside the auction; commit before responding. Preparation supplies the serving path; outcomes feed the feedback path. Required local reservation durability remains on the serving path even though reporting and training happen asynchronously.

Suppose we operate West and East regions. We place bidder capacity near the exchange endpoints we serve, with regional feature caches and model replicas. We measure the actual network path rather than infer latency from a region’s name. Data-location restrictions can also limit where a request may be processed.

The campaign configuration should arrive as a coherent snapshot, the point our campaign lifecycle article made when it separated the configuration system from the serving system. A model expecting feature schema 12 must not silently consume schema 11 because two updates arrived in different orders. We publish a manifest describing compatible configuration, feature transformation, model, and policy versions, and we validate and warm the bundle before activating it.

A last-known-good bundle can keep serving through a publishing outage only within its validity rules. An emergency campaign pause, expired authorization, or required policy update can make an otherwise functional old bundle unusable.

Regional isolation also needs an explicit recovery plan. Redirecting all West traffic into East can overload East and exceed its permitted spending allocation. Failover should depend on spare capacity, warmed artifacts, valid policy state, and independently available spending authority.

---

2. Turn the Exchange Allowance into a Local Deadline

OpenRTB’s tmax in the request specification includes network latency, so only part of that allowance is compute time for the model. Our OpenRTB article budgeted this journey from the exchange’s side; here we look inside the bidder’s share of it, with a trace that includes reservation and queueing, which is why the stages differ.

For our 120 ms example, plan for 15 ms inbound travel, 20 ms outbound travel, and a 10 ms safety margin. That leaves 75 ms from bidder receipt to sending the response:

120 − 15 − 20 − 10 = 75 ms local allowance

Inbound travel cannot be recovered from tmax alone. In production, derive a conservative local allowance from the integration’s timing contract and measured network behavior. Use a monotonic clock for local elapsed time; do not compare unrelated machines’ monotonic timestamps.

Here is one successful request’s local trace:

Stage	Local interval	Duration
Admission, parsing, and initial policy checks	0–8 ms	8 ms
Permitted feature reads in parallel	8–26 ms	18 ms
Candidate scoring and bid selection	26–46 ms	20 ms
Final eligibility and spending reservation	46–56 ms	10 ms
Encode and send response	56–61 ms	5 ms

One feature read takes 6 ms and another 18 ms; because they run concurrently, the stage takes 18 ms, not 24. Queueing and scheduling are included in the intervals shown.

The response leaves at local time 61 ms and arrives at the exchange at 15 + 61 + 20 = 96 ms. It has 24 ms of observed headroom. Under our planning allowance, local sending must finish by 75 ms, with decision work stopping by 70 ms to leave 5 ms for encoding and sending.

Figure 2. A 120 ms auction leaves a smaller local work window. The upper row is a planning budget; the lower row is one observed execution, magnified from the 75 ms local allowance. The local clock starts when the bidder receives the request. Reserved network time is an estimate, not a delivery guarantee.

We pass the remaining allowance downstream. A feature call starting at local time 8 ms cannot receive a fresh 75 ms timeout. Its timeout is bounded by both its stage allocation and the time left for dependent work. The gRPC deadline guide describes propagation and cancellation; applications must also stop work they spawned when the request is cancelled.

Cancellation does not undo a completed side effect. If a spending reservation timed out after committing, retrying with a new reservation ID can create a second hold. We use an idempotent operation key and resolve uncertain results before assuming nothing happened.

Do not add each service’s p99 latency and label the sum an end-to-end p99. Tail events can be dependent, stages can overlap, and traffic mixtures differ. We measure whole-request deadlines under representative load, then use stage traces to explain where the time went.

---

3. Bound Work Before Queues Consume the Budget

A request waiting 50 ms in a queue has already spent that time, even if the model itself takes only 5 ms. An overloaded bidder can report excellent inference latency while missing most auctions.

Little’s law says that the average number of requests in a stable system equals the arrival rate times the average residence time. For a workload of 20,000 requests per second and 50 ms average residence time, that gives about 1,000 requests in the system on average. If the same admitted throughput instead requires 100 ms residence time, the implied average is 2,000. The relationship holds as an accounting identity for stable averages and makes no capacity forecast for an overloaded transient.

We set limits on in-flight work, queue length, candidate count, and per-request fan-out. We reject work that has insufficient remaining time before it consumes expensive scoring resources, and we maintain a cheap overload response path independent of the dependency that is already failing.

The Google SRE discussion of overload describes degraded responses and load shedding. For our bidder, the appropriate degradation is specific: fewer eligible candidates, an approved simpler model, or a prompt no-bid. We cannot degrade away a required policy or spending check.

Retries deserve particular restraint: they can amplify the overload that caused the timeout, and an optional feature timeout rarely justifies several fresh requests within the same auction. If we retry a call, we bound the attempt count, respect the original deadline, and ensure repeated operations cannot duplicate side effects.

When failures persist, a circuit breaker can stop repeatedly calling the unhealthy dependency, with controlled probes to detect recovery. Separate capacity pools for critical eligibility work and optional enrichment prevent enrichment from consuming every worker or connection.

Model batching is another tradeoff: larger batches can improve throughput while adding wait time to individual requests. We evaluate the batch queue’s contribution to the auction deadline. Settling the serving choice needs a deadline-miss rate alongside the throughput benchmark.

---

4. A Cache Hit Needs Meaning, Freshness, and Permission

The bidder’s feature store may return a value quickly and still return the wrong input for the decision.

For each feature, we define its key, type, transformation version, event timestamp, freshness limit, and permitted uses. We distinguish a missing value from zero. A missing purchase count is unknown; treating it as zero purchases changes model meaning.

Freshness limits should follow consequences. A broad page-category classification may tolerate a longer refresh interval than an account suspension or campaign pause. A single cache TTL for every kind of state hides these different risks.

We avoid turning every cache miss into a synchronous database query. Many simultaneous misses can overwhelm the origin store just when latency is already poor. We precompute common features, bound refresh work, and document the missing-feature path. If a fallback model needs a different feature set, it gets its own validated schema.

Permission is an input to routing

We apply the initial policy decision before fetching restricted features or joining identifiers. Checking permission after enrichment is too late to prevent the enrichment itself.

Our example has two explicitly reviewed routes: a personalized route requiring particular signals and permissions, and a contextual route using an independently permitted set of inputs. “Contextual” names a model configuration; every incoming field still needs its own permission to be retained or used.

Missing or unhealthy component	Possible response in this design
Optional contextual feature	Use its documented default or a compatible fallback model
Personalized feature service	Use the approved contextual route if independently eligible
Evidence required to authorize an identifier join	Do not perform the join; use another independently permitted route or no-bid
Required creative approval or campaign status	No bid for the affected candidate
Spending authority	No bid until sufficient authority can be established

These are design policies; the organization’s reviewed policy implementation must encode the applicable legal obligations. An absent or unreadable signal is not an affirmative authorization.

Consent state can change while caches remain warm. We include invalidation and bounded validity for permission-derived state and recheck relevant critical state before emission. A revocation arriving after emission cannot recall an auction response, so propagation guarantees and the permitted exposure window need an explicit design. A freshness target also needs detection of stalled updates.

---

5. Two Regions Cannot Spend the Same Remaining Rupee

The campaign lifecycle article established shared budget constraints and separated confirmed cost, outstanding exposure, and available room. Distribution adds a race: West and East can both read “₹1,000 remaining” and each decide that a ₹700 purchase fits. Local arithmetic is correct; the combined promise is not.

One design is a centralized reservation service. Another distributes bounded spending rights ahead of auctions. Either requires coordination somewhere; moving it out of the request path changes only when it occurs.

For an isolated media-budget example, partition ₹1,000 of remaining authority into ₹400 for West, ₹300 for East, and ₹300 unallocated. Each region atomically checks and reserves against its own grant. The grants are non-overlapping, so neither region treats the other’s unused allocation as locally available.

Figure 3. Partition spending authority before regional bidding. Regional grants partition one authority envelope. Settled charges plus outstanding reservations plus available rights remain within each grant, and a timeout alone does not return an unresolved hold to the pool. This media-only example does not remove separate checks for fees or the campaign’s all-in ceiling.

Before returning the ₹85 CPM bid from our recurring auction, we reserve the maximum media liability allowed by its price and settlement rules. In this single-impression first-price case, that is ₹85 / 1,000 = ₹0.085, assuming no additional media charge. We represent this exactly in suitably fine fixed-point units; rounding each impression to whole paise would distort the ledger. Fees need their own treatment under the campaign’s total ceiling.

A reservation is not spend. A verified billable event converts an appropriate hold into a charge. A confirmed loss can release it. Missing feedback leaves uncertainty that must be resolved under the integration’s rules. Figure 3 shows a later snapshot: West has ₹120 settled, ₹80 held, and ₹200 still available within its original ₹400 grant, and East has ₹50 settled, ₹100 held, and ₹150 available within its ₹300. Settling an ₹80 hold moves it between categories without adding another ₹80 to the grant.

Lease expiry alone must not recycle money that an old worker can still commit or that an emitted bid can still incur. Reallocation needs fencing of old grant holders, durable accounting for outstanding bids, and a defined reconciliation process. Otherwise “failover” can create two owners of the same authority.

This design trades some utilization for bounded exposure. West might temporarily decline bids while East has unused rights. That is preferable to presenting a hard ceiling while silently assuming every replica’s delayed counter is current.

Frequency caps have a similar concurrency problem that money grants leave unsolved. A strict cap for a particular identifier needs a defined ownership or coordination strategy. If a cap is approximate or identity-scoped, expose that limitation rather than advertise a person-level guarantee the system cannot enforce.

---

6. A Response Is Not a Durable History of What Happened

We separate the record that a bid was emitted from later evidence that it won, rendered, became billable, or produced an attributed conversion. Our OpenRTB notification discussion explains why those events are distinct.

A compact internal decision receipt could look like this:

{
  "decision_id": "decision-713",
  "auction_id": "auction-8f3a1c",
  "imp_id": "slot-1",
  "region": "west",
  "config_version": "campaign-42",
  "model_version": "contextual-17",
  "feature_schema": "context-v12",
  "policy_version": "policy-9",
  "route": "contextual",
  "outcome": "bid",
  "elapsed_ms": 61,
  "bid_cpm": "85.000",
  "currency": "INR",
  "reservation_id": "reserve-713"
}

The decimal price is stored as a string to preserve its intended representation; most JSON parsers would otherwise read it into a binary float, the same rounding hazard section 5 raised for the ledger. The receipt identifies the decision and versions; it omits raw identifiers, full page URLs, and feature payloads. Even these join keys need access and retention controls.

The reliable record boundary must be explicit. If the bidder sends a response and crashes before writing anything durable, asynchronous logging cannot reconstruct the missing decision. In this design, the reservation and minimal decision record commit durably before emission, with an outbox for later event publication. Analytics need not finish before the response, but required durability contributes to the serving budget.

An outbox closes the gap between the local state change and publication to the event system. It does not atomically combine our database transaction with delivery to the exchange. A crash can leave a durable reservation for a bid never received. We preserve that uncertainty and reconcile it rather than counting every reserved bid as a win.

Duplicate delivery must not duplicate business effects

Our OpenRTB article introduced idempotent processing for exactly this case; here we add what the bidder’s ledger must do about it. Imagine these callbacks arrive after the decision:

Arrival	Event time	Meaning	Ledger effect
10:00:11	10:00:04	Verified billable event bill-713	Settle the matched ₹0.085 hold once
10:00:12	10:00:04	Replay of bill-713	No second charge
10:00:14	10:00:00.125	Delayed win notification	Record the win; no additional charge

For this example, the billable event contains sufficient verified information to match and settle the reservation independently of the delayed win callback. Otherwise it would enter a pending reconciliation state.

Figure 4. Arrival order does not determine what we charge. Arrival order is not event order. Deduplication uses the event’s identity and kind; settlement applies a business rule rather than charging once per received message.

A message broker cannot infer that two payloads represent the same business event. We define producer identity, event type, stable event ID, and revision semantics. A correction needs an explicit revision or adjustment, not accidental deletion as a duplicate. We couple the deduplication decision with the ledger mutation atomically; otherwise a crash between them can still lose or repeat an effect.

Kafka’s delivery-semantics documentation distinguishes guarantees within Kafka processing from effects in external systems. A transactional stream does not automatically make an unrelated billing database exactly-once. The sink must participate in an appropriate protocol or make repeated operations idempotent.

---

7. Let Late Data Correct the Story Without Rewriting the Past

Event time describes when an event occurred; processing time describes when a system handled it. Both belong in the pipeline. The distinction matters when a conversion reaches us long after the auction that preceded it.

A watermark is a stream processor’s estimate of event-time progress; older events can still arrive after it. Flink’s time model explains this mechanism. We choose explicitly what happens to events arriving after a reporting window is emitted: update the aggregate, send an adjustment, retain them for reconciliation, or reject them under a documented rule.

A dashboard window can close while the financial record stays open to correction. Reporting, billing, and training can need different correction horizons. Deduplication state must also survive the replay horizon we promise; deleting it after a day while replaying a week of callbacks reintroduces old charges unless another layer prevents them.

Feedback is selected by our own bidding

The model influences which bids we send. Bids influence which auctions we win. Wins influence which impressions and outcomes we can observe. Training on those outcomes feeds the next model.

Figure 5. The bidding policy selects its own future observations. The bidder changes the data it later learns from. Missing outcomes on unbought opportunities are not negative conversion labels.

An apparent conversion-rate improvement can therefore reflect buying a different mix of inventory. We record the deployed policy and relevant selection information, distinguish unmatured labels from negatives, and evaluate changes using comparable cohorts. The attribution and incrementality article explains why an attributed outcome is also different from an outcome caused by advertising.

Training features must match information available at decision time. A feature’s event timestamp can precede an auction even though the system only received it afterward. A timestamp join alone may leak that later knowledge into training. We preserve availability or ingestion history where needed to reconstruct what the bidder could actually know.

Feast’s point-in-time join documentation describes selecting historical feature values relative to entity timestamps and freshness windows. We still need to ensure our historical data models delayed arrival and our actual serving transformations correctly.

We monitor both input changes and outcome changes: missingness, feature age, score distributions, calibration on mature labels, and results by supply segment. A model cannot repair a feature pipeline that silently changed units. Retraining automatically on a broken feed can institutionalize the incident.

---

8. Measure Usable Decisions, Not Just Fast HTTP Responses

A no-bid sent promptly because no campaign is eligible is a valid decision. A bid rejected for malformed creative metadata is wasted demand. A late bid with a high score misses the same deadline as a late low score.

We keep separate counters for received requests, admission rejects, policy rejections, no eligible candidate, missing required state, intentional no-bids, emitted bids, and partner-reported late or invalid responses, and we define denominators before calling a percentage “availability”.

For example, reporting latency only for emitted bids hides requests abandoned while waiting for features. A system can improve that latency statistic by giving up on harder traffic. We pair it with deadline misses and outcome counts over all relevant admitted requests, and we account separately for shedding before admission.

We instrument stage duration, queue wait, remaining budget at each dependency call, feature freshness, fallback route, reservation uncertainty, and artifact versions, and we slice by exchange, region, and route while keeping high-cardinality request IDs in controlled traces or receipts rather than unbounded metric labels.

We use the partner’s evidence where available. Local “sent before deadline” differs from “received before deadline,” and absence of a win notification alone does not prove a timeout or a loss. The ₹92 bid arriving at 135 ms in the OpenRTB deadline example is the concrete failure we are trying to distinguish.

Operational dashboards should also expose event lag and reconciliation backlog. The bidder can remain fast while delayed billable events make spend appear low. A controller that responds by raising participation may then accelerate into an understated liability position.

The most useful alert points to a broken promise: unusable responses, stale required state, uncertain liabilities beyond a bound, or labels too incomplete to support an optimization decision. CPU usage helps explain those failures without defining them.

---

9. Roll Out a Compatible Decision System

We treat the model, transformations, feature schema, candidate rules, and compatible configuration as a release bundle. We check shape and type compatibility, warm caches, and replay sanitized representative requests before exposing the bundle to live spending.

A shadow deployment computes a candidate result without emitting a second bid or creating spending reservations. We isolate its resource budget to keep the evaluation itself from causing production timeouts. Shadow scores reveal disagreements, but cannot directly reveal the win and conversion outcomes that would have followed bids we never sent.

Then we use a bounded canary with a comparable control population and define technical guardrails such as deadline-miss rate and invalid responses, plus spending, policy, and quality checks. The SRE guide to canary releases emphasizes evaluating a limited deployment before broader exposure.

Immediate latency checks and delayed conversion checks run on different clocks. Do not promote solely because five minutes of canary traffic has no mature purchase labels. Shared campaign budgets can also make treatment and control interfere, so the assignment and resource allocation should match the claim the experiment intends to support.

A rollback must restore a compatible bundle, not only the model file. We keep formats readable across the transition and record which version made each decision. Current campaign pauses, exhausted budget, and privacy restrictions must remain effective: a rollback must never resurrect an old authorization snapshot.

It also cannot recall a bid already emitted or erase a valid charge. After a faulty release, we stop the harmful behavior, preserve the relevant evidence under retention controls, and reconcile outstanding decisions separately from restoring serving health.

---

10. Walk Through the Feature-Service Incident

Return to our opening request. At local time 8 ms, two permitted feature reads begin. One returns at 14 ms. The other stalls, reaching its allocated cutoff at 26 ms.

The bidder cancels that read and records an optional-feature timeout. Because this request independently qualifies for our reviewed contextual fallback, it runs the compatible local model. If the missing dependency instead established required permission or spending authority, this branch would terminate in a no-bid.

After scoring, the bidder rechecks eligibility, reserves the possible charge against West’s valid grant, and durably records the decision. It sends at local time 61 ms. Under the stipulated network timings, the exchange receives it at 96 ms. The missing feature changed the decision route; it did not extend the auction’s deadline.

If the reservation’s result is uncertain at the decision cutoff, we send a no-bid rather than an unbacked bid, record the unresolved operation, and reconcile the possible hold. If the healthy region lacks capacity, we do not redirect the entire incident into it. If reporting is delayed, we treat unreported spend as outstanding, never as newly available money.

This is the architecture’s practical contract. Some information can arrive later and correct reporting. Some work can be simplified. Some state must be established before we make a commitment. The system must know which category each dependency belongs to before the incident begins.

The money ledger benefits from bounded reservations and duplicate-safe settlement. The data ledger benefits from permission checks before enrichment, minimized decision receipts, and retention rules that cover caches, traces, replay storage, and training datasets. Debugging an outage never justifies logging every identifier or replaying data beyond its permitted use.

For Priya, those boundaries determine what the platform does with her page visit, not merely how many milliseconds the DSP saves. In the next article, we investigate the browser-visible side of that same flow: which requests leave the page, what they carry, and which downstream behavior remains outside the observation.

---

Resources

• IAB Tech Lab, OpenRTB 2.6 BidRequest: tmax and the response allowance (section 2).
• gRPC, deadlines: deadline propagation and cancellation (section 2).
• Google SRE book, handling overload, and SRE workbook, canarying releases: load shedding and bounded rollouts (sections 3 and 9).
• Apache Kafka, message delivery semantics: what exactly-once does and does not cover (section 6).
• Apache Flink, timely stream processing: event time, processing time, and watermarks (section 7).
• Feast, point-in-time joins: leakage-free historical features (section 7).