Market Data

Learning outcomes

Every price you have ever seen on a screen, every fill your order ever got, every risk number your firm reports at the close, traces back to a stream of tiny messages flowing out of exchanges faster than you can blink. Market data is the bloodstream of trading infrastructure. Get it right and the rest of the stack has firm ground to stand on. Get it wrong, even subtly, and you will quote stale prices, cross your own book, fail an audit, or pay a licensing bill with an extra zero you did not expect.

After studying this page, you can:

• Explain what market data is, how it differs from reference data, and why a trading system treats the two completely differently.
• Distinguish quotes from trades, and read level 1, level 2, and level 3 data, saying exactly what each level shows and hides.
• Describe the consolidated tape and the role of a securities information processor, and explain why a direct exchange feed beats it on latency but costs far more.
• Trace a market-data message from the exchange through a feed handler that normalizes, sequences, detects gaps, recovers, and conflates, and say what each stage protects against.
• Explain why market data is distributed over multicast, and what that choice buys and costs compared to point-to-point delivery.
• Reason about timestamps and clock synchronization, and say why a few microseconds of clock error can corrupt a strategy or break a regulatory reconstruction.
• Build the core of an order book from incremental updates, and name the invariants that tell you it has gone wrong (stale ticks, crossed books, dropped sequences).
• Treat market-data licensing and entitlements as a first-class legal and cost concern, not an afterthought, and recognize entitlement leakage when you see it.

Before we dive in

You do not need to have worked at an exchange to follow this. We will define each term the first time it appears, and build up from the simplest idea.

A security is a tradable instrument: a share of stock, a bond, a futures contract, an option. A venue or exchange is a place where buyers and sellers meet to trade a security, for example Nasdaq or the New York Stock Exchange. An order is an instruction to buy or sell a quantity of a security at some price. The bid is the highest price someone is currently willing to pay; the ask (also called the offer) is the lowest price someone is currently willing to sell at. The gap between them is the spread. A quote is the current bid and ask. A trade (also called a print or an execution) is what happens when a buy and a sell actually match and change hands.

Market data is the stream of messages a venue publishes describing this activity: quotes changing, trades printing, orders arriving and cancelling. A tick is one such update, one small message. A feed is the channel the venue publishes its ticks on. A feed handler is the software your firm runs to receive, decode, and make sense of a feed. An order book (or just book) is the live, sorted picture of all resting orders for a security: all the bids on one side, all the asks on the other.

One convention before we start. The numbers, message rates, and latencies in this page are realistic orders of magnitude drawn from US equities and listed derivatives. Exact figures move year to year and venue to venue, so treat them as the right shape, not a spec to quote back. The principles underneath them do not move.

Mental Model

The wrong model, and almost every newcomer holds it, is that “the price” of a stock is a single number that the market hands you, the way a thermometer hands you a temperature. You picture one true price, published by one authority, that everyone sees at the same instant.

None of that is true, and the gap between that picture and reality is where most market-data bugs are born. There is no single price. At any instant there is a best bid and a best ask, and they are different numbers; the “price” a casual app shows you is usually the last trade, which is already in the past. There is no single authority: the same security trades on a dozen venues at once, each publishing its own view, and what you call “the market” is a stitched-together composite of all of them. And there is no single instant: every consumer sees the data after a delay that depends on where they sit, how they receive it, and how much they paid, so two firms can hold genuinely different, both-correct pictures of the market at the “same” moment.

Here is the model to hold instead. Picture market data as a river of events, not a snapshot of a state. The venue does not publish “the price”; it publishes a relentless stream of small facts in order: a bid appeared at 10.01, a hundred shares traded at 10.00, that bid was cancelled, a new ask appeared at 10.02. Your job is never to ask “what is the price.” Your job is to receive every event, in order, with nothing dropped, and replay them to reconstruct the current state for yourself. The “price” is just a query against the state you rebuilt. Once you see market data as an ordered event stream that each consumer independently replays, every rule below (sequencing, gap recovery, clock sync, the order book itself) falls out as a consequence of getting that replay right.

Breaking it down

The core teaching runs in twelve steps. The first five establish what the data is and where it comes from. The middle four are the engineering of receiving it correctly. The last three cover time, the live book you build from it, and the legal and operational realities that surround it.

1. What market data is and why everything downstream depends on it

Start from first principles. A market exists so that buyers and sellers can discover a price and exchange a security. For that to work, participants need to see what is on offer. Market data is simply the venue broadcasting that visibility: here is what people want to buy and sell, here is what just traded. Without it, you would be bidding blind.

Everything in a trading firm sits downstream of this stream. The pricing engine needs current quotes to value a position. The execution system needs the book to decide where to send an order. The risk system needs a mark for every holding to compute exposure. The compliance system needs an accurate record of the market state at the moment of each trade to prove the trade was reasonable. The surveillance team needs to replay the tape to investigate a suspicious print. If the market-data layer is late, lossy, or wrong, every one of those systems is late, lossy, or wrong in turn, and the errors are expensive: a stale price means a bad fill, a dropped tick means a book that disagrees with reality, a clock that drifts means a regulatory reconstruction that does not hold up.

This is why market data is treated as a hard real-time, correctness-critical discipline rather than a convenience. The data is high volume (a busy US equities day can push tens of millions of messages per second across all venues at peak), it is ordered (a cancel only makes sense after the order it cancels), and it is unforgiving (there is no “eventually consistent” for a price you are about to trade on). The rest of this page is, in effect, the set of techniques the industry developed to receive a firehose of ordered events fast enough and faithfully enough that everything downstream can trust it.

2. Quotes trades and the three levels of depth

Two kinds of fact dominate a feed, and confusing them is a classic beginner error. A quote is intent: someone is willing to buy or sell at a price, but nothing has happened yet. A trade is history: a buy and a sell actually matched and changed hands at a price. Quotes update constantly as participants post and pull orders; trades print only when two orders cross. The last trade tells you where the market was; the quote tells you where it is now. A risk mark usually uses the quote (the mid-point between bid and ask) precisely because the last trade can be stale.

How much of the book you get to see is described by three levels, and the level you license determines both what you can do and what you pay.

The crucial point is that the levels are nested: level 2 contains level 1, and level 3 contains level 2. They are not different data, they are different resolutions of the same book. A level 3 feed (often delivered as a stream of order add, modify, and delete messages) is the ground truth; level 2 is that ground truth aggregated by price; level 1 is the very top of that aggregation. Higher levels cost more, carry far more messages, and demand more from your feed handler, which is exactly why a firm licenses the lowest level that supports what it actually does.

3. The consolidated tape and the SIP

A single US-listed stock can trade on more than a dozen exchanges and dozens of alternative venues at once. If every consumer had to subscribe to every venue and stitch the views together just to answer “what is the national best price,” the market would be a mess of fragmented, inconsistent pictures, and smaller participants would be locked out. So the regulatory framework requires a consolidated tape: a single combined stream that merges quotes and trades from all the lit venues into one public view.

The machinery that produces it is the securities information processor, the SIP. In US equities there are two SIP plans by historical convention: one centered on NYSE-listed names and one on Nasdaq-listed names. Each venue sends its quotes and trades to the relevant SIP, the SIP consolidates them, computes the national best bid and offer (the NBBO, the best bid and best ask across all venues), and republishes that consolidated stream to the public. The NBBO is the reference price the whole market is measured against: a broker’s duty to get you the best available price is defined against it, and a trade is judged “through” the market if it executes outside it.

flowchart LR
  V1["Venue A<br/>(quotes + trades)"] --> SIP["SIP<br/>consolidate all venues<br/>compute NBBO"]
  V2["Venue B<br/>(quotes + trades)"] --> SIP
  V3["Venue C<br/>(quotes + trades)"] --> SIP
  SIP --> Tape["Consolidated tape<br/>(public NBBO stream)"]
  Tape --> Broker["Brokers, apps,<br/>risk, surveillance"]

Two things make the SIP both essential and limited. It is essential because it is the egalitarian, authoritative public record: everyone, large or small, can see the same consolidated tape and the same NBBO, and regulation leans on it as the official measure of the market. It is limited because consolidation takes time. The SIP must receive from every venue, merge, compute the NBBO, and rebroadcast, and each of those steps adds latency. For a buy-and-hold investor that latency is invisible and irrelevant. For a firm trying to trade faster than the next firm, the SIP is too slow, and that gap is the whole reason direct feeds exist.

4. SIP feeds versus direct exchange feeds

Here is one of the defining trade-offs in trading infrastructure, and it is a clean example of how a business and latency reality forces an engineering decision.

The SIP is convenient and authoritative but, by construction, slower: it aggregates many venues and republishes, so a consumer reading the SIP sees the consolidated picture some amount of time after the underlying events happened. A direct feed is the raw stream a single exchange publishes for its own market, with no consolidation step in the path. A firm that subscribes directly to each exchange and consolidates the feeds itself, in its own machines sitting in the exchange’s data center, sees each venue’s data sooner than the SIP can deliver the merged version. The difference can be the margin between a strategy that works and one that does not.

But direct feeds are not free, in two distinct senses. First, money: you pay each exchange a licensing fee for its direct feed, you pay for cross-connects and co-location space in each venue’s data center, you pay for the hardware and the engineering to consolidate the feeds yourself, and you take on the legal entitlement obligations for every feed. A full direct-feed setup across all major venues is a seven-figure annual commitment before a single trade. Second, complexity: when you consolidate yourself, you own the correctness. You must build your own NBBO from many independent streams, each with its own format, its own sequencing, its own gaps, its own clock, and you must do it fast and faithfully or you have simply bought a faster way to be wrong.

So firms choose by need. A retail broker, a long-horizon asset manager, or a risk system that marks positions at the close is well served by the SIP: it is cheaper, simpler, and authoritative, and the latency does not matter to them. A market maker, a latency-sensitive proprietary trading firm, or any execution algorithm competing for queue position takes direct feeds because the SIP’s latency would erase its edge. Many firms run both: direct feeds for the trading path where speed pays, and the SIP for display, compliance, and the official NBBO of record.

5. Reference data is not market data

This distinction trips up newcomers constantly, and conflating the two leads to real bugs, so pin it down now. Market data is the fast-moving stream of prices and trades we have been describing: it changes thousands of times a second and is meaningless a moment later. Reference data is the slow-moving, descriptive information that identifies and defines the instrument: its symbol, its full name, the venue it lists on, its currency, its tick size (the minimum price increment), its lot size, the corporate identifiers (like an ISIN or a CUSIP), and for a derivative its contract terms (strike, expiry, multiplier). Reference data changes rarely, on the order of once a day or less, through a controlled process.

The difference is not just speed, it is how you must treat each. Market data is consumed as an ephemeral stream and never trusted as durable. Reference data is consumed as a slowly changing, authoritative dataset, loaded and cached, and treated as a controlled master record because everything keys off it. If your reference data says a stock’s tick size is one cent when the venue changed it to half a cent, your order book will round prices wrong and your orders may be rejected. If your reference data has the wrong multiplier on a futures contract, every position value built on it is off by that factor. Market-data bugs are usually loud and immediate (a price looks wrong); reference data bugs are quiet and pervasive (everything is subtly off and you do not know why).

flowchart TB
  subgraph ref["Reference data (slow, controlled, authoritative)"]
    R["Symbol, name, ISIN/CUSIP,<br/>currency, tick size, lot size,<br/>contract terms, listing venue"]
  end
  subgraph mkt["Market data (fast, ephemeral, real-time)"]
    M["Quotes, trades, book updates,<br/>thousands of ticks per second"]
  end
  R --> Use["Trading, pricing,<br/>risk, settlement"]
  M --> Use

A clean way to remember it: reference data tells you WHAT the instrument is; market data tells you what is HAPPENING to it right now. The trading system needs both, joins them constantly (a tick is just a price until you attach the reference data that gives it meaning), and must never confuse the lifecycle of one for the other. A common production architecture loads reference data at start of day from a controlled master, then layers the live market-data stream on top of it, so a tick for symbol XYZ is enriched with the reference record for XYZ before anything downstream uses it.

6. Inside a feed handler sequencing gaps and recovery

Now we get to the engineering heart of the topic: the feed handler, the software that turns a venue’s raw stream into something correct your systems can use. A feed handler has four jobs, and each defends against a specific failure.

First, normalization. Every venue speaks its own wire format with its own message types, field encodings, and quirks. The feed handler decodes the venue-specific bytes and translates them into your firm’s single internal representation, so the rest of your stack sees one consistent shape regardless of which venue a tick came from. Without this, every downstream system would have to know every venue’s format, an unmaintainable mess.

Second, sequencing. A correct stream is an ordered stream: a cancel only makes sense after the order it cancels, a book update only makes sense applied in order. Venues stamp each message with a monotonically increasing sequence number so a consumer can tell the intended order and, critically, detect when something is missing. The feed handler tracks the expected next sequence number per channel and checks every message against it.

Third, gap detection. Because the data is high volume and often delivered over a protocol that does not guarantee delivery (we will see why in the multicast section), a message can simply not arrive. The sequence numbers make this detectable: if you have processed sequence 1000 and the next message you receive is 1003, you know 1001 and 1002 are missing, a gap. The feed handler must never just apply 1003 as if nothing happened, because the book it builds would now be wrong, silently.

Fourth, recovery. On detecting a gap, the feed handler must fill it before trusting the stream again. Venues provide recovery mechanisms: a retransmission service you can ask to resend the specific missing sequence numbers, and a periodic or on-demand snapshot that gives the full current state so you can rebuild from scratch rather than chasing individual messages. A handler typically requests retransmission for a small gap and falls back to a full snapshot for a large one or when retransmission itself fails.

stateDiagram-v2
  [*] --> InSync: subscribed, building book
  InSync --> InSync: seq == expected, apply update
  InSync --> Gap: seq > expected, messages missing
  Gap --> Recovering: request retransmission or snapshot
  Recovering --> InSync: gap filled, sequence continuous again
  Recovering --> Snapshot: retransmit failed or gap too large
  Snapshot --> InSync: full state rebuilt from snapshot
  note right of Gap
    Never apply an out-of-order update.
    A gap means the book is suspect
    until it is filled.
  end note

Walk a single feed handler through a gap and back to health.

The lesson underneath all four jobs is the one from the Mental Model: the feed handler’s true responsibility is faithful, in-order replay of an event stream. Normalization makes the events legible, sequencing tells you the order, gap detection tells you when the replay is broken, and recovery repairs it. A handler that skips any of these is not fast, it is wrong, and a book built on a missed message disagrees with the market in a way no amount of speed can fix.

7. Conflation and the cost of keeping up

Sometimes you genuinely cannot, or do not want to, process every single tick, and the honest engineering answer is conflation: deliberately collapsing several updates into one so a slow consumer sees the latest state without having to replay every intermediate change.

The motivation is a mismatch of rates. A venue can publish updates for a fast-moving symbol faster than a human display can refresh or a slow consumer can absorb. If a price ticks fifty times in the second your dashboard refreshes, painting all fifty is pointless work; the viewer only cares about where it ended up. Conflation says: collapse those fifty updates into the single most recent state and deliver that. The consumer stays current without drowning.

The critical distinction, and the source of real bugs when it is misunderstood, is what you may and may not conflate. State you may conflate, because only the latest value matters: the current best bid, the current best ask, the current top-of-book size. Showing the latest quote and skipping the intermediate ones loses nothing a display viewer needs. Events you must not conflate, because each one is a distinct fact that matters in its own right: every individual trade print (each is a real execution with its own price, size, and time), and every sequence-numbered book delta if you are reconstructing the exact book. Conflate the quote, never the trades.

Conflation is also a pressure-relief valve for the system itself. When a consumer cannot keep up and its input queue grows, you face a choice: buffer unboundedly (latency climbs until you run out of memory), drop messages blindly (you corrupt the book), or conflate (keep only the latest state per symbol and stay current). For a state-style feed, conflation is the principled answer: the consumer is always current, never unboundedly behind, at the cost of not seeing every intermediate value, which for a state view it never needed.

8. Multicast and how a feed reaches a thousand listeners

A single venue must deliver the same firehose of ticks to hundreds or thousands of subscribing firms simultaneously, with the lowest possible latency and the same data to all of them. Sending a separate copy to each subscriber (point-to-point, or unicast) does not scale: the venue would have to serialize and send the stream a thousand times over, the slowest path would lag the fastest, and the bandwidth out of the venue would be a thousand times the stream itself. So real-time market data is overwhelmingly distributed by multicast.

Multicast lets a sender publish one copy of each message to a group address, and the network itself duplicates that single packet to every subscriber that has joined the group. The venue sends each tick once; the switches in the data center fan it out to all listeners. This is why a co-located firm can receive a venue’s feed with minimal latency and why every subscriber sees the same packet at essentially the same time: there is one stream, replicated by the network, not a thousand serialized copies.

flowchart LR
  EX["Exchange<br/>matching engine"] --> PUB["Publisher<br/>(one copy per tick)"]
  PUB --> NET["Network fabric<br/>(replicates packet<br/>to the group)"]
  NET --> S1["Subscriber 1"]
  NET --> S2["Subscriber 2"]
  NET --> S3["Subscriber 3"]
  NET --> S4["Subscriber N"]

The trade-off multicast makes is the reason the whole previous section on gaps exists. Multicast in this setting runs over a protocol that does not guarantee delivery: there is no per-subscriber acknowledgement and no automatic retransmission, because acknowledging to thousands of subscribers would destroy the very scalability that makes multicast worth using. So a packet can be dropped in transit and the sender will never know. That is precisely why sequence numbers and a recovery channel are mandatory: the fast path is a best-effort multicast stream, and the safety net is a separate retransmission or snapshot service the handler reaches for the moment a sequence gap appears. Many venues even publish the same data on two independent multicast lines (an A feed and a B feed) so a handler can listen to both and fill a gap on one from the other before resorting to a retransmission request, a technique called line arbitration.

So the architecture is a deliberate pairing: an unreliable but extremely fast and scalable multicast stream for the common case, and a reliable but slower recovery path for the rare case of a missing message. You get the latency and fan-out of multicast and the correctness of guaranteed delivery, by using each where it fits.

9. Timestamps clocks and why nanoseconds matter

Time is not a detail in market data; it is load-bearing, and a surprising number of serious problems are really clock problems wearing a disguise.

Consider what timestamps do. A single tick can carry several: the time the venue’s matching engine generated the event, the time the venue’s publisher sent it, the time your handler received it, and the time your application acted on it. The differences between these timestamps are how you measure your own latency, diagnose where time is being lost, and prove the order in which things happened. If you are consolidating feeds from several venues into one NBBO, you must order events from different venues correctly, and you can only do that if their timestamps are comparable, which means the venues’ and your clocks must agree to within a tight tolerance.

Now the failure that hides here. If your clock drifts from the venue’s by even a few microseconds, two consequences follow. First, your latency measurements become fiction: you might compute a negative latency (you “received” a message before it was “sent”) because the two timestamps came from clocks that disagree, which is nonsense that poisons every downstream calculation. Second, your event ordering across venues becomes wrong: you might place a trade on venue A before a quote on venue B when in reality it was the other way around, and a strategy or a surveillance reconstruction built on that ordering reaches a false conclusion. At the speeds modern trading runs, the entire meaningful action can happen inside a window smaller than a millisecond, so a clock error of microseconds is not a rounding issue, it is a correctness issue.

This is why clock synchronization is a first-class discipline in trading infrastructure. Firms discipline their clocks against a precise reference (commonly using the Precision Time Protocol, PTP, which achieves sub-microsecond synchronization across a data-center network, far tighter than the older Network Time Protocol, NTP, can manage), often anchored to GPS-derived time. Regulators reinforce it: rules in both the US and Europe require trading clocks to be synchronized to a reference within a specified tolerance and trade events to be timestamped at a specified granularity, precisely so that a regulator can reconstruct the sequence of events across venues after the fact. The European MiFID II clock-synchronization rules, for instance, mandate timestamp granularity and maximum divergence from a reference clock that tighten as a participant’s activity gets faster.

The takeaway is that in market data, a clock is not infrastructure plumbing you can ignore; it is part of the data’s meaning. A tick without a trustworthy, comparable timestamp is a fact whose place in the order of events you cannot defend, and order is everything.

10. Building a real-time order book

The order book is where everything in this page comes together: it is the live state you reconstruct by replaying the event stream, and building one correctly is the canonical market-data engineering problem.

A book is two sorted ladders. On the bid side, prices descend from the highest (the best bid) downward; on the ask side, prices ascend from the lowest (the best ask) upward. At each price level sits the resting quantity. You build this book not from a snapshot you trust forever but from a stream of incremental updates: an add puts a new order or size at a price, a modify changes the size at a price, a delete removes it. You start from a snapshot to seed the book, then apply every sequenced delta in order, and the book stays current.

The animation below shows the whole shape of book maintenance at once: a seed snapshot, the steady application of in-order deltas, the gap-detection branch that diverts to recovery, and the continuous invariant checks that guard correctness. The full structure is visible from the first frame; watch the flow walk the main path and step into the recovery branch when a sequence gap appears.

A correct book maintains invariants on every update, and checking them is how you catch corruption before it reaches a strategy. The best bid must be strictly less than the best ask: if they cross (the best bid is equal to or higher than the best ask on the same venue) something is wrong, because in reality those orders would have matched and traded. No price level may hold a negative quantity. The sequence must be continuous; a hole means the book is suspect. And the book must not go stale: if no update has arrived for a symbol in a window where you expect activity, you may be silently disconnected, and a book frozen in the past is worse than no book at all, because it looks alive.

ASK (ascending)              BID (descending)
  10.04   x 500                10.01  x 300   <- best bid
  10.03   x 200                10.00  x 800
  10.02   x 100  <- best ask    9.99  x 150
                                9.98  x 400

  invariant: best ask (10.02) > best bid (10.01)   OK
  crossed book example: best bid 10.02 >= best ask 10.02  -> ALARM

The reason this matters so much is that the order book is the surface every trading decision touches. A smart order router reads it to decide where to send an order; a market maker reads it to set quotes; a strategy reads it to estimate impact. If the book is stale, crossed, or built on a dropped sequence, every one of those decisions is made against a fiction. The invariants are not academic: they are the alarms that stop a corrupted book from quietly steering real orders into bad fills.

11. Licensing entitlements and the business of data

Here is the part engineers most often discover too late, and it is as load-bearing as any line of code: market data is licensed, and using it carries strict legal and financial obligations. Exchanges derive enormous revenue from selling their data, and they enforce the terms aggressively, with audits and penalties. Treating entitlements as an afterthought is how a firm ends up with a back-bill or a breach.

The core concept is entitlement: a record of who is permitted to receive and use which data, under what license, and for what purpose. The dimensions multiply quickly. A firm is licensed per feed (each exchange’s direct feed is a separate product with its own fee) and per level (level 1 is cheaper than level 3). Within the firm, usage is licensed per user and per purpose: the exchanges draw a hard line between display use (a human looking at a screen) and non-display use (a machine consuming the data, for example an algorithm or a risk engine), and they distinguish professional from non-professional users, with very different fees. There are per-device and per-user fees, fees for redistributing data to clients, and separate fees for using data to derive a product. The same tick can carry wildly different costs depending on who looks at it and why.

The engineering consequence is that an entitlement system is a real, security-critical component, not a billing detail. Before any consumer receives a tick, the system must check that this user, on this device, for this purpose, is entitled to this feed at this level, and it must keep an accurate, auditable count for reporting, because the exchange will audit those counts and bill against them. The cost pressure is intense and shapes architecture: firms minimize non-display licenses, carefully segregate display from non-display paths, and design so that data never flows to an unentitled consumer, because every unentitled consumer is either a compliance breach or an unbudgeted bill.

The failure to avoid here is entitlement leakage: data reaching a consumer who is not licensed for it. It happens when a developer wires a non-display system to a display-only feed, when a feed meant for one desk is quietly shared with another, when test environments pull production data without the license, or when redistribution to clients exceeds what was reported. Leakage is doubly dangerous because it is invisible until an audit finds it, at which point the firm owes back-fees and may have breached its license agreement. This is why entitlement enforcement belongs at the gate, checked on every subscription, logged for audit, and owned as a first-class concern rather than bolted on.

12. Failure modes and the controls that catch them

A market-data system can fail in ways that are quiet and dangerous precisely because the data keeps flowing and looks alive. Knowing each failure and its specific control is the difference between an engineer who trusts the stream blindly and one who builds the right guards around it.

The pattern across the list is the same one that runs through the whole page. Market data is an ordered event stream that each consumer independently replays, and almost every failure is a way that replay can be corrupted, dropped, frozen, mis-timed, mis-collapsed, or mis-routed, while still looking superficially fine. The controls are the guards that make corruption announce itself: sequence numbers turn a silent drop into a detected gap, invariants turn a corrupted book into an alarm, staleness watchdogs turn a frozen feed into a flag, disciplined clocks turn time into a trustworthy order, and entitlement gates turn a quiet leak into an enforced boundary. A trading firm does not survive on a fast feed; it survives on a feed it can prove is correct, current, in order, and licensed.

Mastery Questions

1. A new strategy at your firm consistently computes a “negative latency” for ticks from one particular venue: it appears to receive messages a few microseconds before the venue says it sent them. The strategy author suspects a bug in the latency calculation. What is the more likely cause, why does it matter beyond the dashboard, and how would you fix it?

   Answer. The far more likely cause is clock disagreement, not a calculation bug. A negative latency means receive-time minus send-time came out below zero, which is physically impossible for a single real event, so the two timestamps must be coming from clocks that do not agree: your receiving clock is running behind the venue’s sending clock by more than the true network latency. It matters well beyond the dashboard because the same disagreement corrupts everything that depends on comparing timestamps. If you consolidate this venue’s feed with others into an NBBO, you will order its events wrong relative to other venues; if a regulator asks you to reconstruct the sequence of events around a trade, your timeline will be indefensible; and any strategy that reasons about the age of a quote will misjudge how fresh its data is. The fix is clock discipline, not arithmetic: synchronize your clocks against a precise reference (PTP across the data-center network, anchored to GPS-derived time) to sub-microsecond tolerance, monitor for divergence continuously, and alarm whenever timestamps imply an impossible ordering. The negative latency is the symptom; the undisciplined clock is the disease.

2. Your firm currently consumes the SIP for everything and a product team proposes switching the entire trading path to direct exchange feeds to reduce latency. A risk manager asks whether this is purely an upgrade. What trade-offs do you lay out, and how do you decide which systems should move and which should stay on the SIP?

   Answer. It is not purely an upgrade; it is a trade of latency against cost and correctness ownership. Direct feeds are faster because they skip the SIP’s consolidation step, which is decisive for any latency-sensitive system: a market maker, an execution algorithm competing for queue position, anything where the SIP’s added delay would erase the edge. But the move carries two real costs. First, money: each exchange charges a licensing fee for its direct feed, plus co-location, cross-connects, hardware, and the engineering to consolidate many feeds yourself, which together is a large, recurring commitment with heavier entitlement obligations. Second, and more subtle, you now own correctness: the SIP hands you an authoritative consolidated NBBO, but on direct feeds you must build your own NBBO from many independent streams, each with its own format, sequencing, gaps, and clock, and if you get that wrong you have bought a faster way to be wrong. So you decide per system by latency sensitivity. Move the trading and execution path, where microseconds pay, onto direct feeds. Keep display, compliance, surveillance, and end-of-day risk marking on the SIP, where the latency is irrelevant and the authoritative public NBBO of record is exactly what you want. Many firms run both deliberately, and that hybrid is usually the right answer rather than an all-or-nothing switch.

3. A developer, to make a slow internal risk dashboard keep up during volatile markets, adds conflation to the market-data delivery layer so the dashboard only ever gets the latest update per symbol. It works for the quote display, but the trade-blotter section of the same dashboard starts missing executions. Explain precisely what went wrong and how to fix it without giving up the performance win.

   Answer. The developer applied conflation indiscriminately, and conflation is only safe for state, not for events. A quote is state: only the latest bid and ask matter, so collapsing fifty intermediate quote updates into the most recent one loses nothing the display needs, which is why the quote section works. A trade print is an event: each execution is a distinct, real fact with its own price, size, and time, and collapsing several trades into “the latest” erases the earlier ones, which is exactly why the blotter starts missing executions. The two share a delivery path, and a single conflation rule that treats everything as collapsible state corrupts the event stream. The fix preserves the performance win by conflating selectively: conflate the state-style data (quotes, top-of-book size) so the display stays current under load, but deliver the event-style data (every trade print, and any sequenced book delta if a consumer reconstructs the book) in full, never collapsed. The principle to encode in the delivery layer is a hard rule: conflate state, never events. The dashboard then keeps its lag-free quote display and a complete, correct blotter at the same time.

4. During an exchange audit, your firm is found to have a non-display risk engine reading from a feed that was licensed only for display use on the trading desk. No data ever left the building and the engine only computed internal risk numbers. The engineers are confused about why this is a serious problem. Explain the issue and the architectural change that prevents it from recurring.

   Answer. This is entitlement leakage, and it is a serious problem regardless of whether data left the building, because the exchange licenses data by purpose, not just by location. Display use (a human reading a screen) and non-display use (a machine consuming the data, like a risk engine or an algorithm) are separately licensed and separately billed, and non-display is typically far more expensive precisely because it scales without a human in the loop. A non-display system reading a display-only feed is consuming data outside its license, so even though nothing was redistributed, the firm has been using data it did not pay the correct fee for, which the exchange will treat as a breach and a back-bill. The danger is that it was invisible until the audit: the data flowed and the numbers looked right, so nothing alarmed. The architectural fix is to enforce entitlements at the subscription gate rather than trusting consumers to behave: every subscription is checked against who the consumer is, what device or process it runs on, and for what purpose, so a non-display process is structurally prevented from binding to a display-only feed; segregate display and non-display paths so they cannot cross-wire; and log every grant for audit and reconcile reported counts against actual consumption. Entitlement enforcement belongs at the gate and as a first-class concern, not as a billing reconciliation discovered after the fact.