On this path · System Design 4. Design Scalable Price-Quote and Instrument Search
Design Scalable Price-Quote and Instrument Search
Elasticsearch retrieval, recommendations, and a sub-40-millisecond latency budget.
Learning outcomes
The real Trade Republic goal is to let a customer find their next investment in under forty milliseconds. Search and recommendation here are the same system: the user types a query or browses a category, and the platform returns relevant instruments ranked by a blend of text match, popularity, and personalization, with a live price quote attached to each result. Trade Republic serves this through Elasticsearch. This page teaches you to design that system end to end: the index model that makes retrieval fast, the latency budget that makes forty milliseconds achievable, the caching and ranking that make results relevant, and the read path that stitches them together.
After studying this page, you can:
- Break the forty-millisecond budget into its stages and defend where each millisecond goes.
- Design the Elasticsearch index model: fields, analyzers, and the denormalization that lets one query return instrument, quote, and ranking signal.
- Choose between a denormalized single-index and a joined two-index design and name what each costs.
- Place caches at the right boundaries and reason about invalidation when prices move.
- Design a ranking pipeline that blends text relevance, popularity, and personalization without exploding latency.
- Handle corporate actions and splits as index-update correctness problems.
- Anticipate the Staff-engineer probes on freshness, cache staleness, and the write path.
Before we dive in
Search, in a consumer app, is where a customer discovers what to buy. The query “electric vehicle” should return the relevant ETFs and equities, ranked so the most likely matches are on top, each with a current price so the customer can act immediately. The engineering problem is that relevance and freshness pull against latency. A fully fresh, fully personalized result that joins the live price feed against the full instrument catalog at query time is slow. A fully precomputed, fully cached result is fast but stale and generic. The forty-millisecond target is the negotiated settlement between these forces, and the design is the budget that makes the settlement hold.
The origin of the published Trade Republic design is the recognition that search and recommendation are the same retrieval problem. Whether the user typed a query or opened a curated category, the system runs a retrieval against an Elasticsearch index, scores the results, and returns them with a price. The deep lesson is that Elasticsearch is doing two jobs at once: text retrieval for typed queries, and filtered retrieval for category browsing and recommendations. Unifying them on one index, with ranking signals embedded as fields, is what makes the system both fast and general.
The senior insight is that the forty-millisecond budget is an end-to-end contract, and hitting it requires splitting the work between a write path that is allowed to be slow and a read path that must be fast. The write path keeps the index fresh and the ranking signals current; the read path does as little work as possible, mostly reading from cache and from a pre-scored index. The read path’s job is to stay out of the way of the budget, and every millisecond spent there must be earned.
The product UX that the budget enables
The forty-millisecond contract is not an arbitrary number; it derives from the product’s user experience. When a user types a query, the search results must appear before the user’s next keystroke, which is typically around a hundred milliseconds for a fast typist. The forty-millisecond server budget plus network round-trip time leaves the user waiting about a hundred milliseconds total, which feels instantaneous. A result that takes two hundred milliseconds feels sluggish and the user re-types or abandons the search.
The search surface has three entry points: the search bar where the user types a free-text query, the category browse where the user taps a category such as “ETFs” or “Tech Stocks,” and the recommendation carousel where the system suggests instruments based on the user’s portfolio and recent activity. All three use the same read path: the search service receives the input, checks the cache, queries Elasticsearch if needed, enriches with price, and returns ranked results. The different entry points differ only in the query construction: the search bar uses text analysis, the category browse uses a filter query, and the carousel uses a personalization-based function score.
The product requirement that constrains the design most is that every result must carry a current price. The price is the most important signal for a customer deciding whether to tap: a stock at 150 euros is different from the same stock at 151 euros. The price must be within seconds of current, not minutes, because a price that is minutes old is misleading. This requirement drives the price cache design and the feeder that keeps it fresh, and it is the constraint that makes the design different from a generic search system where freshness is a nice-to-have.
Clarify requirements and the latency budget
Functional requirements. The system accepts a free-text query or a category filter and returns a ranked list of instruments, each with its name, identifier, type, and a current price quote. It supports recommendations: for a user with a portfolio, it returns instruments similar to holdings or trending in their segment. It keeps results fresh as prices move and as the catalog changes through corporate actions.
Non-functional requirements. End-to-end latency from query to result is under forty milliseconds at the serving tier. Throughput must handle the query rate of millions of users browsing and searching, with a peak concentrated in market-open and news-burst windows. Freshness for the price is near-real-time; freshness for ranking signals is tolerably soft, updated every few minutes. Availability must be high, because search is the front door of the product.
The latency budget is the design’s spine, so break it open. Forty milliseconds is not much, and a naive query that joins an instrument catalog, a price feed, and a personalization store at request time will blow it. Allocate the budget defensively.
The allocation reveals the strategy. The cache must absorb the majority of repeated queries, because a cache hit skips the Elasticsearch stage entirely. The Elasticsearch query, when it runs, must be tight: a single, well-modeled query against a denormalized index, not a join across multiple stores. The price enrichment must be a hot-cache read, not a fetch from the streaming pipeline, because the streaming pipeline’s latency is variable. The sum holds only if each stage respects its slice, and the design is the discipline that enforces it.
Capacity in round numbers. Assume millions of users with a meaningful fraction searching or browsing at peak, so on the order of tens of thousands of queries per second at peak. The catalog is tens of thousands of instruments, which is small enough to fit a hot price cache in memory. The query pattern is skewed: a small set of queries and categories accounts for most of the traffic, which is what makes caching effective.
High-level architecture
The system has two paths, and keeping them separate is the key decision. The write path builds and refreshes the index and the caches; the read path serves queries against them.
The write path has three feeders. The catalog feeder reads the instrument master and updates the Elasticsearch index when instruments are added, delisted, or changed by a corporate action. The price feeder reads the computed price from the streaming pipeline and updates a hot price cache, keyed by instrument id. The ranking-signal feeder periodically recomputes popularity, trend, and personalization features and writes them as fields in the index. The write path is allowed to be slow on the order of seconds to minutes because freshness tolerances are soft, which is what keeps the read path fast.
The read path is the forty-millisecond contract. The client sends a query to the API gateway, which authenticates and routes to the search service. The service checks a result cache keyed by the query and the user’s personalization segment; on a hit, it returns immediately. On a miss, it runs a single Elasticsearch query against the denormalized index, which returns instruments already scored by the embedded ranking signals. The service enriches each result with the current price from the hot price cache, applies any final personalization re-ranking, and returns.
The architecture’s core rule is that the read path never calls the streaming pipeline. The streaming pipeline computes prices continually, but its latency is variable and unsuited to a forty-millisecond budget. Instead, the price feeder reads the latest computed price and writes it to a hot cache, and the read path reads from that cache. This decouples the search latency from the streaming latency, which is what makes the budget hold under load.
Deep dive: the index model and the read path
The index model is where the forty-millisecond budget is won or lost, because the Elasticsearch query is the largest slice. The senior choice is to denormalize: each document in the index is one instrument, carrying its searchable text fields, its type and category filters, its ranking signals, and a snapshot of its current price. Denormalizing the ranking signals into the document is what lets one query both retrieve and score, avoiding a second lookup or a join.
The index mapping strategy
The Elasticsearch index mapping is the contract between the write path and the read path. Each field is mapped with the correct type and analyzer to support the read path’s queries without runtime transformation.
The name_analyzed field uses a text type with a language-specific analyzer chosen by the instrument’s primary market. The analyzer handles lowercasing, stemming, and stop-word removal. The field also has a sub-field with a standard analyzer for exact-match queries, so that a query for “APPLE” matches the exact name even if the language analyzer would stem it to “appl.” The sub-field is indexed separately and adds to the index size, but it eliminates a class of false negatives where the stemmer removes a meaningful suffix.
The synonyms field is a text type with a custom analyzer that reads the synonym file and applies the synonym rules at index time. Applying synonyms at index time rather than query time is deliberate: it ensures that the synonym expansion is consistent across all queries, and it avoids the query-time overhead of synonym expansion. The cost is that a synonym change requires a reindex, which is acceptable because synonym changes are infrequent and the reindex is fast.
The ticker_edge field is a keyword type with an edge ngram filter that indexes prefixes of the ticker. The edge ngram length is set to a minimum of two and a maximum of ten, so a query for “MS” matches “MSFT” and “MSI.” The prefix matching is fast because the edge ngram is indexed for fast lookup, not computed at query time.
The ranking signal fields are numeric types with doc_values enabled for efficient sorting and function-score computation. The popularity_score and trend_score are single-precision floats, and the personalization segment score is a small integer that the function-score query maps to a boost weight. The score fields are updated by partial updates that change only these fields without reindexing the full document.
The text fields use the right analyzers. The instrument name and synonym fields use a language analyzer that handles case folding, stemming, and basic synonyms, so “EV” matches “electric vehicle.” The identifier fields, like ISIN and ticker, use a keyword analyzer with edge ngrams for prefix matching, so a partial ticker still matches. Getting the analyzers right at index time is far cheaper than fixing them at query time, because the analyzer runs once per document at write, not once per query at read.
The ranking signals are embedded as numeric fields: a popularity score derived from recent search and trade volume, a trend score derived from the rate of change of that volume, and a category fit score for recommendations. Embedding them means the Elasticsearch query’s scoring function can blend them without a second lookup, which is the denormalization payoff. The one signal that cannot be embedded is personalization, because it depends on the user; that term is applied at query time as a function score using a user-segment field.
The price is a denormalized field too, but with a freshness caveat. The price in the document is a snapshot from the price feeder, and it may lag the true streaming price by seconds. For the search results list, this is acceptable, because the user is browsing, not executing. When the user opens an instrument’s detail page, the live streaming price takes over. The distinction is important: search shows a near-real-time price, and the detail page shows the live price, and conflating them breaks either the budget or the freshness.
The read path’s discipline is to run one query. The query combines a text clause against the name and synonym fields, a filter clause against the type and category fields, and a function-score clause that blends the embedded ranking signals with the personalization term. The query requests only the fields needed for the result list, not the full document, which cuts serialization. Pagination uses the search-after cursor, not deep pagination, because deep pagination in Elasticsearch is expensive and rarely useful in a discovery interface.
The query construction in detail
The Elasticsearch query is built from the user’s input and the context. For a free-text query, the search service constructs a multi-match query against the name_analyzed and synonyms fields, with the type set to “most_fields” so that a match in both fields scores higher than a match in one. The analyzer on the name_analyzed field is a language-specific analyzer that lowercases, stems, and handles common synonyms. For a ticker prefix match, a separate bool clause uses a prefix query on the ticker_edge field, which uses an edge ngram filter for fast partial matching.
The filter clause restricts the results to instruments the user can trade. Not all instruments are available to all users: some instruments require a suitability check or are restricted in certain jurisdictions. The filter clause is a terms filter on the type and category fields, precomputed per user segment and applied by the search service. The filter reduces the candidate set and speeds up the query, because Elasticsearch evaluates filters before scoring.
The function-score clause applies the ranking signals. The base score is the text relevance. The popularity_score is multiplied by a weight and added, so a popular instrument that matches the text partially may outrank an unpopular instrument that matches perfectly. The trend_score is similarly weighted, with a recent spike in interest boosting the score. The personalization term is a per-user boost computed from the user’s segment and applied as a filter-weight function: instruments in categories the user has previously browsed or held get a small boost. The weights are tuned offline by evaluating search click-through data against held-out queries.
Synonyms and their management
Synonyms are critical for a good search experience. “EV” should match “electric vehicle,” “MSFT” should match “Microsoft,” and “bonds” should match “fixed income.” The synonym set is managed as a configuration file that the index build reads and applies to the synonym field. The synonym file is versioned and deployed with the index schema, so a synonym change requires a reindex.
The danger with synonyms is the false positive: a synonym that matches an unintended instrument. “AAPL” is a synonym for “Apple,” but “AP” or “APPL” are common misspellings that should not be synonyms because they match too broadly. The synonym management includes a review step where a human validates each new synonym against the instrument catalog before it is deployed. A senior designs the review step into the write path rather than as an afterthought, because a bad synonym that surfaces an irrelevant instrument degrades user trust in the search results.
Query-time personalization versus offline personalization
The ranking pipeline blends two personalization strategies. Offline personalization computes per-user segment scores that are written as a field in the Elasticsearch document. The segment score is derived from the user’s portfolio, watchlist, and browsing history, aggregated into a segment, and recomputed periodically by a batch job. A user who holds mostly technology ETFs sees a small score boost on other technology ETFs. The offline computation runs every few hours and writes the segment score to a secondary index that the search service reads.
Query-time personalization applies a light re-rank on the top candidates returned by Elasticsearch. The re-rank is a simple weighted adjustment: it increases the score of instruments in categories the user has previously interacted with and decreases instruments the user has explicitly dismissed. The re-rank operates only on the top-N candidates, typically a few dozen, so it adds negligible latency. The re-rank weights are stored in the user’s session and updated on each interaction.
The offline and online personalization serve different freshness needs. The offline score captures the user’s long-term preferences and changes slowly. The query-time re-rank captures the user’s immediate session behavior and changes quickly. A senior distinguishes the two rather than treating personalization as a single problem, because the two strategies have different latency budgets, update frequencies, and failure modes.
Deep dive: caching, ranking, and personalization
The result cache is the largest latency win, because it can skip the Elasticsearch stage entirely. Key the cache by a normalized form of the query plus the user’s personalization segment, so queries from users in the same segment share a cached result. The hit rate is high because query traffic is skewed: a small set of queries and categories accounts for most of the volume, and those are served from cache. Invalidation is by time-to-live, because ranking signals and prices are soft-fresh; a cached result that is a minute old is still acceptable for browsing.
The price cache is separate from the result cache and is keyed by instrument id. It holds the latest computed price from the streaming pipeline, written by the price feeder. The read path reads it during enrichment, so the price on a cached result can be refreshed even when the result list itself is cached. This separation matters: the result cache captures the slow-changing ranking and text results, and the price cache captures the fast-changing price, and they are invalidated on different schedules.
The result cache is keyed by query and segment and invalidated on a minute-scale TTL, because ranking signals change slowly. The price cache is keyed by instrument and updated continuously by the price feeder, because prices change fast. Splitting them lets the search results stay cached while the price stays fresh, which is the combination that holds the forty-millisecond budget.
Ranking is a pipeline, not a single score. The first stage is the Elasticsearch retrieval, which applies the text match and the function score from embedded signals and returns the top candidates. The second stage is a light personalization re-rank, which adjusts the order based on the user’s segment and recent activity. The re-rank is deliberately light, operating only on the candidates already retrieved, because a heavy personalization model at query time would blow the budget. Heavy personalization is computed offline and written as a field; the query-time re-rank only reads it.
Recommendations are the same pipeline with a different input. Instead of a text query, the input is a category filter or a similarity-to-portfolio filter, and the retrieval returns candidates scored by the embedded ranking signals. The same index, the same read path, the same budget. Unifying search and recommendation on one index is the efficiency that makes the system affordable, because it avoids maintaining two retrieval systems.
The ranking pipeline tuning and A/B testing
The ranking weights, the coefficients that determine how much text relevance, popularity, trend, and personalization contribute to the final score, are not set once. They are tuned continuously through A/B testing. A control group of users sees the current ranking, and a treatment group sees a ranking with adjusted weights. The click-through rate, the conversion rate to the instrument detail page, and the instrument add-to-watchlist rate are compared. A weight change that improves click-through without degrading conversion is promoted.
The A/B test infrastructure is part of the search service. The service reads the user’s experiment group from a cookie or a user-profile field and applies the corresponding ranking weights. The experiment assignment is cached for the session, so the user sees a consistent ranking across queries. The A/B test results are logged to a separate analytics table and analyzed offline.
The ranking pipeline also has a manual override mechanism for editorial curation. When the editorial team wants to promote a specific instrument for a query, for example, promoting a newly listed ETF for the “sustainable investing” query, the override is a separate Elasticsearch document that matches the query with a very high boost score. The override document is indexed in a separate index that the search service queries alongside the main index and merges into the result set. The override has an expiration date, after which it is automatically removed. The manual override mechanism is kept deliberately simple: a search service developer adds the override document, the editorial team validates it in staging, and it is promoted to production through the normal deployment pipeline.
The ranking pipeline is the part of the system that product managers and data scientists care about most. A senior engineer who designs the pipeline to be tunable without code changes, through configuration and A/B test flags, has built a system that the product team can improve without engineering involvement for every change. The pipeline is not just a technical interface for querying; it is a product interface that the product team uses to optimize the discovery experience.
Data model
The Elasticsearch document is the central entity, and its fields are chosen to serve the read path in one query.
The document’s embedded ranking signals are what make the single-query design work. If they lived in a separate store, the read path would need a join, which would blow the budget. Embedding them trades write-path cost, updating the signals periodically, for read-path speed, and for a search system the read path is the budget.
The search click feedback loop
Every search interaction generates signals that improve future ranking. When a user taps a search result and navigates to the instrument detail page, that click is a signal that the result was relevant. When a user scrolls past several results without tapping, that is a signal that the ranking may need adjustment.
The click signal is collected by the search service and written to a Kafka clickstream topic. A batch job reads the topic every few minutes, aggregates the click count per query-and-result pair, and updates the popularity_score field in the Elasticsearch index. An instrument that receives many clicks for a query sees its score for that query increase through the popularity_score weight in the function-score query.
The feedback loop is the mechanism that makes the ranking adaptive without manual tuning. When a new instrument is added to the index, its popularity_score starts at a default value and rises as customers discover and click on it. When an instrument’s relevance declines, for example, a stock that has fallen out of the news, its click rate drops and its popularity_score decays through a half-life formula that reduces the score by a configurable fraction per day without clicks.
A senior ensures the feedback loop is visible to the product team through a dashboard that shows the top queries, their click-through rates, and the most-clicked results per query. The dashboard is the product team’s interface to the ranking system, and it lets them spot anomalies, such as a query that returns no popular result, without needing an engineer to investigate.
Multi-language support
Trade Republic operates in multiple European markets, and the search index must support multiple languages. Each language has its own analyzer: German uses a German analyzer with compound word decomposition, French uses a French analyzer with elision handling, and Italian uses an Italian analyzer with stem merging. The instrument name field is indexed separately for each language, and the query analyzer is selected based on the user’s locale preference.
The language selection affects the synonym set: a synonym that works in English may not apply in German. The synonym file is partitioned by language, and the index build applies the correct synonym set for each language field. The query-time analyzer selection is driven by the user’s Accept-Language header or the app locale setting, and the Elasticsearch query targets the corresponding language field. An instrument that has different names in different languages, for example a stock that trades under a local name in Germany, is indexed with both names so it appears in searches from either locale.
The multi-language index increases the index size proportionally to the number of supported languages. For a catalog of tens of thousands of instruments and half a dozen languages, the index size grows by a factor of the language count plus the synonym storage. This is still modest, on the order of a few gigabytes, and the index fits in the Elasticsearch node’s memory for fast access. A senior accounts for the language factor in the index size estimate and ensures the Elasticsearch cluster has sufficient memory for the working set.
The write path: from instrument master to index
The write path starts when a change occurs in the instrument master. The catalog feeder, a Kafka consumer, reads the instrument-change topic and applies the change to the Elasticsearch index. The feeder is a separate deployment from the search service, because the write path has different scaling characteristics from the read path: the write path is throughput-sensitive and latency-flexible, while the read path is latency-sensitive and throughput-elastic.
The catalog feeder handles three types of changes: new instrument, where it adds a document to the index with default ranking signals; instrument update, where it updates the document’s text fields and reference data; and instrument delisting, where it removes the document from the index. The removal is a soft delete: the document is marked as inactive and filtered out by the query’s filter clause, so that it remains in the index for historical queries that may reference the instrument’s ISIN. The soft delete is preferred over a hard delete because a hard delete would cause a 404 on the instrument detail page for a delisted instrument that the customer might still hold.
The price feeder is a separate Kafka consumer that reads the computed-price topic from the streaming pipeline and updates the price cache. The price feeder must be fast because the price cache is the source of fresh prices for the search result list. It is a simple consumer that reads the latest price per instrument and writes it to the cache with a TTL equal to the expected update interval. The price feeder does not update the Elasticsearch document’s price_snapshot field because the snapshot is written by the ranking-signal feeder on a different cadence and is used only as a display fallback.
The ranking-signal feeder recomputes the popularity and trend scores every few minutes and applies them as partial updates to the Elasticsearch documents. The partial update updates only the ranking fields, not the full document, and is a lightweight operation on the index. The feeder reads the recent search and trade volume from an analytics store, computes the scores, and updates the index in batches of a few hundred instruments per batch. The batch update is rate-limited to stay within the Elasticsearch cluster’s indexing capacity.
Data model refresh cadence
The index data comes from two sources: the instrument master, which changes rarely, and the ranking signals, which change frequently. The instrument master update triggers a reindex that rebuilds the entire index, which takes minutes for a catalog of tens of thousands of instruments. The ranking signal update is a partial update that modifies only the ranking fields for the affected instruments, which takes seconds.
The reindex cadence is driven by the instrument master change rate, which is low, a few changes per day. The ranking signal update cadence is driven by the popularity and trend recomputation frequency, which is every few minutes during market hours. The reindex is a batch job that runs on a schedule and is triggered immediately when a corporate action or a new instrument requires it. The partial update is a streaming consumer that reads the ranking signal feed and applies the updates to the index in near real time.
A senior distinguishes between the two update paths and monitors them separately. A reindex that fails stalls all index updates, including the partial ranking updates, because the index is locked during reindex. The mitigation is to use Elasticsearch’s alias swap so that the reindex builds a new index while the old index continues serving queries, and the partial updates are applied to the old index until the swap. This makes the reindex transparent to the read path.
Tradeoffs and alternatives
A denormalized single index versus a joined two-index design is the central tradeoff. The denormalized design embeds ranking signals and the price snapshot in the instrument document, so one query retrieves and scores. The joined design keeps instruments in one index and signals in another, joining at query time. The denormalized design wins on read latency, which is the budget; it loses on write amplification, because updating a signal rewrites the document. For a forty-millisecond search budget, the read-path win dominates, and denormalization is correct. The write-path cost is bounded because the catalog is only tens of thousands of documents.
Reading the price from the streaming pipeline at query time is the alternative to the price cache. It is fresher but slower and variable, because the streaming pipeline’s latency depends on load. The forty-millisecond budget cannot tolerate that variability, so the hot price cache is the production choice. The trade is a few seconds of price staleness on the search list, which is acceptable for browsing.
A dedicated recommendation engine separate from search is the alternative to unifying them on one index. It allows a more specialized model but doubles the operational surface and the cost. Unifying them works because both are filtered retrieval against the same instrument catalog, and the ranking signals serve both. The trade is a less specialized recommendation model, which is acceptable given the budget and cost wins.
A cache-aside strategy for the result cache versus a write-through cache is a tradeoff that affects consistency. The result cache is cache-aside: the search service checks the cache on a miss, queries Elasticsearch, and writes the result to the cache with a TTL. This is simple and works well for the read-heavy workload. A write-through cache, where the write path updates the cache when the index changes, would keep the cache fresher but would require the write path to know about the cache entries, which adds coupling. At the minute-scale TTL of the result cache, the cache-aside strategy is correct, and the cache staleness is bounded by the TTL.
An auto-complete feature, where the search service suggests completions as the user types, is a separate concern that the design must accommodate but not be constrained by. Auto-complete queries must be even faster than search, because they fire on every keystroke, which is roughly every hundred milliseconds. The auto-complete path reads from a dedicated Elasticsearch completion suggester field, not from the main query path. The completion field is built from the instrument name and ticker and is updated on the same reindex schedule as the main index. The auto-complete results are cached aggressively because the same partial query is typed by many users.
The completion suggester has its own latency budget: ten milliseconds, not forty. To hit this budget, the auto-complete path bypasses the ranking pipeline entirely and returns the top completions sorted by a precomputed popularity score. The result is a list of instrument names and identifiers that the user can tap to navigate to the instrument detail page or to trigger a full search. The auto-complete path is isolated from the main search path so that a slow auto-complete does not degrade the main search latency.
A SQL database with full-text indexes instead of Elasticsearch is the alternative for smaller catalogs. Postgres full-text search handles tens of thousands of documents competently. The reason to use Elasticsearch is the ranking and scoring flexibility: function-score queries blending multiple signals are Elasticsearch’s strength, and Postgres full-text ranking is comparatively limited. At Trade Republic’s scale and ranking complexity, Elasticsearch is the production choice, which is why the published architecture uses it.
Failure modes and what breaks at scale
An Elasticsearch cluster outage takes down search. The graceful degradation is a cached fallback: the result cache, which is in a separate store, can serve stale results for the most common queries while the cluster recovers. The detail page, which depends on the streaming pipeline and not on search, continues to work. Designing so search degrades rather than collapses requires the cache to be independent of Elasticsearch, which is why they are separate stores.
A stale price cache produces a search result with an outdated price. This is tolerable for browsing but must be bounded: the price feeder must update the cache frequently, and the result must indicate its freshness so the user knows to check the detail page before acting. The failure mode to avoid is a stale price that looks live, which misleads the user; a stale price that is honestly marked is acceptable.
A corporate action, such as a stock split, changes the price scale and the symbol. If the index is not updated, a search returns the instrument with a pre-split price and possibly a stale identifier. The catalog feeder must consume corporate-action events and update the index, and the price cache must be invalidated for the affected instrument so the next feeder write repopulates it. This is the same corporate-action correctness concern that affects the alert system, and the index is another consumer of the reference-data feed.
A cache stampede, when a popular query expires and many requests miss simultaneously, can overload Elasticsearch. The mitigations are request coalescing, so concurrent misses for the same key run one Elasticsearch query and share the result, and staggered TTLs, so popular keys do not expire at the same instant. These are standard cache-engineering concerns, and naming them shows depth.
A personalization model that drifts can produce irrelevant recommendations, which is a relevance failure rather than a latency failure. The mitigation is offline evaluation of the model against held-out data and monitoring of click-through and conversion metrics. This is a product-quality concern that touches the system because personalization is part of the ranking pipeline.
A cache poisoning attack, where a malicious query generates a cached result that serves incorrect data to later users, is a security concern for the result cache. The cache is keyed by the normalized query and the user’s personalization segment, not by the raw query string, which prevents trivial injection attacks where a query with special characters produces a different cache key. The cache value is the Elasticsearch result, which already carries only permitted fields, so even if the cache is poisoned, the malicious content is limited to the public instrument data. A senior does not treat the cache as a security boundary, but designs it so that a cache poisoning incident leaks nothing that the attacker could not get by querying the search service directly.
A slow Elasticsearch cluster due to heavy indexing during a corporate-action event, when many instruments are updated at once, increases the query latency and threatens the forty-millisecond budget. The index update should be throttled to limit the impact on query performance. Elasticsearch supports dynamic index settings that allow the search service to reduce the indexing throughput during high query load. The tradeoff is that the index freshness lags during the throttle, which is acceptable because the index update can wait seconds or minutes, while the query latency must stay under forty milliseconds every millisecond of the day.
A price staleness failure, where the price cache is not updated for an instrument for an extended period, causes the search result to show a price that is significantly outdated. The price feeder is a Kafka consumer that reads the computed price from the streaming pipeline’s Kafka topic and writes it to the price cache. If the price feeder falls behind, the price cache grows stale. The mitigation is to monitor the price feeder lag and to set a maximum staleness TTL on the price cache entry: if the entry is older than the TTL, the search service treats it as a cache miss and falls back to a direct read of the latest computed price from the streaming pipeline, accepting the latency variability. The fallback is logged so the team can investigate the feeder lag.
The result cache hit rate is the single most important operational metric for the read path. A falling hit rate means that the query distribution is shifting to long-tail queries that the cache does not cover, or that the TTL is too short for the query patterns. The senior response is not to increase the TTL blindly, because a longer TTL increases staleness. The response is to analyze the miss queries, identify patterns that could be pre-cached, and adjust the cache key design. For example, if users in the same personalization segment frequently issue the same long-tail query, the cache should segregate segments by the query prefix rather than the full query, so that the miss for one user populates the cache for other users in the same segment.
The price cache miss rate is a separate metric that monitors the price feeder’s health. A rising price cache miss rate means the feeder is not keeping up with the price update rate, or the feeder consumer group has rebalanced and is catching up. The miss rate is a leading indicator of price staleness, and the team alerts on it before users notice that the search results show old prices.
Operational concerns
Observability keys off the budget. Track the per-stage latency: gateway, cache lookup, Elasticsearch query, enrichment, serialization. Track the cache hit rate per stage, because a falling hit rate signals that the cache is mis-keyed or that traffic is shifting to long-tail queries. Track the Elasticsearch query latency distribution and slow-query log, because a regression there consumes the budget first.
Index freshness is monitored as the lag between a catalog change and its reflection in the index. For corporate actions, the lag must be small enough that a split is reflected before the market opens on the ex-date. For ranking signals, the lag is softer but should be bounded to minutes. Monitoring this lag is how the team knows the write path is healthy.
Capacity planning keys off the query rate and the cache hit rate. The Elasticsearch cluster is sized off the miss rate, not the total query rate, because the cache absorbs the hits. A common error is to size the cluster off the total rate and over-provision, or off the miss rate alone and under-provision when the hit rate drops during a novel event that drives new queries. Size for the peak miss rate with headroom.
Deployment safety for the index means using the alias-and-swap pattern: build a new index, validate it, then swap the alias so traffic shifts atomically. This avoids the partial-consistency window of updating an index in place and gives a clean rollback path. For a search system that is the front door of the product, zero-downtime reindexing is a baseline expectation.
Read-path testing and verification
The forty-millisecond budget depends on the read path being correct under every query pattern. Testing the read path has three layers.
Unit tests verify that the Elasticsearch query construction produces the correct Elasticsearch Query DSL for each input type: a free-text query, a ticker prefix, a category filter, and a recommendation request. The tests assert the query structure, the analyzer assignment, and the function-score weights. They run without an Elasticsearch cluster by validating the generated JSON against a schema.
Integration tests run against a real Elasticsearch cluster with a test index of a few hundred instruments. They submit queries and verify the result set, the ordering, and the latency. The latency assertion is not the absolute value, which depends on the test environment, but the absence of outliers: the p99 latency must be within a small multiple of the expected production latency.
Canary tests run against the production cluster with a fraction of the query traffic. They compare the search results from the new version against the old version and alert on differences. The canary runs for a few minutes before the new version is promoted, and it catches regressions in ranking, filtering, or personalization that unit and integration tests miss.
A senior also tests the write path’s impact on the read path. A reindexing operation that temporarily degrades query performance must be detected before the reindex reaches production. The team runs a reindex against a shadow Elasticsearch cluster, measures the query latency under reindex load, and validates that the latency remains within budget before promoting the reindex procedure.
Defending the design to a Staff engineer
A Staff engineer will probe the budget, and you should walk it stage by stage. The forty milliseconds split into network, gateway, cache, Elasticsearch, enrichment, and serialization, with cache and Elasticsearch dominating. The strategy is to maximize cache hit rate for the skewed query distribution and to make the Elasticsearch query tight: one query, denormalized document, embedded ranking signals, search-after pagination. The price is a hot-cache read, never a streaming-pipeline call, so the budget is decoupled from streaming latency.
The freshness defense is the two-path split. The write path keeps the index and the price cache fresh on soft schedules: ranking signals every few minutes, prices continuously. The read path never does fresh work; it reads from the caches and the pre-scored index. This is what holds the budget under load, because the read path’s work is bounded by cache and index reads, not by computation.
The relevance defense is the ranking pipeline: Elasticsearch retrieval with a function score blending embedded signals, followed by a light query-time personalization re-rank on the retrieved candidates. Heavy personalization is computed offline and written as a field. The trade is a less specialized model than a dedicated engine, but the cost and latency win of unifying search and recommendation on one index is worth it.
The Staff engineer will also probe the edge case of a new instrument that is not yet in the index. The answer is that the catalog feeder writes the new instrument to the index within seconds of its creation in the instrument master. During the window before the index write, the instrument is not searchable. The window is acceptable because new instruments are rarely urgent for the customer; they can navigate to the instrument by ticker through the auto-complete path, which reads from a different index built from the same catalog feeder. The auto-complete index is built more frequently, so the new instrument appears in auto-complete within seconds even if it is not yet in the full search index.
The zero-result query is another edge case the Staff engineer probes. A query that returns no results is a disappointment for the user. The system handles zero-result queries by logging the query for analysis, returning a suggested correction if the query is a close misspelling of a known instrument, and returning popular instruments in the user’s segment as a fallback. The suggested correction is computed by Elasticsearch’s phrase suggester on the name field, and the fallback is a query against the same index with the personalization ranking but no text filter. The zero-result rate is monitored as a signal that the index or the synonym set needs improvement.
The system splits a slow write path that keeps a denormalized Elasticsearch index and a hot price cache fresh from a fast read path that serves queries in forty milliseconds by reading from a result cache, running one tight query against the pre-scored index, and enriching from the price cache, never from the streaming pipeline.
Common mistakes candidates make
- Reading the live price from the streaming pipeline at query time. That couples the search latency to streaming load and blows the forty-millisecond budget. Read from a hot price cache updated by a feeder.
- Running multiple queries or joins in the read path. Each adds latency and consumes the budget. Denormalize ranking signals into the document and run one query.
- Applying heavy personalization at query time. A heavy model at query time blows the budget. Compute personalization offline and write it as a field; apply only a light re-rank at query time.
- Caching the price with the result. The price changes fast and the ranking changes slowly. Split the result cache and the price cache on different freshness schedules.
- Ignoring corporate actions on the index. A split rescales the price and the symbol; an un-updated index returns a stale, misleading result. The catalog feeder must consume corporate-action events.
- Using deep pagination. Deep pagination in Elasticsearch is expensive and rarely useful in discovery. Use search-after cursors.
- Single-caching by query alone. Users in different personalization segments need different results. Key the result cache by query plus segment.
- Updating the index in place without alias-and-swap. That creates a partial-consistency window. Use the alias pattern for zero-downtime reindexing.
- Not distinguishing the reindex cadence from the partial update cadence. A reindex failure stalls all index updates. Use Elasticsearch’s alias swap so the reindex builds a new index while the old index continues serving, and apply partial updates to the old index until the swap.
- Treating auto-complete and search as the same query path. Auto-complete needs ten-millisecond latency and bypasses ranking. Use a dedicated completion suggester field and a separate query path with aggressive caching.
- Monitoring only the total query latency without per-segment breakdown. A falling cache hit rate in one segment may be hidden by steady hit rates in others. Monitor the hit rate per personalization segment and per query type.
- Not handling the zero-result query gracefully. A query that returns no results is a disappointment. Return a suggested correction or popular instruments as a fallback, and log the query for synonym analysis.
What the interviewer asks next
Expect to be pushed on freshness. The question: the price in the search result is a cache snapshot; how stale can it be, and how do you keep it acceptable? The answer is that the price feeder updates the hot cache continuously from the streaming pipeline, the cache entry carries a timestamp, and the result indicates freshness; for browsing, seconds of staleness is fine, and the detail page shows the live streaming price for action.
A second follow-up: what happens when Elasticsearch is down? The answer is that the result cache, being a separate store, serves the most common queries as stale results while the cluster recovers, and the detail page continues to work because it does not depend on search. Search degrades rather than collapses.
A third, deeper probe: a stock splits two-for-one overnight. Walk the index and cache update. The catalog feeder consumes the corporate-action event before market open, updates the instrument document with the new price scale and identifier, rebuilds the synonym and ticker edge fields, and invalidates the price cache entry so the next feeder write repopulates it. Any cached search result containing the instrument is allowed to expire on its TTL. The detail page, reading from the streaming pipeline and the updated reference data, shows the post-split price at open. The split is reflected everywhere before the user sees it.
Mastery Questions
Question
Why does the read path read the price from a hot cache updated by a feeder, rather than directly from the streaming pipeline?
Answer
The streaming pipeline's latency is variable and depends on load, so calling it at query time would make the forty-millisecond budget unholdable. The price feeder reads the latest computed price from the pipeline and writes it to a hot cache keyed by instrument, and the read path reads from that cache. This decouples search latency from streaming latency. The trade is a few seconds of price staleness on the search list, which is acceptable for browsing; the detail page shows the live price.
Sources & evidence5 claims · 2 cited
Design based on Trade Republic engineering blog (src_tr_40ms) and interview context (src_tr_interview); design reasoning beyond sources is internal-reasoning and carries no claim.
- Trade Republic serves instrument search and recommendations in under 40 milliseconds using Elasticsearch with a denormalized index model where each document carries searchable text, ranking signals, and a price snapshot, allowing one query to both retrieve and score.verified
- The search read path splits the 40ms latency budget across network, gateway, result cache lookup, Elasticsearch query, price enrichment, and serialization, and the strategy maximizes cache hit rate for the skewed query distribution.verified
- Reading the live price from the streaming pipeline at query time couples search latency to streaming load and blows the 40ms budget; the production choice reads from a hot price cache updated by a feeder, accepting seconds of staleness for the browsing use case.verified
- The write path for search is split from the read path: catalog feeder, price feeder, and ranking-signal feeder update the Elasticsearch index and price cache on soft schedules while the read path serves queries in 40ms by reading from cache and a pre-scored index.verified
- Corporate actions such as stock splits change the price scale and symbol; if the search index is not updated before the ex-date, search returns instruments with stale pre-split prices, which is a correctness failure that the catalog feeder must prevent by consuming corporate-action events.verified
Cited sources
- Finding your next investment in less than 40 milliseconds · Trade Republic Engineering
- How to Pass the Trade Republic Software Engineer Interview (JobMentis) · JobMentis (interview outcome database)