Regulatory Reporting

Learning outcomes

If you build or operate anything that touches regulated markets, regulatory reporting is the part of the system that the firm can least afford to get wrong and the part that engineers most often treat as an afterthought. A trading engine that goes down for an hour is an incident. A reporting pipeline that silently drops two percent of its records for six months is a multi-million-dollar enforcement action, a remediation program, and a line in the regulator’s annual report with your firm’s name on it. The mechanics are not glamorous, but they are unforgiving, and the engineers who understand them are the ones a firm trusts with its license to operate.

After studying this page, you can:

• Explain why regulators require firms to report data at all, what that data buys the market, and why the obligation falls on the firm rather than on the regulator to go fetch it.
• Name the major reporting regimes (trade and order reporting, transaction reporting, financial and operational reporting, position and large-trader reporting, and anti-money-laundering filings), say who reports what to whom under each, and place a new regime into that map when you meet one.
• Define the golden-source problem and explain why data lineage, not clever transformation, is the hard part of a reporting platform.
• Separate the three independent obligations a report carries (it must be accurate, it must be complete, and it must be on time) and explain why a system can satisfy any two while failing the third.
• Walk a record through the full pipeline (extract, transform, validate, submit, acknowledge, reconcile, amend) and say what each stage protects against.
• Design a reporting platform that treats every submission as reversible, reconciles against what the regulator actually received, and can resubmit a corrected record without losing the audit trail.
• Predict how a reporting failure happens, what it costs, and which control would have caught it, and distinguish the fundamental obligation that every regime shares from the jurisdiction-specific conventions layered on top.

Before we dive in

You do not need a compliance background. We will define each term as it appears and lean on a few plain ideas you already have.

A regulator is a government or government-empowered body that supervises a market, for example a securities commission, a central bank, or a market-conduct authority. A reporting regime is a specific set of rules that says which events a firm must report, in what format, on what deadline, and to whom. A report or a submission is the file or message a firm sends to satisfy a regime. A trade is an executed transaction (a buy or a sell that happened), an order is an instruction to trade that may or may not execute, and a position is the net holding a firm or client carries in an instrument at a point in time. A reportable event is anything a regime says must be reported: an execution, an order, a new position, a suspicious cash deposit.

Two structural words matter throughout. The golden source is the one system a firm designates as authoritative for a given piece of data, so that when two systems disagree, there is a defined answer about which one is right. Data lineage is the recorded chain of where a value came from and every transformation it passed through on its way to a report, so that for any field in any submission you can answer the regulator’s favorite question: where did this number come from?

One convention before we start. Specific deadlines, field counts, and penalty figures below are drawn from the real regimes as they have stood in recent years, but every regime revises its rules, its formats, and its thresholds on its own schedule. Treat the exact numbers as illustrative of how the regimes work, and always confirm the current rule text for any regime you actually report under. The mechanics, the pipeline, and the failure modes are stable; the precise thresholds drift.

Mental Model

The wrong model, and it is the one most engineers start with, is that regulatory reporting is an export job. You have the data in your systems, the regulator wants a copy, so you write a nightly batch that pulls the records, maps them to the required format, and emails or uploads a file. In this picture the report is a side effect of the real system, something the data team bolts on at the end, and the goal is just to produce a file that parses.

That model fails in production for a reason worth internalizing. A report is not a copy of your data; it is a legal assertion about your activity, made to a supervisor with the power to fine you, and it is judged against what actually happened in the market, not against what your database happens to contain. The regulator already has, or can get, the other side of many of your trades: the venue reported the execution, the counterparty reported the same transaction, the clearing house reported the position. Your submission is checked against theirs. When they do not match, the burden is on you to explain the gap, and “our export script had a bug” is not a defense, it is an admission.

So hold this model instead. Regulatory reporting is a continuous reconciliation between your firm’s claim about its activity and the market’s independent record of that same activity, conducted under deadline and recorded with full lineage. The output file is the smallest part. The hard parts are knowing which events are reportable, sourcing each field from the one system entitled to provide it, proving where every value came from, getting it submitted before the clock runs out, confirming the regulator actually received and accepted it, and correcting it through a controlled amendment when it was wrong, all while never being able to quietly delete the evidence of a mistake. Once you see reporting as reconciliation rather than export, every design decision below stops looking like bureaucracy and starts looking like the only way the system could possibly work.

Breaking it down

The teaching runs in twelve steps. The first four establish why reporting exists and what makes it hard. The middle four build the machine: the pipeline, the formats, the platform, and the correction lifecycle. The last four widen out to the people, the failures, the scaling path, and the line between universal principle and local convention.

1. Why regulators demand reporting at all

Start with the regulator’s problem, because the obligation is a direct answer to it. A regulator is responsible for the integrity of a market it cannot see directly. Trades happen inside thousands of private firms, across dozens of venues, in microseconds. When something goes wrong, a manipulation scheme, a flash crash, a firm taking on risk it cannot cover, an account laundering proceeds of crime, the regulator needs to reconstruct what happened across all those firms and venues, after the fact, with enough fidelity to act. It cannot do that by asking nicely after the event, because by then the records may be gone, inconsistent, or doctored. So it requires firms to report as they go, in a defined format, on a defined clock, and it holds the firm legally responsible for the accuracy of what it sends.

This is the first principle: the cost of visibility is pushed onto the firm because the firm is the only party that has the data at the moment it is created. The regulator could in theory build the world’s largest data-collection apparatus and pull everything itself, but it would still depend on the firm’s systems to produce the raw events, so the cheaper and more enforceable design is to mandate that each firm report its own activity and to audit those reports against each other and against venue and clearing records. The whole edifice is a distributed surveillance system where the regulator owns the rules and the cross-checks, and the regulated firms own the production of the data.

That framing explains three things that otherwise look arbitrary. It explains why the firm, not the regulator, is liable when a number is wrong: the firm is the source. It explains why completeness is policed as harshly as accuracy: a missing report is a hole in the regulator’s reconstruction, and a hole it cannot even see, because it does not know what it did not receive. And it explains why deadlines are hard rather than soft: a surveillance system that lets firms report whenever convenient cannot detect anything in time to intervene.

flowchart LR
  E["Reportable event<br/>(trade, order,<br/>position, cash flow)"] --> F["Reporting firm<br/>(extracts, formats,<br/>submits, owns accuracy)"]
  F --> R["Regulator or<br/>trade repository<br/>(collects, validates,<br/>acknowledges)"]
  V["Venue and clearing<br/>independent records"] --> R
  C["Counterparty<br/>independent record"] --> R
  R --> S["Surveillance and<br/>cross-firm reconciliation"]

2. The map of what is reported and to whom

Newcomers drown in the alphabet soup of regimes. The way out is to see that almost every regime is one of a small number of types, defined by what kind of event it captures. Learn the types and the named regimes become examples you can slot into place.

Trade and order reporting captures market activity, what was executed and, in some regimes, the full lifecycle of every order that led to it. The Consolidated Audit Trail (CAT) in the United States is the extreme case: it requires broker-dealers and exchanges to report the entire lifecycle of every order in National Market System securities and listed options, every origination, route, modification, cancellation, and execution, time-stamped, so the regulator can reconstruct the order book. The Trade Reporting and Compliance Engine (TRACE), run by FINRA, collects reports of over-the-counter trades in eligible fixed-income securities, with the goal of bringing post-trade transparency to a bond market that has historically been opaque.

Transaction reporting is the European and adjacent family that captures the economic details of completed transactions for market-abuse surveillance and systemic-risk monitoring. Under MiFIR, investment firms report the details of executed transactions in financial instruments to their national competent authority, typically by the end of the next working day, with a record carrying dozens of fields per transaction. EMIR requires both counterparties to a derivative to report it to a registered trade repository, which introduces the two-sided reporting and reconciliation problem that we return to later. SFTR extends the same pattern to securities financing transactions such as repos and stock loans. The common thread is a structured, field-rich record per transaction, delivered to a repository that validates and reconciles.

Financial and operational reporting is about the firm’s own health rather than its trades. The FOCUS Report (Financial and Operational Combined Uniform Single report) that US broker-dealers file with FINRA is the canonical example: a periodic report of the firm’s financial condition and, critically, its net capital, the regulatory measure of whether the firm has enough liquid capital to wind down without harming customers. These reports answer a different question from trade reports: not what did you trade, but are you solvent and operationally sound.

Position and large-trader reporting captures concentration and exposure. When a single participant accumulates a position large enough to matter for market integrity, regimes require it to be reported, so the regulator can see where risk is piling up. Large-trader reporting and position-limit reporting in futures and options markets are the canonical cases, and the motivation is squarely systemic: a position invisible until it blows up is exactly the thing post-crisis reform was built to prevent.

Financial-crime filings are a different animal: filings to financial-intelligence units about activity that may indicate crime. In the United States, firms file Suspicious Activity Reports (SARs) and Currency Transaction Reports (CTRs) with FinCEN, the Financial Crimes Enforcement Network. A CTR is mechanical, triggered by cash transactions over a threshold; a SAR is judgmental, filed when staff or systems flag activity as suspicious. These carry their own confidentiality rules (a firm generally may not tell a customer a SAR was filed) and their own deadlines, and they sit alongside, not inside, the market-reporting regimes.

The payoff of this map is practical. When you meet a regime you have never seen, you do not start from zero: you ask which family it belongs to, and that tells you its shape, its likely cadence, and the kind of cross-check it will be reconciled against. A new derivative transaction regime in another jurisdiction will look a great deal like EMIR; a new market abuse regime will look like MiFIR; a new crime-filing regime will look like the FinCEN filings. The regimes proliferate, but the families are few.

3. The golden-source problem and data lineage

Here is the part nobody warns junior engineers about. The hard problem in reporting is almost never the format. Mapping your data to an XML schema is tedious but tractable. The hard problem is knowing which of your own systems holds the true value for each field, because in any real firm the same fact lives in several places that disagree.

Consider one field: the price of a trade. The execution system recorded a price. The risk system received a slightly different price because it applied a rounding convention. The books-and-records system shows a third value because an adjustment was booked overnight. The client-facing statement shows a fourth because it nets a fee. All four are “the price” in some sense, and a report demands exactly one. The golden-source problem is the discipline of designating, for every reportable field, the one system that is authoritative, and routing the report to draw that field only from that system, so that the value in the submission is defensible.

This is not a one-time mapping exercise; it is a standing governance commitment. Systems change, fields get added, an upstream team quietly starts overwriting a value, and your report begins drawing a wrong number from a system that used to be right. So a serious reporting platform does not just read the golden source, it records lineage: for every field in every submission, it stores where the value was read from, what transformations were applied, and when. When the regulator asks why a price was reported as 100.25, you can answer with the specific source record, the transformation, and the timestamp, rather than shrugging at a black-box export.

flowchart TB
  EX["Execution system<br/>price = 100.25"] --> G{"Golden-source<br/>policy: which<br/>system is<br/>authoritative<br/>for price?"}
  RK["Risk system<br/>price = 100.3"] --> G
  BR["Books and records<br/>price = 100.24"] --> G
  G -->|"execution is golden<br/>for trade price"| L["Lineage record:<br/>value, source,<br/>transform, timestamp"]
  L --> RPT["Reportable field<br/>price = 100.25<br/>(defensible)"]

The golden-source problem is why two firms with identical trading systems can have wildly different reporting quality. The one that treated reporting as an export sometime after the fact wired each field to whatever system was convenient, and now cannot explain its own numbers. The one that treated reporting as reconciliation designed a golden-source map and a lineage store from the start, and can answer any question about any field in any submission years later. Lineage is the unglamorous foundation, and it is the single thing most worth getting right early, because retrofitting it after the firm has scaled is brutal.

4. Accuracy completeness and timeliness as three separate obligations

Engineers tend to collapse “report correctly” into one goal. Regulators do not. A report carries three independent obligations, and a system can satisfy any two while failing the third, which is exactly how firms get surprised.

Accuracy means each reported value is correct: the right price, the right quantity, the right instrument identifier, the right counterparty. Completeness means every reportable event is reported and no field is missing: not 98 percent of trades, all of them, with no silently dropped records. Timeliness means the report arrives before its deadline: by end of next working day for MiFIR transaction reporting, within a defined window for CAT, within the statutory period for a SAR. These pull against each other. A pipeline tuned for timeliness might submit before late-arriving corrections settle, hurting accuracy. A pipeline that waits to perfect accuracy might blow the deadline, failing timeliness. A pipeline that filters out records it cannot confidently format might be accurate and timely on what it sends, while quietly failing completeness on what it dropped.

The completeness failure is the most insidious because it is invisible from inside your own system. If you drop two percent of records, your submissions all look fine, they parse, they balance, they get acknowledged. Nothing in your own world tells you the two percent is gone. The gap only appears when the regulator reconciles your reports against the venue’s record, or the counterparty’s, and finds transactions they have that you never reported. This is the same lesson the ledger track teaches about reconciliation: internal consistency does not prove external correctness. Your reporting system can be perfectly self-consistent and still be missing a chunk of reality, and only a cross-check against an independent record reveals it.

Because the three obligations are independent, a serious platform measures and monitors each separately. It tracks a submission’s timeliness against the deadline, its accuracy against the golden source and against acknowledgments, and its completeness against a control total derived independently of the reporting path. Collapsing them into one “did the report go out” metric is how completeness gaps live undetected for months.

5. The reporting pipeline end to end

Now the machine. Almost every reporting regime, whatever its format and deadline, is served by the same pipeline of seven stages. Each stage exists to defend against a specific failure, and skipping any one of them is how firms end up in enforcement.

The stages are extract, transform, validate, submit, acknowledge, reconcile, and amend. Extract pulls the reportable events from the golden sources. Transform maps them into the regime’s required format and enriches them with reference data (legal entity identifiers, instrument identifiers). Validate checks the record against the regime’s schema and business rules before sending, so you catch errors you can fix rather than errors the regulator catches for you. Submit transmits the file or message to the regulator or repository. Acknowledge is the regulator’s response: accepted, or rejected with reasons, and a report you sent but that was rejected is a report you did not file. Reconcile compares what you intended to report, what you submitted, and what the regulator acknowledged, and (for two-sided regimes) what the counterparty reported, to find gaps and breaks. Amend corrects errors found at any stage through a controlled resubmission that preserves the history.

Two stages deserve emphasis because they are the ones immature pipelines skip. The first is acknowledge. A startling number of reporting incidents come from firms that submitted files and assumed success, never processing the rejection messages, so records they believed were filed were actually bounced for months. Submission without consuming the acknowledgment is not reporting; it is hoping. The second is reconcile. Without an independent control total to reconcile against, completeness is unverifiable, and completeness is the obligation most likely to fail silently. A pipeline that extracts, transforms, validates, and submits, but neither truly consumes acknowledgments nor reconciles, is the classic recipe for an enforcement action that lands years after the gap opened.

6. Standards and formats that carry the data

The format is the easy part, but it is not a free part, and the industry has converged on a small set of standards worth knowing. The dominant direction of travel is toward ISO 20022, an international standard that defines a common dictionary of financial business concepts and expresses messages, historically in XML and increasingly with other encodings, from that shared dictionary. The point of ISO 20022 is not the angle brackets; it is the shared semantics: a “settlement amount” or a “counterparty identifier” means the same modeled thing across messages and jurisdictions, so a firm that models its data once can serve many message types without redefining every field per regime.

Underneath the message standard sit the identifier standards that make cross-firm reconciliation possible at all. A Legal Entity Identifier (LEI) is a 20-character code that uniquely identifies a legal entity that is party to a financial transaction, so that when your report names a counterparty and the counterparty’s report names you, both reports point to the same globally unique code rather than to inconsistent free-text names. An ISIN (International Securities Identification Number) identifies a specific security the same way. These identifiers are the join keys of the entire reporting system: without a shared LEI, the regulator cannot match your report of a trade with Firm X to Firm X’s report of the same trade, and the two-sided reconciliation that regimes like EMIR depend on collapses.

{
  "transactionIdentifier": "TXN-2026-0001847",
  "executingEntityLEI": "5493001KJTIIGC8Y1R12",
  "counterpartyLEI": "549300XYZ123ABC45678",
  "instrumentISIN": "US0378331005",
  "tradeDateTime": "2026-06-03T14:22:07.512Z",
  "quantity": 1500,
  "price": { "amount": "189.4400", "currency": "USD" },
  "side": "BUY",
  "venue": "XNAS"
}

A reporting record is, at its core, exactly this shape: a stable transaction identifier so a later amendment can point back to it, the parties expressed as LEIs so both sides can be matched, the instrument as an ISIN, an exact timestamp, and the economic terms with explicit currency and precision. The XML the regime actually demands is a more verbose encoding of the same content. The reason to understand the standards is not to hand-write XML; it is to appreciate that the entire surveillance system rests on shared identifiers and shared semantics, and that a firm which models its data to those standards internally pays the cost once and reports everywhere, while a firm that maps ad hoc per regime pays it again and again and reconciles poorly.

7. Engineering a reporting platform

With the pipeline and standards in hand, the platform’s architecture follows from a few hard requirements. Every submission must be reversible and correctable, because you will need to amend. Every value must carry lineage, because you must be able to explain it. Every record must be tracked through its full lifecycle from extracted to acknowledged to reconciled, because an unacknowledged record is an open obligation. And the whole thing must be idempotent and replayable, because pipelines fail mid-run and you must be able to re-run without double-reporting or losing records.

These requirements drive a now-standard shape. The reportable events are captured as an append-only event store, never edited in place, so the firm’s claim about its activity has a complete, immutable history exactly as a ledger does. On top of that store, the platform derives reports rather than storing them as the truth, so a report can always be regenerated from the events and the golden-source rules in force at the time. Each submission gets a unique submission identifier and a per-record status that moves through a state machine. Reference data (LEIs, ISINs, the golden-source map itself) is versioned, so a report produced today and a report regenerated next year both reflect the rules that were in force on the reporting date, not today’s rules.

stateDiagram-v2
  [*] --> Extracted: event captured with lineage
  Extracted --> Validated: passes schema and business rules
  Extracted --> Repair: fails validation
  Repair --> Validated: fixed and re-validated
  Validated --> Submitted: transmitted to regulator
  Submitted --> Accepted: acknowledgment received
  Submitted --> Rejected: acknowledgment with error code
  Rejected --> Repair: investigate and fix
  Accepted --> Reconciled: matches independent record
  Accepted --> Break: mismatch found in reconciliation
  Break --> Amended: corrected resubmission, linked to original
  Amended --> Submitted: amendment transmitted
  Reconciled --> [*]

The animation below shows the whole platform as one flow, declared up front: events extracted from golden sources, transformed and validated, submitted and acknowledged, then reconciled against the regulator’s record and the counterparty’s, with the amendment loop that any break or rejection feeds into. Watch how every path that finds an error routes back through a controlled amendment rather than an in-place edit, and how the reconcile stage is the only place completeness is actually proven.

One design decision separates platforms that survive an audit from those that do not: never mutate a submitted record. Just as a trustworthy ledger appends a reversing entry rather than erasing a posting, a trustworthy reporting platform issues an amendment that supersedes the prior record and links to it, leaving both the original error and its correction permanently in the trail. When the regulator asks what you reported and when, the honest, complete answer is in the history, and the firm that can produce that history cleanly is treated very differently from the firm that cannot say what it filed last March.

8. Reconciliation resubmission and the amendment lifecycle

Reconciliation is where reporting earns its keep, so it deserves its own step. There are three reconciliations a serious platform runs, and they catch different failures.

The first is intent-to-submission: did every event you extracted actually make it into a submitted record? This catches records lost inside your own pipeline, the silent completeness killer. The second is submission-to-acknowledgment: did every record you submitted get accepted? This catches records the regulator rejected that you never noticed, so you stop believing you filed something you did not. The third, for two-sided regimes like EMIR, is your-side-to-counterparty-side: does your report of the trade match the counterparty’s report of the same trade, joined on a shared trade identifier and the two LEIs? This catches disagreements about the economics, a price or quantity or date that the two of you booked differently, which the regulator will otherwise flag as a break and push back to both of you.

flowchart TB
  I["Events extracted<br/>(intent)"] -->|"recon 1:<br/>any lost in pipeline?"| SUB["Records submitted"]
  SUB -->|"recon 2:<br/>any rejected unnoticed?"| ACK["Records acknowledged"]
  ACK -->|"recon 3:<br/>do both sides agree?"| CP["Counterparty's report"]
  SUB --> BR1["Break: open obligation"]
  ACK --> BR2["Break: investigate amend"]
  CP --> BR3["Break: economic mismatch"]

When any reconciliation finds a break, you correct it through the amendment lifecycle, and the discipline here is exactly the immutability discipline from the ledger world. You do not edit the wrong record in place. You submit a new record that the regime recognizes as an amendment (typically carrying an action type such as amend, cancel, or correct, and a reference to the original transaction identifier), and the platform keeps both. A cancellation followed by a fresh report, or a correction that supersedes, leaves a trail of the original, the correction, and the link between them. This matters for two reasons. It keeps the audit trail honest, so the firm can always show its full reporting history. And it matches what the regulator’s systems expect: regimes define how an amendment must reference its original precisely because they too need to reconcile your correction against what they previously received.

The deeper lesson is the same one that runs through every part of financial infrastructure: the record of what happened, including what went wrong, is more valuable than a tidy record that pretends nothing did. A reporting platform that can never lie about its own past, because it physically cannot overwrite it, is the only kind a regulator and an auditor can trust.

9. The participants and their conflicting incentives

Reporting is not one party’s problem; it is a system with several participants whose incentives only partly align, and understanding the friction explains a lot of how the ecosystem actually behaves.

The reporting firm wants to meet its obligations at the lowest cost and risk. Its genuine interest is in clean, complete, defensible reporting, because the downside of failure is severe, but its day-to-day pressure is to ship product features, and reporting competes with revenue work for engineering time. This tension is why reporting is chronically under-invested in until an incident, and why the firms that do it well treat it as core infrastructure rather than a compliance chore.

The regulator or trade repository wants complete, accurate, timely, comparable data from every firm so it can surveil the market and detect problems. Its lever is enforcement: fines and remediation orders that make under-investment more expensive than investment. It also has an interest in standardization (shared identifiers, common formats) because comparable data across firms is what makes cross-firm reconciliation possible. A trade repository sits between firm and regulator for some regimes, collecting and validating reports and offering the two-sided reconciliation that EMIR and similar regimes require.

The vendor is the third participant, because most firms do not build the entire stack themselves. Reporting vendors sell connectivity, format mapping, validation, and submission as a service, and they exist because the format-and-channel work is undifferentiated across firms and genuinely hard to keep current as regimes change. The vendor’s incentive is to abstract the regime’s complexity, which is valuable, but it introduces a trap: a firm can outsource the work but not the liability. The obligation and the legal responsibility for accuracy stay with the firm, so a firm that treats a vendor as a black box, and stops reconciling and checking lineage because “the vendor handles it,” has outsourced its competence while keeping its liability. The mature posture is to use vendors for the undifferentiated heavy lifting while retaining the golden-source map, the lineage, and the reconciliation in-house, because those are the things you cannot delegate without delegating your defensibility.

10. Failure modes fines and what actually goes wrong

Reporting failures are not exotic. They cluster into a handful of patterns, each with a characteristic cause, a characteristic way it stays hidden, and a characteristic control that would have caught it. Knowing the catalog is how you build the right defenses instead of discovering them after an enforcement action.

The economic point behind the catalog is that these failures are expensive out of proportion to the engineering cost of preventing them. Regulators across major jurisdictions have levied substantial fines specifically for transaction-reporting failures, often where firms mis-reported or failed to report large volumes of transactions over extended periods, and the penalties have run into the millions and tens of millions for the most serious cases. The fine, moreover, is rarely the largest cost. A reporting failure typically triggers a mandated remediation program (re-reporting years of data, hiring consultants, rebuilding the pipeline under supervision), heightened ongoing scrutiny, and reputational damage with both regulators and counterparties. Set against that, the cost of building lineage, reconciliation, and acknowledgment handling correctly from the start is small, which is the entire business case for treating reporting as core infrastructure rather than a bolt-on.

11. Scaling from a startup to a global institution

The reporting burden does not arrive all at once; it accretes as a firm grows, crosses borders, and adds products, and the architecture has to scale with it without being rebuilt each time.

A small firm under a single regime can run a genuinely simple pipeline: a scheduled job that extracts the day’s events, maps them to the one required format, validates, submits through a vendor, and consumes the acknowledgments. The discipline that matters even at this stage is not scale but shape: capture events append-only with lineage, treat acknowledgments as first-class, and reconcile against at least one independent count. A startup that gets the shape right pays little for it and is ready to grow; one that ships an export script accrues debt that compounds viciously.

As the firm adds regimes and jurisdictions, the combinatorial pressure hits. The same trade may now be reportable under several regimes at once (a single derivative might trigger both a transaction report and a position report in different formats to different recipients), and each regime has its own deadline, format, identifiers, and reconciliation. The architecture that survives this separates a regime-independent core (the append-only event store, the golden-source map, the lineage, the reconciliation engine) from regime-specific adapters (the format mapping, the channel, the schedule for each regime). One canonical model of the firm’s activity feeds many adapters, so adding a regime is adding an adapter, not rebuilding the pipeline. This is the ISO 20022 insight at the architecture level: model once, report many.

flowchart LR
  EV["Append-only event store<br/>(canonical activity model)"] --> CORE["Regime-independent core<br/>golden-source map, lineage,<br/>reconciliation engine"]
  CORE --> A1["MiFIR adapter<br/>(format, channel, deadline)"]
  CORE --> A2["EMIR adapter<br/>(format, channel, deadline)"]
  CORE --> A3["CAT adapter<br/>(format, channel, deadline)"]
  CORE --> A4["FinCEN adapter<br/>(format, channel, deadline)"]
  A1 --> NCA["National competent authority"]
  A2 --> TR["Trade repository"]
  A3 --> CATS["CAT processor"]
  A4 --> FIN["FinCEN"]

At global-institution scale, two further realities dominate. Volume means the reconciliation and acknowledgment-processing path, not the submission path, becomes the engineering challenge, because reconciling millions of records per day against multiple independent sources is harder than producing them. And governance becomes as important as code: a large institution needs a controlled, audited process for changing the golden-source map and the regime adapters, because an uncontrolled change to which system feeds a field is exactly how a firm that was reporting correctly starts reporting wrong without anyone deciding to. The engineering scales by separation of concerns; the correctness scales by governance over the rules that drive the engineering.

12. Fundamental obligations versus jurisdiction-specific regimes

Finish by separating what is universal from what is local, because a senior engineer who can tell them apart can move between regimes and even jurisdictions without relearning everything, while one who cannot treats every regime as a fresh mystery.

The fundamental obligations are the same everywhere, because they fall out of what reporting is for. Every regime requires that you report the events it deems reportable (completeness), that the values be correct (accuracy), that the report arrive by a deadline (timeliness), that you be able to show where each value came from (lineage and defensibility), and that the firm, not the regulator, bear responsibility for all of the above. Every serious regime is reconciled against some independent record, whether a venue, a counterparty, or a clearing house, which is why internal consistency is never enough. And every regime expects corrections to be controlled and traceable rather than silent in-place edits. These are not coincidences across regimes; they are the necessary structure of any system that asks firms to surveil themselves on the regulator’s behalf.

The jurisdiction-specific conventions are everything else, and they are where the work and the variation live: which exact events are in scope, the precise field set and format, the specific identifiers, the exact deadline, the submission channel, whether reporting is one-sided or two-sided, whether a third-party repository sits in the middle, and the penalty structure. These differ by regime and by jurisdiction and change over time, and they are genuinely fiddly, but they are configuration over a stable machine, not a different machine.

flowchart TB
  subgraph universal["Fundamental obligations (everywhere)"]
    U1["Complete: every reportable event"]
    U2["Accurate: correct values"]
    U3["Timely: before the deadline"]
    U4["Defensible: lineage for every field"]
    U5["Firm-owned: liability stays with the firm"]
    U6["Reconciled: checked against an independent record"]
    U7["Corrected, never erased: controlled amendments"]
  end
  subgraph local["Jurisdiction-specific conventions (configuration)"]
    L1["Which events are in scope"]
    L2["Field set and format"]
    L3["Identifiers and channel"]
    L4["Exact deadline"]
    L5["One-sided vs two-sided; repository in the middle"]
    L6["Penalty structure"]
  end
  universal --> local

This is the same distinction that runs through all of financial infrastructure: there is a small set of load-bearing principles, and a large set of institution-specific and jurisdiction-specific conventions built on top. Master the principles and the conventions become learnable details rather than an endless catalog. The engineer who internalizes that completeness, accuracy, timeliness, lineage, firm liability, independent reconciliation, and controlled correction are the invariants, and that everything else is a parameter, can build a reporting platform that absorbs a new regime as a new adapter, and can read an unfamiliar rulebook and already know what shape the answer has to take.

Mastery Questions

1. Your firm’s MiFIR transaction reports for the last quarter all parsed cleanly, were submitted before the next-day deadline, and every single one was acknowledged as accepted by the national competent authority. Your monitoring is green across the board. Should you be confident the firm met its reporting obligations, and if not, what would you check?

   Answer. No, you should not be confident, because everything you listed measures only two of the three obligations and a self-referential version of the third. Clean parsing and acceptance show the records you sent were well-formed and accurate enough to pass the regulator’s validation; submission before the deadline shows timeliness; but none of it shows completeness, because completeness is about the events you did not send, and a pipeline that silently dropped records will report nothing about them and acknowledge nothing about them. Green monitoring on submitted records is exactly the blind spot that lets a completeness gap live for months. What you check is reconciliation against an independent count: take the day’s executions from the golden source (and, where available, the venue’s record and the counterparty’s reports) and confirm that the number of reportable events equals the number of records submitted and accepted. The records that exist upstream but never appear in a submission are the ones quietly failing the obligation, and they are invisible from inside the submission path. Internal consistency, all your reports being fine, never proves external correctness, that all your reportable events were actually reported.

2. A junior engineer fixes a mis-reported trade price by updating the record in your reporting database and resubmitting the corrected value, and is pleased that the regulator now has the right number. Walk through why this is the wrong way to correct a report even though the final value is correct, and what the right approach is.

   Answer. The final value being correct is not the standard; the standard is that the firm can always show, truthfully and completely, what it reported and when. By updating in place, the engineer has destroyed that. The firm’s own history now shows only the corrected price, so it can no longer prove what it originally filed, which is itself a defensibility failure. The regulator’s systems hold the prior, wrong copy and now receive a new value with no defined link explaining that this is a correction of that specific earlier record, so the regulator’s reconciliation sees an unexplained discrepancy rather than a clean amendment. And an auditor asking what was reported on the trade date gets a rewritten answer, which is worse than the original error because it looks like concealment. The right approach is a controlled amendment: submit a new record that the regime recognizes as a correction, carrying the appropriate action type and a reference to the original transaction identifier, so it supersedes the prior record while both remain in the audit trail, linked. This mirrors the ledger discipline of appending a reversing entry rather than erasing a posting: the record of the mistake and its correction is more valuable than a tidy record that hides that anything went wrong, and it is also what the regulator’s own reconciliation expects.

3. Your firm is small, reports under a single regime today, and a vendor handles the format mapping, submission channel, and acknowledgment plumbing. The CTO argues that since the vendor owns the hard parts, the firm can keep its own pipeline as a thin export script and invest engineering elsewhere. What is the flaw in this reasoning, and what should the firm own regardless of the vendor?

   Answer. The flaw is conflating outsourcing the work with outsourcing the liability. The legal responsibility for accurate, complete, timely, defensible reporting stays with the firm no matter how much of the plumbing a vendor runs, so a thin export script feeding a black-box vendor leaves the firm holding all of the liability with none of the controls that make it defensible. Specifically, the vendor can map and transmit and collect acknowledgments, but it cannot tell the firm which of the firm’s own systems is the authoritative source for each field (the golden-source problem is internal to the firm), it cannot prove the lineage of values it never saw the origins of, and it cannot reconcile the firm’s reports against an independent count of the firm’s own activity, because it only sees what the export script chose to hand it. So the firm should own, regardless of the vendor, three things: the golden-source map (which system is authoritative for each reportable field), the lineage record (where every reported value came from and how it was transformed), and the reconciliation against independent records (intent-to-submission for completeness, submission-to-acknowledgment for filing confirmation, and, where the regime is two-sided, the match against the counterparty). Those are precisely the things that fail silently and that determine whether the firm can defend itself, and none of them can be delegated without delegating the firm’s own defensibility. Use the vendor for the undifferentiated, fast- changing format and channel work; keep the defensibility in-house.