A bot did something last quarter that I could not explain, and the log I reached for was a wall of plaintext lines that told me the order got sent and filled but not one word about why. I could see the trade. I could not reconstruct the decision behind it. That gap, between having a record of what happened and having a record of why it happened, is the whole reason I now treat logging as a first-class part of a trading system rather than something you bolt on after the strategy works.
The uncomfortable part is that most logging in trading code is written for the happy path. You log that the signal fired, you log that the order filled, and it reads fine when you skim it live. Then something weird happens months later, you open the log to do forensics, and you realize every line was written to be glanced at, not queried. So this is the schema I have settled on, the correlation-ID discipline that ties it together, and the boring retention math that keeps the whole thing affordable when you are producing millions of events.
The five stages you actually need to record
A trade is not one event. It is a chain of them, and the reconstruction only works if you capture each link separately and can walk from any point to any other. I think of it as five stages, and I log each one as its own structured record.
- Signal. The raw input that woke the strategy up. A price cross, a funding-rate shift, a new disclosure landing in your feed, whatever it is. Record the input values as they were at that instant, not a summary. If the signal came from a data source, record the source and the timestamp the data itself carried, which is often different from when you received it.
- Decision. The strategy's verdict and, more importantly, the reasoning inputs. Which rule matched, what the position sizing math produced, what risk checks passed or were skipped, and what the strategy chose not to do. The decision record is the one everybody forgets, and it is the one you will want most.
- Order. The intent you sent to the exchange. Symbol, side, size, type, price, time-in-force, and the client order ID you generated. This is what you asked for.
- Ack. The exchange's acknowledgment that it received the order, with the exchange-assigned order ID. This is where you learn the exchange took the order, rejected it, or went quiet.
- Fill. The execution reports, and there can be several per order. Each partial fill, its price, its size, its fees, and the venue timestamp. This is what actually happened, and it rarely matches the order exactly.
The reason to keep these separate instead of collapsing them into one fat trade record is that the interesting failures live in the gaps. An order with no ack. An ack with no fill. A fill at a price nowhere near what the decision assumed. If you only store the final outcome, every one of those gaps becomes invisible.
Correlation IDs are the whole trick
None of the five records above is useful in isolation. What makes reconstruction possible is being able to pull every event tied to a single trade and lay them out in order. That means a shared correlation ID stamped on every record from signal to fill.
The rule of thumb I use is that a correlation ID is minted at the signal, the earliest possible moment, and it rides along through every downstream record. When one signal fans out into several orders, each order gets its own order-level ID as well, but they all keep the parent correlation ID so you can still gather the whole family. Practically, that means a query like give me everything with correlation ID abc123 returns the signal, the decision, all the orders, all the acks, and all the fills, and you can read the trade like a story.
A few things I have learned to stamp on every record, not just the correlation ID:
- A stable strategy identifier and version. When you change the strategy, the old logs should still tell you which version produced them, because otherwise you cannot tell whether a weird trade was a bug or the old behavior working as designed.
- The client order ID, generated by you and idempotent, so a retry does not create a duplicate order and your logs do not lie about how many orders you sent.
- Two timestamps per event where they differ: when your system observed it, and when the venue says it happened. Clock skew between your box and the exchange is real, and if you only keep one timestamp you will misorder events during exactly the fast-moving windows where ordering matters most.
Make it structured, and log the numbers you already have
Plaintext is the enemy here. If your logs are human-sentence strings, every forensic session turns into regex archaeology. Structured logs, one JSON object per event with consistent field names, mean you can filter and aggregate instead of grepping. The signal record and the fill record should share field names wherever they describe the same thing, so a symbol is always in a field called symbol and never sometimes ticker.
The single highest-value habit is recording the numbers the decision was based on, at decision time, inside the decision record. Not the price now, the price the strategy saw. Not the balance now, the balance it sized against. When a trade looks wrong months later, the first question is always whether the strategy was working from bad data or was working correctly from data that has since moved. You can only answer that if you froze the inputs into the log at the moment of the decision. Reconstructing them afterward from historical data is guesswork, because you rarely know exactly which snapshot the code was holding.
One concrete failure mode worth internalizing: an order fills at a price that surprises you, and the order log shows the price you sent, and the fill log shows the price you got, but nowhere is there a record of the market state the strategy used to pick that price. So you cannot tell whether the strategy made a bad choice or the market moved between decision and execution. Log the reference price and the spread the decision used, and that entire class of mystery disappears.
Retention and cost without drowning
High-frequency systems produce a lot of events, and the naive answer of keep everything forever at full fidelity gets expensive fast. The approach that has held up for me is tiering by both age and importance.
Split your events into two buckets. Decision-critical events, the five stages above, are the forensic record and you keep them a long time, typically long enough to cover whatever dispute or audit window you actually care about, often measured in years for anything touching real money. Then there is high-volume operational noise, things like per-tick market-data snapshots and heartbeat pings, which are enormous in aggregate and rarely useful past a short window. Those you keep hot for days to weeks, then drop or heavily downsample.
For storage, a rough tiering that works:
- Hot tier, roughly the last few weeks, in something you can query fast and interactively. This is where live debugging happens.
- Warm tier, older than that, compressed and moved to cheap object storage, still queryable but slower and fine for occasional forensics.
- Cold or deleted for the pure operational noise once its short window passes.
Two things keep the bill sane. Compression, because structured logs with repeated field names compress extremely well, often shrinking by a large multiple. And partitioning by date and strategy, so a forensic query three months later scans one day of one strategy instead of your entire history. The mistake I see is treating all events as equally precious. Your fills deserve years. Your raw tick feed does not, and paying to store it forever is how logging quietly becomes a large infrastructure cost.
The test for whether you have got this right is simple. Pick any trade from a few months ago, take only its correlation ID, and see if you can answer why it took this trade without opening any other tool. If you can walk from the signal through the decision inputs to the order, the ack, and the fills, and it reads like a coherent account, you are done. If you find yourself inferring the reasoning from the outcome, you logged the what and skipped the why, and the next surprising trade is going to cost you a long afternoon you did not need to spend.