Wes makes a good point that’s worth highlighting. Most of the time a financial / HFT / low-latency system is doing….. nothing. It’s waiting to react. It’s when it wants to react that the timing
constraints come into play (typically). In that sense it’s much like the brake circuit on a vehicle. It’s not active, until it is.
This (or an adjacent idea) was a big talking point back when SG14 was discussing [[likely]] and [[unlikely]]. What got standardized (P0479) was basically the existing practice in GCC, and with very underspecified behavior to boot; but a big deal was made in SG14 of the fact that
- "This branch happens 99% of the time / this is the hot path" does not necessarily imply "we must save every possible cycle"
- "This branch happens 1% of the time / this is the cold path / this is the exceptional path" does not necessarily imply "we can afford to deoptimize this path"
In fact (it was claimed at the time), in HFT it's often exactly the opposite: it's the exceptional, once-a-week-at-most, very-cold codepath that must always be ready to execute the fastest.
I have no firsthand knowledge of whether this is true, and would be interested to hear more concrete details from Staffan, Wes, et al. if possible — how true is this? what's an example? what mechanisms are used to "keep cold code hot" in practice?
And did the standardization of [[likely]] and [[unlikely]] in C++20 actually help anyone in this space (or elsewhere)?
–Arthur