Yes in what we call our hot path, it's true most of the time you're just in a busy loop with no action.

When a market event occurs that triggers some action then you need to respond with some events in your busy loop in a time that is under your alpha decay (SLA). For example generate a series of orders that need to go out to the market. Yes we use likely and unlikely quite a bit in our core infrastructure code. We are very happy with these new features.

And given the nature of low latency systems, you are correct about keeping our 'hot path' hot.

So what we typically do is basically have timers that will fire every so often which will warm our cache, by reading in the next blocks of memory where our events will be stored ready for processing (orders ready for risk checks for example to go out to exchanges ) SO this is how we reconcile the fact that our hot paths have infrequent random events that require low latency SLAs.

Thanks, Wes

On Mon, Oct 3, 2022 at 5:23 PM Arthur O'Dwyer via SG14 <sg14@lists.isocpp.org> wrote:

On Mon, Oct 3, 2022 at 3:29 PM Tjernstrom, Staffan via SG14 <sg14@lists.isocpp.org> wrote:

Wes makes a good point that’s worth highlighting. Most of the time a financial / HFT / low-latency system is doing….. nothing. It’s waiting to react. It’s when it wants to react that the timing constraints come into play (typically). In that sense it’s much like the brake circuit on a vehicle. It’s not active, until it is.

This (or an adjacent idea) was a big talking point back when SG14 was discussing [[likely]] and [[unlikely]]. What got standardized (P0479) was basically the existing practice in GCC, and with very underspecified behavior to boot; but a big deal was made in SG14 of the fact that
- "This branch happens 99% of the time / this is the hot path" does not necessarily imply "we must save every possible cycle"
- "This branch happens 1% of the time / this is the cold path / this is the exceptional path" does not necessarily imply "we can afford to deoptimize this path"
In fact (it was claimed at the time), in HFT it's often exactly the opposite: it's the exceptional, once-a-week-at-most, very-cold codepath that must always be ready to execute the fastest.

I have no firsthand knowledge of whether this is true, and would be interested to hear more concrete details from Staffan, Wes, et al. if possible — how true is this? what's an example? what mechanisms are used to "keep cold code hot" in practice?
And did the standardization of [[likely]] and [[unlikely]] in C++20 actually help anyone in this space (or elsewhere)?

–Arthur
_______________________________________________
SG14 mailing list
SG14@lists.isocpp.org
https://lists.isocpp.org/mailman/listinfo.cgi/sg14

Wesley C. Maness
347-387-7162