Hello,

I would like to provide better benchmarking numbers for the effects of converting all contracts in P2900 to assume statements. The executive summary is that I had mixed results with Eigen and could argue this is a bug in clang's optimizer.

This is not a proposal to modify P2900. Specifically this is listed in P2900 as something not proposed: 2.3 Features Not Proposed: The ability to assume that an unchecked contract predicate would evaluate to true and to allow the compiler to optimize based on that assumption, i.e., the assume semantic.

To provide a benchmark, my immediate concern was finding a representative project. Most well known high-performance projects already use __builtin_unreachable() (a.k.a. [[assume(false)]]) and it didn't seem too credible to experiment on something that has already been carefully instrumented that way.

Meanwhile P2646R0, the paper that successfully lobbied for the inclusion of [[assume]], used a synthetic benchmark that is not representative of widespread use across a non-trivial program either.

In the end, I settled on the Eigen math library because it is well known, has no assume statements and has a benchmark. The results were mixed and not too encouraging. Eigen may not have been an ideal test subject either as it has been carefully optimized too. Please consider this a request for comments, as I would prefer a benchmark that would be easily recognized as a good example of this technique being applied to a "normal, well written, performance sensitive application."

These numbers were acquired by converting all asserts to __builtin_assume() and then using AI to add some more. The command line was: clang++-18 -O3 -march=native. This is AMD with AVX-512, AVX2/FMA.

Largest improvements (assume is faster):
  - bench_gemm float: nearly all cases 1-9% faster, up to -8.53% at 192x192
  - TRMV_float_Lower/512: -8.46%
  - VectorNorm_double/65536: -14.01%
  - VectorMinCoeff_float/4096: -7.59%, /262144: -7.43%

  Largest regressions (assume is slower):
  - BlockRead_float/512/64: +78.29% - extreme outlier, possible cache effect
  - TRSV_float_Lower/512: +18.78%, /128: +17.61% - consistent, likely real
  - TRMV_float_UnitUpper/1024: +18.89%, TRMV_float_Upper/1024: +18.51%
  - TRMV_double_UnitUpper/2048: +15.63%
  - Dot_cfloat/1048576: +15.76%, Dot_double/65536: +15.78%
  - VectorLpNormInf_double/1048576: +20.62% - worst case overall

The assume hints help float GEMM consistently but hurt triangular solvers (TRSV/TRMV Upper) and some large-vector reductions significantly. The __builtin_assume hints appear to misguide the compiler's vectorization decisions for certain upper-triangular traversal patterns and large strided reductions.

The reality is that Eigen has use cases that are evidence of clang using optimization hints to deoptimize a program instead. Arguably this is a bug in clang's optimizer. This means it is advisable to keep an eye on your generated assembly and timing numbers as you add assume hints. I have seen some reports from embedded systems programmers saying that __builtin_unreachable() is extremely important to them for reducing code size. And the widespread use of __builtin_unreachable() also shows their importance. However, it is hard to encourage wholesale global use today based on these numbers. Attached is the full benchmark data.

Again, if you have a suggestion for a benchmark that is not already optimized with assume semantics, is open source and would be particularly credible as an example of whole program optimization, let me know.

All the best,

Adrian Johnston