I debated sending this to the list, but since my reply is off topic and potentially could be seen as an attack (though I don't mean it that way I figured I'm better send this privately. Feel free to forward to others - I think you will find it helps you avoid jumping into a world of hurt.

On Sat, Aug 30, 2025, at 00:40, Tek, Robert Mate via Std-Proposals wrote:

>
> Mind you, when I say 'unit test', I am not necessarily referring to
> 'proper' unit testing of a single class in isolation, but rather any

 A CLASS IS NOT A UNIT. I don't know what unit means - I've pressed people on this and never got a satisfactory definition (and thus I quit using the term unit or integration tests as they are too meaningless), but everyone seems to agree a class is not a unit. A class can be multiple units without violation the single responsibility principal.. More often a unit is a collection of classes and sometimes even modules, and yet still not an integration test (again, whatever that is).

In general a class is a terrible thing to test in isolation. The purpose of a test is to ensure that no matter what some invariant will never change. Unfortunately we have no way of saying "here is what I mean to test, but the ABI/function signatures may change", and so our test frameworks all end up locking in not just the logic we care about, but also the API and thus prevent changing that code in the future.

Thus the smallest "unit" needs to be large enough that whatever refactoring you may want to make in the future (when some future requirement has changed). Now the concern about too large is valid - you need most of your tests to be fast and reliably give the same result 100% of the time. So you do need to find that balance, but be very careful of the temptation to go too small - it will hurt you in the future.

One of the largest values of an automated test is when it alerts you to a problem by failing. 80% of your tests (I don't have a study on this, but 80% feels right) will never again fail after the initial failure in TDD and you could safely delete them - but there is no way to know which 20% will fail and so we keep them all. In my experience the large the test the more likely it is to fail on real bugs (they fail on flakiness too), so make sure you have enough large tests - while of course avoiding the issues with large tests - this is often a compromise.

We have learned some rules over the years.

UI is partially fashion/fads and so it changes often, so you should always test just below the UI. There are some great UI testing tools which you should use to do "integration tests" as well if you have such a tool, but in general test below the UI. This is also a good thing because if you have a strong UI layer division you can add other UIs and this is often useful.

Anything that has '"many" users won't be changed anyway, even if they are internal. While you could in theory make a change in practice anytime you look at it you discover all those places that need to be touched and give up (even with automatic refactoring you still need to manually test it all - I believe in automated tests, but good manual testing is still very important if you want high quality). As such you may as well test that extensively as it won't change anyway. I would test "strlen" as a unit because despite how bad c strings are, they are not going anywhere, but if I was implementing new a string interface for my company I'd test it mostly through the code that uses it until I have a lot of users at which point the interface can't/won't change anymore.

Any interface with multiple implementations becomes a place to break in tests. You may want to test behavior that is hard to get with the real implementations, but a fake (I avoid mocks - mocks are an assertion that a function will be called - almost never something you care about!) can provide it. I've also had some luck writing code to test the implementations of an interface provide that contract, but not enough to tell you how to do it.

Next I'd look at the org chart. There is always friction around code that is used by other teams, and changing those interfaces becomes political. Thus I would always test my code to others before handing it off, changes will become hard anyway (even if we are designing the interface together it will be locked in ways that I'd refactor internals code). The farther the team is from me on the org chart the more testing I will do.

> kind of testing that can be done with unit testing frameworks such as
> Google Test or Catch2, because we all know that many times, these tests
> are more on the level of integration tests, or sit somewhere above
> strict unit testing level on the testing pyramid. Regardless, any such
> tests are welcome, and better than not testing at all, and (in my
> experience) are far preferred than release testing (testing the entire
> codebase all at once).

Just remember you still need extensive manual release testing. Automated tests long term save more money than they cost, and they speed up release, but they only get a minority of the issues manual testing will catch.

> I do read the list of upcoming C++ language and library features from
> time to time (using C++ 20 at my workplace as of today), and I get the
> feeling that 99% of them will be of no use to me, or the average
> programmer, but rather library authors, or compiler vendors, so I
> decided to try my luck here.

There is good reason for that. Most of us do not need anywhere near as generic as library code and so we don't need that, while the library does. Most of us don't write code where a 0.1% improvement would have a noticeable effect, but for library code that small a change will make a big difference to somebody, and even you personally don't see the difference it will save millions of dollars every year in the electric bills (across many different companies - Facebook alone has hinted that they can save 6 figures from changes of that magnitude) Most of us don't write the complex algorithms - we look for them in our library.