On Tue, May 30, 2023 at 4:44 PM Daniel Ruoso <daniel_at_[hidden]> wrote:

> Em sex., 26 de mai. de 2023 às 11:57, Mathias Stearn <
> redbeard0531+isocpp_at_[hidden]> escreveu:
>
>> Sorry, there was a detail that I forgot to mention explicitly that is
>> important to understanding how this answers your questions. The key is that
>> this leans *very* heavily on ninja's restat=1 reature and a write_if_changed
>> function
>> <https://github.com/RedBeard0531/cxx_modules_builder/blob/e13174805992c7aa10e8fbc9b1452cbfc0f66ecc/modules_builder.py#L92-L99> in
>> the scanner to be sure to only touch files if their contents change. This
>> means that even if we need to redo a scan (which we would because all scans
>> depend on the yaml file holding the list of importable headers), we *don't*
>> need to recompile unless it actually would affect that compile.
>>
>
> This is a massive change in requirements for build systems.
>
> Either the tool driving the build supports the `restat` feature (as ninja
> does), or the build system has to use the checksum of the contents to drive
> target invalidation (as scons does).
>

Please don't move the goalposts. You asked me what was needed to support
importable headers *in ninja*, so that is what I provided. This is
especially notable in the context that where I was saying that different
build strategies will apply for different build systems (specifically the
execution component like make/ninja/msbuild). *Of course* a strategy that
was specifically designed for ninja to take advantage of restat will not
work unmodified with make which doesn't.

I freely admit that I am not an expert in Make. In fact, I avoid it like
the plague wherever possible. However, I suspect that you can still make
this work there, just with a different approach. It will likely involve
dynamic build nodes (which Make supports but ninja doesn't), and may
involve some dirty tricks like using order-only deps and not declaring all
outputs, and instead relying on Make to evaluate timestamps lazily.


> It probably means that we need the compiler to support an option to
> only overwrite the file if the contents are different. The alternative
> being that the build rule has to generate an intermediate file which then
> only gets renamed over the old one if the contents are not the same.
>

I believe you only *need* this for the scanner, not necesarily for the
compiler. Of course it would be *nice* if the compiler would only modify
the BMI files if their contents change (and hopefully doesn't put nonsense
like timestamps in there so that they always change), but that is more
about enabling more-minimal incremental builds than we have today with
headers, rather than being a fundamental requirement. And as you point out,
a build system can usually work around that by using its own shadow outputs
and adding a copy-if-changed mechanism.


> It essentially means that `make` is no longer a viable implementation to
> drive a C++ build (not even generated make, as CMake does). As I mentioned
> before, we use ninja for development builds, but it doesn't satisfy all our
> requirements for the production builds (e.g.:
> https://github.com/ninja-build/ninja/issues/1139 ).
>

You also said that you'd be willing to move to ninja if they took that
patch. Given that you have a working (I assume?) patch that meets your
needs, I suspect that using a patched ninja is a tiny amount of work
relative to the other build system work required to support modules.


> And that's the crux of what the paper is pointing at: Only a subset of
> build systems in use today will be able to viably support Importable
> Headers, which will exclude systems where Named Modules could be
> implemented.
>

I'm still not sure how any of this is specific to header units. The scan
step is already required for named modules, and if you have a single
megascan as a normal (ie not order-only) dep of every compile, AND you are
unable to do the equivalent of restat=1, it is going to hurt.


> I guess the fundamental question is whether we have consensus on "GNU Make
> is no longer a viable C++ build driver". I, personally, strongly oppose
> that direction.
>

Personally, I'd love if we could decouple from broken build systems of the
past. There is so much fundamentally wrong with Make, that I'd love to see
adoption of modules, and the need to do build system work anyway, be used
as a reason to wholesale move off of it. I'm not saying that it *can't* be
made to work, or even work well, I'm just saying that we should instead
invest any effort in less fundamentally broken tools.


>
> daniel
>
>