On 5/24/23 2:48 PM, Daniel Ruoso wrote:
> Em qua., 24 de mai. de 2023 às 11:06, Tom Honermann
> <tom_at_[hidden]> escreveu:
>
>> Concretely, what would happen is that if I do an `apt-get update &&
>> apt-get dist-upgrade` it could result in a full clean build in my
>> incremental build because the list of importable headers or the
>> arguments to those could change.
>
> We seem to have a clear disagreement regarding this.
>
> Yes. That much is true.
>
> Let me try to break it down in steps, so we identify where the
> disagreement is:
>
> 1. The list of header units and their arguments (at least the ones
> used in that translation unit) are an input to the dependency scanning
> process.
>
> 2. The dependency file is used to generate the module map.
>
> 3. The module map is an input to the translation itself.
>
> 4. For a significant number of build systems, an invalidation on the
> beginning of a chain unconditionally invalidates all downstream targets.
Or beginning anywhere within the chain, but yes.
>
> So far so good?
I believe so.
> Now, let's start with the clean build.
(There seems to be a missing step 5; I'll just assume a classic
off-by-one error).
>
> 6. We don't yet know which importable headers could be possibly used
> by any translation unit, we just have a clean checkout of the code and
> libraries installed on the system.
>
> 7. The example I presented elsewhere on this thread hopefully shows
> that the dependency scanning needs to know the list of importable
> headers before the dependency scanning starts.
>
> 8. Therefore, the only option is to give the entire list of
> importable headers and their arguments to the initial dependency scanning.
Correct. Or equivalently, the dependency scanning can consist of two
phases in which the first phase identifies importable headers by
scanning for metadata (e.g., Clang's module maps or Microsoft's
header-units.json files).
>
> In other words, your proposal presumes that the dependency chain that
> started from the clean build somehow gets replaced by a narrower set
> of inputs. I don't see how that would work in ninja, for instance.

I haven't offered a proposal. I'm not sure where the idea of a narrowing
set of inputs comes from. I think the only thing that I described that
sounds similar is the case where a header file is removed but where
there is still dependency information that refers to it. I didn't mean
to suggest that such a scenario be automatically resolved; in my
experience that usually requires a manual action to regenerate
dependency information for the impacted TUs (e.g., removing generated
".d" files). These kinds of issues have never occurred with enough
frequency for me to have been worth trying to automate. I suspect that
the cases of importable headers becoming non-importable is similarly an
unusual occurrence that might not be worth automating.

>
> Would you mind elaborating how you see that happening?

Let me ask a question first. From a build system perspective, do you see
there being a difference in requirements for header units and for
generated header files?

I see only one slight difference; for header units, there is an
additional step to map an import to a specific build (a specific BMI
variant) of a header unit. Thus, the dependency scanning has to resolve
an imported header to a set of header unit builds from which one is
selected. Crucially, the build system must know which variants it will
build (and how to build them) and the dependency information must
reflect the variants. I believe this is also true for named modules
since variants might be necessary for them as well (particularly when it
is desirable for some declarations to not be present such as when
targeting a minimal deployment version and wanting to ensure no
unintended uses of features that require a later deployment are used;
e.g., for macros like _XOPEN_SOURCE).

Tom.

> daniel