[Replying to multiple emails.]

Gabriel Dos Reis via Modules <modules_at_[hidden]> writes:

> > On Apr 11, 2019, at 7:31 AM, Boris Kolpackov <boris_at_[hidden]> wrote:
> >
> > My point is I don't think the assumption that header
> > units will only be used with headers that rarely
> > change (and therefore can be handled with a "total
> > precompile") is valid.
>
> Hmm, I don’t get it.
>
> Header units are nothing but formalization of PCHs. The long standing
> tradition, practice and recommendation is that you put in PCHs only
> stuff that rarely change - at least for the inner loop of development.

My understanding of the state of PCHs is they only saw wide adoption
on MSVC where one can only have one PCH per TU. That would explain
this tradition/practice/recommendation.


> Do you have new data that would invalidate that recommendation?

Google's use of Clang modules, which, if I understand correctly,
are pretty much header units.


Gabriel Dos Reis via Modules <modules_at_[hidden]> writes:

> I am worried about an elaborate scheme.

I think we are long past worrying about that? ;-)


> If you believe dynamic resolution of BMI is inevitable, please walk
> me slowly through the steps that led you there.

Ok, let me try. Imagine you are a build system trying to compile
a TU that contains an import declaration like this:

import hello.core;

Before you can compile such a TU you need to make sure you have
the BMI for the hello module ready and up-to-date. Since the set
of imports in each TU changes frequently, in most cases it would
be burdensome to expect the user to supply this information in a
way that is easy for you to consume.

Therefore, you are generally expected to extract this import set
from the TU. While not exactly trivial, it is doable: you first
preprocess the TU and then parse the result looking for the
import declarations. Or, you can check if your compiler can do
this for you (which is the original topic of this discussion).

Ok, normal modules are good: you can obtain the import set, make
sure the BMIs are ready, and compile the TU.

Now let's say the TU contains an import declaration like this:

import <hello/core.hpp>;

The above approach of obtaining the import set (preprocess and
parse) no longer works because header units may export macros
and such macros may affect further preprocessing, for example:

import <hello/core.hpp>; // exports HELLO_CORE

#ifdef HELLO_CORE
import <hello/extra.hpp>;
#endif

So, in a nutshell, you need the header unit BMIs as you are
trying to discover the BMIs that you need.

Several solutions have been proposed which I have listed in
my original email:

Boris Kolpackov <boris_at_[hidden]> writes:

> 1. Use a dynamic module mapper to supply these BMIs as pre-
> processing progresses. This is the GCC's Module Mapper
> approach (P1184).
>
> 2. Assume the compiler takes care of it somehow. As to how,
> there were several ideas:
>
> b) The compiler builds the header unit BMIs on the fly in
> some kind of a cache.
>
> a) The compiler still preprocesses the header emulating
> the import semantics (macro isolation, etc).
>
> 3. Have all the header units pre-compiled at some kind of a
> pre-build step (since they can import each other, their
> dependency information will have to be pre-specified).
>
> 4. Anything I am missing/forgetting?

To me only the first approach (dynamic mapper) seems like a
realistic option for a general-purpose build system (other
options may be appropriate for compiler or codebase-specific
build systems). Specifically, (2) feels too much like
brittle compiler magic while (3) is too restrictive.