On 2/8/19 10:46 AM, Matthew Woehlke wrote:
> On 08/02/2019 00.33, Tom Honermann wrote:
>> A couple of people have countered that build system updates are required
>> (presumably in excess of just adding '-fmodules' to Clang invocations)
>> in order to enable support for Clang Modules. This seems trivially
>> incorrect to me, so long as one does not consider Clang's
>> module.modulemap files to be part of the build system (I don't; they are
>> tool specific configuration files that the build system does not need to
>> be aware of).
> I think the objection here is that modules as they are currently
> proposed encode this information in the source files rather than any
> central file (if I am understanding the clang model correctly). This
> would make the modulemap something that should be tool-generated, and
> thus, *should* be something that the build system cares about. Moreover,
> in order to do so, the build system has to scan many source files, some
> of which *may not exist yet*.
Yes.
>
> If I understand it correctly, the clang approach is to instead encode
> all of the export information in the modulemap. This means that tooling
> is ensured to already have a central location that describes how to find
> a module given its name. Combined with build-(or parse-)on-demand, this
> does indeed mean that builds (and other tooling problems) Just Work,
> albeit at a cost of some parallelism. However, it should not be hard for
> tooling to make improvements there.

Yes. There are a couple of parallelism costs that you could be
referring to here, but I'm not sure which one(s). Could you elaborate?

One is the cost of compiler invocations synchronizing over updates to a
central module artifact repository. Another would be if the module map
files are generated. In my experience, Clang module map files tend not
to be generated.

>
> The critical difference is that the modulemap essentially spells out the
> "harder" half of dependency extraction. I think if we were to take this
> approach, i.e. that the information as to what files generate what
> modules is NOT in the source files themselves, but is instead in a
> separate file, this would greatly simplify or eliminate many of the
> problems that have folks concerned. (I'll note, however, that describing
> *what* to export can still be done in the source file, which would be a
> significant improvement on the current clang model.)

With Clang modules, the module names don't currently matter for C++
since imports are specified via #include (Objective-C has an import
declaration that uses the module name, and that mechanism could be
extended to C++).

I agree that the ability to limit exports in source is desirable.

>
> One drawback of this approach is that it is difficult to use the
> preprocessor to affect names of module exports, although it could be
> argued whether that is a bad or a *good* thing. OTOH this could possibly
> be solved by the modulemap itself being subject to the preprocessor.
I don't think this is an issue with Clang modules since a different set
of macro definitions will result in a distinct module artifact being
produced and used (the set of variants can be trimmed via configuration
macro declarations in the module map).
>
>> I'd like to hear more details from those that believe
>> build system updates are required to take advantage of Clang modules.
>> What changes do you believe build systems have required?
> I don't know if anyone (other than clang itself?) is using clang modules
> with CMake, but I suspect the main changes that would be *desirable*,
> though not necessarily required, would relate to creating build edges
> for modules so that they can be built once, ahead of time. Right now, I
> would guess that the build tool could end up trying to build in parallel
> several TU's that share imports, which would lead to building those
> modules multiple times. This *works*, but at least in theory could be
> more efficient.
I agree and Clang supports this today. The '-fno-implicit-modules',
'-fmodule-file=[<name>=]<file>', and
'-fprebuilt-module-path=<directory>' options can be used when extrinsic
module management is desired.
>
> But as far as I can figure out, if modules are built on demand, and the
> compiler can always determine how to build a module from nothing but the
> input source and compile flags (or information indirectly provided via
> the same), then both builds and tooling can work with minimal or no changes.
Exactly! And as I claim, this is a key reason for the success of Clang
modules.
>
> The main reason we don't have that situation currently is that we don't
> have the second part, and per above, a large reason *why* we don't is
> because the information that maps source files to modules is scattered
> across all of those source files.

Yes. But also, the separate compilation model *can* result in a
situation where separate compilation is required. E.g., when the module
source and module consumer require different compiler options (that
don't impact ABI). Use of implicit module artifacts prevents that
situation from arising.

Tom.

>
> On 08/02/2019 00.58, Mathias Stearn wrote:
>> I would hope you are marking the module.modulemap as an input
>> dependency of every compilation that uses -fmodules.
> If *that* is a/the change we need to make for build systems to "support
> modules", that is *laughably* trivial compared to "the Fortran way".
>
> That said, you're assuming that clang doesn't mention the modulemap as a
> required include file... because I think that would be sufficient, and
> wouldn't be surprised if that *is* already happening.
>