On Fri, Feb 08, 2019 at 21:33:37 -0800, JF Bastien wrote:
> On Fri, Feb 8, 2019 at 1:59 PM Ben Boeckel <ben.boeckel_at_[hidden]> wrote:
>
> > On Fri, Feb 08, 2019 at 13:34:43 -0800, JF Bastien wrote:
> > > Let us know if you have any feedback!
> >
> > §2
> > > It is no slower to determine the dependencies of a modular C++ file
> > > than a normal C++ file today, and current proposals aimed at
> > > restricting the preamble wouldn’t make a scanner any faster.
> >
> > Header dependencies are subtly, but crucially, different than module
> > dependencies. One can determine header dependencies *while compiling*
> > the file (though is not necessarily how it is implemented) and can be
> > done at any time once the source file is up-to-date. The first run does
> > not need this information because we already know we need to run the
> > output; storing discovered dependency information is a nice side effect.
> >
> > Module dependencies must be present *during compile*, so must be
> > determined at build time (since source files may not be available before
> > the build has started) *before* compilation starts.
> >
>
> Maybe it would help anchor our discussion if we had a graph of "build
> activity over time" where we quantify parallelism versus sequentialism, and
> what each operation is doing? Put another way, a step which blocks many
> others is fine as long as it's pretty fast. How fast would dependency
> scanning have to be on a project of LLVM's size to make you comfortable?

It depends on the topology of the build graph. A very "wide" one can
probably saturate all the CPUs with scan steps anyways. A very "narrow"
one doesn't have enough available parallelism to really discuss anyways.

> It might also be useful to separate concerns between clean build and
> incremental build. The costs won't be the same in both, so we should
> probably discuss them independently (or rather, in the same paper but not
> the same paragraph).

Agreed. We'll work on mentioning that in the next revision.

> §2.2
> > > <clang-scan-deps>
> > Assuming there is consensus on D1483§7, I think this tool can be worked
> > to satisfy it. There may be issues around having the tool emulate other
> > compilers, but we have experience with that as well[1]. I think we may
> > even be able to drop some features from the list (e.g., pcm generation
> > at scan time and "installed listeners") in your paper.
> >
>
> We're not saying that this specific implementation should be the only one
> (or that it shouldn't!). I'd certainly be interested in seeing other
> projects implement similar tools.

Experimentation would be nice. However, popular uptake of such
experiments can place pressure on build tools like CMake to support
them.

> > §2.3
> > > <mapping files>
> >
> > The approach we describe doesn't require mapping files at all. Other
> > tools may find them useful however. I'm thinking mainly static analysis
> > tools (versus those that piggyback on the compiler like IWYU and
> > `clang-tidy`) since they can't just say "run us with your build tool".
> >
>
> That can certainly be an optional thing.

We describe that build tools don't need this (in principle). Other tools
may want it. I think having support in build tools for creating common
formats may be useful (e.g., see `compile_commands.json`).

> §2.4
> > > We believe modules should be built (not shipped or distributed) as
> > > part of a build, and potentially shared in the same environment for
> > > other compiler invocations that end up using non-conflicting setups.
> >
> > Linux distros aren't going to like this… Nor are ports-based systems
> > (like Homebrew). Go, Rust, and other languages can get away with it
> > because given source code, how to build it is dictated by convention and
> > available tooling. C++ is a wild west of solutions for the "source ->
> > binary" transformation and given a set of sources, there's no "good way"
> > to just know how to compile it today.
> >
> > That said, I don't think it's something the language standard can
> > dictate, but compilers can work together to provide something shippable
> > beside compiled libraries.
> >
>
> I don't think we're talking about the same thing: our paper talks about
> shipping something the compiler created between source code and a native
> binaries, and we don't think that's necessary.

OK, that sounds fine.

> Linux, Homebrew, and other platforms (such as, say, the one I support)
> currently ship headers with native binaries. We believe that modules allows
> them to continue doing so, both for their own code which yields said native
> binaries, as well as for developers on these platforms which link to the
> native binaries (by referring to the headers) using modules for their own
> code.

I think we're eventually going to see projects wanting to not have
headers at all and that means that `-devel` packages will need
*something*. But I'm personally leaning towards it being a compiler
thing, not a language thing. IIRC, there was mention on Reddit of a
format Microsoft was working on for distilling a module down to an
interface description. Getting *that* as a shippable and consumable
format would be great.

In any case, I think there is some general confusion among those
involved with Tooling at what problems modules is expected to solve.
I think we can probably all be more explicit when we say conflated terms
like "dependency" and "distribution".

--Ben