On Sat, Feb 09, 2019 at 00:01:07 -0800, Scott Wardle wrote:
> I think you are total right that is what I could use here. I would
> love a diagram that shows what processes or stages are needed for the
> use cases you are thinking of clean build vs incremental or maybe some
> others.

I'll look at adding two possible implementation (1:1:1 source:scan:ddi
versus N:1:N where N can be all-at-once or per-target) graphs.

> Here is what I was thinking of:
> - Clean builds vs incremental builds
> - Linux processes are cheap vs windows less process more threads
> - Multi computer build, what data is pushed over the network what data is pulled over the network.
> -Object/Module BMI/Binary/Module Map caching vs no caching.

These are probably good as notes on when one approach might be preferred
over the other. I'm more than willing to go over build graphs on a
whiteboard at Kona, but I don't know that multiple pages of incremental
diagrams while trying to describe execution strategies of them is going
to be easy to digest.

> Maybe even making this more concrete and talk about command lines of some of these use cases:
> At least we should talk about:
> -does modules change anything with:
> -Include paths -I<dir> vs -isystem
> -Object/Library path -L<dir>
> -are there overlap with:
> -module BMI paths (-fmodules-cache-path=<directory> vs -fprebuilt-module-path=<directory>)
> -module map path/files -fmodule-map-file=

Eh, these might be to low-level for what we're describing. We list what
is important for compilers to provide in §7.1. The actual flag spellings
aren't (that) important. In any case, I would hope that it would not.
Changing semantics of flags as fundamental as `-I` or `-L` based on
`-fmodules` or `-std=c++2a` is not going to be fun for build tools to
implement.

> -Artifact Hashing (?? How do dependency work with this? see
> what the process writes out and assume dependency? maybe I
> don’t understand this.)

This is strictly a dependency detection strategy. `mtime` is common, but
hashing before saying "dirty" is another viable strategy.

The rest of this is off-topic here, but I'll add my 2¢.

> Note the use case I am trying to understand is EA uses include paths
> as a layer enforcement mechanism. IE lower layer rendering can’t
> include high level gameplay. But gameplay can include rendering. We
> currently have a different set of includes for each library. A game is
> built out of about 400 to 300 of these libraries. Since we know what
> library uses what other libraries we can use this to understand what
> includes path are necessary. These include path dependencies are
> different than a libraries linkage dependency. You might use a header
> from a library but as you only use inline functions you don’t need to
> link to it and therefore you don’t need to build it first. This can be
> a good speed up when building DLLs.

For CMake, there is a potential to add `$<COMPILE_ONLY>` genex to add
usage requirements, but ignore the target at link time. Other build
tools could certainly implement analogous semantics.

    https://gitlab.kitware.com/cmake/cmake/issues/18049#note_496112

> What I am worried about with the EA include path layering enforcement
> is:
> -We are very close to running out of command line (on windows) as we
> will have 100s of include paths. (A high level, application level
> modules will need just about every library after all.). With modules
> I am not sure what is the equivalent of include paths are but it would
> seem like we need 2x the command line for module paths if not more.
> -We have had this system for a long time so we probably have duplicate
> include files names. if we reduced the number of include paths we
> might hit these problems.

Response files help a lot here.

> -We have 100s of include paths. I worry this is not very efficient.
> If the OS has a good directory cache maybe this is good enough however
> it could be be very slow otherwise. I am not sure if other company do
> this type of thing.

VTK's build can have ~100 `-I` flags for parts using "lots" of VTK. It
certainly has *an* effect, but I don't know how much number-wise.

--Ben