On Mon, Mar 4, 2019, 7:09 PM Ben Boeckel <ben.boeckel_at_[hidden]> wrote:

>
> Personally, my gut reaction is that this belongs at the build executor
> level, not the compiler.


I don't think the build executor should need to know how to tell which
changes within a BMI should or should not trigger downstream rebuilds.

Currently, most of these do mtime-based
> detection. Easy to implement at the expense of potential excess work.
> Also easy to force the executor to redo something. Smarter executors might
> do
> content hashing detection of changes (I believe that bazel and similar
> tools effectively do this). Really smart ones might support a
> `content_hash = somecmd` to compute it for any output rule (possibly
> with some built-in). This would also potentially work with things like
> abidiff for object files and shared libraries too. I don't know how one
> would implement `rebuild_hash` semantics without backdating mtimes in
> existing executors which seems really finicky.
>

At least for ninja it is easy. You just have an intermediate output.hash
file that everything depends on and you have a restat=1 rule that updates
it only when needed. I've done this and it works quite well
https://github.com/RedBeard0531/mongo_module_ninja/commit/f19916bbb1d2f7c8b18d39f38559de72ba486cd2#diff-2955d5257f635de1df53f55a171ca5c7
.


> I don't know that we can shoehorn this information into the output of
> the scan step and expect it to actually be implemented before we know
> what actually affects BMI output (cf. Clang assuming all flags affect
> BMI output and not even "trying" to guess).
>

Sure. I was thinking that this would be in the output of every compiler
invocation, whether scanning, extracting a BMI, or compiling to an object
file. Each stage would output information relevant to itself, and needed
for later stages. You will need to get deps during compilation, not just
the earlier stages, since it is legal to #include in the private module
fragment, which hopefully would never even be lexed by earlier invocations.

Also, I didn't notice this at first, but it looks like your proposed scan
output is telling the build system where the compiler will be storing its
outputs. It seems better to have it list the logical outputs it will
generate, with later stages being told *exactly* where each file should be
written. We already have a large problem with one case where the compiler
decides on its own where to write files (split dwarf .dwo files) so I
really don't want to introduce more cases like it. The build system tells
the compiler where to put its output, not the other way around.