Date: Mon, 4 Mar 2019 20:35:35 -0500
I believe the intent is for the compiler to emit where it was told,
implicitly or explicitly, where the output is to go, and what it believes
are the dependencies. The problem is make, where exact spelling counts.
On Mon, Mar 4, 2019, 20:30 Mathias Stearn <redbeard0531+isocpp_at_[hidden]>
wrote:
>
>
> On Mon, Mar 4, 2019, 7:09 PM Ben Boeckel <ben.boeckel_at_[hidden]> wrote:
>
>>
>> Personally, my gut reaction is that this belongs at the build executor
>> level, not the compiler.
>
>
> I don't think the build executor should need to know how to tell which
> changes within a BMI should or should not trigger downstream rebuilds.
>
> Currently, most of these do mtime-based
>> detection. Easy to implement at the expense of potential excess work.
>> Also easy to force the executor to redo something. Smarter executors
>> might do
>> content hashing detection of changes (I believe that bazel and similar
>> tools effectively do this). Really smart ones might support a
>> `content_hash = somecmd` to compute it for any output rule (possibly
>> with some built-in). This would also potentially work with things like
>> abidiff for object files and shared libraries too. I don't know how one
>> would implement `rebuild_hash` semantics without backdating mtimes in
>> existing executors which seems really finicky.
>>
>
> At least for ninja it is easy. You just have an intermediate output.hash
> file that everything depends on and you have a restat=1 rule that updates
> it only when needed. I've done this and it works quite well
> https://github.com/RedBeard0531/mongo_module_ninja/commit/f19916bbb1d2f7c8b18d39f38559de72ba486cd2#diff-2955d5257f635de1df53f55a171ca5c7
> .
>
>
>> I don't know that we can shoehorn this information into the output of
>> the scan step and expect it to actually be implemented before we know
>> what actually affects BMI output (cf. Clang assuming all flags affect
>> BMI output and not even "trying" to guess).
>>
>
> Sure. I was thinking that this would be in the output of every compiler
> invocation, whether scanning, extracting a BMI, or compiling to an object
> file. Each stage would output information relevant to itself, and needed
> for later stages. You will need to get deps during compilation, not just
> the earlier stages, since it is legal to #include in the private module
> fragment, which hopefully would never even be lexed by earlier invocations.
>
> Also, I didn't notice this at first, but it looks like your proposed scan
> output is telling the build system where the compiler will be storing its
> outputs. It seems better to have it list the logical outputs it will
> generate, with later stages being told *exactly* where each file should be
> written. We already have a large problem with one case where the compiler
> decides on its own where to write files (split dwarf .dwo files) so I
> really don't want to introduce more cases like it. The build system tells
> the compiler where to put its output, not the other way around.
> _______________________________________________
> Modules mailing list
> Modules_at_[hidden]
> Subscription: http://lists.isocpp.org/mailman/listinfo.cgi/modules
> Link to this post: http://lists.isocpp.org/modules/2019/03/0128.php
>
implicitly or explicitly, where the output is to go, and what it believes
are the dependencies. The problem is make, where exact spelling counts.
On Mon, Mar 4, 2019, 20:30 Mathias Stearn <redbeard0531+isocpp_at_[hidden]>
wrote:
>
>
> On Mon, Mar 4, 2019, 7:09 PM Ben Boeckel <ben.boeckel_at_[hidden]> wrote:
>
>>
>> Personally, my gut reaction is that this belongs at the build executor
>> level, not the compiler.
>
>
> I don't think the build executor should need to know how to tell which
> changes within a BMI should or should not trigger downstream rebuilds.
>
> Currently, most of these do mtime-based
>> detection. Easy to implement at the expense of potential excess work.
>> Also easy to force the executor to redo something. Smarter executors
>> might do
>> content hashing detection of changes (I believe that bazel and similar
>> tools effectively do this). Really smart ones might support a
>> `content_hash = somecmd` to compute it for any output rule (possibly
>> with some built-in). This would also potentially work with things like
>> abidiff for object files and shared libraries too. I don't know how one
>> would implement `rebuild_hash` semantics without backdating mtimes in
>> existing executors which seems really finicky.
>>
>
> At least for ninja it is easy. You just have an intermediate output.hash
> file that everything depends on and you have a restat=1 rule that updates
> it only when needed. I've done this and it works quite well
> https://github.com/RedBeard0531/mongo_module_ninja/commit/f19916bbb1d2f7c8b18d39f38559de72ba486cd2#diff-2955d5257f635de1df53f55a171ca5c7
> .
>
>
>> I don't know that we can shoehorn this information into the output of
>> the scan step and expect it to actually be implemented before we know
>> what actually affects BMI output (cf. Clang assuming all flags affect
>> BMI output and not even "trying" to guess).
>>
>
> Sure. I was thinking that this would be in the output of every compiler
> invocation, whether scanning, extracting a BMI, or compiling to an object
> file. Each stage would output information relevant to itself, and needed
> for later stages. You will need to get deps during compilation, not just
> the earlier stages, since it is legal to #include in the private module
> fragment, which hopefully would never even be lexed by earlier invocations.
>
> Also, I didn't notice this at first, but it looks like your proposed scan
> output is telling the build system where the compiler will be storing its
> outputs. It seems better to have it list the logical outputs it will
> generate, with later stages being told *exactly* where each file should be
> written. We already have a large problem with one case where the compiler
> decides on its own where to write files (split dwarf .dwo files) so I
> really don't want to introduce more cases like it. The build system tells
> the compiler where to put its output, not the other way around.
> _______________________________________________
> Modules mailing list
> Modules_at_[hidden]
> Subscription: http://lists.isocpp.org/mailman/listinfo.cgi/modules
> Link to this post: http://lists.isocpp.org/modules/2019/03/0128.php
>
Received on 2019-03-05 02:35:50