C++ Logo

sg15

Advanced search

Re: [Tooling] [ Modules and Tools ] Tracking Random Dependency Information

From: Manuel Klimek <klimek_at_[hidden]>
Date: Thu, 3 Jan 2019 15:28:30 +0100
On Wed, Jan 2, 2019 at 7:13 PM Mathias Stearn <redbeard0531_at_[hidden]>
wrote:

>
>
> On Wed, Jan 2, 2019 at 8:41 AM Manuel Klimek <klimek_at_[hidden]> wrote:
>
>> On Wed, Dec 19, 2018 at 10:30 AM Colby Pike <vectorofbool_at_[hidden]>
>> wrote:
>>
>>>
>>> > It doesn't fit well with implementing support for distributed
>>> compilation or ignorable change detection.
>>>
>>> It works perfectly fine. For distributed build, you hand the TU to the
>>> distributed tool and it will spit back the dependency information. For
>>> ignorable changes, nothing about a just-in-time dependency info model
>>> requires timestamps be used as the determiner of "out-of-date"-ness.
>>>
>>
>> If you want to sandbox execution so that an action is only executed with
>> the exact set of inputs it needs (for example, in order to send the minimum
>> number of inputs to a remote execution node), this seems like a
>> chicken-and-egg problem.
>>
>>
> If you use the ICECC model of passing a preprocessed[1] file to the remote
> host which compiles in a chroot, then it is already fully sandboxed. It
> only has access to the single stream of input text. I think this is
> probably the best way to handle distributing with std::embed as well, at
> least for small files, embedding them directly in the single stream using
> some length-delimited binary format.
>

Two problems:
1. preprocessing takes time, and I need to distribute it to make the build
fast enough
2. we do sandboxing for reproducability guarantees, thus preprocessing
needs to be sandboxed, too

The other problem with discovered inputs is that we want to be able to run
all tests that can be possibly affected by a change, and for that we need
fairly precise dependency information in a way that's cheap enough to
compute.


> [1] Technically, it is only *partially* preprocessed using
> -fdirectives-only (gcc) or -frewrite-includes (clang). This delays macro
> expansion so that the remote node can generate the correct warnings and
> debug info.
> _______________________________________________
> Tooling mailing list
> Tooling_at_[hidden]
> http://www.open-std.org/mailman/listinfo/tooling
>

Received on 2019-01-03 15:28:48