> How do you invoke the compiler without knowing the dependency information?

There's an important distinction that hasn't been made in this thread yet,
which I find extremely important: There are two types of dependencies:
"Run-before" dependencies, and "effects-the-output" dependencies. For
example, building of an object file must "run-before" the task of "link the
executable" *if* that object file is used as an input to the link. However,
the object file itself is an "effects-the-output" dependency of the
executable.

Header files and resource files passed to std::embed() are
"effects-the-output" dependencies, not "run-before" dependencies.
"run-before" is a relationship between build graph edges, while
"effects-the-output" is a relationship between build graph nodes.

Ninja (and other tools) have already solved the problem of determining the
"effects-the-output" dependencies by asking the edge (compiler/linker) to
generate this information while it runs the edge, not ahead-of-time. This
permits you to execute an edge when you are unsure of the dependencies, and
stashing them away for the next execution of the build graph. This first "I
dunno" will be on the first execution of the graph, when the outputs are
already non-existent, so it doesn't even matter what the
"effects-the-output" dependencies are.

The purpose of the paper (as I understand it) isn't to prevent that
catastrophic "everything is building out-of-order and there are inputs
missing to the compiler and the build turns red," problems. Those are easy,
and any tool can assure they don't happen with simple naive dep scanning. I
have no qualm with that. The problem I'm afraid of is the "Oops. I silently
missed that you modified this file and I didn't rebuild the relevant
translation units and now you're shipping three-week old code because the
build system had a hiccup."

On Wed, Dec 19, 2018 at 12:49 AM Peter Bindels <dascandy_at_[hidden]> wrote:

> > If someone could get all of them 100% on-board with doing full semantic
> analysis for dependency analysis, then I could just tank this paper
> entirely. I would actually be 200% on board with that. But, I don't see it
> happening anytime soon.
>
> This is putting the cart behind the horse. How do you invoke the compiler
> without knowing the dependency information? How do you read a code base if
> the actual files found depend on your include paths? Wasn't this part of
> the original problem - having code that does what you write down & intend,
> rather than depend on context for meaning?
>
> I do not believe that doing full semantic analysis for dependency analysis
> is even an option. That would equally imply that to compile the source code
> you need to have fully compiled it - how do you ever begin?
>
> Unless you want to go back to the dark ages of "put everything in your
> build files, including all component to component dependencies and include
> paths" - which is copying what your includes and imports already do into
> another format that is hard to maintain and read.
>
> I, for one, have stopped writing build files about 4 years ago and I
> haven't missed them one bit.
>
>
>
> On Wed, 19 Dec 2018 at 04:24, Colby Pike <vectorofbool_at_[hidden]> wrote:
>
>> Sorry to take so long to get back on this one.
>>
>> > If someone could get all of them 100% on-board with doing full semantic
>> analysis for dependency analysis, then I could just tank this paper
>> entirely. I would actually be 200% on board with that. But, I don't see it
>> happening anytime soon.
>>
>> I believe convincing can be done in this regard. I'm not proposing that
>> implementers add support for this as part of the preprocessor, but rather
>> as another compiler option. Modern build tools such as Ninja grab the
>> dependency information for a translation unit as part of invoking the
>> compiler command, not by running the preprocessor separately. I'm preparing
>> a post on how Ninja builds and performs dependency analysis, which I
>> believe is the currently optimal way to do it. There may be some tweaks
>> with the addition on modules, but it will still be applicable.
>>
>> In short: Implicit dependency information is generated on-the-fly by the
>> edge command as part of regular edge execution. Not only is semantic
>> analysis executed, but the entire compilation process.
>>
>> On Tue, Dec 4, 2018 at 6:50 PM JeanHeyd Meneide <phdofthehouse_at_[hidden]>
>> wrote:
>>
>>> Dear SG15,
>>>
>>> To address one of the previous concerns, I've made sure to
>>> specifically add a call out a new Word of Power in this proposal:
>>> https://thephd.github.io/vendor/future_cxx/papers/d1130.html#wording-intent.
>>> It's called *resource-location* or *resource-locations*. This will give
>>> std::embed and other proposals a well-defined way of referring to a lookup
>>> mechanism. It is still implementation-defined where this comes from, but
>>> just like include paths I expect this to come from a Resource Path command
>>> line parameter that build systems can tap into.
>>>
>>> If you have any other concerns or questions, please do let me know!
>>>
>>> Sincerely,
>>> JeanHeyd Meneide
>>>
>>> On Mon, Dec 3, 2018 at 6:13 AM JeanHeyd Meneide <phdofthehouse_at_[hidden]>
>>> wrote:
>>>
>>>> > That's insufficient: to be usable, it must also say that it is
>>>> > *the same* for all resource strings, regardless of which facility
>>>> > (std::embed, etc) they are used in.
>>>>
>>>> Resource lookup will be exactly what is written in this paper, so long
>>>> as I can get consensus and move this through the Committee. As the author
>>>> of std::embed, I have the privilege of moving these papers in sync. Should
>>>> p1130 be accepted, std::embed will be updated to specifically call out
>>>> Resource Lookup, as defined in [module.requires] (or whatever it ends up
>>>> being).
>>>>
>>> _______________________________________________
>>> Tooling mailing list
>>> Tooling_at_[hidden]
>>> http://www.open-std.org/mailman/listinfo/tooling
>>>
>> _______________________________________________
>> Tooling mailing list
>> Tooling_at_[hidden]
>> http://www.open-std.org/mailman/listinfo/tooling
>>
> _______________________________________________
> Tooling mailing list
> Tooling_at_[hidden]
> http://www.open-std.org/mailman/listinfo/tooling
>