On Wed, Sep 26, 2018 at 11:14 AM Rene Rivera <grafikrobot_at_[hidden]> wrote:

> On Wed, Sep 19, 2018 at 11:09 AM JF Bastien <cxx_at_[hidden]> wrote:
>
>> On Tue, Sep 18, 2018 at 7:41 PM Rene Rivera <grafikrobot_at_[hidden]>
>> wrote:
>>
>>> On Tue, Sep 18, 2018 at 12:20 AM JF Bastien <cxx_at_[hidden]> wrote:
>>>
>>>> On Mon, Sep 17, 2018 at 3:37 PM Rene Rivera <grafikrobot_at_[hidden]>
>>>> wrote:
>>>>
>>>>> On Mon, Sep 17, 2018 at 5:04 PM JF Bastien <cxx_at_[hidden]> wrote:
>>>>>
>>>>>> On Mon, Sep 17, 2018 at 1:38 PM Rene Rivera <grafikrobot_at_[hidden]>
>>>>>> wrote:
>>>>>>
>>>>>>> On Mon, Sep 17, 2018 at 3:31 PM Tony V E <tvaneerd_at_[hidden]> wrote:
>>>>>>>
>>>>>>>> Also, if you give me a function called std::compile(), that
>>>>>>>> compiles code, it seems I can now write a compiler?
>>>>>>>>
>>>>>>>> int main(int argc, char ** argv)
>>>>>>>> {
>>>>>>>> return std::compile(argc, argv);
>>>>>>>> }
>>>>>>>>
>>>>>>>> Wow, that was easy.
>>>>>>>> Can the paper explain what I'm misunderstanding? (Or maybe it does
>>>>>>>> explain, but I missed it)
>>>>>>>>
>>>>>>>
>>>>>>> That's a correct understanding. And that's the one example I use in
>>>>>>> my implementation <
>>>>>>> https://github.com/bfgroup/std_cpp/blob/master/example/std_cpp.cpp>.
>>>>>>> I do try and explain the goals in the paper. In that it serves a dual
>>>>>>> purpose. But mainly it's a way to standardize the compiler options.
>>>>>>>
>>>>>>
>>>>>> Right, otherwise you'd have to do:
>>>>>>
>>>>>>
>>>>>> int main(int argc, char** argv) {
>>>>>> std::system((std::string("clang ") + argv[1]).c_str());
>>>>>> return 0;
>>>>>> }
>>>>>>
>>>>>>
>>>>>> ;-)
>>>>>>
>>>>>> More seriously, the selection of compiler options you've chosen seem
>>>>>> semi-random.
>>>>>>
>>>>>
>>>>> First it's not random :-) It's the minimal to get basic actual
>>>>> compiling working and to show highlight some of the differences in link
>>>>> compatibility. Second, it's very incomplete. I'll keep adding options as I
>>>>> implement them from now until the mailing deadline (and keep implementing
>>>>> them afterwards for an R1 paper -- and so on). Third, I hope I can get some
>>>>> volunteers to help in adding options.
>>>>>
>>>>>
>>>>>> It would be useful to have a survey of existing compilers and their
>>>>>> options, and some criteria to determine which should be supported here and
>>>>>> which shouldn't.
>>>>>>
>>>>>
>>>>> Selection criteria is indeed a hard problem. What's actually needed
>>>>> for core? And what can be delegated to the vendor specific realm?
>>>>>
>>>>
>>>> For context, I've added options in the past, removed some. It's not a
>>>> big deal. It would be a big deal if the standard added and removed some.
>>>> That's worrying.
>>>>
>>>
>>> Yes, it would be worrying. It might be the case that we need another
>>> level of options defined: core options in the IS, common options in an SD,
>>> and vendor options in SDs. Having the common options in an SD would allow
>>> for the experience gathering lifetime like TSs do for library features.
>>>
>>> Further, what's the expectation of the result of calling this function?
>>>>>> Can I actually execute any code? How? Can you make sure that you take into
>>>>>> account the restrictions various platforms have, such as requiring code
>>>>>> authentication. It goes way beyond linkers.
>>>>>>
>>>>>
>>>>> The proposal doesn't, intentionally, say anything about executing
>>>>> code. This is one of those "implementation defined" areas. Just like it is
>>>>> currently in the standard. If it's possible to execute code in some manner,
>>>>> either indirectly through an std::system equivalent or directly through
>>>>> JIT/DLL, is left for the implementor/compiler to document.
>>>>>
>>>>
>>>> So a valid implementation always returns true, does nothing?
>>>>
>>>
>>> Yes. Returning "true" would get coerced to "1", since the function
>>> returns an in. Which would indicate the error of not being implemented.
>>> Although ideally it would also output an error message to that effect (if
>>> possible).
>>>
>>
>> I meant the opposite: a valid implementation can always say "success" and
>> do nothing AFAICT.
>>
>
> I'll make that clear.. To say that an implementation should indicate an
> error if it's not implemented.
>

A valid implementation of std;:compile is then to always return "not
implemented"? That addresses some of my concerns.


system seems like a precedent for what you suggest, and in my experience
>>>>>> system isn't a precedent people want us to repeat.
>>>>>>
>>>>>
>>>>> "std::system" is what the sample implementation uses.. but it's
>>>>> possible you could implement it as a direct library call (easily doable for
>>>>> Clang, for example). And ideally production implementations would use
>>>>> something more robust than std::system ;-)
>>>>>
>>>>
>>>> What's the upside of a library call?
>>>>
>>>
>>> One upside is that it's a known viable mechanism that can be
>>> standardized.
>>>
>>
>> What's the upside of this, when build systems already handle this?
>>
>
> Build systems barely handle it.
>

I disagree: we manage to build code today. Works pretty well. The
difficulty of build systems isn't mapping compiler options, it's all the
other stuff around it.


But it's not just build systems. It's anything that needs some
> understanding of the options, for example code editors, binary package
> managers, code analyzers, and so on.
>

Could your paper explain how these are helped by your proposal?


Keep in mind that standardization has downsides, such as making any
>> additions very slow, and changes nearly impossible. If I add a new option
>> tomorrow, nobody can use it with a standards-based mechanism. Who would
>> want to use this mechanism, given such limits?
>>
>
> Yes, and I think that using a combination of the IS and SDs we can avoid
> the downsides. I.e. we can have a "stable" set of options in the IS
> combined with a more fluid set of options in SDs.
>

An SD is still way slower than the turnaround I have today.


I also mention the in the paper runtime code compilation to target special
>>> runtimes in the prior art (NVRTC). There's also the possibility of tighter
>>> integration with build system to optimize build performance.
>>>
>>
>> Do you have numbers to back up such a claim?
>>
>
> I don't personally (I could with enough work -- would be awesome if
> someone helped me out on this). But Daniel Dunbar has done some work in
> this area that shows significant performance gains <
> https://www.youtube.com/watch?v=b_T-eCToX1I>.
>

I'm not sure that was Daniel's point, but I can ask him next week.


I get the impression you're going down a rabbit hole...
>>>>
>>>> How is the design different from something like this:
>>>>
>>>> https://clang.llvm.org/docs/DriverInternals.html
>>>>
>>>> ?
>>>>
>>>
>>> One similarity is that both compile source :-) Another, is of course,
>>> that the clang driver (or front end) somewhat simulates the gcc front end.
>>> That pattern is not uncommon.. I mention one of them in the prior art
>>> section. Others are the various msvc compatible compilers, like Intel.
>>>
>>
>> I'd expect not just a list of prior-art, but a comparison of what each
>> does.
>>
>
> Good idea.. I'm not going to have time to write that comparison for R0
> though :-(
>

Understood, could you leave a "future work" item for it?


Maybe the code will help explain what I have in mind:
>>>>
>>>> https://github.com/llvm-mirror/clang/blob/master/tools/driver/driver.cpp
>>>>
>>>>
>>>> Or, put another way, why can't a Python script be used to frob
>>>> command-line parameters in the way you propose?
>>>>
>>>
>>> It could. Although that would be a tool and as such not something we
>>> could put into the IS.
>>>
>>
>> That raises a wider question: is SG15 constrained to putting things into
>> the IS? Is it constrained to adopting pure C++ solutions to all problems?
>> You seem to answer "yes" to both questions.
>>
>
> No I did not answer "yes" to both. I explicitly mention using SDs as a
> viable method also. But unless something changes in the way WG21 works I
> don't see any other way outside out of the IS to have anything enforceable.
>

I'd like to see SG15 do things outside IS / SD.

Could you answer the other question: it sounds like you absolutely want
this in C++, not elsewhere.


Have there been such tools in the past?
>>>>
>>>
>>> Sure. I mention clang-cl in the prior art section, for example. And if I
>>> understand correctly Isabella Muerte is working on something like that
>>> also. And some build systems do essentially the same job if they in any
>>> form abstract the compiler options. That doesn't remove the desirability of
>>> having a standard set of options and a library interface. In the contrary
>>> it increases the desirability as there are obviously many instances of
>>> duplicated effort that we could could help with while increases
>>> interoperation of tools and build understanding of build products.
>>>
>>>
>>>> Have they succeeded?
>>>>
>>>
>>> Some have.
>>>
>>
>> I'd expect the paper to list these.
>>
>
> Okay.
>
> What were their approaches?
>>>>
>>>
>>> I think I already answered that.. in that they simulate another such
>>> tool. Which helps in interoperation of their tool with others (i.e. being
>>> able to substitute one compiler for another in an IDE or build system
>>> transparently).
>>>
>>> How is this different from, say, how CMake does things? It tries to
>>>> abstract away some flags and it does so much more than your proposal.
>>>>
>>>
>>> Not that different.. Although this would be more comprehensive
>>>
>>
>> As it stands, the draft is way less comprehensive than CMake.
>>
>
> As it stands, being a work in progress draft (or proposal), yes it's less
> comprehensive than almost every other build system. But I don't think it
> needs to be comprehensive at this point. Perhaps by the time this gets to
> R2 it will be more comprehensive that most build systems.
>

OK, could you add this as a goal of the paper? It would have helped me
understand where you're going with this.



> and the compiler vendors are in a better position to define their option
>>> "abstraction".
>>>
>>
>> Sure, all compilers have their own command-line parameters at the moment.
>> Standardizing them seems to remove that "better position" from compiler
>> vendors, or at a minimum impose a bunch more work on them.
>>
>
> I don't think it adds much work to compiler vendors, after all they
> already implement the option parsing. It does add work for tool vendors as
> they will have to adjust to the new options.
>

Sure the mapping isn't hard, the extra constraints are an annoyance. What
adds a huge burden, were we to adopt this, is that my standard library now
*must* include a compiler. On every single desktop, laptop, and device we
ship.



> Why would I want your proposal over CMake?
>>>>
>>>
>>> You would not choose it over CMake, or any other build system. But it
>>> would make the build system much easier to implement.
>>>
>>
>> Is that a problem worth solving compared to other tooling problems?
>>
>
> Anything that makes our tools more consistent to help build understanding
> is worth it, IMO.
>

What I'm asking is for a prioritization. Given limited time and resources,
compared to other tooling problems, which would SG15 pick first?

-- 
> -- Rene Rivera
> -- Grafik - Don't Assume Anything
> -- Robot Dreams - http://robot-dreams.net
>
> _______________________________________________
> Tooling mailing list
> Tooling_at_[hidden]
> http://www.open-std.org/mailman/listinfo/tooling
>