On Thu, Jan 10, 2019 at 2:18 PM Corentin <corentin.jabot_at_[hidden]> wrote:
>
> > When I say "implementation experience for modules" I mean: I have code, I compile it with a compiler. That's pretty much the whole story.
>
> That's exactly my point, this is not the whole story: You most likely have a build system invoking the compiler for you.

It's still the whole story: I compiled code (yes, using a build
system). I got an executable. That's the implementation experience I
want to hear about, no hypotheticals. Let's stop playing word games
and let's get to a point where everyone can do this. I find it highly
worrying that SG15 keeps saying "vendors have experience but we
don't!". Yes, vendors have been working on this for a while and have
experience, not all of which they can share. How do we get everyone on
the same footing? Talking about it, or getting code working?


> We know that modules work great at the compiler level and I don't think anyone has issues with modules as a language feature.
>
> However, import statements impact the order in which individual translations units need to be built by the build system ( cmake, msbuild, b2, pick your poison).
> We know that this works if dependencies are specified manually, but we know that it is not manageable to do so above a handful of modules.
>
> Instead, the build system manage the dependency graph which brings a lot of interesting challenges that are not the responsibility of compilers (as we understand them in C++) so there are a lot more pieces
> in the equation (and modules do not behave at all like headers from a build system standpoint).
>
> So we cannot Godbolt away the concerns we may have about modules, unfortunately.
>
> Now, I have thought a lot about the issue, I think this is (one of the problem) needing solving, and I humbly suggest a few things that I think would alleviate some of the issues.
>
> I might be wrong about the problem or misguided about the solution.
> However, writing the tools and having large enough codebases to play with is a significant investment which could occupy several people for a few months if they had nothing else to do.

Whereas talking about it is... free? I'm happy to bow out if you just
want to talk, but don't expect me to be super receptive to arguments
along the lines of "I don't have what's needed to try my ideas out and
understand if they're worthwhile... but how about we adopt the changes
I propose?".


> Working on theory, I think it is still preferable to be too restrictive ( and then lift the restriction ) than not enough and not be able to do anything about it (restricting the way modules are named or mapped after C++20 ships would be a massive breaking change.
>
> So am I sure something needs to be done regarding the way modules are mapped to files? No, even if I strongly suspect so.
> But, are you sure it is in fact not necessary?
>
>
> On Thu, 10 Jan 2019 at 22:26 JF Bastien <cxx_at_[hidden]> wrote:
>>
>> On Thu, Jan 10, 2019 at 1:18 PM Corentin <corentin.jabot_at_[hidden]> wrote:
>> >
>> > It is a chicken and egg problem.
>>
>> I don't think you're answering the question I asked, but you are
>> pointing out a problem: SG15 doesn't have a large modularized
>> codebase. Why not focus efforts on creating one, so that SG15's
>> efforts can be grounded in facts?
>>
>>
>> > Integrating modules in build systems will be a tedious endeavor - especially in meta build systems,
>> > and there is little incentive to do that before modules get merged.
>> >
>> > It would be rather inconvenient to realize modules are not reasonably consumed by build system _after_ the IS is published.
>> >
>> > While having large modularized projects and build systems would be nice, it's a luxury we don't seem to have.
>>
>> You can create one. Or you can take the luxury of talk and suggest
>> restrictions to modules, without grounding that talk in facts. I'm
>> saying that you want more than just talk.
>>
>>
>> > And I think taking a long look at module toolability (both in term of current tools. and in term of the 10 years goals/hopes this group has for tooling) falls in SG15 purview.
>>
>> Totally agree, but you can't tool with talk. At least when it comes to
>> C++, you need code.
>>
>>
>> > While my own experience is limited, I trust this group has enough build system experience and knowledge to foresee eventual issues and correct for them.
>> > We can also extrapolate from the well-known issue of headers name collision that exists and the guidelines that arose from that.
>> >
>> > It is important to be explicit about what we mean when we say "implementation experience" when talking about modules because they exist at the border between language and tools
>> > and having compiler implementation experience isn't the whole story.
>>
>> When I say "implementation experience for modules" I mean: I have
>> code, I compile it with a compiler. That's pretty much the whole
>> story.
>>
>>
>> > On Thu, 10 Jan 2019 at 21:39 JF Bastien <cxx_at_[hidden]> wrote:
>> >>
>> >> On Thu, Jan 10, 2019 at 11:52 AM Corentin <corentin.jabot_at_[hidden]> wrote:
>> >> >
>> >> > I suspect it will be a while (several years) before we start to see large projects transitioning fully to modules and consumed as such by tools doing automatic dependency scanning etc.
>> >> > My understanding is that there is little large scale implementation experience as far as tooling and build systems are concerned ( there is plenty _compilers_ implementation experience, and some build system implementation (and usage of modules) experience - mostly in build2).
>> >>
>> >> Given your statement above, why is it useful for SG15 to discuss
>> >> enforcing mapping? Specifically, how will that discussion be grounded
>> >> in facts? It seems to me like facts need a compiler implementation,
>> >> and a codebase to try it out on. Feel free to talk about hypotheticals
>> >> all you want, but in this case code wins.
>> >>
>> >> I think you want to refocus your approach: what are you trying to
>> >> achieve? What's the advantage that SG15 has, which the modules SG and
>> >> now EWG don't have? How can SG15 contribute module's success?
>> >>
>> >>
>> >> > On Thu, 10 Jan 2019 at 18:47 JF Bastien <cxx_at_[hidden]> wrote:
>> >> >>
>> >> >> On Thu, Jan 10, 2019 at 6:53 AM Corentin <corentin.jabot_at_[hidden]> wrote:
>> >> >> >
>> >> >> > Hello.
>> >> >> > I would like to suggest two modules related proposals that I think SG15 should look at.
>> >> >> >
>> >> >> > - Compiler enforced mapping between module names and module interface file (resource) name.
>> >> >>
>> >> >> Why does SG15 need to do this, versus someone implementing it in an
>> >> >> open-source toolchain, trying it out, and bringing what using it
>> >> >> taught them to SG15?
>> >> >>
>> >> >>
>> >> >> > Currently, modules interfaces can be declared in any file - which makes dependency scanning more tedious than it needs to be and have performance implications
>> >> >> > (The build system needs to open all files to gather a list of modules) - notably when the build system tries to start building while the dependency graph isn't yet complete.
>> >> >> >
>> >> >> > Tools ( ide, code servers, indexers, refactoring) may also greatly benefit from an easier way to locate the source file which declares a module.
>> >> >> >
>> >> >> > The specifics of the mapping are open to bikeshedding. However, I think we would have better luck sticking to something simple like <module identifier> <=> <file name>.<extension>
>> >> >> > (The standardese would mention resource identifier rather than filename)
>> >> >> >
>> >> >> > - A standing document giving guidelines for modules naming.
>> >> >> >
>> >> >> > The goal is to take everything the community had to learn the hard way about header naming over the past 30 years and apply it to modules by providing a set of guidelines
>> >> >> > that could be partially enforced by build system vendors.
>> >> >> > Encouraging consistency and uniqueness of module identifiers across the industry is I think a necessary step towards sane package management.
>> >> >> > Note that the standard requires uniqueness of modules identifiers within (the standard definition of) a program but says little about a way to ensure this uniqueness.
>> >> >> >
>> >> >> > Here is a rough draft of what I think would be good guidelines, partially inspired by what is done by other languages facing similar issues.
>> >> >> >
>> >> >> > Prefix module names with an entity and/or a project name to prevent modules from different companies, entities and projects of declaring the same module names.
>> >> >> > Exported top-level namespaces should have a name identic to the project name used as part of the name of the module(s) from which it is exported.
>> >> >> > Do not export multiple top-level namespaces
>> >> >> > Do not export entities in the global namespace outside of the global module fragment.
>> >> >> > Organize modules hierarchically. For example, if both modules example.foo and example.foo.bar exist as part of the public API of example, example.foo should reexport example.foo.bar
>> >> >> > Avoid common names such as util and core for module name prefix and top-level namespace names.
>> >> >> > Use lower-case module names
>> >> >> > Do not use characters outside of the basic source character set in module name identifiers.
>> >> >> >
>> >> >> > My hope is that these 2 proposals (whose impact on the standard is minimal) would make it easier for current tooling to deal with modules
>> >> >> > while making possible for example to design dependency managers and build systems able to work at the module level.
>> >> >> >
>> >> >> > I'd love to gather feedback and opinions before going further in that direction.
>> >> >> > Thanks a lot!
>> >> >> >
>> >> >> > Corentin
>> >> >> >
>> >> >> > PS: For a bit of background, I talked about these issues there
>> >> >> >
>> >> >> > https://cor3ntin.github.io/posts/modules_mapping/
>> >> >> > https://cor3ntin.github.io/posts/modules_naming/
>> >> >> >
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > Tooling mailing list
>> >> >> > Tooling_at_[hidden]
>> >> >> > http://www.open-std.org/mailman/listinfo/tooling
>> >> >> _______________________________________________
>> >> >> Tooling mailing list
>> >> >> Tooling_at_[hidden]
>> >> >> http://www.open-std.org/mailman/listinfo/tooling
>> >> >
>> >> > _______________________________________________
>> >> > Tooling mailing list
>> >> > Tooling_at_[hidden]
>> >> > http://www.open-std.org/mailman/listinfo/tooling
>> >> _______________________________________________
>> >> Tooling mailing list
>> >> Tooling_at_[hidden]
>> >> http://www.open-std.org/mailman/listinfo/tooling
>> >
>> > _______________________________________________
>> > Tooling mailing list
>> > Tooling_at_[hidden]
>> > http://www.open-std.org/mailman/listinfo/tooling
>> _______________________________________________
>> Tooling mailing list
>> Tooling_at_[hidden]
>> http://www.open-std.org/mailman/listinfo/tooling
>
> _______________________________________________
> Tooling mailing list
> Tooling_at_[hidden]
> http://www.open-std.org/mailman/listinfo/tooling