As I said, I want to talk about the problem rather than a specific solution.

Now, we can all assume that modules as specified will be supported without
pain by all build systems, IDEs, refactoring tools etc in a way that is
efficient, scalable, distributable, easy to use and compatible with SG15
dependencies manager ambitions.

But, I have enough reservation that I would not be comfortable voting for a
merge of the proposal any time soon.
If people proposing changes to the proposal have a burden of proof, maybe
so should the people that keep on insisting that everything is perfect as
it is?

Modules are by nature quite invasive. Not only will they require
module-aware build system to be usable, but so will any downstream users.
Sticking "-std=c++20" on gcc won't let people magically use modules
everywhere.
There is a risk that the feature will be ignored or make libraries that
elect to use modules unusable. And modules have enough benefits to offer
that we should do everything we can to avoid that.


So maybe you are right that we need more implementation.


On Thu, 10 Jan 2019 at 23:33 JF Bastien <cxx_at_[hidden]> wrote:

> On Thu, Jan 10, 2019 at 2:18 PM Corentin <corentin.jabot_at_[hidden]> wrote:
> >
> > > When I say "implementation experience for modules" I mean: I have
> code, I compile it with a compiler. That's pretty much the whole story.
> >
> > That's exactly my point, this is not the whole story: You most likely
> have a build system invoking the compiler for you.
>
> It's still the whole story: I compiled code (yes, using a build
> system). I got an executable. That's the implementation experience I
> want to hear about, no hypotheticals. Let's stop playing word games
> and let's get to a point where everyone can do this. I find it highly
> worrying that SG15 keeps saying "vendors have experience but we
> don't!". Yes, vendors have been working on this for a while and have
> experience, not all of which they can share. How do we get everyone on
> the same footing? Talking about it, or getting code working?
>
>
> > We know that modules work great at the compiler level and I don't think
> anyone has issues with modules as a language feature.
> >
> > However, import statements impact the order in which individual
> translations units need to be built by the build system ( cmake, msbuild,
> b2, pick your poison).
> > We know that this works if dependencies are specified manually, but we
> know that it is not manageable to do so above a handful of modules.
> >
> > Instead, the build system manage the dependency graph which brings a lot
> of interesting challenges that are not the responsibility of compilers (as
> we understand them in C++) so there are a lot more pieces
> > in the equation (and modules do not behave at all like headers from a
> build system standpoint).
> >
> > So we cannot Godbolt away the concerns we may have about modules,
> unfortunately.
> >
> > Now, I have thought a lot about the issue, I think this is (one of the
> problem) needing solving, and I humbly suggest a few things that I think
> would alleviate some of the issues.
> >
> > I might be wrong about the problem or misguided about the solution.
> > However, writing the tools and having large enough codebases to play
> with is a significant investment which could occupy several people for a
> few months if they had nothing else to do.
>
> Whereas talking about it is... free? I'm happy to bow out if you just
> want to talk, but don't expect me to be super receptive to arguments
> along the lines of "I don't have what's needed to try my ideas out and
> understand if they're worthwhile... but how about we adopt the changes
> I propose?".
>
>
> > Working on theory, I think it is still preferable to be too restrictive
> ( and then lift the restriction ) than not enough and not be able to do
> anything about it (restricting the way modules are named or mapped after
> C++20 ships would be a massive breaking change.
> >
> > So am I sure something needs to be done regarding the way modules are
> mapped to files? No, even if I strongly suspect so.
> > But, are you sure it is in fact not necessary?
> >
> >
> > On Thu, 10 Jan 2019 at 22:26 JF Bastien <cxx_at_[hidden]> wrote:
> >>
> >> On Thu, Jan 10, 2019 at 1:18 PM Corentin <corentin.jabot_at_[hidden]>
> wrote:
> >> >
> >> > It is a chicken and egg problem.
> >>
> >> I don't think you're answering the question I asked, but you are
> >> pointing out a problem: SG15 doesn't have a large modularized
> >> codebase. Why not focus efforts on creating one, so that SG15's
> >> efforts can be grounded in facts?
> >>
> >>
> >> > Integrating modules in build systems will be a tedious endeavor -
> especially in meta build systems,
> >> > and there is little incentive to do that before modules get merged.
> >> >
> >> > It would be rather inconvenient to realize modules are not reasonably
> consumed by build system _after_ the IS is published.
> >> >
> >> > While having large modularized projects and build systems would be
> nice, it's a luxury we don't seem to have.
> >>
> >> You can create one. Or you can take the luxury of talk and suggest
> >> restrictions to modules, without grounding that talk in facts. I'm
> >> saying that you want more than just talk.
> >>
> >>
> >> > And I think taking a long look at module toolability (both in term of
> current tools. and in term of the 10 years goals/hopes this group has for
> tooling) falls in SG15 purview.
> >>
> >> Totally agree, but you can't tool with talk. At least when it comes to
> >> C++, you need code.
> >>
> >>
> >> > While my own experience is limited, I trust this group has enough
> build system experience and knowledge to foresee eventual issues and
> correct for them.
> >> > We can also extrapolate from the well-known issue of headers name
> collision that exists and the guidelines that arose from that.
> >> >
> >> > It is important to be explicit about what we mean when we say
> "implementation experience" when talking about modules because they exist
> at the border between language and tools
> >> > and having compiler implementation experience isn't the whole story.
> >>
> >> When I say "implementation experience for modules" I mean: I have
> >> code, I compile it with a compiler. That's pretty much the whole
> >> story.
> >>
> >>
> >> > On Thu, 10 Jan 2019 at 21:39 JF Bastien <cxx_at_[hidden]> wrote:
> >> >>
> >> >> On Thu, Jan 10, 2019 at 11:52 AM Corentin <corentin.jabot_at_[hidden]>
> wrote:
> >> >> >
> >> >> > I suspect it will be a while (several years) before we start to
> see large projects transitioning fully to modules and consumed as such by
> tools doing automatic dependency scanning etc.
> >> >> > My understanding is that there is little large scale
> implementation experience as far as tooling and build systems are concerned
> ( there is plenty _compilers_ implementation experience, and some build
> system implementation (and usage of modules) experience - mostly in build2).
> >> >>
> >> >> Given your statement above, why is it useful for SG15 to discuss
> >> >> enforcing mapping? Specifically, how will that discussion be grounded
> >> >> in facts? It seems to me like facts need a compiler implementation,
> >> >> and a codebase to try it out on. Feel free to talk about
> hypotheticals
> >> >> all you want, but in this case code wins.
> >> >>
> >> >> I think you want to refocus your approach: what are you trying to
> >> >> achieve? What's the advantage that SG15 has, which the modules SG and
> >> >> now EWG don't have? How can SG15 contribute module's success?
> >> >>
> >> >>
> >> >> > On Thu, 10 Jan 2019 at 18:47 JF Bastien <cxx_at_[hidden]> wrote:
> >> >> >>
> >> >> >> On Thu, Jan 10, 2019 at 6:53 AM Corentin <
> corentin.jabot_at_[hidden]> wrote:
> >> >> >> >
> >> >> >> > Hello.
> >> >> >> > I would like to suggest two modules related proposals that I
> think SG15 should look at.
> >> >> >> >
> >> >> >> > - Compiler enforced mapping between module names and module
> interface file (resource) name.
> >> >> >>
> >> >> >> Why does SG15 need to do this, versus someone implementing it in
> an
> >> >> >> open-source toolchain, trying it out, and bringing what using it
> >> >> >> taught them to SG15?
> >> >> >>
> >> >> >>
> >> >> >> > Currently, modules interfaces can be declared in any file -
> which makes dependency scanning more tedious than it needs to be and have
> performance implications
> >> >> >> > (The build system needs to open all files to gather a list of
> modules) - notably when the build system tries to start building while the
> dependency graph isn't yet complete.
> >> >> >> >
> >> >> >> > Tools ( ide, code servers, indexers, refactoring) may also
> greatly benefit from an easier way to locate the source file which declares
> a module.
> >> >> >> >
> >> >> >> > The specifics of the mapping are open to bikeshedding. However,
> I think we would have better luck sticking to something simple like <module
> identifier> <=> <file name>.<extension>
> >> >> >> > (The standardese would mention resource identifier rather than
> filename)
> >> >> >> >
> >> >> >> > - A standing document giving guidelines for modules naming.
> >> >> >> >
> >> >> >> > The goal is to take everything the community had to learn the
> hard way about header naming over the past 30 years and apply it to modules
> by providing a set of guidelines
> >> >> >> > that could be partially enforced by build system vendors.
> >> >> >> > Encouraging consistency and uniqueness of module identifiers
> across the industry is I think a necessary step towards sane package
> management.
> >> >> >> > Note that the standard requires uniqueness of modules
> identifiers within (the standard definition of) a program but says little
> about a way to ensure this uniqueness.
> >> >> >> >
> >> >> >> > Here is a rough draft of what I think would be good guidelines,
> partially inspired by what is done by other languages facing similar issues.
> >> >> >> >
> >> >> >> > Prefix module names with an entity and/or a project name to
> prevent modules from different companies, entities and projects of
> declaring the same module names.
> >> >> >> > Exported top-level namespaces should have a name identic to the
> project name used as part of the name of the module(s) from which it is
> exported.
> >> >> >> > Do not export multiple top-level namespaces
> >> >> >> > Do not export entities in the global namespace outside of the
> global module fragment.
> >> >> >> > Organize modules hierarchically. For example, if both modules
> example.foo and example.foo.bar exist as part of the public API of example,
> example.foo should reexport example.foo.bar
> >> >> >> > Avoid common names such as util and core for module name prefix
> and top-level namespace names.
> >> >> >> > Use lower-case module names
> >> >> >> > Do not use characters outside of the basic source character set
> in module name identifiers.
> >> >> >> >
> >> >> >> > My hope is that these 2 proposals (whose impact on the standard
> is minimal) would make it easier for current tooling to deal with modules
> >> >> >> > while making possible for example to design dependency managers
> and build systems able to work at the module level.
> >> >> >> >
> >> >> >> > I'd love to gather feedback and opinions before going further
> in that direction.
> >> >> >> > Thanks a lot!
> >> >> >> >
> >> >> >> > Corentin
> >> >> >> >
> >> >> >> > PS: For a bit of background, I talked about these issues there
> >> >> >> >
> >> >> >> > https://cor3ntin.github.io/posts/modules_mapping/
> >> >> >> > https://cor3ntin.github.io/posts/modules_naming/
> >> >> >> >
> >> >> >> >
> >> >> >> > _______________________________________________
> >> >> >> > Tooling mailing list
> >> >> >> > Tooling_at_[hidden]
> >> >> >> > http://www.open-std.org/mailman/listinfo/tooling
> >> >> >> _______________________________________________
> >> >> >> Tooling mailing list
> >> >> >> Tooling_at_[hidden]
> >> >> >> http://www.open-std.org/mailman/listinfo/tooling
> >> >> >
> >> >> > _______________________________________________
> >> >> > Tooling mailing list
> >> >> > Tooling_at_[hidden]
> >> >> > http://www.open-std.org/mailman/listinfo/tooling
> >> >> _______________________________________________
> >> >> Tooling mailing list
> >> >> Tooling_at_[hidden]
> >> >> http://www.open-std.org/mailman/listinfo/tooling
> >> >
> >> > _______________________________________________
> >> > Tooling mailing list
> >> > Tooling_at_[hidden]
> >> > http://www.open-std.org/mailman/listinfo/tooling
> >> _______________________________________________
> >> Tooling mailing list
> >> Tooling_at_[hidden]
> >> http://www.open-std.org/mailman/listinfo/tooling
> >
> > _______________________________________________
> > Tooling mailing list
> > Tooling_at_[hidden]
> > http://www.open-std.org/mailman/listinfo/tooling
> _______________________________________________
> Tooling mailing list
> Tooling_at_[hidden]
> http://www.open-std.org/mailman/listinfo/tooling
>