Date: Fri, 15 Oct 2021 20:12:53 +0000
* [1] which wouldn't preclude the support for module map files, in fact I can see a build system build a module map file for the local modules with a fallback to path-based lookup for external libraries
Why is the module map file approach being discounted or not pursued in favor of the current proposal?
-- Gaby
From: Daniel Ruoso <daniel_at_[hidden]>
Sent: Friday, October 15, 2021 12:46 PM
To: sg15_at_[hidden]
Cc: Gabriel Dos Reis <gdr_at_[hidden]>
Subject: Re: [SG15] P2473R0: Distributing C++ Module Libraries
On Fri, Oct 15, 2021 at 2:28 PM Gabriel Dos Reis via SG15 <sg15_at_[hidden]<mailto:sg15_at_[hidden]>> wrote:
I would be extremely hard pressed to recommend practices that effectively shut down an evolution path where a given source file can have more than one module interface specification.
I don't have an answer to that, except that I think this space is already complex enough with a 1-1 mapping from name to interface specification. IMHO, limiting unnecessary complexity should also be a goal, so I guess I would like to hear arguments for what is the problem that can't be solved without that extra complexity.
What we need is a mapping. This proposal suggests to encode that mapping into the host filesystem. I am skeptical that is the best practice to recommend in the context of C++.
To be clear, this proposal suggests to encode that mapping in the filesystem in the context of distributing libraries as pre-built artifacts. It specifically states that the internal representation inside the build system is out of the scope of this proposal.
In fact, one of the concrete suggestions in the proposal is that the build system should perform the discovery up-front and make that information available in the same way the compilation database does today. The output of that discovery process is a complete mapping of all the modules that are relevant for this particular project -- which then the build system can make use of in whichever way it wants.
The point of the proposal is precisely to come up with a reasonable way to produce that mapping in the absence of a standardized package manager.
Now, it's true that the proposal *also* makes the case that *if* the compilers support the path-based-lookup[1] a simple GNU Makefile would be able to drive the build of a project that consumes and produces C++ modules, just by having a step that generates the mapping and then translates it to a make dependency syntax[2].
Another reason why the path-based lookup greatly reduces the complexity of the build system is that it's easier to avoid creating a bottleneck in the caching ability of build systems making use of the Remote Execution API[3]. Since, realistically, the build system ends up having to assemble a map file for each translation (either to produce the bmi or to produce a .o), and then keep those up-to-date as modules change location. A simpler build system would be able to just rely on the path-based-lookup and still do the right thing.
[1] which wouldn't preclude the support for module map files, in fact I can see a build system build a module map file for the local modules with a fallback to path-based lookup for external libraries
[2] the accompanying code at https://github.com/bloomberg/p2473<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbloomberg%2Fp2473&data=04%7C01%7Cgdr%40microsoft.com%7Cf84a100fc5a24b6c2abb08d9901465fe%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637699239564775144%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=KBU3lDFFYFHRmEfn3fOADKR1m%2BJupODoNSOW1r5VC1U%3D&reserved=0> serves as proof-of-concept for that
[3] https://bazel.build/remote-execution-services.html<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbazel.build%2Fremote-execution-services.html&data=04%7C01%7Cgdr%40microsoft.com%7Cf84a100fc5a24b6c2abb08d9901465fe%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637699239564785137%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=lk%2FbkIyk8dpl7dAf9iedJU1AAD%2FOYOtk1u8GPVrOHKw%3D&reserved=0>
Why is the module map file approach being discounted or not pursued in favor of the current proposal?
-- Gaby
From: Daniel Ruoso <daniel_at_[hidden]>
Sent: Friday, October 15, 2021 12:46 PM
To: sg15_at_[hidden]
Cc: Gabriel Dos Reis <gdr_at_[hidden]>
Subject: Re: [SG15] P2473R0: Distributing C++ Module Libraries
On Fri, Oct 15, 2021 at 2:28 PM Gabriel Dos Reis via SG15 <sg15_at_[hidden]<mailto:sg15_at_[hidden]>> wrote:
I would be extremely hard pressed to recommend practices that effectively shut down an evolution path where a given source file can have more than one module interface specification.
I don't have an answer to that, except that I think this space is already complex enough with a 1-1 mapping from name to interface specification. IMHO, limiting unnecessary complexity should also be a goal, so I guess I would like to hear arguments for what is the problem that can't be solved without that extra complexity.
What we need is a mapping. This proposal suggests to encode that mapping into the host filesystem. I am skeptical that is the best practice to recommend in the context of C++.
To be clear, this proposal suggests to encode that mapping in the filesystem in the context of distributing libraries as pre-built artifacts. It specifically states that the internal representation inside the build system is out of the scope of this proposal.
In fact, one of the concrete suggestions in the proposal is that the build system should perform the discovery up-front and make that information available in the same way the compilation database does today. The output of that discovery process is a complete mapping of all the modules that are relevant for this particular project -- which then the build system can make use of in whichever way it wants.
The point of the proposal is precisely to come up with a reasonable way to produce that mapping in the absence of a standardized package manager.
Now, it's true that the proposal *also* makes the case that *if* the compilers support the path-based-lookup[1] a simple GNU Makefile would be able to drive the build of a project that consumes and produces C++ modules, just by having a step that generates the mapping and then translates it to a make dependency syntax[2].
Another reason why the path-based lookup greatly reduces the complexity of the build system is that it's easier to avoid creating a bottleneck in the caching ability of build systems making use of the Remote Execution API[3]. Since, realistically, the build system ends up having to assemble a map file for each translation (either to produce the bmi or to produce a .o), and then keep those up-to-date as modules change location. A simpler build system would be able to just rely on the path-based-lookup and still do the right thing.
[1] which wouldn't preclude the support for module map files, in fact I can see a build system build a module map file for the local modules with a fallback to path-based lookup for external libraries
[2] the accompanying code at https://github.com/bloomberg/p2473<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbloomberg%2Fp2473&data=04%7C01%7Cgdr%40microsoft.com%7Cf84a100fc5a24b6c2abb08d9901465fe%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637699239564775144%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=KBU3lDFFYFHRmEfn3fOADKR1m%2BJupODoNSOW1r5VC1U%3D&reserved=0> serves as proof-of-concept for that
[3] https://bazel.build/remote-execution-services.html<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbazel.build%2Fremote-execution-services.html&data=04%7C01%7Cgdr%40microsoft.com%7Cf84a100fc5a24b6c2abb08d9901465fe%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637699239564785137%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=lk%2FbkIyk8dpl7dAf9iedJU1AAD%2FOYOtk1u8GPVrOHKw%3D&reserved=0>
Received on 2021-10-15 15:12:58