Date: Sun, 17 Apr 2022 09:02:24 -0400
On 4/16/22 2:57 PM, Gabriel Dos Reis via Ext wrote:
> [Nathan]
>> True, but IIUC that is how clang's implicit module machinery already
>> works. using the module map, the compiler will fork to compile an
>> implicit module (equivalent of a header unit). I don't know how
>> concurrency concerns are addressed. (I don't have data on how this
>> scheme behaves in practice)
> "Implicit build of module" a la Clang (which was originally implemented for Clang Modules) is known to cause problems for build systems and dependency management at scale, which is why they introduced an "explicit build". Last time I heard, organizations using the existing "implicit build" system would love to move away from it, and move towards the "explicit build" approach.
>
> I fully appreciate that "implicit build" works for nice little example. I am hoping that issue can be solved by other means (either using an IDE or a pre-scanner to construct a sort of module map) than moving build system logics into the compilers.
My impression is a little different. When working on Coverity, I saw
many examples of implicit modules being successfully used by Coverity
customers (primarily with Objective-C, primarily in conjunction with
Apple provided frameworks). I think the desire for explicit builds is
driven by a desire to improve efficiency over what implicit builds can
provide (implicit builds often result in the same interface units being
built multiple times subject to implementation ability to recognize when
certain command line options are irrelevant to the module being built;
an admittedly difficult problem). Thus there is a tradeoff; willingness
to accept a less efficient solution vs expending additional effort to
modify the build system to support explicit module builds. I acknowledge
that providing an explicit build solution that is easy to deploy is a
win-win for everyone. But I don't think that implies that implicit
modules causes problems for build systems and dependency management at
scale; there are other related factors.
Tom.
>
> -- Gaby
>
> -----Original Message-----
> From: Ext <ext-bounces_at_[hidden]> On Behalf Of Nathan Sidwell via Ext
> Sent: Friday, April 15, 2022 4:51 AM
> To: ext_at_[hidden]; Boris Kolpackov <boris_at_[hidden]>
> Cc: Nathan Sidwell <nathan_at_[hidden]>; WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
> Subject: Re: [isocpp-ext] Can we expect that all C++ source files can have the same suffix?
>
> On 4/14/22 16:18, Ben Boeckel via Ext wrote:
>> On Thu, Apr 14, 2022 at 15:09:35 +0200, Boris Kolpackov wrote:
>>> Ben Boeckel via Ext <ext_at_[hidden]> writes:
>>>> Need? No. Even with different suffixes, a scanner is still necessary to
>>>> know what order files need to be compiled in.
>>> A (pre-)scanner is not the only way to do it. Another approach is a
>>> module mapper (implemented in GCC, being implemented in Clang, used
>>> in build2).
>> Well, an active module mapper. CMake just creates the static files which
>> handle the GCC's module mapper protocol.
>>
>>> With the mapper approach you don't need to pre-scan everything but
>>> you do need to map module names to module interface files. Having
>>> a separate extension for interface files helps narrow down the
>>> candidate pool.
>> While true, I have doubts about the scalability of holding umpteen
>> compiler instances around while executing more to find the bottleneck
>> TU. Unsatisifiable requests also end up taking way more resources
>> without smarter caching mechanisms to store that information across
> True, but IIUC that is how clang's implicit module machinery already
> works. using the module map, the compiler will fork to compile an
> implicit module (equivalent of a header unit). I don't know how
> concurrency concerns are addressed. (I don't have data on how this
> scheme behaves in practice)
>
>> attempts. Which is fine if you're implementing your own build executor I
>> suppose. Getting Make or Ninja to understand it without complicated
>> stamp juggling doesn't sound fun though.
>>
>> --Ben
>> _______________________________________________
>> Ext mailing list
>> Ext_at_[hidden]
>> Subscription: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isocpp.org%2Fmailman%2Flistinfo.cgi%2Fext&data=05%7C01%7Cgdr%40microsoft.com%7C9a138e11724044d7887008da1ed62878%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637856202406675963%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=q0AVwnQZPknmwPtvNCJFft2eh2GrcmujwBpeCv%2FHnBQ%3D&reserved=0
>> Link to this post: https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.isocpp.org%2Fext%2F2022%2F04%2F19012.php&data=05%7C01%7Cgdr%40microsoft.com%7C9a138e11724044d7887008da1ed62878%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637856202406675963%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Biw77eOgwB%2BJC3bdCXasKP1GB7bM83IYUfreevV7IZA%3D&reserved=0
>>
>
> [Nathan]
>> True, but IIUC that is how clang's implicit module machinery already
>> works. using the module map, the compiler will fork to compile an
>> implicit module (equivalent of a header unit). I don't know how
>> concurrency concerns are addressed. (I don't have data on how this
>> scheme behaves in practice)
> "Implicit build of module" a la Clang (which was originally implemented for Clang Modules) is known to cause problems for build systems and dependency management at scale, which is why they introduced an "explicit build". Last time I heard, organizations using the existing "implicit build" system would love to move away from it, and move towards the "explicit build" approach.
>
> I fully appreciate that "implicit build" works for nice little example. I am hoping that issue can be solved by other means (either using an IDE or a pre-scanner to construct a sort of module map) than moving build system logics into the compilers.
My impression is a little different. When working on Coverity, I saw
many examples of implicit modules being successfully used by Coverity
customers (primarily with Objective-C, primarily in conjunction with
Apple provided frameworks). I think the desire for explicit builds is
driven by a desire to improve efficiency over what implicit builds can
provide (implicit builds often result in the same interface units being
built multiple times subject to implementation ability to recognize when
certain command line options are irrelevant to the module being built;
an admittedly difficult problem). Thus there is a tradeoff; willingness
to accept a less efficient solution vs expending additional effort to
modify the build system to support explicit module builds. I acknowledge
that providing an explicit build solution that is easy to deploy is a
win-win for everyone. But I don't think that implies that implicit
modules causes problems for build systems and dependency management at
scale; there are other related factors.
Tom.
>
> -- Gaby
>
> -----Original Message-----
> From: Ext <ext-bounces_at_[hidden]> On Behalf Of Nathan Sidwell via Ext
> Sent: Friday, April 15, 2022 4:51 AM
> To: ext_at_[hidden]; Boris Kolpackov <boris_at_[hidden]>
> Cc: Nathan Sidwell <nathan_at_[hidden]>; WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
> Subject: Re: [isocpp-ext] Can we expect that all C++ source files can have the same suffix?
>
> On 4/14/22 16:18, Ben Boeckel via Ext wrote:
>> On Thu, Apr 14, 2022 at 15:09:35 +0200, Boris Kolpackov wrote:
>>> Ben Boeckel via Ext <ext_at_[hidden]> writes:
>>>> Need? No. Even with different suffixes, a scanner is still necessary to
>>>> know what order files need to be compiled in.
>>> A (pre-)scanner is not the only way to do it. Another approach is a
>>> module mapper (implemented in GCC, being implemented in Clang, used
>>> in build2).
>> Well, an active module mapper. CMake just creates the static files which
>> handle the GCC's module mapper protocol.
>>
>>> With the mapper approach you don't need to pre-scan everything but
>>> you do need to map module names to module interface files. Having
>>> a separate extension for interface files helps narrow down the
>>> candidate pool.
>> While true, I have doubts about the scalability of holding umpteen
>> compiler instances around while executing more to find the bottleneck
>> TU. Unsatisifiable requests also end up taking way more resources
>> without smarter caching mechanisms to store that information across
> True, but IIUC that is how clang's implicit module machinery already
> works. using the module map, the compiler will fork to compile an
> implicit module (equivalent of a header unit). I don't know how
> concurrency concerns are addressed. (I don't have data on how this
> scheme behaves in practice)
>
>> attempts. Which is fine if you're implementing your own build executor I
>> suppose. Getting Make or Ninja to understand it without complicated
>> stamp juggling doesn't sound fun though.
>>
>> --Ben
>> _______________________________________________
>> Ext mailing list
>> Ext_at_[hidden]
>> Subscription: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isocpp.org%2Fmailman%2Flistinfo.cgi%2Fext&data=05%7C01%7Cgdr%40microsoft.com%7C9a138e11724044d7887008da1ed62878%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637856202406675963%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=q0AVwnQZPknmwPtvNCJFft2eh2GrcmujwBpeCv%2FHnBQ%3D&reserved=0
>> Link to this post: https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.isocpp.org%2Fext%2F2022%2F04%2F19012.php&data=05%7C01%7Cgdr%40microsoft.com%7C9a138e11724044d7887008da1ed62878%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637856202406675963%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Biw77eOgwB%2BJC3bdCXasKP1GB7bM83IYUfreevV7IZA%3D&reserved=0
>>
>
Received on 2022-04-17 13:11:47