Date: Tue, 19 Apr 2022 12:22:56 -0400
On 4/19/22 3:34 AM, Jayesh Badwaik via Ext wrote:
> Why should standard library be special in this sense?
For the same reason that compiler drivers implicitly add system include
paths for the standard library unless directed otherwise. It improves
the out-of-the-box experience and provides significant benefit for
programmers.
I do expect that, assuming SG15 eventually produces a TR that specifies
an implementation agnostic method of discovering modules, their
requirements, and their dependencies, that the standard library would
implement those methods the same as any other library. For me that
increases the importance (and potential) for compiler drivers to provide
that support, at least at a minimally functional level.
Tom.
>
> On Tue, Apr 19, 2022, 05:24 Tom Honermann via Ext
> <ext_at_[hidden]> wrote:
>
> On 4/17/22 10:13 AM, Gabriel Dos Reis wrote:
> > [Tom]
> >> But I don't think that implies that implicit
> >> modules causes problems for build systems and dependency
> management at
> >> scale; there are other related factors.
> > Cache invalidation.
>
> If by this you mean that cached module artifacts may become out of
> date
> with respect to the source files used to construct them, I don't
> believe
> that to be a fundamental limitation of the implicit module approach.
> Implementations can index artifacts by hashes of their inputs
> (including
> command line options, file and path names, file timestamps, file
> contents, etc...), observe when a cached artifact is older than
> one of
> its inputs, and then generate a replacement. Cached artifacts can be
> garbage collected to reduce cache size. In cases where many source
> files
> contribute to a module, that implies a need to frequently stat() a
> potentially large set of source files, but the costs of doing so can
> likely be mitigated by a service that listens for and caches
> filesystem
> updates for a select set of files. Avoiding violations of the ODR
> would
> still require that a build system ensure that source files that
> contribute to module artifacts appear as dependencies for TUs that
> consume those modules (or be assumed to be static as is often the
> case
> for system headers); there is no avoiding that requirement.
>
> With respect to Nico's concerns, I'm not very concerned about the
> ability to compile a bunch of source files together in a single
> command
> line invocation. I'm much more concerned about the ability to
> compile a
> single TU that imports (portions of) the standard library. If, going
> forward, a build system is required to compile the following example
> because it is necessary to prebuild modules for the standard library
> using whatever compiler options are being used to compile a single
> source file into an executable, then I think we'll have done the C++
> community quite a disservice.
>
> import std;
> int main(int argc, char **argv) {
> std::cout << "Hello!\n";
> }
>
> Tom.
>
> >
> > -- Gaby
> >
> >
> > -----Original Message-----
> > From: Tom Honermann <tom_at_[hidden]>
> > Sent: Sunday, April 17, 2022 6:02 AM
> > To: ext_at_[hidden]; Boris Kolpackov <boris_at_[hidden]>
> > Cc: Gabriel Dos Reis <gdr_at_[hidden]>; Nathan Sidwell
> <nathan_at_[hidden]>; WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
> > Subject: Re: [isocpp-ext] Can we expect that all C++ source
> files can have the same suffix?
> >
> > On 4/16/22 2:57 PM, Gabriel Dos Reis via Ext wrote:
> >> [Nathan]
> >>> True, but IIUC that is how clang's implicit module machinery
> already
> >>> works. using the module map, the compiler will fork to compile an
> >>> implicit module (equivalent of a header unit). I don't know how
> >>> concurrency concerns are addressed. (I don't have data on how this
> >>> scheme behaves in practice)
> >> "Implicit build of module" a la Clang (which was originally
> implemented for Clang Modules) is known to cause problems for
> build systems and dependency management at scale, which is why
> they introduced an "explicit build". Last time I heard,
> organizations using the existing "implicit build" system would
> love to move away from it, and move towards the "explicit build"
> approach.
> >>
> >> I fully appreciate that "implicit build" works for nice little
> example. I am hoping that issue can be solved by other means
> (either using an IDE or a pre-scanner to construct a sort of
> module map) than moving build system logics into the compilers.
> > My impression is a little different. When working on Coverity, I saw
> > many examples of implicit modules being successfully used by
> Coverity
> > customers (primarily with Objective-C, primarily in conjunction with
> > Apple provided frameworks). I think the desire for explicit
> builds is
> > driven by a desire to improve efficiency over what implicit
> builds can
> > provide (implicit builds often result in the same interface
> units being
> > built multiple times subject to implementation ability to
> recognize when
> > certain command line options are irrelevant to the module being
> built;
> > an admittedly difficult problem). Thus there is a tradeoff;
> willingness
> > to accept a less efficient solution vs expending additional
> effort to
> > modify the build system to support explicit module builds. I
> acknowledge
> > that providing an explicit build solution that is easy to deploy
> is a
> > win-win for everyone. But I don't think that implies that implicit
> > modules causes problems for build systems and dependency
> management at
> > scale; there are other related factors.
> >
> > Tom.
> >
> >> -- Gaby
> >>
> >> -----Original Message-----
> >> From: Ext <ext-bounces_at_[hidden]> On Behalf Of Nathan
> Sidwell via Ext
> >> Sent: Friday, April 15, 2022 4:51 AM
> >> To: ext_at_[hidden]; Boris Kolpackov <boris_at_[hidden]>
> >> Cc: Nathan Sidwell <nathan_at_[hidden]>; WG21 Tooling Study Group
> SG15 <tooling_at_[hidden]>
> >> Subject: Re: [isocpp-ext] Can we expect that all C++ source
> files can have the same suffix?
> >>
> >> On 4/14/22 16:18, Ben Boeckel via Ext wrote:
> >>> On Thu, Apr 14, 2022 at 15:09:35 +0200, Boris Kolpackov wrote:
> >>>> Ben Boeckel via Ext <ext_at_[hidden]> writes:
> >>>>> Need? No. Even with different suffixes, a scanner is still
> necessary to
> >>>>> know what order files need to be compiled in.
> >>>> A (pre-)scanner is not the only way to do it. Another
> approach is a
> >>>> module mapper (implemented in GCC, being implemented in
> Clang, used
> >>>> in build2).
> >>> Well, an active module mapper. CMake just creates the static
> files which
> >>> handle the GCC's module mapper protocol.
> >>>
> >>>> With the mapper approach you don't need to pre-scan
> everything but
> >>>> you do need to map module names to module interface files. Having
> >>>> a separate extension for interface files helps narrow down the
> >>>> candidate pool.
> >>> While true, I have doubts about the scalability of holding umpteen
> >>> compiler instances around while executing more to find the
> bottleneck
> >>> TU. Unsatisifiable requests also end up taking way more resources
> >>> without smarter caching mechanisms to store that information
> across
> >> True, but IIUC that is how clang's implicit module machinery
> already
> >> works. using the module map, the compiler will fork to compile an
> >> implicit module (equivalent of a header unit). I don't know how
> >> concurrency concerns are addressed. (I don't have data on how this
> >> scheme behaves in practice)
> >>
> >>> attempts. Which is fine if you're implementing your own build
> executor I
> >>> suppose. Getting Make or Ninja to understand it without
> complicated
> >>> stamp juggling doesn't sound fun though.
> >>>
> >>> --Ben
> >>> _______________________________________________
> >>> Ext mailing list
> >>> Ext_at_[hidden]
> >>> Subscription:
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isocpp.org%2Fmailman%2Flistinfo.cgi%2Fext&data=05%7C01%7Cgdr%40microsoft.com%7Cf747c6c5ab2740313c2008da2072866f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637857973514339517%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HSScxdrumTl31rWDGpgNnUpv%2F6dQBbO3kGTNsbGjF7I%3D&reserved=0
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isocpp.org%2Fmailman%2Flistinfo.cgi%2Fext&data=05%7C01%7Cgdr%40microsoft.com%7Cf747c6c5ab2740313c2008da2072866f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637857973514339517%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HSScxdrumTl31rWDGpgNnUpv%2F6dQBbO3kGTNsbGjF7I%3D&reserved=0>
> >>> Link to this post:
> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.isocpp.org%2Fext%2F2022%2F04%2F19012.php&data=05%7C01%7Cgdr%40microsoft.com%7Cf747c6c5ab2740313c2008da2072866f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637857973514339517%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=XnkgcgS8%2FM%2BSY%2BIwbgg97NqlTH7qkO5Htjqy1C9Tc10%3D&reserved=0
> <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.isocpp.org%2Fext%2F2022%2F04%2F19012.php&data=05%7C01%7Cgdr%40microsoft.com%7Cf747c6c5ab2740313c2008da2072866f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637857973514339517%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=XnkgcgS8%2FM%2BSY%2BIwbgg97NqlTH7qkO5Htjqy1C9Tc10%3D&reserved=0>
> >>>
> _______________________________________________
> Ext mailing list
> Ext_at_[hidden]
> Subscription: https://lists.isocpp.org/mailman/listinfo.cgi/ext
> Link to this post: http://lists.isocpp.org/ext/2022/04/19060.php
>
>
> _______________________________________________
> Ext mailing list
> Ext_at_[hidden]
> Subscription:https://lists.isocpp.org/mailman/listinfo.cgi/ext
> Link to this post:http://lists.isocpp.org/ext/2022/04/19061.php
> Why should standard library be special in this sense?
For the same reason that compiler drivers implicitly add system include
paths for the standard library unless directed otherwise. It improves
the out-of-the-box experience and provides significant benefit for
programmers.
I do expect that, assuming SG15 eventually produces a TR that specifies
an implementation agnostic method of discovering modules, their
requirements, and their dependencies, that the standard library would
implement those methods the same as any other library. For me that
increases the importance (and potential) for compiler drivers to provide
that support, at least at a minimally functional level.
Tom.
>
> On Tue, Apr 19, 2022, 05:24 Tom Honermann via Ext
> <ext_at_[hidden]> wrote:
>
> On 4/17/22 10:13 AM, Gabriel Dos Reis wrote:
> > [Tom]
> >> But I don't think that implies that implicit
> >> modules causes problems for build systems and dependency
> management at
> >> scale; there are other related factors.
> > Cache invalidation.
>
> If by this you mean that cached module artifacts may become out of
> date
> with respect to the source files used to construct them, I don't
> believe
> that to be a fundamental limitation of the implicit module approach.
> Implementations can index artifacts by hashes of their inputs
> (including
> command line options, file and path names, file timestamps, file
> contents, etc...), observe when a cached artifact is older than
> one of
> its inputs, and then generate a replacement. Cached artifacts can be
> garbage collected to reduce cache size. In cases where many source
> files
> contribute to a module, that implies a need to frequently stat() a
> potentially large set of source files, but the costs of doing so can
> likely be mitigated by a service that listens for and caches
> filesystem
> updates for a select set of files. Avoiding violations of the ODR
> would
> still require that a build system ensure that source files that
> contribute to module artifacts appear as dependencies for TUs that
> consume those modules (or be assumed to be static as is often the
> case
> for system headers); there is no avoiding that requirement.
>
> With respect to Nico's concerns, I'm not very concerned about the
> ability to compile a bunch of source files together in a single
> command
> line invocation. I'm much more concerned about the ability to
> compile a
> single TU that imports (portions of) the standard library. If, going
> forward, a build system is required to compile the following example
> because it is necessary to prebuild modules for the standard library
> using whatever compiler options are being used to compile a single
> source file into an executable, then I think we'll have done the C++
> community quite a disservice.
>
> import std;
> int main(int argc, char **argv) {
> std::cout << "Hello!\n";
> }
>
> Tom.
>
> >
> > -- Gaby
> >
> >
> > -----Original Message-----
> > From: Tom Honermann <tom_at_[hidden]>
> > Sent: Sunday, April 17, 2022 6:02 AM
> > To: ext_at_[hidden]; Boris Kolpackov <boris_at_[hidden]>
> > Cc: Gabriel Dos Reis <gdr_at_[hidden]>; Nathan Sidwell
> <nathan_at_[hidden]>; WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
> > Subject: Re: [isocpp-ext] Can we expect that all C++ source
> files can have the same suffix?
> >
> > On 4/16/22 2:57 PM, Gabriel Dos Reis via Ext wrote:
> >> [Nathan]
> >>> True, but IIUC that is how clang's implicit module machinery
> already
> >>> works. using the module map, the compiler will fork to compile an
> >>> implicit module (equivalent of a header unit). I don't know how
> >>> concurrency concerns are addressed. (I don't have data on how this
> >>> scheme behaves in practice)
> >> "Implicit build of module" a la Clang (which was originally
> implemented for Clang Modules) is known to cause problems for
> build systems and dependency management at scale, which is why
> they introduced an "explicit build". Last time I heard,
> organizations using the existing "implicit build" system would
> love to move away from it, and move towards the "explicit build"
> approach.
> >>
> >> I fully appreciate that "implicit build" works for nice little
> example. I am hoping that issue can be solved by other means
> (either using an IDE or a pre-scanner to construct a sort of
> module map) than moving build system logics into the compilers.
> > My impression is a little different. When working on Coverity, I saw
> > many examples of implicit modules being successfully used by
> Coverity
> > customers (primarily with Objective-C, primarily in conjunction with
> > Apple provided frameworks). I think the desire for explicit
> builds is
> > driven by a desire to improve efficiency over what implicit
> builds can
> > provide (implicit builds often result in the same interface
> units being
> > built multiple times subject to implementation ability to
> recognize when
> > certain command line options are irrelevant to the module being
> built;
> > an admittedly difficult problem). Thus there is a tradeoff;
> willingness
> > to accept a less efficient solution vs expending additional
> effort to
> > modify the build system to support explicit module builds. I
> acknowledge
> > that providing an explicit build solution that is easy to deploy
> is a
> > win-win for everyone. But I don't think that implies that implicit
> > modules causes problems for build systems and dependency
> management at
> > scale; there are other related factors.
> >
> > Tom.
> >
> >> -- Gaby
> >>
> >> -----Original Message-----
> >> From: Ext <ext-bounces_at_[hidden]> On Behalf Of Nathan
> Sidwell via Ext
> >> Sent: Friday, April 15, 2022 4:51 AM
> >> To: ext_at_[hidden]; Boris Kolpackov <boris_at_[hidden]>
> >> Cc: Nathan Sidwell <nathan_at_[hidden]>; WG21 Tooling Study Group
> SG15 <tooling_at_[hidden]>
> >> Subject: Re: [isocpp-ext] Can we expect that all C++ source
> files can have the same suffix?
> >>
> >> On 4/14/22 16:18, Ben Boeckel via Ext wrote:
> >>> On Thu, Apr 14, 2022 at 15:09:35 +0200, Boris Kolpackov wrote:
> >>>> Ben Boeckel via Ext <ext_at_[hidden]> writes:
> >>>>> Need? No. Even with different suffixes, a scanner is still
> necessary to
> >>>>> know what order files need to be compiled in.
> >>>> A (pre-)scanner is not the only way to do it. Another
> approach is a
> >>>> module mapper (implemented in GCC, being implemented in
> Clang, used
> >>>> in build2).
> >>> Well, an active module mapper. CMake just creates the static
> files which
> >>> handle the GCC's module mapper protocol.
> >>>
> >>>> With the mapper approach you don't need to pre-scan
> everything but
> >>>> you do need to map module names to module interface files. Having
> >>>> a separate extension for interface files helps narrow down the
> >>>> candidate pool.
> >>> While true, I have doubts about the scalability of holding umpteen
> >>> compiler instances around while executing more to find the
> bottleneck
> >>> TU. Unsatisifiable requests also end up taking way more resources
> >>> without smarter caching mechanisms to store that information
> across
> >> True, but IIUC that is how clang's implicit module machinery
> already
> >> works. using the module map, the compiler will fork to compile an
> >> implicit module (equivalent of a header unit). I don't know how
> >> concurrency concerns are addressed. (I don't have data on how this
> >> scheme behaves in practice)
> >>
> >>> attempts. Which is fine if you're implementing your own build
> executor I
> >>> suppose. Getting Make or Ninja to understand it without
> complicated
> >>> stamp juggling doesn't sound fun though.
> >>>
> >>> --Ben
> >>> _______________________________________________
> >>> Ext mailing list
> >>> Ext_at_[hidden]
> >>> Subscription:
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isocpp.org%2Fmailman%2Flistinfo.cgi%2Fext&data=05%7C01%7Cgdr%40microsoft.com%7Cf747c6c5ab2740313c2008da2072866f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637857973514339517%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HSScxdrumTl31rWDGpgNnUpv%2F6dQBbO3kGTNsbGjF7I%3D&reserved=0
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isocpp.org%2Fmailman%2Flistinfo.cgi%2Fext&data=05%7C01%7Cgdr%40microsoft.com%7Cf747c6c5ab2740313c2008da2072866f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637857973514339517%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HSScxdrumTl31rWDGpgNnUpv%2F6dQBbO3kGTNsbGjF7I%3D&reserved=0>
> >>> Link to this post:
> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.isocpp.org%2Fext%2F2022%2F04%2F19012.php&data=05%7C01%7Cgdr%40microsoft.com%7Cf747c6c5ab2740313c2008da2072866f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637857973514339517%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=XnkgcgS8%2FM%2BSY%2BIwbgg97NqlTH7qkO5Htjqy1C9Tc10%3D&reserved=0
> <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.isocpp.org%2Fext%2F2022%2F04%2F19012.php&data=05%7C01%7Cgdr%40microsoft.com%7Cf747c6c5ab2740313c2008da2072866f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637857973514339517%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=XnkgcgS8%2FM%2BSY%2BIwbgg97NqlTH7qkO5Htjqy1C9Tc10%3D&reserved=0>
> >>>
> _______________________________________________
> Ext mailing list
> Ext_at_[hidden]
> Subscription: https://lists.isocpp.org/mailman/listinfo.cgi/ext
> Link to this post: http://lists.isocpp.org/ext/2022/04/19060.php
>
>
> _______________________________________________
> Ext mailing list
> Ext_at_[hidden]
> Subscription:https://lists.isocpp.org/mailman/listinfo.cgi/ext
> Link to this post:http://lists.isocpp.org/ext/2022/04/19061.php
Received on 2022-04-19 16:22:58