Date: Sat, 16 Apr 2022 19:31:39 +0000
[Boris]
> Sure, it will be difficult to implement efficiently in legacy build
> tools like Make and Ninja. But the solution space is not limited to
> legacy build systems.
You're right that the solution space isn't limited to existing build systems. However, they can't be ignored either. For example, Office engineers exploring header units for example didn't need to throw away their existing build systems -- that would have been a simple non-starter. And their build system isn't something I would call "legacy" either.
-- Gaby
-----Original Message-----
From: Ext <ext-bounces_at_[hidden]> On Behalf Of Boris Kolpackov via Ext
Sent: Friday, April 15, 2022 4:30 AM
To: Ben Boeckel <ben.boeckel_at_[hidden]>
Cc: Boris Kolpackov <boris_at_[hidden]>; Ben Boeckel via Ext <ext_at_[hidden]>; WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
Subject: Re: [isocpp-ext] Can we expect that all C++ source files can have the same suffix?
Ben Boeckel <ben.boeckel_at_[hidden]> writes:
> On Thu, Apr 14, 2022 at 15:09:35 +0200, Boris Kolpackov wrote:
>
> > With the mapper approach you don't need to pre-scan everything but
> > you do need to map module names to module interface files. Having
> > a separate extension for interface files helps narrow down the
> > candidate pool.
>
> While true, I have doubts about the scalability of holding umpteen
> compiler instances around while executing more to find the bottleneck
> TU.
Likewise, I have scalability doubts of the "pre-scan the world"
approach.
The nested compiler invocations is a potential issue, that's true.
I, however, believe the following will mitigate it:
1. The dependency depth of real-world software appears to be quite
low (I believe Rene looked into this and concluded that header
dependency depth rarely exceeds 10, IIRC).
2. The memory consumption (the primary concern) of both the compiler
that got suspended waiting for the BMI and the compiler that will
be executed to produce the BMI will be low. IME, the bulk of the
memory consumption happens during the code generation phase. The
suspended compiler hasn't reached that phase yet while the BMI-
producing compiler presumably can produce it without requiring much
memory at which point the suspended compiler can proceed.
It's possible that the BMI producing compiler will then proceed
(in parallel with the unblocked compiler) to code generation and
consume a large amount of memory. But with the module mapper
protocol (at least as implemented in GCC), the build system can
effectively suspend it (by delaying the reply to the "BMI is ready"
request) until later (e.g., until the unblocked compiler exits).
This is of course all conjecture and only experience will tell for
sure.
> Unsatisifiable requests also end up taking way more resources
> without smarter caching mechanisms to store that information across
> attempts. Which is fine if you're implementing your own build executor I
> suppose. Getting Make or Ninja to understand it without complicated
> stamp juggling doesn't sound fun though.
Sure, it will be difficult to implement efficiently in legacy build
tools like Make and Ninja. But the solution space is not limited to
legacy build systems.
_______________________________________________
Ext mailing list
Ext_at_[hidden]
Subscription: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isocpp.org%2Fmailman%2Flistinfo.cgi%2Fext&data=05%7C01%7Cgdr%40microsoft.com%7C47e9eebc6afc49028cb708da1ed34c0a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637856190118386092%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=s6r7dqfO9J356CHWzbNd1EoJ15xSxc12N2awfy%2F2%2F0k%3D&reserved=0
Link to this post: https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.isocpp.org%2Fext%2F2022%2F04%2F19014.php&data=05%7C01%7Cgdr%40microsoft.com%7C47e9eebc6afc49028cb708da1ed34c0a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637856190118386092%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=utsiUnxvKjG9OB9XrbtUMMzl8DvhsNJoFYEpkNQBVpQ%3D&reserved=0
> Sure, it will be difficult to implement efficiently in legacy build
> tools like Make and Ninja. But the solution space is not limited to
> legacy build systems.
You're right that the solution space isn't limited to existing build systems. However, they can't be ignored either. For example, Office engineers exploring header units for example didn't need to throw away their existing build systems -- that would have been a simple non-starter. And their build system isn't something I would call "legacy" either.
-- Gaby
-----Original Message-----
From: Ext <ext-bounces_at_[hidden]> On Behalf Of Boris Kolpackov via Ext
Sent: Friday, April 15, 2022 4:30 AM
To: Ben Boeckel <ben.boeckel_at_[hidden]>
Cc: Boris Kolpackov <boris_at_[hidden]>; Ben Boeckel via Ext <ext_at_[hidden]>; WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
Subject: Re: [isocpp-ext] Can we expect that all C++ source files can have the same suffix?
Ben Boeckel <ben.boeckel_at_[hidden]> writes:
> On Thu, Apr 14, 2022 at 15:09:35 +0200, Boris Kolpackov wrote:
>
> > With the mapper approach you don't need to pre-scan everything but
> > you do need to map module names to module interface files. Having
> > a separate extension for interface files helps narrow down the
> > candidate pool.
>
> While true, I have doubts about the scalability of holding umpteen
> compiler instances around while executing more to find the bottleneck
> TU.
Likewise, I have scalability doubts of the "pre-scan the world"
approach.
The nested compiler invocations is a potential issue, that's true.
I, however, believe the following will mitigate it:
1. The dependency depth of real-world software appears to be quite
low (I believe Rene looked into this and concluded that header
dependency depth rarely exceeds 10, IIRC).
2. The memory consumption (the primary concern) of both the compiler
that got suspended waiting for the BMI and the compiler that will
be executed to produce the BMI will be low. IME, the bulk of the
memory consumption happens during the code generation phase. The
suspended compiler hasn't reached that phase yet while the BMI-
producing compiler presumably can produce it without requiring much
memory at which point the suspended compiler can proceed.
It's possible that the BMI producing compiler will then proceed
(in parallel with the unblocked compiler) to code generation and
consume a large amount of memory. But with the module mapper
protocol (at least as implemented in GCC), the build system can
effectively suspend it (by delaying the reply to the "BMI is ready"
request) until later (e.g., until the unblocked compiler exits).
This is of course all conjecture and only experience will tell for
sure.
> Unsatisifiable requests also end up taking way more resources
> without smarter caching mechanisms to store that information across
> attempts. Which is fine if you're implementing your own build executor I
> suppose. Getting Make or Ninja to understand it without complicated
> stamp juggling doesn't sound fun though.
Sure, it will be difficult to implement efficiently in legacy build
tools like Make and Ninja. But the solution space is not limited to
legacy build systems.
_______________________________________________
Ext mailing list
Ext_at_[hidden]
Subscription: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isocpp.org%2Fmailman%2Flistinfo.cgi%2Fext&data=05%7C01%7Cgdr%40microsoft.com%7C47e9eebc6afc49028cb708da1ed34c0a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637856190118386092%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=s6r7dqfO9J356CHWzbNd1EoJ15xSxc12N2awfy%2F2%2F0k%3D&reserved=0
Link to this post: https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.isocpp.org%2Fext%2F2022%2F04%2F19014.php&data=05%7C01%7Cgdr%40microsoft.com%7C47e9eebc6afc49028cb708da1ed34c0a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637856190118386092%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=utsiUnxvKjG9OB9XrbtUMMzl8DvhsNJoFYEpkNQBVpQ%3D&reserved=0
Received on 2022-04-16 19:31:42