I messaged a few days ago. I understand if there is a lack of interest, but
I would appreciate it if you convey the reasons.

I think the following might be your reasons.

i) Insignificant compilation speed-up of 5% once the project is fully
converted to modules.

I think this is incorrect as for projects like UE5 there will be a lot of
accompanying header units with the library modules as UE5 has lots of
compile definitions in its code base. For such projects, the final
improvement with this approach could be greater than 10%. So, while it
results in blazing speeds during the conversion process to C++20 modules
when the project solely consists of C++20 header units, it will also result
in noticeable final improvements.

ii) This approach has lots of trade-offs.

I think that besides the compiler crashing, all others can be very
effectively managed by the build-systems. I will consider them as
nonissues. As far as the compiler crashing is concerned, I think we all can
bet that with future releases, all compilers will become very stable and
crash-proof as their automated testing increases if they are not already
there.

If any implementation is interested, please message me.

I look forward to your thoughts.

Best,
Hassan Sajjad

On Sat, Sep 30, 2023 at 5:43 AM Hassan Sajjad <hassan.sajjad069_at_[hidden]>
wrote:

> Hi.
>
> Following the guidelines of the procedure
> https://isocpp.org/std/the-life-of-an-iso-proposal here, I request the
> SG15 chair to convene the SG15 meeting with my paper
> https://isocpp.org/files/papers/P2978R0.html as the agenda.
>
> My understanding of procedures is that I will have to present the paper.
> Basically, go through the paper. And take part in the discussion with the
> SG15 audience and try to address their concerns. After the presentation, a
> poll is conducted.
>
> If consensus is achieved, the paper is forwarded to the wording group.
> But, in my case after consensus, the paper won't be forwarded as it is not
> a C++ standard or library paper. It is true for the few other SG15 papers
> as well e.g.
> https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p1689r5.html#_motivation.
> The tooling papers just remain there meaning that the conforming
> implementation can use these as a reference if they want to support the
> specific feature/workflow in the future.
>
> Please guide the next steps if my understanding is incorrect.
>
> Best,
> Hassan Sajjad
>
> On Wed, Sep 27, 2023 at 10:55 AM Hassan Sajjad <hassan.sajjad069_at_[hidden]>
> wrote:
>
>> Hi.
>>
>> I think you have plenty of rigor here to justify a numbered ISO paper to
>>> drive an agenda item for the Tooling Study Group (SG-15)
>>>
>>
>> My paper received the number https://isocpp.org/files/papers/P2978R0.html
>>
>> Best,
>> Hassan Sajjad
>>
>> On Tue, Sep 12, 2023 at 7:00 PM Bret Brown <mail_at_[hidden]> wrote:
>>
>>> I think you have plenty of rigor here to justify a numbered ISO paper to
>>> drive an agenda item for the Tooling Study Group (SG-15). Though I expect
>>> finding some dedicated attention from the author of the papers you cite
>>> would be a productive use of time beforehand, so at least you two are on
>>> the same line of thinking before starting a larger group discussion. But
>>> it's worth submitting a paper to drive an interactive discussion even if
>>> you cannot reasonably arrange that sidebar discussion.
>>>
>>> As to the compiler categorizing flags as local or not, I'm open to the
>>> idea I suppose. Though I suspect there are a lot of strange edge cases that
>>> would justify a feature in which authored information is respected about
>>> which preprocessor definitions are explicitly local and which ones
>>> explicitly are not.
>>>
>>> But I agree with you that in most cases, nobody should want to maintain
>>> detailed build system configurations about how to categorize each flag.
>>> Though I expect that normally what will happen is people won't consider
>>> this at all and use whatever defaults the build system supports, which I
>>> expect to be non-local flags since that's the more conservative option with
>>> respect to accurate parsing.
>>>
>>> Bret
>>>
>>> On Mon, Sep 11, 2023, 10:02 Hassan Sajjad <hassan.sajjad069_at_[hidden]>
>>> wrote:
>>>
>>>> Hi.
>>>>
>>>> I emailed this https://lists.isocpp.org/sg15/2023/09/2040.php a few
>>>> days ago.
>>>> I am looking forward for your review.
>>>>
>>>> Best,
>>>> Hassan Sajjad
>>>>
>>>> On Tue, Sep 5, 2023, 09:46 Hassan Sajjad <hassan.sajjad069_at_[hidden]>
>>>> wrote:
>>>>
>>>>> Hi.
>>>>>
>>>>> Please share your thoughts on this.
>>>>>
>>>>> Best,
>>>>> Hassan Sajjad
>>>>>
>>>>>
>>>>> On Sun, Sep 3, 2023, 04:57 Hassan Sajjad <hassan.sajjad069_at_[hidden]>
>>>>> wrote:
>>>>>
>>>>>> Hi.
>>>>>>
>>>>>> First of all, great job on your writeups. They are very well
>>>>>>> considered. I appreciate how much effort you're putting into communicating
>>>>>>> your ideas and developing better C++ tools.
>>>>>>>
>>>>>>
>>>>>> Thank you. This complement means a lot. It would be an immense
>>>>>> pleasure if I or my project could be helpful in improving C++ tooling.
>>>>>>
>>>>>> I think your design could work
>>>>>>>
>>>>>>
>>>>>> This was so good to hear :). Thank you again.
>>>>>>
>>>>>>
>>>>>>> though I expect the node identities of the imported entities need
>>>>>>> to be tweaked to better model a combination of the file and the flags used
>>>>>>> to parse the file. The build system would ideally know when to just reuse a
>>>>>>> particular parse of a particular entity, especially when optimizing for
>>>>>>> build speed as you are.
>>>>>>>
>>>>>>
>>>>>> With the new modifications my build system now covers 50% of the use
>>>>>> case you have described in your email. Please see the link
>>>>>> https://github.com/HassanSajjad-302/Example12/blob/main/hmake.cpp.
>>>>>>
>>>>>> This might appear big, but without comments, I think this is quite
>>>>>> concise for what it is achieving. This covers the scenarios where in a
>>>>>> mono-repo style project a dependency builds with different flags than the
>>>>>> consumer e.g. when the dependency supports only the c++20 but the project
>>>>>> is using c++23 or e.g. a dependency has to be built by a different compiler
>>>>>> for performance reasons.
>>>>>>
>>>>>> However, this is a manual approach, and we can not detect whether the
>>>>>> ifc files from a prebuilt target are compatible for use in a target we are
>>>>>> building or not.
>>>>>>
>>>>>> This paper
>>>>>> https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2581r2.pdf
>>>>>> provides good guidance on how this can be achieved in the build system.
>>>>>>
>>>>>> It is mentioned in the paper
>>>>>>
>>>>>> Those categories, however, represent higher-level semantics. It is
>>>>>>> not the case that the build system can introspect the command line used to
>>>>>>> produce a BMI on another project and decide which arguments fall on which
>>>>>>> of the categories. They need to be authored by the engineer maintaining the
>>>>>>> build system.
>>>>>>
>>>>>>
>>>>>> This paper proposes that build systems should offer a mechanism to
>>>>>>> identify which of the options used in a translation unit are a Local
>>>>>>> Preprocessor Argument.
>>>>>>
>>>>>>
>>>>>> It is also mentioned
>>>>>>
>>>>>> The scope of compatibility for consuming an existing built module
>>>>>>> interface file is defined by the union of the Basic Toolchain Configuration
>>>>>>> Arguments and the BMI-Sensitive arguments. And it should explicitly exclude
>>>>>>> Local Preprocessor Arguments. Those arguments are the ones that should be
>>>>>>> used to define the compatibility identifier.
>>>>>>
>>>>>>
>>>>>> The build system should take the compiler invocation for a
>>>>>>> translation unit, remove the Local Preprocessor Arguments and the reference
>>>>>>> to the specific translation unit and invoke the compiler in that mode in
>>>>>>> order to obtain the compatibility identifier for BMIs produced and consumed
>>>>>>> by that translation unit.
>>>>>>
>>>>>>
>>>>>> I was thinking that instead of changing the build system to have an
>>>>>> API to allow the user to specify the flags of different categories this job
>>>>>> is also given to the compiler. So, the compiler when invoked with a
>>>>>> specific option with the full compile command excluding input files,
>>>>>> outputs:
>>>>>> 1) Concatenation of Basic Toolchain Configuration Arguments and
>>>>>> BMI-Sensitive Arguments.
>>>>>> 2) Compiler Definitions
>>>>>> 3) Include Directories
>>>>>> 4) Other Arguments
>>>>>> 5) Compatibility Identifier
>>>>>>
>>>>>> Following the example mentioned in the paper, I broke Local
>>>>>> Preprocessor Arguments into 2 and 3. Can there be more besides these? Also,
>>>>>> maybe 5 can be left out and 1 can be used instead.
>>>>>>
>>>>>> Exempting the build-system of the responsibility of categorization of
>>>>>> different flags keeps the user-facing API of the build-system more concise
>>>>>> as the user does not need to specify the compiler flags in 4 different
>>>>>> categories instead of 1.
>>>>>>
>>>>>> It seems that there is acceptance regarding the dynamic loading of
>>>>>> the compiler shared-library and the mentioned API. Would it be appropriate
>>>>>> to prepare a formal proposal for consideration?
>>>>>>
>>>>>> Best,
>>>>>> Hassan Sajjad
>>>>>>
>>>>>> On Sat, Aug 26, 2023 at 3:48 AM Bret Brown <mail_at_[hidden]>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Hassan,
>>>>>>>
>>>>>>> First of all, great job on your writeups. They are very well
>>>>>>> considered. I appreciate how much effort you're putting into communicating
>>>>>>> your ideas and developing better C++ tools.
>>>>>>>
>>>>>>> I have one concern with the last design you shared with us. As far
>>>>>>> As I understand currently, we expect that both imported header units and
>>>>>>> public module interfaces will need to be parsed multiple times per build
>>>>>>> graph. This has been discussed in the ISO Tooling Study Group a bit
>>>>>>> already, so it's not a new isea. In short, it's expected that in typical
>>>>>>> scenarios, the build system would need to model multiple parses of a
>>>>>>> particular importable entity. This is because, unfortunately, some
>>>>>>> compilation options of the importing translation unit will need to be used
>>>>>>> to parse the imported translation unit.
>>>>>>>
>>>>>>> I think your design could work, though I expect the node identities
>>>>>>> of the imported entities need to be tweaked to better model a combination
>>>>>>> of the file and the flags used to parse the file. The build system would
>>>>>>> ideally know when to just reuse a particular parse of a particular entity,
>>>>>>> especially when optimizing for build speed as you are.
>>>>>>>
>>>>>>> Daniel is hinting at this challenge upthread in his reference to
>>>>>>> preprocessor state. You can read more about that in his ISO papers that
>>>>>>> have been published in the last year, including
>>>>>>> https://wg22.link/p2898. Daniel also elaborated on this with
>>>>>>> diagrams in his C++Now talk this spring. You can find it here:
>>>>>>> https://youtu.be/_LGR0U5Opdg?si=H92b0eyGQd8Vsr0v. Note that Daniel
>>>>>>> said upthread that his concerns are satisfied by some new ideas that were
>>>>>>> discussed in Varna. But the need for build systems to model multiple parses
>>>>>>> per imported interface remains given current understanding.
>>>>>>>
>>>>>>> I suspect certain codebases that are carefully governed might have
>>>>>>> exactly one parse per imported unit, but a build system that wants to
>>>>>>> support things like existing package management ecosystems, among other
>>>>>>> examples, would need to consider a more complicated approach.
>>>>>>>
>>>>>>> Bret Brown
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 24, 2023, 23:53 Hassan Sajjad via SG15 <
>>>>>>> sg15_at_[hidden]> wrote:
>>>>>>>
>>>>>>>> Thanks for reaching out. I have gone through the thread and I am
>>>>>>>>> still a little confused probably because the approach is quite
>>>>>>>>> different
>>>>>>>>> from what we have been used to.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Thank you so much for commenting. Yes, it is a quite different
>>>>>>>> approach.
>>>>>>>>
>>>>>>>> It seems that there are two aspects that you are proposing: a)
>>>>>>>>> the
>>>>>>>>> way the build system describes the build rules; b) how we can
>>>>>>>>> efficiently translate the build rules into action. IIUC in some
>>>>>>>>> languages (duck typing predominantly) they use the language itself
>>>>>>>>> to
>>>>>>>>> describe the rules, too.
>>>>>>>>>
>>>>>>>>
>>>>>>>> I just wanted to let you know that I am not commenting on what
>>>>>>>> language the build-system should use. I am just proposing a new approach
>>>>>>>> with the potential of a good speed-up. I am only partially aware of the
>>>>>>>> other build-systems design, specifically the design around C++20 modules /
>>>>>>>> header-units. Because this is a different approach, non-trivial changes
>>>>>>>> might be needed in other build-systems to support it.
>>>>>>>>
>>>>>>>> The second part is more interesting to me. AFAICT your approach
>>>>>>>>> inverts the build graph /somehow/. Is the aim to provide a "symbol
>>>>>>>>> registry" which upon of the symbol to compile the relevant
>>>>>>>>> subgraph? I'd
>>>>>>>>> appreciate if you could describe more verbosely (with examples
>>>>>>>>> maybe)
>>>>>>>>> how the process works.
>>>>>>>>>
>>>>>>>>
>>>>>>>> My pleasures.
>>>>>>>>
>>>>>>>> Please see the attached document. In it, I analyze how I plan to
>>>>>>>> add support for this in my build-system. I also provide an example. Based
>>>>>>>> on the analysis, I am confident in my ability to implement this within one
>>>>>>>> month if it's approved and a compiler with an API becomes available. While
>>>>>>>> I currently see no issues, I acknowledge the possibility of limitations and
>>>>>>>> potential errors. I am currently awaiting feedback from other stakeholders
>>>>>>>> before proceeding.
>>>>>>>>
>>>>>>>> A slight correction in my above email. I mentioned that my
>>>>>>>> build-system now is limitation-free with the adoption of the new consensus.
>>>>>>>> Well, it is for a clean build. But there is a small bug for rebuild which
>>>>>>>> will be fixed soon.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Hassan Sajjad
>>>>>>>>
>>>>>>>> On Tue, Aug 22, 2023 at 8:49 PM Vassil Vassilev <
>>>>>>>> v.g.vassilev_at_[hidden]> wrote:
>>>>>>>>
>>>>>>>>> Hi Hassan,
>>>>>>>>>
>>>>>>>>> Thanks for reaching out. I have gone through the thread and I
>>>>>>>>> am
>>>>>>>>> still a little confused probably because the approach is quite
>>>>>>>>> different
>>>>>>>>> from what we have been used to.
>>>>>>>>>
>>>>>>>>> It seems that there are two aspects that you are proposing: a)
>>>>>>>>> the
>>>>>>>>> way the build system describes the build rules; b) how we can
>>>>>>>>> efficiently translate the build rules into action. IIUC in some
>>>>>>>>> languages (duck typing predominantly) they use the language itself
>>>>>>>>> to
>>>>>>>>> describe the rules, too.
>>>>>>>>>
>>>>>>>>> The second part is more interesting to me. AFAICT your approach
>>>>>>>>> inverts the build graph /somehow/. Is the aim to provide a "symbol
>>>>>>>>> registry" which upon of the symbol to compile the relevant
>>>>>>>>> subgraph? I'd
>>>>>>>>> appreciate if you could describe more verbosely (with examples
>>>>>>>>> maybe)
>>>>>>>>> how the process works.
>>>>>>>>>
>>>>>>>>> Best, Vassil
>>>>>>>>>
>>>>>>>>> On 7/29/23 3:28 PM, Hassan Sajjad via SG15 wrote:
>>>>>>>>> > Hi.
>>>>>>>>> >
>>>>>>>>> > I will like to showcase my build-system HMake
>>>>>>>>> > https://github.com/HassanSajjad-302/HMake.
>>>>>>>>> >
>>>>>>>>> > It has C++20 modules and header-units support. It also supports
>>>>>>>>> > drop-in header-files to header-units replacement.
>>>>>>>>> >
>>>>>>>>> > With it, with MSVC I compiled an SFML example with C++ 20
>>>>>>>>> header-units.
>>>>>>>>> > https://github.com/HassanSajjad-302/SFML
>>>>>>>>> >
>>>>>>>>> > HMake however has a flaw and there is no easy solution for it.
>>>>>>>>> To fix
>>>>>>>>> > this, I will like to propose a new way to compile source files.
>>>>>>>>> >
>>>>>>>>> https://github.com/HassanSajjad-302/HMake/wiki/HMake-Flaw-And-Its-Possible-Fixes
>>>>>>>>> >
>>>>>>>>> > I am very confident that the adoption of this will result
>>>>>>>>> in flawless
>>>>>>>>> > module and header-unit support in HMake which will translate to
>>>>>>>>> a very
>>>>>>>>> > good user experience while converting their code base to C++20
>>>>>>>>> modules.
>>>>>>>>> >
>>>>>>>>> > Please share your thoughts.
>>>>>>>>> >
>>>>>>>>> > Best,
>>>>>>>>> > Hassan Sajjad
>>>>>>>>> >
>>>>>>>>> > _______________________________________________
>>>>>>>>> > SG15 mailing list
>>>>>>>>> > SG15_at_[hidden]
>>>>>>>>> > https://lists.isocpp.org/mailman/listinfo.cgi/sg15
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>> SG15 mailing list
>>>>>>>> SG15_at_[hidden]
>>>>>>>> https://lists.isocpp.org/mailman/listinfo.cgi/sg15
>>>>>>>>
>>>>>>>