Date: Tue, 5 Mar 2019 17:03:45 +0000
If the flattened file format doesn’t need to be parse-able by third party tools, then why go through the trouble of generating it in the first place - requiring additional language extensions at the source level?
— Gaby
> On Mar 5, 2019, at 8:43 AM, Ben Craig <ben.craig_at_[hidden]om> wrote:
>
> If the flattened file format doesn't need to be parse-able by third party tools, then the compiler can do whatever it wants with the text format, so long as the compiler itself can consume the resulting file on a different machine. Abandoning the parse-able requirement means that you could instead do...
>
> #pragma module begin(foo)
> /* base64 encoded implicitly generated BMI */
> #pragma module end(foo)
>
> No cross-vendor spec would be necessary for such an approach.
>
>> -----Original Message-----
>> From: Modules <modules-bounces_at_lists.isocpp.org> On Behalf Of Gabriel
>> Dos Reis via Modules
>> Sent: Tuesday, March 5, 2019 10:19 AM
>> To: modules_at_lists.isocpp.org
>> Cc: Gabriel Dos Reis <gdr_at_microsoft.com>; WG21 Tooling Study Group SG15
>> <tooling_at_open-std.org>; Nathan Sidwell <nathan_at_acm.org>
>> Subject: [EXTERNAL] Re: [isocpp-modules] Round2: Path to modules with old
>> bad build systems
>>
>>
>>
>>> On Mar 5, 2019, at 8:12 AM, Ben Craig <ben.craig_at_ni.com> wrote:
>>>
>>> I will concede that static analysis tools and other tools that try to parse C++
>> probably don't need a textual inclusion format, since they most likely need to
>> be able to parse and understand the pragmas anyway. If module mapping is
>> sufficiently straightforward, then those tools can do module lookup the same
>> as a compiler. Those tools already need to do include lookup in the same
>> way that compilers do.
>>
>> I haven’t yet seen a mapping that isn’t straightforward :-)
>>
>>>
>>> I think the textual inclusion format will still be very useful to distribution
>> and caching tools though, as they don't need to understand the code. Those
>> tools frequently lean on the compiler's preprocessor today, and don't know
>> how to do include lookups.
>>
>> I don’t know there is an actual spec of such flattened text file that isn’t at
>> least as involved as the module spec - if not more involved.
>>
>> I worry that we may be creating a more complex problem than the issue we
>> are set to address.
>>
>> — Gaby
>>
>>>
>>>> -----Original Message-----
>>>> From: Nathan Sidwell <nathanmsidwell_at_gmail.com> On Behalf Of Nathan
>>>> Sidwell
>>>> Sent: Tuesday, March 5, 2019 6:04 AM
>>>> To: Ben Craig <ben.craig_at_[hidden]>; modules_at_lists.isocpp.org; WG21
>>>> Tooling Study Group SG15 <tooling_at_open-std.org>
>>>> Subject: [EXTERNAL] Re: [isocpp-modules] Round2: Path to modules with
>>>> old bad build systems
>>>>
>>>>> On 3/4/19 10:02 AM, Ben Craig wrote:
>>>>> I do mean textual inclusion, though I can be convinced otherwise.
>>>>> Textual
>>>> inclusion (with extra generated pragmas) should make it much easier
>>>> to keep tools like distcc and cppcheck happy in the short term. I
>>>> suspect that those tools don't want to crack open a BMI to figure out
>>>> which other BMIs need to be found.
>>>>>
>>>>> Tools that (think they can) parse C++ will still need to understand
>>>>> these
>>>> pragmas in order to provide the right macro, visibility, and
>>>> reachability behaviors, so some work will still be required on their
>>>> part, but at least they won't need to understand new binary formats.
>>>>
>>>> Correct, tools consuming such #pragma-marked flattened source will
>>>> need to understand modules at a fundamental level. As such, why not
>>>> implement the same mechanisms to find module source as the compiler?
>>>> That'll give them more information to perform code analysis with.
>>>>
>>>> nathan
>>>>
>>>>>>> On 3/2/19 1:03 PM, Ben Craig wrote:
>>>>>>>
>>>>>>> Some quick notes on this implementation strategy:
>>>>>>> * Uses TEXTUAL inclusion
>>>>>>> * Compiler assumes that the build system knows nothing of BMIs
>>>>>>> * Compiler needs to be able to do module mapping with minimal
>>>>>>> input from users.
>>>>>>
>>>>>> Do you literally mean textual inclusion or do you really mean
>>>>>> dynamically produce an internal-only BMI object?
>>>>>>
>>>>>> nathan
>>>>>>
>>>>>> --
>>>>>> Nathan Sidwell
>>>>
>>>>
>>>> --
>>>> Nathan Sidwell
>>> _______________________________________________
>>> Modules mailing list
>>> Modules_at_lists.isocpp.org
>>> Subscription:
>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__nam06.safelinks.p&data=02%7C01%7Cgdr%40microsoft.com%7Cb10248c40d104406a58008d6a1899f80%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636874009800024740&sdata=p7A5pBCX07tWsIhQL6SmfcP6iunVFWqvUY1vKEw69t8%3D&reserved=0
>>> rotection.outlook.com_-3Furl-3Dhttp-253A-252F-252Flists.isocpp.org-252
>>> Fmailman-252Flistinfo.cgi-252Fmodules-26amp-3Bdata-3D02-257C01-
>> 257Cgdr
>>> -2540microsoft.com-257Ca5f3d89c3d7d4346707f08d6a1854be8-
>> 257C72f988bf86
>>> f141af91ab2d7cd011db47-257C1-257C0-257C636873991217371177-26amp-
>> 3Bsdat
>>> a-3DRwfoGFvU8Df-252F2llvXjbpAiIIG-252B0RxL4Su8ewJeoypCw-253D-
>> 26amp-3Br
>>> eserved-
>> 3D0&d=DwIGaQ&c=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y
>>> 8mub81SfUi-
>> UCZRX0Vl1g&m=AdhXKMaBKONGion_X4H9mmw08XjCtqqGoeW9bs7cVuQ&
>> s=
>>> ofQHenjonX7d85yaK2-6F3x_myxVyMcitwJo9rUo8gI&e=
>>> Link to this post:
>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__nam06.safelinks.p&data=02%7C01%7Cgdr%40microsoft.com%7Cb10248c40d104406a58008d6a1899f80%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636874009800024740&sdata=p7A5pBCX07tWsIhQL6SmfcP6iunVFWqvUY1vKEw69t8%3D&reserved=0
>>> rotection.outlook.com_-3Furl-3Dhttp-253A-252F-252Flists.isocpp.org-252
>>> Fmodules-252F2019-252F03-252F0145.php-26amp-3Bdata-3D02-257C01-
>> 257Cgdr
>>> -2540microsoft.com-257Ca5f3d89c3d7d4346707f08d6a1854be8-
>> 257C72f988bf86
>>> f141af91ab2d7cd011db47-257C1-257C0-257C636873991217371177-26amp-
>> 3Bsdat
>>> a-3DOuOtS6D49Sy5EhRl8VUpv-252ByE5esxGZ7dvPvf-252Be1zr1c-253D-
>> 26amp-3Br
>>> eserved-
>> 3D0&d=DwIGaQ&c=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y
>>> 8mub81SfUi-
>> UCZRX0Vl1g&m=AdhXKMaBKONGion_X4H9mmw08XjCtqqGoeW9bs7cVuQ&
>> s=
>>> sg8E8Gx-jIbRDIRalv20OlbIkHaNJbyC3TwX8-ueRDU&e=
>> _______________________________________________
>> Modules mailing list
>> Modules_at_[hidden]
>> Subscription: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-&data=02%7C01%7Cgdr%40microsoft.com%7Cb10248c40d104406a58008d6a1899f80%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636874009800024740&sdata=vJxYGAlfPqfe%2BB6HOBD4JCjOZSEP9Sc1fKDJt8ehEu4%3D&reserved=0
>> 3A__lists.isocpp.org_mailman_listinfo.cgi_modules&d=DwIGaQ&c=I_0YwoK
>> y7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y8mub81SfUi-
>> UCZRX0Vl1g&m=AdhXKMaBKONGion_X4H9mmw08XjCtqqGoeW9bs7cVuQ&
>> s=gmG6dYi-y2pNIFyGrUsRGlDKCKRKWf4WQQlvzQDWM-s&e=
>> Link to this post: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-&data=02%7C01%7Cgdr%40microsoft.com%7Cb10248c40d104406a58008d6a1899f80%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636874009800034748&sdata=123o9Z3tNcVY%2FTsR6VbHITcMCHsPMk3459OhqCqpGwk%3D&reserved=0
>> 3A__lists.isocpp.org_modules_2019_03_0147.php&d=DwIGaQ&c=I_0YwoKy
>> 7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y8mub81SfUi-
>> UCZRX0Vl1g&m=AdhXKMaBKONGion_X4H9mmw08XjCtqqGoeW9bs7cVuQ&
>> s=51GVUSm-w3aQdhBiPoTvrNnOE6QlopXYmzwNAYDSU5o&e=
— Gaby
> On Mar 5, 2019, at 8:43 AM, Ben Craig <ben.craig_at_[hidden]om> wrote:
>
> If the flattened file format doesn't need to be parse-able by third party tools, then the compiler can do whatever it wants with the text format, so long as the compiler itself can consume the resulting file on a different machine. Abandoning the parse-able requirement means that you could instead do...
>
> #pragma module begin(foo)
> /* base64 encoded implicitly generated BMI */
> #pragma module end(foo)
>
> No cross-vendor spec would be necessary for such an approach.
>
>> -----Original Message-----
>> From: Modules <modules-bounces_at_lists.isocpp.org> On Behalf Of Gabriel
>> Dos Reis via Modules
>> Sent: Tuesday, March 5, 2019 10:19 AM
>> To: modules_at_lists.isocpp.org
>> Cc: Gabriel Dos Reis <gdr_at_microsoft.com>; WG21 Tooling Study Group SG15
>> <tooling_at_open-std.org>; Nathan Sidwell <nathan_at_acm.org>
>> Subject: [EXTERNAL] Re: [isocpp-modules] Round2: Path to modules with old
>> bad build systems
>>
>>
>>
>>> On Mar 5, 2019, at 8:12 AM, Ben Craig <ben.craig_at_ni.com> wrote:
>>>
>>> I will concede that static analysis tools and other tools that try to parse C++
>> probably don't need a textual inclusion format, since they most likely need to
>> be able to parse and understand the pragmas anyway. If module mapping is
>> sufficiently straightforward, then those tools can do module lookup the same
>> as a compiler. Those tools already need to do include lookup in the same
>> way that compilers do.
>>
>> I haven’t yet seen a mapping that isn’t straightforward :-)
>>
>>>
>>> I think the textual inclusion format will still be very useful to distribution
>> and caching tools though, as they don't need to understand the code. Those
>> tools frequently lean on the compiler's preprocessor today, and don't know
>> how to do include lookups.
>>
>> I don’t know there is an actual spec of such flattened text file that isn’t at
>> least as involved as the module spec - if not more involved.
>>
>> I worry that we may be creating a more complex problem than the issue we
>> are set to address.
>>
>> — Gaby
>>
>>>
>>>> -----Original Message-----
>>>> From: Nathan Sidwell <nathanmsidwell_at_gmail.com> On Behalf Of Nathan
>>>> Sidwell
>>>> Sent: Tuesday, March 5, 2019 6:04 AM
>>>> To: Ben Craig <ben.craig_at_[hidden]>; modules_at_lists.isocpp.org; WG21
>>>> Tooling Study Group SG15 <tooling_at_open-std.org>
>>>> Subject: [EXTERNAL] Re: [isocpp-modules] Round2: Path to modules with
>>>> old bad build systems
>>>>
>>>>> On 3/4/19 10:02 AM, Ben Craig wrote:
>>>>> I do mean textual inclusion, though I can be convinced otherwise.
>>>>> Textual
>>>> inclusion (with extra generated pragmas) should make it much easier
>>>> to keep tools like distcc and cppcheck happy in the short term. I
>>>> suspect that those tools don't want to crack open a BMI to figure out
>>>> which other BMIs need to be found.
>>>>>
>>>>> Tools that (think they can) parse C++ will still need to understand
>>>>> these
>>>> pragmas in order to provide the right macro, visibility, and
>>>> reachability behaviors, so some work will still be required on their
>>>> part, but at least they won't need to understand new binary formats.
>>>>
>>>> Correct, tools consuming such #pragma-marked flattened source will
>>>> need to understand modules at a fundamental level. As such, why not
>>>> implement the same mechanisms to find module source as the compiler?
>>>> That'll give them more information to perform code analysis with.
>>>>
>>>> nathan
>>>>
>>>>>>> On 3/2/19 1:03 PM, Ben Craig wrote:
>>>>>>>
>>>>>>> Some quick notes on this implementation strategy:
>>>>>>> * Uses TEXTUAL inclusion
>>>>>>> * Compiler assumes that the build system knows nothing of BMIs
>>>>>>> * Compiler needs to be able to do module mapping with minimal
>>>>>>> input from users.
>>>>>>
>>>>>> Do you literally mean textual inclusion or do you really mean
>>>>>> dynamically produce an internal-only BMI object?
>>>>>>
>>>>>> nathan
>>>>>>
>>>>>> --
>>>>>> Nathan Sidwell
>>>>
>>>>
>>>> --
>>>> Nathan Sidwell
>>> _______________________________________________
>>> Modules mailing list
>>> Modules_at_lists.isocpp.org
>>> Subscription:
>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__nam06.safelinks.p&data=02%7C01%7Cgdr%40microsoft.com%7Cb10248c40d104406a58008d6a1899f80%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636874009800024740&sdata=p7A5pBCX07tWsIhQL6SmfcP6iunVFWqvUY1vKEw69t8%3D&reserved=0
>>> rotection.outlook.com_-3Furl-3Dhttp-253A-252F-252Flists.isocpp.org-252
>>> Fmailman-252Flistinfo.cgi-252Fmodules-26amp-3Bdata-3D02-257C01-
>> 257Cgdr
>>> -2540microsoft.com-257Ca5f3d89c3d7d4346707f08d6a1854be8-
>> 257C72f988bf86
>>> f141af91ab2d7cd011db47-257C1-257C0-257C636873991217371177-26amp-
>> 3Bsdat
>>> a-3DRwfoGFvU8Df-252F2llvXjbpAiIIG-252B0RxL4Su8ewJeoypCw-253D-
>> 26amp-3Br
>>> eserved-
>> 3D0&d=DwIGaQ&c=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y
>>> 8mub81SfUi-
>> UCZRX0Vl1g&m=AdhXKMaBKONGion_X4H9mmw08XjCtqqGoeW9bs7cVuQ&
>> s=
>>> ofQHenjonX7d85yaK2-6F3x_myxVyMcitwJo9rUo8gI&e=
>>> Link to this post:
>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__nam06.safelinks.p&data=02%7C01%7Cgdr%40microsoft.com%7Cb10248c40d104406a58008d6a1899f80%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636874009800024740&sdata=p7A5pBCX07tWsIhQL6SmfcP6iunVFWqvUY1vKEw69t8%3D&reserved=0
>>> rotection.outlook.com_-3Furl-3Dhttp-253A-252F-252Flists.isocpp.org-252
>>> Fmodules-252F2019-252F03-252F0145.php-26amp-3Bdata-3D02-257C01-
>> 257Cgdr
>>> -2540microsoft.com-257Ca5f3d89c3d7d4346707f08d6a1854be8-
>> 257C72f988bf86
>>> f141af91ab2d7cd011db47-257C1-257C0-257C636873991217371177-26amp-
>> 3Bsdat
>>> a-3DOuOtS6D49Sy5EhRl8VUpv-252ByE5esxGZ7dvPvf-252Be1zr1c-253D-
>> 26amp-3Br
>>> eserved-
>> 3D0&d=DwIGaQ&c=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y
>>> 8mub81SfUi-
>> UCZRX0Vl1g&m=AdhXKMaBKONGion_X4H9mmw08XjCtqqGoeW9bs7cVuQ&
>> s=
>>> sg8E8Gx-jIbRDIRalv20OlbIkHaNJbyC3TwX8-ueRDU&e=
>> _______________________________________________
>> Modules mailing list
>> Modules_at_[hidden]
>> Subscription: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-&data=02%7C01%7Cgdr%40microsoft.com%7Cb10248c40d104406a58008d6a1899f80%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636874009800024740&sdata=vJxYGAlfPqfe%2BB6HOBD4JCjOZSEP9Sc1fKDJt8ehEu4%3D&reserved=0
>> 3A__lists.isocpp.org_mailman_listinfo.cgi_modules&d=DwIGaQ&c=I_0YwoK
>> y7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y8mub81SfUi-
>> UCZRX0Vl1g&m=AdhXKMaBKONGion_X4H9mmw08XjCtqqGoeW9bs7cVuQ&
>> s=gmG6dYi-y2pNIFyGrUsRGlDKCKRKWf4WQQlvzQDWM-s&e=
>> Link to this post: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-&data=02%7C01%7Cgdr%40microsoft.com%7Cb10248c40d104406a58008d6a1899f80%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636874009800034748&sdata=123o9Z3tNcVY%2FTsR6VbHITcMCHsPMk3459OhqCqpGwk%3D&reserved=0
>> 3A__lists.isocpp.org_modules_2019_03_0147.php&d=DwIGaQ&c=I_0YwoKy
>> 7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y8mub81SfUi-
>> UCZRX0Vl1g&m=AdhXKMaBKONGion_X4H9mmw08XjCtqqGoeW9bs7cVuQ&
>> s=51GVUSm-w3aQdhBiPoTvrNnOE6QlopXYmzwNAYDSU5o&e=
Received on 2019-03-05 18:03:48