If the flattened file format doesn't need to be parse-able by third party tools, then the compiler can do whatever it wants with the text format, so long as the compiler itself can consume the resulting file on a different machine. Abandoning the parse-able requirement means that you could instead do...

#pragma module begin(foo)
/* base64 encoded implicitly generated BMI */
#pragma module end(foo)

No cross-vendor spec would be necessary for such an approach.

> -----Original Message-----
> From: Modules <modules-bounces_at_lists.isocpp.org> On Behalf Of Gabriel
> Dos Reis via Modules
> Sent: Tuesday, March 5, 2019 10:19 AM
> To: modules_at_[hidden]
> Cc: Gabriel Dos Reis <gdr_at_microsoft.com>; WG21 Tooling Study Group SG15
> <tooling_at_[hidden]>; Nathan Sidwell <nathan_at_acm.org>
> Subject: [EXTERNAL] Re: [isocpp-modules] Round2: Path to modules with old
> bad build systems
>
>
>
> > On Mar 5, 2019, at 8:12 AM, Ben Craig <ben.craig_at_ni.com> wrote:
> >
> > I will concede that static analysis tools and other tools that try to parse C++
> probably don't need a textual inclusion format, since they most likely need to
> be able to parse and understand the pragmas anyway. If module mapping is
> sufficiently straightforward, then those tools can do module lookup the same
> as a compiler. Those tools already need to do include lookup in the same
> way that compilers do.
>
> I haven’t yet seen a mapping that isn’t straightforward :-)
>
> >
> > I think the textual inclusion format will still be very useful to distribution
> and caching tools though, as they don't need to understand the code. Those
> tools frequently lean on the compiler's preprocessor today, and don't know
> how to do include lookups.
>
> I don’t know there is an actual spec of such flattened text file that isn’t at
> least as involved as the module spec - if not more involved.
>
> I worry that we may be creating a more complex problem than the issue we
> are set to address.
>
> — Gaby
>
> >
> >> -----Original Message-----
> >> From: Nathan Sidwell <nathanmsidwell_at_gmail.com> On Behalf Of Nathan
> >> Sidwell
> >> Sent: Tuesday, March 5, 2019 6:04 AM
> >> To: Ben Craig <ben.craig_at_ni.com>; modules_at_lists.isocpp.org; WG21
> >> Tooling Study Group SG15 <tooling_at_[hidden]rg>
> >> Subject: [EXTERNAL] Re: [isocpp-modules] Round2: Path to modules with
> >> old bad build systems
> >>
> >>> On 3/4/19 10:02 AM, Ben Craig wrote:
> >>> I do mean textual inclusion, though I can be convinced otherwise.
> >>> Textual
> >> inclusion (with extra generated pragmas) should make it much easier
> >> to keep tools like distcc and cppcheck happy in the short term. I
> >> suspect that those tools don't want to crack open a BMI to figure out
> >> which other BMIs need to be found.
> >>>
> >>> Tools that (think they can) parse C++ will still need to understand
> >>> these
> >> pragmas in order to provide the right macro, visibility, and
> >> reachability behaviors, so some work will still be required on their
> >> part, but at least they won't need to understand new binary formats.
> >>
> >> Correct, tools consuming such #pragma-marked flattened source will
> >> need to understand modules at a fundamental level. As such, why not
> >> implement the same mechanisms to find module source as the compiler?
> >> That'll give them more information to perform code analysis with.
> >>
> >> nathan
> >>
> >>>>> On 3/2/19 1:03 PM, Ben Craig wrote:
> >>>>>
> >>>>> Some quick notes on this implementation strategy:
> >>>>> * Uses TEXTUAL inclusion
> >>>>> * Compiler assumes that the build system knows nothing of BMIs
> >>>>> * Compiler needs to be able to do module mapping with minimal
> >>>>> input from users.
> >>>>
> >>>> Do you literally mean textual inclusion or do you really mean
> >>>> dynamically produce an internal-only BMI object?
> >>>>
> >>>> nathan
> >>>>
> >>>> --
> >>>> Nathan Sidwell
> >>
> >>
> >> --
> >> Nathan Sidwell
> > _______________________________________________
> > Modules mailing list
> > Modules_at_[hidden]
> > Subscription:
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__nam06.safelinks.p
> > rotection.outlook.com_-3Furl-3Dhttp-253A-252F-252Flists.isocpp.org-252
> > Fmailman-252Flistinfo.cgi-252Fmodules-26amp-3Bdata-3D02-257C01-
> 257Cgdr
> > -2540microsoft.com-257Ca5f3d89c3d7d4346707f08d6a1854be8-
> 257C72f988bf86
> > f141af91ab2d7cd011db47-257C1-257C0-257C636873991217371177-26amp-
> 3Bsdat
> > a-3DRwfoGFvU8Df-252F2llvXjbpAiIIG-252B0RxL4Su8ewJeoypCw-253D-
> 26amp-3Br
> > eserved-
> 3D0&d=DwIGaQ&c=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y
> > 8mub81SfUi-
> UCZRX0Vl1g&m=AdhXKMaBKONGion_X4H9mmw08XjCtqqGoeW9bs7cVuQ&
> s=
> > ofQHenjonX7d85yaK2-6F3x_myxVyMcitwJo9rUo8gI&e=
> > Link to this post:
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__nam06.safelinks.p
> > rotection.outlook.com_-3Furl-3Dhttp-253A-252F-252Flists.isocpp.org-252
> > Fmodules-252F2019-252F03-252F0145.php-26amp-3Bdata-3D02-257C01-
> 257Cgdr
> > -2540microsoft.com-257Ca5f3d89c3d7d4346707f08d6a1854be8-
> 257C72f988bf86
> > f141af91ab2d7cd011db47-257C1-257C0-257C636873991217371177-26amp-
> 3Bsdat
> > a-3DOuOtS6D49Sy5EhRl8VUpv-252ByE5esxGZ7dvPvf-252Be1zr1c-253D-
> 26amp-3Br
> > eserved-
> 3D0&d=DwIGaQ&c=I_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y
> > 8mub81SfUi-
> UCZRX0Vl1g&m=AdhXKMaBKONGion_X4H9mmw08XjCtqqGoeW9bs7cVuQ&
> s=
> > sg8E8Gx-jIbRDIRalv20OlbIkHaNJbyC3TwX8-ueRDU&e=
> _______________________________________________
> Modules mailing list
> Modules_at_[hidden]
> Subscription: https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__lists.isocpp.org_mailman_listinfo.cgi_modules&d=DwIGaQ&c=I_0YwoK
> y7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y8mub81SfUi-
> UCZRX0Vl1g&m=AdhXKMaBKONGion_X4H9mmw08XjCtqqGoeW9bs7cVuQ&
> s=gmG6dYi-y2pNIFyGrUsRGlDKCKRKWf4WQQlvzQDWM-s&e=
> Link to this post: https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__lists.isocpp.org_modules_2019_03_0147.php&d=DwIGaQ&c=I_0YwoKy
> 7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA&r=y8mub81SfUi-
> UCZRX0Vl1g&m=AdhXKMaBKONGion_X4H9mmw08XjCtqqGoeW9bs7cVuQ&
> s=51GVUSm-w3aQdhBiPoTvrNnOE6QlopXYmzwNAYDSU5o&e=