C++ Logo

sg15

Advanced search

Re: Proposal for module metadata format to be used by the std library and others

From: Tom Honermann <tom_at_[hidden]>
Date: Wed, 13 Dec 2023 17:58:35 -0500
On 12/13/23 4:28 PM, Bret Brown via SG15 wrote:
> I'm confused about including compile options for multiple front ends.
> We should be describing the assumptions baked into how the standard
> library was built. The library presumably was built with one specific
> compiler.
Built with one, yes, but consumable by many. For example, on a typical
Linux machine, gcc and both the clang and clang-cl drivers can be used
with the provided libstdc++ installation. Likewise, MSVC, Clang, oneAPI,
and other compilers can be used with Microsoft's standard library on
Windows.
>
> Note that parses often do change in relevant ways depending on which
> compiler is doing the parsing. Also which flags, like language
> standard selection, are appropriate could vary depending on the
> version of a given toolchain. And certainly not every standard library
> will support every single possible tool that might parse the std module.
Absolutely. The goal of this metadata is to enable the construction of
BMIs needed for a given tool chain.
>
> I'm not opposed to these files suggesting parse rules for use by
> specific alternative tools or toolchains, but that feels more like an
> extension for now. As long as we're confident that nonstandard content
> can be added to these files (like a vendor section), we can keep it
> simple for now. If this turns out to be valuable or even essential, we
> can revisit.

I think this is fundamental to the existing ecosystem. Tool chains are
more interwoven than they used to be.

Tom.

>
> Bret
>
> On Wed, Dec 13, 2023, 14:38 Ben Boeckel via SG15
> <sg15_at_[hidden]> wrote:
>
> On Wed, Dec 13, 2023 at 13:45:45 -0500, Tom Honermann via SG15 wrote:
> > On 12/12/23 4:56 PM, Daniel Ruoso via SG15 wrote:
> > > * Module discovery should be subordinate to ABI decisions
> having been
> > > made already. The outcome is that we don't expect the module
> metadata
> > > to be used to discover which ABI settings are available for a
> given
> > > standard library or how to choose between them. The way that this
> > > manifests is that, following the lead from P2577R2, we expect the
> > > libraries to ship those metadata files on a one-to-one mapping
> with
> > > the library binary used as input to the linker.
> > I think someone mentioned it yesterday, but this will presumably
> have to
> > account for multilib libraries in some way.
>
> I assume this is "universal" binaries on Apple or the (nigh-unused)
> FatELF where multiple architectures live inside one artifact
> rather than
> Red Hat "multilib" /usr/lib vs /usr/lib64 or Debian "multiarch"
> /usr/$triplet. I'm not sure where `.xcframework` artifacts fall here
> off-hand.
>
> > > * Relative file paths: Any non-absolute path described in
> this file
> > > will be presumed to have the directory where the metadata file
> was
> > > found as the base for the lookup.
> > I think it will be necessary to support paths that are relative
> to some
> > other parameterized location to accommodate dependencies on
> header files
> > or modules provided by other projects/packages. This would help to
> > reduce the otherwise common practice of a build system having to
> > concatenate include paths for essentially every project it knows
> about
> > when building any package it knows about.
>
> Yeah, that gets really close to package management… Relocatable
> installs
> want to say "find this dep again" while non-relocatable installs
> are ok
> with "hard-code this dependency path for now and always".
>
> > > Since the goal of this proposal is to evaluate specific usage,
> I'll
> > > will prioritize describing the file with examples, rather than
> writing
> > > a JSON schema for it. The final design should still be encoded
> that
> > > way, but I feel the format of json schema would make this
> conversation
> > > harder to maintain.
> > I agree. JSON schema is great ... for later :)
>
> It is certainly finicky and hard to keep up-to-date as schema changes
> are being made. Though tooling better than Vim is probably available
> too…
>
> > > * is-interface (optional, default to true): This describes
> whether
> > > this contributes to the external interface of the module, the
> same
> > > semantics of P1689 applies.
> > Do implementation module units need to be mentioned at all? I'm not
> > opposed to allowing for them, but if I'm following correctly,
> since the
> > metadata file is consumed to satisfy module imports, they don't
> > contribute to anything that is relevant to import. I would expect
> > is-interface to always be true.
>
> This was added to P1689R5. Looking at the history of its hosted
> repo, it
> looks like it was intended to help guide whether transitive usage was
> required, however, it is used in MSVC's module mapper to know
> whether to
> use `-interface` or `internalPartition` because `module M:part;` *can*
> be an implementation unit there (and is without `-internalPartition`
> being passed).
>
> I'm not sure it is useful here as implementation units don't need
> to be
> described anywhere.
>
> > > * local-arguments (optional), an object describing arguments
> that
> > > should be applied for translating this particular importable
> unit, but
> > > that doesn't need to be in the compilation of the translation
> unit
> > > importing this module:
> >
> > Since local-arguments (which I presume to mean compiler command
> line
> > options) are necessarily implementation specific, I think this
> should
> > either be generalized or named such that it reflects an
> implementation
> > dependency. Perhaps:
> >
> > "local-arguments": [
> > { "gcc-compatible": [ "-fconstexpr-depth=512" ] },
> > { "cl-compatible": [ "/constexpr:depth512" ] },
> > { "circle": [ "--ftemplate-depth=768" ] }
> > ]
>
> I would prefer these just be the "frontend family" name rather than
> saying "compatible". Classic Intel (though its implementation of C++20
> modules is unlikely to ever exist) is *mostly* gcc compatible, but
> should have a way to do fallback. As compilers port to being Clang
> variants, I suspect some compatibility with their old flags will
> persist in some form or another.
>
> Also note that for things like this flag, you want some smarter
> logic to
> do `max()` over all specified values instead of just slapping them
> together. But that is a whole different can of worms and not something
> to tackle here.
>
> --Ben
> _______________________________________________
> SG15 mailing list
> SG15_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg15
>
>
> _______________________________________________
> SG15 mailing list
> SG15_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg15

Received on 2023-12-13 22:58:39