Date: Wed, 13 Dec 2023 17:53:28 -0500
On 12/13/23 2:38 PM, Ben Boeckel via SG15 wrote:
> On Wed, Dec 13, 2023 at 13:45:45 -0500, Tom Honermann via SG15 wrote:
>> On 12/12/23 4:56 PM, Daniel Ruoso via SG15 wrote:
>>> * Module discovery should be subordinate to ABI decisions having been
>>> made already. The outcome is that we don't expect the module metadata
>>> to be used to discover which ABI settings are available for a given
>>> standard library or how to choose between them. The way that this
>>> manifests is that, following the lead from P2577R2, we expect the
>>> libraries to ship those metadata files on a one-to-one mapping with
>>> the library binary used as input to the linker.
>> I think someone mentioned it yesterday, but this will presumably have to
>> account for multilib libraries in some way.
> I assume this is "universal" binaries on Apple or the (nigh-unused)
> FatELF where multiple architectures live inside one artifact rather than
> Red Hat "multilib" /usr/lib vs /usr/lib64 or Debian "multiarch"
> /usr/$triplet. I'm not sure where `.xcframework` artifacts fall here
> off-hand.
Yes, "universal" binaries, but not just those. While true that multilib
support is not often used (at least, not for different target
architectures) on x86 based Linux systems, they are used elsewhere. See
this Multilib RFC <https://discourse.llvm.org/t/rfc-multilib/67494> for
LLVM for an example of relatively recent discussion. I'm not sure if
those uses have much impact for modules though.
We also need to think about how to support C++ language extensions for
heterogeneous programming like OpenMP, OpenCL, CUDA, HIP, and SYCL. A
single compiler invocation for these languages results in the same
primary source file being separately compiled (in practice, this is
technically implementation detail, but we need to be aware of it) for
the host and one or more devices. Distinct BMIs will be required for
each of these compilations because of differences in predefined
preprocessor macros (at least, probably for other reasons as well).
>
>>> * Relative file paths: Any non-absolute path described in this file
>>> will be presumed to have the directory where the metadata file was
>>> found as the base for the lookup.
>> I think it will be necessary to support paths that are relative to some
>> other parameterized location to accommodate dependencies on header files
>> or modules provided by other projects/packages. This would help to
>> reduce the otherwise common practice of a build system having to
>> concatenate include paths for essentially every project it knows about
>> when building any package it knows about.
> Yeah, that gets really close to package management… Relocatable installs
> want to say "find this dep again" while non-relocatable installs are ok
> with "hard-code this dependency path for now and always".
Indeed. As I stated in my reply to Olga, it would be ideal if build
systems provide this information as informed by a package manager.
>
>>> Since the goal of this proposal is to evaluate specific usage, I'll
>>> will prioritize describing the file with examples, rather than writing
>>> a JSON schema for it. The final design should still be encoded that
>>> way, but I feel the format of json schema would make this conversation
>>> harder to maintain.
>> I agree. JSON schema is great ... for later :)
> It is certainly finicky and hard to keep up-to-date as schema changes
> are being made. Though tooling better than Vim is probably available
> too…
>
>>> * is-interface (optional, default to true): This describes whether
>>> this contributes to the external interface of the module, the same
>>> semantics of P1689 applies.
>> Do implementation module units need to be mentioned at all? I'm not
>> opposed to allowing for them, but if I'm following correctly, since the
>> metadata file is consumed to satisfy module imports, they don't
>> contribute to anything that is relevant to import. I would expect
>> is-interface to always be true.
> This was added to P1689R5. Looking at the history of its hosted repo, it
> looks like it was intended to help guide whether transitive usage was
> required, however, it is used in MSVC's module mapper to know whether to
> use `-interface` or `internalPartition` because `module M:part;` *can*
> be an implementation unit there (and is without `-internalPartition`
> being passed).
>
> I'm not sure it is useful here as implementation units don't need to be
> described anywhere.
>
>>> * local-arguments (optional), an object describing arguments that
>>> should be applied for translating this particular importable unit, but
>>> that doesn't need to be in the compilation of the translation unit
>>> importing this module:
>> Since local-arguments (which I presume to mean compiler command line
>> options) are necessarily implementation specific, I think this should
>> either be generalized or named such that it reflects an implementation
>> dependency. Perhaps:
>>
>> "local-arguments": [
>> { "gcc-compatible": [ "-fconstexpr-depth=512" ] },
>> { "cl-compatible": [ "/constexpr:depth512" ] },
>> { "circle": [ "--ftemplate-depth=768" ] }
>> ]
> I would prefer these just be the "frontend family" name rather than
> saying "compatible". Classic Intel (though its implementation of C++20
> modules is unlikely to ever exist) is *mostly* gcc compatible, but
> should have a way to do fallback. As compilers port to being Clang
> variants, I suspect some compatibility with their old flags will
> persist in some form or another.
I don't think frontend family suffices here. Clang provides both gcc and
cl compatible drivers and the build system will choose which one is
being used. There are also many gcc and clang derived compilers that
don't use a gcc compatible driver (Blackberry qnx is an example).
>
> Also note that for things like this flag, you want some smarter logic to
> do `max()` over all specified values instead of just slapping them
> together. But that is a whole different can of worms and not something
> to tackle here.
Agreed. And another case where we need to think carefully in order to
not reinvent cmake.
Tom.
>
> --Ben
> _______________________________________________
> SG15 mailing list
> SG15_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg15
> On Wed, Dec 13, 2023 at 13:45:45 -0500, Tom Honermann via SG15 wrote:
>> On 12/12/23 4:56 PM, Daniel Ruoso via SG15 wrote:
>>> * Module discovery should be subordinate to ABI decisions having been
>>> made already. The outcome is that we don't expect the module metadata
>>> to be used to discover which ABI settings are available for a given
>>> standard library or how to choose between them. The way that this
>>> manifests is that, following the lead from P2577R2, we expect the
>>> libraries to ship those metadata files on a one-to-one mapping with
>>> the library binary used as input to the linker.
>> I think someone mentioned it yesterday, but this will presumably have to
>> account for multilib libraries in some way.
> I assume this is "universal" binaries on Apple or the (nigh-unused)
> FatELF where multiple architectures live inside one artifact rather than
> Red Hat "multilib" /usr/lib vs /usr/lib64 or Debian "multiarch"
> /usr/$triplet. I'm not sure where `.xcframework` artifacts fall here
> off-hand.
Yes, "universal" binaries, but not just those. While true that multilib
support is not often used (at least, not for different target
architectures) on x86 based Linux systems, they are used elsewhere. See
this Multilib RFC <https://discourse.llvm.org/t/rfc-multilib/67494> for
LLVM for an example of relatively recent discussion. I'm not sure if
those uses have much impact for modules though.
We also need to think about how to support C++ language extensions for
heterogeneous programming like OpenMP, OpenCL, CUDA, HIP, and SYCL. A
single compiler invocation for these languages results in the same
primary source file being separately compiled (in practice, this is
technically implementation detail, but we need to be aware of it) for
the host and one or more devices. Distinct BMIs will be required for
each of these compilations because of differences in predefined
preprocessor macros (at least, probably for other reasons as well).
>
>>> * Relative file paths: Any non-absolute path described in this file
>>> will be presumed to have the directory where the metadata file was
>>> found as the base for the lookup.
>> I think it will be necessary to support paths that are relative to some
>> other parameterized location to accommodate dependencies on header files
>> or modules provided by other projects/packages. This would help to
>> reduce the otherwise common practice of a build system having to
>> concatenate include paths for essentially every project it knows about
>> when building any package it knows about.
> Yeah, that gets really close to package management… Relocatable installs
> want to say "find this dep again" while non-relocatable installs are ok
> with "hard-code this dependency path for now and always".
Indeed. As I stated in my reply to Olga, it would be ideal if build
systems provide this information as informed by a package manager.
>
>>> Since the goal of this proposal is to evaluate specific usage, I'll
>>> will prioritize describing the file with examples, rather than writing
>>> a JSON schema for it. The final design should still be encoded that
>>> way, but I feel the format of json schema would make this conversation
>>> harder to maintain.
>> I agree. JSON schema is great ... for later :)
> It is certainly finicky and hard to keep up-to-date as schema changes
> are being made. Though tooling better than Vim is probably available
> too…
>
>>> * is-interface (optional, default to true): This describes whether
>>> this contributes to the external interface of the module, the same
>>> semantics of P1689 applies.
>> Do implementation module units need to be mentioned at all? I'm not
>> opposed to allowing for them, but if I'm following correctly, since the
>> metadata file is consumed to satisfy module imports, they don't
>> contribute to anything that is relevant to import. I would expect
>> is-interface to always be true.
> This was added to P1689R5. Looking at the history of its hosted repo, it
> looks like it was intended to help guide whether transitive usage was
> required, however, it is used in MSVC's module mapper to know whether to
> use `-interface` or `internalPartition` because `module M:part;` *can*
> be an implementation unit there (and is without `-internalPartition`
> being passed).
>
> I'm not sure it is useful here as implementation units don't need to be
> described anywhere.
>
>>> * local-arguments (optional), an object describing arguments that
>>> should be applied for translating this particular importable unit, but
>>> that doesn't need to be in the compilation of the translation unit
>>> importing this module:
>> Since local-arguments (which I presume to mean compiler command line
>> options) are necessarily implementation specific, I think this should
>> either be generalized or named such that it reflects an implementation
>> dependency. Perhaps:
>>
>> "local-arguments": [
>> { "gcc-compatible": [ "-fconstexpr-depth=512" ] },
>> { "cl-compatible": [ "/constexpr:depth512" ] },
>> { "circle": [ "--ftemplate-depth=768" ] }
>> ]
> I would prefer these just be the "frontend family" name rather than
> saying "compatible". Classic Intel (though its implementation of C++20
> modules is unlikely to ever exist) is *mostly* gcc compatible, but
> should have a way to do fallback. As compilers port to being Clang
> variants, I suspect some compatibility with their old flags will
> persist in some form or another.
I don't think frontend family suffices here. Clang provides both gcc and
cl compatible drivers and the build system will choose which one is
being used. There are also many gcc and clang derived compilers that
don't use a gcc compatible driver (Blackberry qnx is an example).
>
> Also note that for things like this flag, you want some smarter logic to
> do `max()` over all specified values instead of just slapping them
> together. But that is a whole different can of worms and not something
> to tackle here.
Agreed. And another case where we need to think carefully in order to
not reinvent cmake.
Tom.
>
> --Ben
> _______________________________________________
> SG15 mailing list
> SG15_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg15
Received on 2023-12-13 22:53:32