On Sun, Jul 02, 2023 at 17:41:29 +0200, Mark de Wever via SG15 wrote:
> I'm a libc++ developer working on implementing the std and the
> std.compat module. I have an experimental version of the std module [1].
> Using the module requires manually building the BMI using the files in
> the local libc++ build directory. This works for early adaptors, but is
> not a solution that can be used in production.
>
> There was an discussion about the installation location on LLVM's
> discourse [2]. There it has been suggested to discuss this in SG15. I
> spoke with Louis Dionne, the maintainer of libc++. He spoke with Michael
> Spencer and it indeed seems in the purview of SG15 to discuss this.
> Louis asked me to send an email to the reflector to give more
> information for a discussion during a telecon.
>
> The goal is to find a good installation directory for (standard) library
> module sources so they can be used by build systems to build the
> required BMIs. This means the library should provide the information
> needed for build systems. There is a p1689 format for modules, which
> seems good starting point for supplying this information.
> With this information users can import the std(.compat) module in their
> projects and it works out of the box, just like including standard
> headers and linking against the standard library.
>
> MSVC has the files std.ixx, std.compat.ixx, and modules.json in their
> source tree. I've no experience with MSVC and what is required to build
> BMIs from these source files.

I believe that, today, one just tells VS "I need the standard modules"
and MSBuild knows what to do. But they're nothing "special" beyond the
implicitness I expect everyone would like from their build system (at
least as far as it knows "this code wants C++23 or newer").

> Libc++ has std.cppm, std/<header>.cppm for every header, and a
> CMakeLists.txt. In order to build the BMI clang needs to use special
> build flags. For example, -Wno-reserved-module-identifier is needed to
> allow building modules with implementation reserved names. AFAIK
> specifying build flags is not possible in the p1689 format without
> vendor extensions.

Nor is it intended to. Note that extensions are, de jure, ignorable. In
CMake terms, this would be part of the relevant `CMake::CXX23` (or
whatever color that bikeshed gets painted) target that references the
modules. Something like:

    add_library(CMake::CXX23 IMPORTED INTERFACE)
    target_compile_options(CMake::CXX23 PRIVATE -Wno-reserved-module-identifier)
    target_sources(CMake::CXX23 INTERFACE FILE_SET TYPE CXX_MODULES FILES std.cppm std.compat.cppm BASE_DIRS …)

Then (as I imagine it at least), CMake would see "oh, you use C++23 and
haven't disabled all modules support, so you implicitly link to
CMake::CXX23" and the support for modules on IMPORTED targets handles
the BMI generation. The WIP MR that enables this and that I plan to
polish later this month is here:

    https://gitlab.kitware.com/cmake/cmake/-/merge_requests/8535

The modules would be scanned (to gather module dependency information)
and BMI'd as needed by consumers.

> When the standard libraries have a good location it seems natural that
> other libraries want to use a similar location. It would be good to have
> a uniform way to store module information, which then can be used by
> build systems to build the required BMI files.
>
> The suggestion on LLVM's discourse for the path would be
>
> ${PREFIX}/usr/share/c++/modules/<library>/

Has the FHS group been contacted about this? I have an affinity towards
`/usr/lib` instead as they are tied to compiled artifacts in that while
the modules may need compiled umpteen ways for BMI usage, there's still
only one `libc++` they're associated with (my motivating thoughts are
that installing multiple libc++ libraries shouldn't conflict under
`/usr/share`).

FHS mailing list info:

    https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch07.html#theFhsMailingList

> For example, Apple uses ${SDKROOT} as ${PREFIX} where it's an empty
> string on Linux systems.

A nitpick, but `${PREFIX}` usually handles the `/usr` on Unix platforms
(e.g., `/usr/local` is impossible with the proposed pattern as-is).
Apple can do whatever they do today to reroot under the SDK as they do
for everything else that gets installed (I suspect there's some DESTDIR
and longer PREFIX settings, but that's just a guess). The SDKs are all
relocatable anyways, so there's not really a set place beyond "what does
the compiler think the SDK root is?" whether that be set via
`xcodeselect` or `DEVELOPER_DIR`).

> The <library> is the name of the library where every name starting with
> the library name is reserved. Some examples:
>
> ${PREFIX}/usr/share/c++/modules/libc++/v1/std.cppm
> ${PREFIX}/usr/share/c++/modules/libc++/v1/std.compat.cppm
>
> ${PREFIX}/usr/share/c++/modules/libstdc++/14/std.cppm
> ${PREFIX}/usr/share/c++/modules/libstdc++/14/std.compat.cppm
>
> ${PREFIX}/usr/share/c++/modules/qt/core.cppm
> ${PREFIX}/usr/share/c++/modules/qt/network.cppm
>
> ${PREFIX}/usr/share/c++/modules/boost181/boost.cppm
> ${PREFIX}/usr/share/c++/modules/boost182/boost.cppm
>
> In the latter case the boost181 and boost182 belong to the boost
> library namespace.

Is there any correlation with SONAME or is it more about the `.so`
linker file name? Boost has…complex library naming rules as it is and is
why I think there's interest in storing (at least a pointer to the
module files) closer to the libraries themselves.

> The proposed path seems not a good path for Windows. I hope the
> Microsoft developers have a suggestion. Would it be possible for other
> libraries to match with MSVC STL does?

I suspect that whatever the Windows SDKs ship with will be influential.
However, I think they just reuse `include`.

> The things I would like to discuss are:
> - Are we happy with the proposed locations on non-Windows systems?

I think asking FHS folks for input is useful here.

> - What would be a good location on Windows?

Again, I think somewhere closer to the `.lib` is useful.

> - What information do build system vendors need to turn the sources into
> BMI files and how is this information provided.

Very SG15-relevant question that needs answered yet.

> [1] https://libcxx.llvm.org/Modules.html
> [2] https://discourse.llvm.org/t/rfc-about-the-default-location-for-std-modules/69191
>
> Cheers,
> Mark
> _______________________________________________
> SG15 mailing list
> SG15_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg15