On Thu, Mar 3, 2022 at 4:34 PM Olga Arkhipova via SG15 <sg15@lists.isocpp.org> wrote:

I think it would be useful to finish the discussion about scenarios and assumptions before we go to solutions


Here is what I believe we’ve got so far


Suppose we have:


Library A: contains Module A

Library B: contains Module B which imports Module A

Code C: imports Module B


The libraries can come in different forms:


  1. Binary Libraries – libraries that contain prebuilt BMIs and static lib (as well as at least some sources)
  2. Source Libraries – libraries that don’t contain any binaries, most of the popular libraries today. There are two main forms today
These two scenarios don't seem to include what I think is the more common scenario that, so far as I understand it, is at the core one being discussed in the proposal:

Code that would currently ship as a binary library - headers and precompiled (static and/or shared) libraries. I think "most of the popular libraries today" ship this way (see LLVM, boost, etc).

The scenario that Bloomberg folks are trying to figure out is what those libraries look like/how they ship in a modules world - which means shipping module interface source and precompiled static/shared libraries (/maybe/ some BMIs that cover some compilers and build configurations, but the more challenging cases/motivation for the proposal are where those BMIs aren't applicable to the consumer's (C's) build) - and probably some kind of build instructions for the module interface source. The hope/desire is to standardize these build instructions, to some degree. Not for building the whole source of library A or B, only the BMIs.

b.1. Header only libraries  – headers are built as a part of the user’s code TUs.

   Note: This form is not applicable to module interfaces as they are different TUs. The module interface source will have to be built explicitly (i.e. it has to be b.2).

b.2. Sources and (some) build instructions  - sources are built on the user’s machine with user’s build tools (producing static or dynamic lib) which then consumed by the user’s build.

  The build is performed either by the package manager or by user’s build explicitly. The result and further usage is similar to Binary Libraries.


Scenarios and assumptions


  1. Main build of C.

1.1   The build needs to have Module B’s and Module A’s BMIs to be able to compile the source that import Module B.

The build needs to

1.1.1.        find BMIs in the  (assuming it can read them) using the information available to that build system


1.1.2.        somehow re-build the BMIs (assuming it cannot read them).


1.2.      The build also needs to find and link static lib of Library B (which contains implementation of Module B), as well as static lib of Library A (which contains implementation of Module A).


  1. Tools outside of the main build (IDE, static analysis, etc.) need to compile/parse C.

Assumption: the main build of C succeeds on the same machine.

For that they need to be able to rebuild Module A and Module B (the assumption is that they never can read BMIs).

Right - including the case where this "tool" might also be another compiler. (eg: someone ships code with GCC BMIs and needs to compile it with Clang - or a different build mode that's not BMI compatible (clang's BMIs are much more constrained than their ABI - there are different flags you can use that would make the BMI not valid to share, but would still be usable with some exsiting shared/static library that the module was an interface to))



What we’ve agreed on:


  1. Any library which contains modules must include module interface sources
  2. If a library contains BMIs, it should also contain information (probably in some json form) sufficient to be able to rebuilt them.
Could we remove the "IF a library contains BMIs" from this statement? "A shipped library should include some portable-ish* means of defining how to build BMIs from its module interface definitions"

* portable-ish in that it might have parts that lets it specify compiler-specific flags (maybe general ones - GCC-like or MSVC-like, etc - enable or disable certain warnings, or define certain preprocessor directives, etc).



I also promised a description of what VS is doing for module’s build, please see attached.

Thanks! Had a bit of a look. Is my understanding correct that MSVC can't consume external modules that don't include an MSVC Project file? (is this true for external dependencies pre-modules, or is this a new/tighter constraint for consuming external libraries?)