On Fri, Mar 4, 2022 at 8:25 AM Olga Arkhipova <olgaark_at_[hidden]> wrote:

> Boost is header only (IIRC), so user just needs to add its directories to
> the Include dirs.
>

Not according to the documentation I linked, so far as I can tell:
https://www.boost.org/doc/libs/1_65_0/more/getting_started/windows.html
<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.boost.org%2Fdoc%2Flibs%2F1_65_0%2Fmore%2Fgetting_started%2Fwindows.html&data=04%7C01%7Colgaark%40microsoft.com%7C997fe9f1494c4152da5708d9fdbc500d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637819807677818219%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=ECH%2BdeX9t1aIdqy26r9plTt9KmKsQ8UexfpO9csTqRA%3D&reserved=0>
-
that describes a binary distribution - with both headers and libraries to
link to.


> LLVM – the recommended way is to use VS installer to install it as well as
> built in msbuild support for it. if you don’t install though VS and don’t
> use msbuild integration for it, you’ll have to do quite a lot of work of
> setting various project properties pointing on binraires, libs, includes,
> compilation options and might be able to make build working, but not all
> IDE features.
>

Fair enough, maybe easier to focus on Boost as a simpler example, then.


> >>What about not-prebuilt-modules - where there's a static/shared library,
> and some cppm files, no (or not usable-by-this-tool) BMI? How does MSVC
> consume those, when they're external to the MSVC projects?
>
>
> For non prebuilt modules – no, there is no support for automatically
> consuming them now and I am not sure what you actually mean by that.
>

OK - this seems like a part of the conversation we really need to focus on,
because it's, as far as I understand, the core scenario this
proposal/discussion is all about.


> All modules have to be built for the compiler to be able to consume them,
> which means that they either become part of the user projects (so user’s
> build produces BMIs for them the same way it does for user modules) or
> modules are built in an external build, so they become external prebuilt
> modules.
>

OK - so maybe a good example would be: What happens if the third party
builds the library with clang, but you want to consume it with MSVC, or the
other way around? The BMIs can't be used (maybe if you're using clang, but
you figure most of your users will use MSVC - maybe you don't build/ship
BMIs at all (also because Clang's are so tightly version locked, they're
likely to be not applicable to your end users anyway)) - so how do we
enable library vendors and library users to coordinate in a relatively
low-cost way that is at least somewhat portable (doesn't require teh vendor
creating IDE-specific project files for every IDE they want to enable their
users to use)

That's the problem this discussion and proposal is trying to solve, as I
understand it. Does this make sense? Can we clarify/dig into this scenario
further in some way to get us all on the same page about this?

- David


>
>
> Olga
>
>
>
>
>
> *From:* David Blaikie <dblaikie_at_[hidden]>
> *Sent:* Friday, March 4, 2022 12:52 AM
> *To:* Olga Arkhipova <olgaark_at_[hidden]>
> *Cc:* David Blaikie via SG15 <sg15_at_[hidden]>
> *Subject:* Re: [SG15] Meeting on Friday March 4th at 9AM Pacific.
>
>
>
> You don't often get email from dblaikie_at_gmail.com. Learn why this is
> important <http://aka.ms/LearnAboutSenderIdentification>
>
> On Thu, Mar 3, 2022 at 10:49 PM Olga Arkhipova <olgaark_at_[hidden]>
> wrote:
>
> >>. Is my understanding correct that MSVC can't consume external modules
> that don't include an MSVC Project file? (is this true for external
> dependencies pre-modules, or is this a new/tighter constraint for consuming
> external libraries?)
>
>
>
> As there is no “well defined” libraries location on Windows, all locations
> for includes, libs, binaries, whatever have to be specified explicitly for
> the build there either by the user or by package managers (via build
> extensions)
>
>
> OK, so if I picked up LLVM or Boost, etc, from an open source build
> without a MSVC project file - I'd probably add some include and library
> paths to my MSVC project?
>
> looks like the boost docs document something like this:
> https://www.boost.org/doc/libs/1_65_0/more/getting_started/windows.html
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.boost.org%2Fdoc%2Flibs%2F1_65_0%2Fmore%2Fgetting_started%2Fwindows.html&data=04%7C01%7Colgaark%40microsoft.com%7C997fe9f1494c4152da5708d9fdbc500d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637819807677818219%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=ECH%2BdeX9t1aIdqy26r9plTt9KmKsQ8UexfpO9csTqRA%3D&reserved=0>
>
> Looks like it's a total of 3 steps:
> 1) Add an include path
> 2) Add a library path
> 3) Add a library name to link to (oh, this is optional if your linker
> supports auto-linking - I guess the headers embed some metadata into the
> objects, and the linker reads that metadata to determine what to link)
>
>
> MSVC can consume prebuilt modules if somebody specifies the BMI locations
> as build options either in the form of
>
> /reference “ModuleName=”Path/To/Module/BMI”
>
> or the BMI search path.
>
>
> What about not-prebuilt-modules - where there's a static/shared library,
> and some cppm files, no (or not usable-by-this-tool) BMI? How does MSVC
> consume those, when they're external to the MSVC projects?
>
>
> In the last case the expectation is that ifc name would exactly match the
> module and it is not working very well due to build not producing what
> compiler expects there - user intervention is needed to change the default
> ifc name to match the module name.
>
>
>
> In any case is it quite tedious work for the user as all module
> dependencies must be fould by the compiler and this is why we’d like to
> have some additional info provided by the libraries which would allow us to
> get “ModuleName=BMI location” info automatically and to reduce user’s work.
> It would be great if we could use the same mechanism that we use today for
> getting the BMI dependencies, i.e. to be able to find json with additional
> info from the BMI location.
>
>
> Right, though for other folks/use cases, they'd like to be able to go from
> ModuleName to cppm location and build commands (since they might not have a
> usable BMI - different compiler, build configuration,
> tool-that-isn't-a-compiler/doesn't-embed-one, etc).
>
>
>
>
> Thanks,
>
> Olga
>
>
>
>
>
> *From:* David Blaikie <dblaikie_at_[hidden]>
> *Sent:* Thursday, March 3, 2022 20:47
> *To:* David Blaikie via SG15 <sg15_at_[hidden]>
> *Cc:* Olga Arkhipova <olgaark_at_[hidden]>
> *Subject:* Re: [SG15] Meeting on Friday March 4th at 9AM Pacific.
>
>
>
> You don't often get email from dblaikie_at_gmail.com. Learn why this is
> important <http://aka.ms/LearnAboutSenderIdentification>
>
> On Thu, Mar 3, 2022 at 4:34 PM Olga Arkhipova via SG15 <
> sg15_at_[hidden]> wrote:
>
> I think it would be useful to finish the discussion about scenarios and
> assumptions before we go to solutions
>
>
>
> Here is what I believe we’ve got so far
>
>
>
> Suppose we have:
>
>
>
> Library A: contains Module A
>
> Library B: contains Module B which imports Module A
>
> Code C: imports Module B
>
>
>
> The libraries can come in different forms:
>
>
>
> 1. *Binary Libraries *– libraries that contain prebuilt BMIs and
> static lib (as well as at least some sources)
> 2. *Source Libraries* – libraries that don’t contain any binaries,
> most of the popular libraries today. There are two main forms today
>
> These two scenarios don't seem to include what I think is the more common
> scenario that, so far as I understand it, is at the core one being
> discussed in the proposal:
>
> Code that would currently ship as a binary library - headers and
> precompiled (static and/or shared) libraries. I think "most of the popular
> libraries today" ship this way (see LLVM, boost, etc).
>
> The scenario that Bloomberg folks are trying to figure out is what those
> libraries look like/how they ship in a modules world - which means shipping
> module interface source and precompiled static/shared libraries (/maybe/
> some BMIs that cover some compilers and build configurations, but the more
> challenging cases/motivation for the proposal are where those BMIs aren't
> applicable to the consumer's (C's) build) - and probably some kind of build
> instructions for the module interface source. The hope/desire is to
> standardize these build instructions, to some degree. Not for building the
> whole source of library A or B, only the BMIs.
>
>
> 1.
>
> *b.1. Header only libraries * – headers are built as a part of the user’s
> code TUs.
>
> * Note*: This form is not applicable to module interfaces as they are
> different TUs. The module interface source will have to be built explicitly
> (i.e. it has to be b.2).
>
> *b.2. Sources and (some) build instructions - *sources are built on the
> user’s machine with user’s build tools (producing static or dynamic lib)
> which then consumed by the user’s build.
>
> The build is performed either by the package manager or by user’s build
> explicitly. The result and further usage is similar to Binary Libraries.
>
>
>
> *Scenarios and assumptions *
>
>
>
> 1. Main build of C.
>
> 1.1 The build needs to have Module B’s and Module A’s BMIs to be able
> to compile the source that import Module B.
>
> The build needs to
>
> 1.1.1. find BMIs in the (assuming it can read them) using the
> information available to that build system
>
> or
>
> 1.1.2. somehow re-build the BMIs (assuming it cannot read them).
>
>
>
> 1.2. The build also needs to find and link static lib of Library B
> (which contains implementation of Module B), as well as static lib of
> Library A (which contains implementation of Module A).
>
>
>
> 1. Tools outside of the main build (IDE, static analysis, etc.) need
> to compile/parse C.
>
> Assumption: the main build of C succeeds on the same machine.
>
> For that they need to be able to rebuild Module A and Module B (the
> assumption is that they never can read BMIs).
>
> Right - including the case where this "tool" might also be another
> compiler. (eg: someone ships code with GCC BMIs and needs to compile it
> with Clang - or a different build mode that's not BMI compatible (clang's
> BMIs are much more constrained than their ABI - there are different flags
> you can use that would make the BMI not valid to share, but would still be
> usable with some exsiting shared/static library that the module was an
> interface to))
>
>
>
>
>
> *What we’ve agreed on:*
>
>
>
> 1. Any library which contains modules must include module interface
> sources
> 2. If a library contains BMIs, it should also contain information
> (probably in some json form) sufficient to be able to rebuilt them.
>
> Could we remove the "IF a library contains BMIs" from this statement? "A
> shipped library should include some portable-ish* means of defining how to
> build BMIs from its module interface definitions"
>
> * portable-ish in that it might have parts that lets it specify
> compiler-specific flags (maybe general ones - GCC-like or MSVC-like, etc -
> enable or disable certain warnings, or define certain preprocessor
> directives, etc).
>
>
>
>
> 1.
>
>
>
>
>
> I also promised a description of what VS is doing for module’s build,
> please see attached.
>
>
> Thanks! Had a bit of a look. Is my understanding correct that MSVC can't
> consume external modules that don't include an MSVC Project file? (is this
> true for external dependencies pre-modules, or is this a new/tighter
> constraint for consuming external libraries?)
>
>