Hi Olga and Boris,
 The idea of making build systems acting more like a compiler is interesting. It reminds me of Rust. In the tutorial of rust, the appearance of `rustc` is really rare. All the examples invoke `cargo` simply. And I am wondering if we can make things more simpler. e.g., we can skip to provide all the source files (may with a special pattern) by treating all files in the workdir as source files. Also even if we can skip a module map for such simple tools. For example, we can assume athe source file `X.cppm` will declare a module `X`.
Let's call the tool as std-tool or std-build. Then we're talking about the conventions in the tool. This won't a convention that require all tools to follow.
The goal of the tool should be helpful to beginners and small projects. It should not be a goal to take the job of cmake for complex and big projects.
Thanks,
Chuanqi
Thanks,
Chuanqi
------------------------------------------------------------------
From: SG15 <sg15_at_[hidden]>
Send Time:2023 Dec. 12 (Tue.) 10:26
To: SG15 <sg15_at_[hidden]>
Cc:Olga Arkhipova <olgaark_at_[hidden]>; Tom Honermann <tom_at_[hidden]>
Subject:Re: [SG15] [Modules] Should the compiler try to build std module implicitly?
>> Perhaps we are approaching this from the wrong end? Maybe instead of the compiler acting more and more like a build system we can make the build system act more like a compiler? That is, a build system's driver can support the "direct compilation" mode that mimics the compiler invocation while under the hood taking care of the above "build system" issues (which would naturally be quite easy).
I think it can be even simpler - what if there is a "simple cpp build" exe which would be able to build cpp sources with modules by invoking the compiler (or whatever other tools) multiple times like build does?
At some point I wrote an experimental command line tool to be used instead of compiler for simple builds to provide similar experience people have today with direct invocation of the compiler, something like
 cppbuild [tool switches] sources=module.ixx;source.cpp [compiler switches]
The command line of this tool might be a little different from the real compiler so the tool does not need to know about all compiler switches to be able to identify the sources, but I think it is simple enough.
The tool invokes the compiler to scan the sources, constructs the dependency graph and builds all given c++ sources and header units.
The idea also was to allow some extensibility to be able to call any compiler assuming this compiler (or related tools) can produce standardized scan data, build a BMI and have a way pass those BMIs as references. At the time of my experimentations only MSVC had modules support, but I believe such extensibility is possible. If we have common/standardized compiler features (and or mapping to compiler specific switches) it will make this extensibility much easier.
I don't think it would be a big problem extending such a tool to build std modules as soon as it knows where to find them (say, via standardized library manifest with well-known location).
And if the tool can work with library manifests, it means that it will work with other libraries too (as soon as it can find their manifests), which I guess is a plus.
It is a good question who will own/maintain such a tool and how people get it.
I think the most logical is to include it into compiler toolset, but it could be other options as well.
Thanks,
Olga
-----Original Message-----
From: SG15 <sg15-bounces_at_[hidden]> On Behalf Of Boris Kolpackov via SG15
Sent: Monday, December 11, 2023 5:00 AM
To: Chuanqi Xu via SG15 <sg15_at_[hidden]>
Cc: Boris Kolpackov <boris_at_[hidden]>; Tom Honermann <tom_at_[hidden]>
Subject: Re: [SG15] [Modules] Should the compiler try to build std module implicitly?
Chuanqi Xu via SG15 <sg15_at_[hidden]> writes:
> In case we would love to do the same thing when writing the object to
> a known location, we need to handle/answer the following things:
>
> (1) Where should be the position for the produced BMI of the std module?
>
> (2) Where should be the position of the produced object file of the std
> module?
>
> (3) Do we want to share the BMIs and object files with other invocations
> in the same system?
>
> (4) If (3) is yes, how can we find a BMI and the object file that match the
> flags and compilers we're using. For example, a BMI for std module
> produced with `-std=c++23` can't be consumed with an invocation with
> `-std=c++20`. Also a BMI produced with clang18 can't be consumed with an
> invocation with clang19. And then how can we record that? This should
> not be an easy problem.
>
> (5) Once the `std.cppm` and the corresponding header files changes, how
> can we know that to remove the existing BMIs and object files.
>
> If we can answer all of the issues, I think we're inventing a new tiny
> build systems. But this is not our goal.
Agree and would also add that if you choose to share BMIs, you will also have to deal with concurrent invocations/file locking.
Perhaps we are approaching this from the wrong end? Maybe instead of the compiler acting more and more like a build system we can make the build system act more like a compiler? That is, a build system's driver can support the "direct compilation" mode that mimics the compiler invocation while under the hood taking care of the above "build system" issues (which would naturally be quite easy).
Specifically, instead of, say:
$ clang++ -c hello.cpp
One would run (using the build2 driver as an example):
$ b -m cxx clang++ -c hello.cpp
(I am just riff'ing on the idea; here `-m cxx` means "ask the build system module that provides the C++ compilation support to perform direct compilation using the arguments that follow".)
Some extra things that immediately follow from this:
- Sensible default so you don't need to specify /EHsc /nologo or
 -fPIC manually.
- We could support change tracking:
$ b -m cxx clang++ hello.cpp impl.cpp
$ touch hello.cpp
$ b -m cxx clang++ hello.cpp impl.cpp # Only recompiles hello.cpp.
- We could support additional operations, such as clean, test,
 install/uninstall:
$ b clean -m cxx clang++ hello.cpp impl.cpp
- We could invent an imaginary compile option for producing static
 libraries:
$ b -m cxx clang++ hello.cpp # Executable.
$ b -m cxx clang++ -shared hello.cpp # Shared library.
$ b -m cxx clang++ -archive hello.cpp # Static library.
In a sense it becomes a form of "build by convention" support where the convention is not how we lay source files on disk and name them but how we invoke the compiler.
I could probably prototype something like this in build2 if there is interest.
_______________________________________________
SG15 mailing list
SG15_at_[hidden]
https://lists.isocpp.org/mailman/listinfo.cgi/sg15
_______________________________________________
SG15 mailing list
SG15_at_[hidden]
https://lists.isocpp.org/mailman/listinfo.cgi/sg15