On Tue, Dec 12, 2023, 08:40 Chuanqi Xu <chuanqi.xcq_at_[hidden]> wrote:

> Hi Ran,
>
> While it is still abstract for the tool you described, I think we still
> need to scan the source codes in the end of the day. Or do I misunderstand
> your points?
>

"scan the source" has two meanings:
1. Look for changes in the code that require recompile.
2. Scan for the required import modules.

For 1 - it is the tool's mission.
For 2 - the compiler already does it.

clang has few tools that already do some of these tasks, it is not abstract
at all.


In addition, I think that separating for small and "real" projects is a bad
idea. Tools should work the same regardless the size of the project. It is
the more options that you use as the project grows - you don't switch tool
at some point. The build system should be there for the project's needs
from day one and on.



Ran.

>
>
>
>
> Thanks,
> Chuanqi
>
> ------------------------------------------------------------------
> From: SG15 <sg15_at_[hidden]>
> Send Time:2023 Dec. 12 (Tue.) 14:21
> To: SG15 <sg15_at_[hidden]>
> Cc:Ran Regev <regev.ran_at_[hidden]>; Tom Honermann <tom_at_[hidden]>
> Subject:Re: [SG15] [Modules] Should the compiler try to build std module
> implicitly?
>
>
> Hi all,
>
> We keep treating the compiler as *end* target in the chain of tools to
> build code. We are trying to prepare everything for it to make it be able
> to compile the code.
>
> But what if the compiler is not the last tool in the chain? What if we
> could standardize a tool, that, for example, given a required module to
> import, and the required compilation flags it needs to be build with,
> simply returns the BMI for it? And if the tool can't find it it returns the
> sources that the module can be built from.
>
> Such a tool can be invoked by the compiler whenever there is a need to
> import a module, any module, not just std.
>
> Few advantages for such a tool:
> - no need to scan the code and know which imports are required, which
> flags were used, the compiler does it.
> - file locations, detection of changes in sources, sensitivity to
> compilation flags - all are left to the implementation of the tool. It is
> QoI if it can manage the complexity or always return just the sources.
>
>
> In the past the compiler used to invoke a few tools to accomplish its
> mission, so it is not a new idea to run things "after" the compiler.
>
>
>
> Ran.
>
>
> On Tue, Dec 12, 2023, 05:29 Gabriel Dos Reis via SG15 <
> sg15_at_[hidden]> wrote:
> Yes. Now get a roomful of experts define "small project" :-)
>
> -- Gaby
>
>
> ------------------------------
> *From:* SG15 <sg15-bounces_at_[hidden]> on behalf of Chuanqi Xu via
> SG15 <sg15_at_[hidden]>
> *Sent:* Monday, December 11, 2023 7:04:11 PM
> *To:* SG15 <sg15_at_[hidden]>
> *Cc:* Chuanqi Xu <chuanqi.xcq_at_[hidden]>; Tom Honermann <
> tom_at_[hidden]>
> *Subject:* Re: [SG15] [Modules] Should the compiler try to build std
> module implicitly?
>
> Hi Olga and Boris,
>
> The idea of making build systems acting more like a compiler is
> interesting. It reminds me of Rust. In the tutorial of rust, the appearance
> of `rustc` is really rare. All the examples invoke `cargo` simply. And I
> am wondering if we can make things more simpler. e.g., we can skip to
> provide all the source files (may with a special pattern) by treating all
> files in the workdir as source files. Also even if we can skip a module map
> for such simple tools. For example, we can assume athe source file `X.cppm`
> will declare a module `X`.
>
> Let's call the tool as std-tool or std-build. Then we're talking about the
> conventions in the tool. This won't a convention that require all tools to
> follow.
>
> The goal of the tool should be helpful to beginners and small projects. It
> should not be a goal to take the job of cmake for complex and big projects.
>
> Thanks,
> Chuanqi
>
>
> Thanks,
> Chuanqi
>
> ------------------------------------------------------------------
> From: SG15 <sg15_at_[hidden]>
> Send Time:2023 Dec. 12 (Tue.) 10:26
> To: SG15 <sg15_at_[hidden]>
> Cc:Olga Arkhipova <olgaark_at_[hidden]>; Tom Honermann <
> tom_at_[hidden]>
> Subject:Re: [SG15] [Modules] Should the compiler try to build std module
> implicitly?
>
> >> Perhaps we are approaching this from the wrong end? Maybe instead of
> the compiler acting more and more like a build system we can make the build
> system act more like a compiler? That is, a build system's driver can
> support the "direct compilation" mode that mimics the compiler invocation
> while under the hood taking care of the above "build system" issues (which
> would naturally be quite easy).
>
> I think it can be even simpler - what if there is a "simple cpp build"
> exe which would be able to build cpp sources with modules by invoking the
> compiler (or whatever other tools) multiple times like build does?
>
> At some point I wrote an experimental command line tool to be used instead
> of compiler for simple builds to provide similar experience people have
> today with direct invocation of the compiler, something like
>
> cppbuild [tool switches] sources=module.ixx;source.cpp [compiler
> switches]
>
> The command line of this tool might be a little different from the real
> compiler so the tool does not need to know about all compiler switches to
> be able to identify the sources, but I think it is simple enough.
>
> The tool invokes the compiler to scan the sources, constructs the
> dependency graph and builds all given c++ sources and header units.
> The idea also was to allow some extensibility to be able to call any
> compiler assuming this compiler (or related tools) can produce standardized
> scan data, build a BMI and have a way pass those BMIs as references. At the
> time of my experimentations only MSVC had modules support, but I believe
> such extensibility is possible. If we have common/standardized compiler
> features (and or mapping to compiler specific switches) it will make this
> extensibility much easier.
>
> I don't think it would be a big problem extending such a tool to build std
> modules as soon as it knows where to find them (say, via standardized
> library manifest with well-known location).
> And if the tool can work with library manifests, it means that it will
> work with other libraries too (as soon as it can find their manifests),
> which I guess is a plus.
>
> It is a good question who will own/maintain such a tool and how people get
> it.
> I think the most logical is to include it into compiler toolset, but it
> could be other options as well.
>
> Thanks,
> Olga
>
>
> -----Original Message-----
> From: SG15 <sg15-bounces_at_[hidden]> On Behalf Of Boris Kolpackov
> via SG15
> Sent: Monday, December 11, 2023 5:00 AM
> To: Chuanqi Xu via SG15 <sg15_at_[hidden]>
> Cc: Boris Kolpackov <boris_at_[hidden]>; Tom Honermann <
> tom_at_[hidden]>
> Subject: Re: [SG15] [Modules] Should the compiler try to build std module
> implicitly?
>
> Chuanqi Xu via SG15 <sg15_at_[hidden]> writes:
>
> > In case we would love to do the same thing when writing the object to
> > a known location, we need to handle/answer the following things:
> >
> > (1) Where should be the position for the produced BMI of the std module?
> >
> > (2) Where should be the position of the produced object file of the std
> > module?
> >
> > (3) Do we want to share the BMIs and object files with other invocations
> > in the same system?
> >
> > (4) If (3) is yes, how can we find a BMI and the object file that match
> the
> > flags and compilers we're using. For example, a BMI for std module
> > produced with `-std=c++23` can't be consumed with an invocation with
> > `-std=c++20`. Also a BMI produced with clang18 can't be consumed
> with an
> > invocation with clang19. And then how can we record that? This should
> > not be an easy problem.
> >
> > (5) Once the `std.cppm` and the corresponding header files changes, how
> > can we know that to remove the existing BMIs and object files.
> >
> > If we can answer all of the issues, I think we're inventing a new tiny
> > build systems. But this is not our goal.
>
> Agree and would also add that if you choose to share BMIs, you will also
> have to deal with concurrent invocations/file locking.
>
> Perhaps we are approaching this from the wrong end? Maybe instead of the
> compiler acting more and more like a build system we can make the build
> system act more like a compiler? That is, a build system's driver can
> support the "direct compilation" mode that mimics the compiler invocation
> while under the hood taking care of the above "build system" issues (which
> would naturally be quite easy).
>
> Specifically, instead of, say:
>
> $ clang++ -c hello.cpp
>
> One would run (using the build2 driver as an example):
>
> $ b -m cxx clang++ -c hello.cpp
>
> (I am just riff'ing on the idea; here `-m cxx` means "ask the build system
> module that provides the C++ compilation support to perform direct
> compilation using the arguments that follow".)
>
> Some extra things that immediately follow from this:
>
>
> - Sensible default so you don't need to specify /EHsc /nologo or
> -fPIC manually.
>
>
> - We could support change tracking:
>
> $ b -m cxx clang++ hello.cpp impl.cpp
> $ touch hello.cpp
> $ b -m cxx clang++ hello.cpp impl.cpp # Only recompiles hello.cpp.
>
>
> - We could support additional operations, such as clean, test,
> install/uninstall:
>
> $ b clean -m cxx clang++ hello.cpp impl.cpp
>
>
> - We could invent an imaginary compile option for producing static
> libraries:
>
> $ b -m cxx clang++ hello.cpp # Executable.
> $ b -m cxx clang++ -shared hello.cpp # Shared library.
> $ b -m cxx clang++ -archive hello.cpp # Static library.
>
>
> In a sense it becomes a form of "build by convention" support where the
> convention is not how we lay source files on disk and name them but how we
> invoke the compiler.
>
> I could probably prototype something like this in build2 if there is
> interest.
> _______________________________________________
> SG15 mailing list
> SG15_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg15
> _______________________________________________
> SG15 mailing list
> SG15_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg15
> _______________________________________________
> SG15 mailing list
> SG15_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg15
>
>