>
> Or the build system may communicate with the compiler itself via
> inter-process communication. This will allow you to sidestep most
> of the issues listed by Tom. FYI:
>

How? Can you please explain?

I think the only issue that this side steps is the compiler crash. Because,
if we want faster compilation, then we will have to preserve the compiler
state and cache file reads. This will lead to similar memory usage. Maybe
we can preserve the compiler state of each compilation in its own process.
So that the build system process is not the victim of the Linux OOM process
killer. But to achieve this we will have to sacrifice the file caching as
well and even then there is no guarantee. Besides, there is a process setup
cost as well.

I have made a few more changes to my proposal

Updated paper:
https://htmlpreview.github.io/?https://github.com/HassanSajjad-302/iso-papers/blob/main/generated/my-paper.html#what-are-the-tradeoffs
Diff:
https://github.com/HassanSajjad-302/iso-papers/commit/81bb020d43fb6686f066f1a58e68da1315da7418?short_path=97ede4f#diff-97ede4f82ab7b62ba59e0eb311dfb817cbc8af90a8e13cd6e26348874d156ac3

These changes now make it possible to do the conventional build model with
a shared library as well. The build system can cache the file reads and
avoids the process setup cost. This leads to similar or even less memory
usage as the same file can be read by multiple threads but there is no
compiler state to save this time. Based on the build system configuration,
the build system can select between scanning or not scanning depending on
the available hardware.

Changes include returning the array of output files with corresponding
paths instead of having fixed variables for headerUnit and module bmi in
returned compile_output. Because the build system constructs the compile
command, it can identify the output file from the corresponding path. This
way it can detect all of BMI, the scanner output file, and the object file.
Previously there was no variable for the scanner output file in the
returned compile_output. Now, I think, it can work with any compile
command. The build system can ignore an output file if it gets an
unexpected output file and error out if it does not get an expected file.

If my proposal is accepted, I see little reason for continuing to maintain
an executable-based scanning build model in my build system.

While the build system can not detect the compiler crash, its output can be
piped to another process. This process can then run the build failure
routine instead.

I am confident that we will lose little to none in this approach.

Best,
Hassan Sajjad

On Sun, Sep 24, 2023 at 8:56 PM Boris Kolpackov <boris_at_[hidden]>
wrote:

> Hassan Sajjad via SG15 <sg15_at_[hidden]> writes:
>
> > The build system may use one-process-per-compilation with inter-process
> > communication.
>
> Or the build system may communicate with the compiler itself via
> inter-process communication. This will allow you to sidestep most
> of the issues listed by Tom. FYI:
>
> https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p1184r2.pdf
>
> https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1842r0.html
>