I think something slightly different from what was proposed would be
useful: a per .cpp/.o output directory. Basically switch from a model where
the compiler dumps auxiliary files when compiling outdir/blah.o to names
like outdir/blah.dwo and instead the compiler is given outdir/blah as a
directory that is private to that compilation and it can put all output
files in there. That way caching, cleaning, distributed builds, etc don't
need to know about all of a compilers different files they just
cache/delete/ship whatever files they find in that directory. This would
cover everything from debug fission, profile data, SARIF, timing reports,
anything that is output by the compiler. Some of these, there is currently
no way to explicitly specify the output file name right now, it can only be
derived from the blah.o name. Obviously, the build system will need to
know about the output files it deals with, especially those that are inputs
to other actions like the .o into linking. But this gives a clean way for
the compiler to introduce new files that the build system doesn't need to
care about but are consumed by other operations (eg .dwo being used by gdb)

A slight extension would be some way to separate "transient" outputs that
the compiler only writes to and could be deleted between invocations from
"in-out" files that the compiler both reads and writes to and needs to
persist. I'm not sure if there are any like this in the .cpp->.o
compilation right now, but I could imagine a compiler wanting to do its own
fine-grained caching so that it can do truely minimal recompiles, eg if you
change a single function, it only needs to codegen and optimize that
function. I think something similar is used in clang's incremental thin
lto: https://clang.llvm.org/docs/ThinLTO.html#incremental, but I don't see
any reason that is fundamentally only applicable to LTO.

I think doing this at the .cpp/.o invocation level rather than the "top
level target" level avoids a lot of the problems of the former. Not the
least of which is defining what is a "top level target" anyway?

On Tue, Nov 26, 2024 at 3:37 PM Iain Sandoe via SG15 <sg15_at_[hidden]>
wrote:

>
>
> > On 26 Nov 2024, at 13:45, Boris Kolpackov <boris_at_[hidden]>
> wrote:
> >
> > Iain Sandoe via SG15 <sg15_at_[hidden]> writes:
> >
> >> AFAIU: the compiler implementers represented (MSVC, clang, GCC)
> commented
> >> that they all have, indeed, have representative mechanisms to do this
> >> already (e.g. specifying where ‘/tmp’ is on unix-like systems, and/or
> >> specifying the placement of saved intermediate results relative to
> working
> >> directory and/or the object output directory).
> >
> > Hm, this to me feel very different to what is proposed. What exists is
> > the ability to specify the directory for transient files that only exist
> > while the compiler is running and are not observable by the build system
> > or the user. Or am I missing something here?
>
> I believe what is proposed (in outline, with a paper hopefully to come) is
> that the place specified would be usable by files that could persist
> between multiple invocations (and, more contentiously, multiple parallel
> invocations) of the tools.
>
> Currently, some of the artefacts specified by similar compiler flags are
> equally intended to be persistent (at least over a build) e.g. dependencies
> and intermediate files.
>
> Iain
> _______________________________________________
> SG15 mailing list
> SG15_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg15
>