Date: Fri, 14 Oct 2022 17:44:18 +0000
[Ben]
> We can't scan sources that don't exist?
Correct. So, the issue has nothing to do with "header units". Even if there was a magic keyword in the source code saying "this is header unit", you would still have this problem.
[gdr]
>> For non-generated source files, would it be conceivable to run the
>> dependency scanning just before generating the build definition?
[Ben]
> No, because if the scanning is not part of the build graph, a change in
> a source requires rerunning the scanning which could mean regenerating
> the build definitions.
I think you misunderstood the suggestion.
The suggestion wasn't to entirely remove the scanning from the build phase.
Rather, to have a pre-evaluation phase of the build graph (before writing it down on disk) where you run the scanner, in addition to the usual run the scan during build for the "inner loop" case.
> Build tools don't universally support "only rerun
> if needed". `ninja` has `restat = 1`, but `make` is specified that if a
> recipe runs, its output is *assumed* to have changed (even if it
> didn't).
That is not the case of Make .PHONY targets that are always run (so, they are not triggered by changes), right?
> How else do you see this working?
The in-source annotation of header units doesn’t help - even if we could magically get it.
I am trying to proceed by elimination here 😊
-- Gaby
-----Original Message-----
From: Ben Boeckel <ben.boeckel_at_[hidden]m>
Sent: Friday, October 14, 2022 4:54 AM
To: Gabriel Dos Reis <gdr_at_microsoft.com>
Cc: sg15_at_[hidden]
Subject: Re: [SG15] Header units again.
On Thu, Oct 13, 2022 at 23:04:05 +0000, Gabriel Dos Reis wrote:
> > - generated source files
>
> For generated source files, how is the problem different from
> importing a named module being imported?
We can't scan sources that don't exist?
> For non-generated source files, would it be conceivable to run the
> dependency scanning just before generating the build definition?
No, because if the scanning is not part of the build graph, a change in
a source requires rerunning the scanning which could mean regenerating
the build definitions. Build tools don't universally support "only rerun
if needed". `ninja` has `restat = 1`, but `make` is specified that if a
recipe runs, its output is *assumed* to have changed (even if it
didn't). How else do you see this working?
> > changing a source file
>
> Only if the change happens after the generation of the build
> definition (e.g. "inner loop"),
The following steps:
1. cmake generates build rules
2. build discovers imports and generates module maps
has a cycle on step 2 when a source is edited. `make` and `ninja` are
good at determining what work needs to be done when a file changes.
You're proposing:
1. cmake generates build rules and dependencies for extant files (and
generated files that exist on a rerun?)
2. build discovers imports from…other generated files
where 1 is now in the cycle when a source file is edited. CMake is *not*
good at picking its generation up from some state and needs to start
again from the top of the `CMakeLists.txt` to build that up again to
reseed any module map files it is responsible for. It is strictly more
complicated *and* less performant in the common case. Not to mention
that scanning of source files is now part of the "generation" step and
this involves the execution of a compiler/tool per TU. Again, `make` and
`ninja` are far better at this task.
> but the gate to that is the timestamp
> of the build definition generation. And this is similar to the
> generated source file scenario, which in my understanding affects also
> importation of named modules. No?
I'm not sure what mental model you have here where this is better in any
way. Do the graphs in this paper make sense to you?
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmathstuf.fedorapeople.org%2Ffortran-modules%2Ffortran-modules.html&data=05%7C01%7Cgdr%40microsoft.com%7Cfa01a10bd634485864d008daaddac139%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638013452308646540%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=fTGRZSgFf1%2BWINNXiJAlTTY8inUdibawxSAC1RNMPkg%3D&reserved=0
--Ben
> We can't scan sources that don't exist?
Correct. So, the issue has nothing to do with "header units". Even if there was a magic keyword in the source code saying "this is header unit", you would still have this problem.
[gdr]
>> For non-generated source files, would it be conceivable to run the
>> dependency scanning just before generating the build definition?
[Ben]
> No, because if the scanning is not part of the build graph, a change in
> a source requires rerunning the scanning which could mean regenerating
> the build definitions.
I think you misunderstood the suggestion.
The suggestion wasn't to entirely remove the scanning from the build phase.
Rather, to have a pre-evaluation phase of the build graph (before writing it down on disk) where you run the scanner, in addition to the usual run the scan during build for the "inner loop" case.
> Build tools don't universally support "only rerun
> if needed". `ninja` has `restat = 1`, but `make` is specified that if a
> recipe runs, its output is *assumed* to have changed (even if it
> didn't).
That is not the case of Make .PHONY targets that are always run (so, they are not triggered by changes), right?
> How else do you see this working?
The in-source annotation of header units doesn’t help - even if we could magically get it.
I am trying to proceed by elimination here 😊
-- Gaby
-----Original Message-----
From: Ben Boeckel <ben.boeckel_at_[hidden]m>
Sent: Friday, October 14, 2022 4:54 AM
To: Gabriel Dos Reis <gdr_at_microsoft.com>
Cc: sg15_at_[hidden]
Subject: Re: [SG15] Header units again.
On Thu, Oct 13, 2022 at 23:04:05 +0000, Gabriel Dos Reis wrote:
> > - generated source files
>
> For generated source files, how is the problem different from
> importing a named module being imported?
We can't scan sources that don't exist?
> For non-generated source files, would it be conceivable to run the
> dependency scanning just before generating the build definition?
No, because if the scanning is not part of the build graph, a change in
a source requires rerunning the scanning which could mean regenerating
the build definitions. Build tools don't universally support "only rerun
if needed". `ninja` has `restat = 1`, but `make` is specified that if a
recipe runs, its output is *assumed* to have changed (even if it
didn't). How else do you see this working?
> > changing a source file
>
> Only if the change happens after the generation of the build
> definition (e.g. "inner loop"),
The following steps:
1. cmake generates build rules
2. build discovers imports and generates module maps
has a cycle on step 2 when a source is edited. `make` and `ninja` are
good at determining what work needs to be done when a file changes.
You're proposing:
1. cmake generates build rules and dependencies for extant files (and
generated files that exist on a rerun?)
2. build discovers imports from…other generated files
where 1 is now in the cycle when a source file is edited. CMake is *not*
good at picking its generation up from some state and needs to start
again from the top of the `CMakeLists.txt` to build that up again to
reseed any module map files it is responsible for. It is strictly more
complicated *and* less performant in the common case. Not to mention
that scanning of source files is now part of the "generation" step and
this involves the execution of a compiler/tool per TU. Again, `make` and
`ninja` are far better at this task.
> but the gate to that is the timestamp
> of the build definition generation. And this is similar to the
> generated source file scenario, which in my understanding affects also
> importation of named modules. No?
I'm not sure what mental model you have here where this is better in any
way. Do the graphs in this paper make sense to you?
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmathstuf.fedorapeople.org%2Ffortran-modules%2Ffortran-modules.html&data=05%7C01%7Cgdr%40microsoft.com%7Cfa01a10bd634485864d008daaddac139%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638013452308646540%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=fTGRZSgFf1%2BWINNXiJAlTTY8inUdibawxSAC1RNMPkg%3D&reserved=0
--Ben
Received on 2022-10-14 17:44:21