Hi Ben,
> would suffice for not rebuilding *successful* consumers while also
> ensuring that consumers that *have not yet succeeded* get updated
> diagnostics. This does mean that any diagnostic-affecting changes would
> need to be included in this declhash (e.g., `[[deprecated]]`
> attributes) as adding a deprecation should cause consumers to re-check
> their usage of any such declarations. I'm not sure what to do with
> consumers that have extant diagnostics…this would make it tough to
> iteratively fix warnings as any implementation edit will leave the
> declhash unchanged and the consumer won't give a fresh set of diagnostic
> output with the changes included. However, is it possible to craft
> something where:
>
> - a diagnostic comes "from" a module that is triggered by some usage
> pattern in the consumer
> - the fix doesn't change the declhash
> - the importer no longer has the diagnostic (or has more diagnostics)
>
> It's late and I'm not well-versed in module details beyond how to build
> them enough to come up with something right now, but if we can come up
> with such a scenario, something even more sophisticated will be
> necessary than the above function.
IIUC, this paragraph is mainly about avoiding re-run of **failed** consumers. Do I understand correctly?
For the fix the warnings case, if we fixed the warnings in the consumers, we need to rebuild the consumers for sure. And if we tried to fix or modify the warnings-related things in the modules, it should be the job of the tool to change.
> The "Hash of declarations" strategy is workable with a `restat = 1`
> feature though. Something like this:
>
> c++ -o pmi.cppm -fmodule-output=pmi.pch.tmp &&
> copy_if_needed pmi.pch.tmp pmi.pch
Oh, I felt you misunderstand my point. The method post here is different from the post in https://discourse.llvm.org/t/rfc-c-20-modules-introduce-thin-bmi-and-decls-hash/74755 <https://discourse.llvm.org/t/rfc-c-20-modules-introduce-thin-bmi-and-decls-hash/74755 >. It is not about the decls hash of **all the declarations in the module**. But only the decls hash used by certain consumer(s). So the working example in my mind is 'we'll always change the BMI; but it is up to the result of the query to change the consumers or not'.
And IIUC, cmake may not be able to do that for the same reason that cmake can't make the `Used Files` strategy.
Thanks,
Chuanqi
Thanks,
Chuanqi
------------------------------------------------------------------
From:Ben Boeckel <ben.boeckel_at_[hidden]>
Send Time:2023 Nov. 22 (Wed.) 10:57
To: SG15 <sg15_at_[hidden]>
Cc:Chuanqi <chuanqi.xcq_at_[hidden]inc.com>
Subject:Re: [SG15] [Modules] [P3057] Two finer-grained compilation models for named modules
On Tue, Nov 21, 2023 at 16:44:51 +0800, Chuanqi Xu via SG15 wrote:
> It is a major concern that incremental build of modules may build too
> many files if the interface units or the included files get changed.
> To solve/mitigate the issue, I presented two solutions. See
> https://isocpp.org/files/papers/P3057R0.html
> <https://isocpp.org/files/papers/P3057R0.html > for details.
> The demo tools in the compiler (clang) side are presented. So the
> build tools are able to do some simple experiments.
> Feedbacks or concerns are highly appreciated.
IIUC, the "Used Files" strategy isn't going to work for `make` or
`ninja` because there's no way to "mask" edges of dependencies.
("a -> b" is "a is used by b" or "b depends on a")
 stdio.h -> pmi.cppm -> pmi.pch -> use.cpp
We'd need a way to specify "`.pch` usage ignores transitive dependencies
for direct consumers" which has no spelling in either tool's syntax.
The "Hash of declarations" strategy is workable with a `restat = 1`
feature though. Something like this:
 c++ -o pmi.cppm -fmodule-output=pmi.pch.tmp &&
 copy_if_needed pmi.pch.tmp pmi.pch
where (in verbose, but defensive, Bourne Shell):
 copy_if_needed () {
 # Grab parameters.
 local in="$1"
 readonly in
 shift
 local out="$1"
 readonly out
 shift
 # Whether to sync the timestamp or not.
 local preserve_mtime=false
 # Whether to update or not.
 local update=false
 if ! [ -f "$out" ]; then
 # No output? Update.
 update=true
 else
 local declhashin
 local declhashout
 # Compute the declhash of each file.
 declhashin="$( declhash "$in" )"
 declhashout="$( declhash "$out" )"
 readonly declhashain
 readonly declhashaout
 if ! [ "$declhashin" = "$declhashout" ]; then
 # Different declhash? Update.
 update=true
 elif ! cmp --quiet "$in" "$out"
 # The file is different; copy, but use the old
 # timestamp.
 update=true
 preserve_mtime=true
 fi
 fi
 readonly update
 readonly preserve_mtime
 if $preserve_mtime; then
 # Use the previous timestamp. Successful builds will be
 # skipped due to `restat = 1`. Failed builds will get new
 # diagnostics as needed though.
 touch -m --reference "$out" "$in"
 fi
 if $update; then
 # Replace the file.
 mv "$in" "$out"
 else
 # Remove the intermediate file.
 rm "$in"
 fi
 }
would suffice for not rebuilding *successful* consumers while also
ensuring that consumers that *have not yet succeeded* get updated
diagnostics. This does mean that any diagnostic-affecting changes would
need to be included in this declhash (e.g., `[[deprecated]]`
attributes) as adding a deprecation should cause consumers to re-check
their usage of any such declarations. I'm not sure what to do with
consumers that have extant diagnostics…this would make it tough to
iteratively fix warnings as any implementation edit will leave the
declhash unchanged and the consumer won't give a fresh set of diagnostic
output with the changes included. However, is it possible to craft
something where:
- a diagnostic comes "from" a module that is triggered by some usage
 pattern in the consumer
- the fix doesn't change the declhash
- the importer no longer has the diagnostic (or has more diagnostics)
It's late and I'm not well-versed in module details beyond how to build
them enough to come up with something right now, but if we can come up
with such a scenario, something even more sophisticated will be
necessary than the above function.
--Ben