C++ Logo

sg15

Advanced search

Re: [Modules] [P3057] Two finer-grained compilation models for named modules

From: Chuanqi Xu <chuanqi.xcq_at_[hidden]>
Date: Wed, 22 Nov 2023 10:21:55 +0800
Hi Dani,
> What you propose is to go even farther: create a dependency on /all/ explicit or implicit importers of a given module interface, right? I just want to make sure that we are all on the same page.
>
> You envision a mechanism to inform build systems about the /actual/ dependencies of a modules, based on the /uses/ in importers and the entities they reference in the module interface. Correct?
Basically correct. But not **a** module interface. It should be workable with multiple module interfaces no matter in the hash based solution or in the used files based solution.
> If I understand you correctly, you speak about primary module interfaces that
>
> * /use/ headers from the global module fragment
> * /export/ using declarations to make names from said headers visible to importers of the module and also create a dependency to /parts/ of the contents of those headers as prescribed in [module.global.frag]/3, taking into account the discarding of entities that are determined as "unused" according to [module.global.frag]/4
> * /emit/ some of the surviving (the step before) declarations into the BMI, and the other survivors into the object file. The target artifact is determined individually by the properties of each declaration and possibly the way how an modules implementation works.
Yeah, and in fact there should be more use cases. I just removed them from the paper since I felt a longer paper is harder to read.
One use case is that we're implementing a big module and the another case is about the mono repo.
In the big module case, imaging a case like:
```
// big_module.cppm
export module big_module;
export import :part1;
export import :part2;
...
export import :part_n;
// impl1.cpp
module big_module;
...
// impl2.cpp
module big_module;
...
// impl_n.cpp
module big_module;
...
```
Currently, in the scanner-based solution, all the implementation units will depend on all the partitions. Then it'll rebuild the whole project after we touch any of the partitions.
---
Another use case may be the monorepo. In a monorepo we have many sub-repos, and it should make sense to wrap these sub-repos into different modules. Then it will imply a big rebuild after we change the interfaces of a module in the earlier place of dependencies.
---
In the previous talk to other people, they suggest that we can/should put as many declarations as we can to the implementation unit and we should keep the interface units small. Yes it is true and correct. But it wont be easier to reach that. On the one hand, the templates are the devil. On the other hand, most existing projects that wants to use modules need to support headers too for a relative long time. In such cases, the rebuilding issue will be a pain for modules. And I guess fewer people will be interested in modules if we can't solve such issues.
----
BTW, the most interesting part of the proposed solutions to me is that we don't need to touch the implementation of the compiler at all. I implemented it by a standalone tool and 2 plugins. So it won't bring any cost to users that don't use it and it won't be a trouble for developers to maintain.
Thanks,
Chuanqi
Thanks,
Chuanqi
------------------------------------------------------------------
From: SG15 <sg15_at_[hidden]>
Send Time:2023 Nov. 22 (Wed.) 01:35
To: SG15 <sg15_at_[hidden]>
Cc:Daniela Engert <dani_at_ngrt.de>
Subject:Re: [SG15] [Modules] [P3057] Two finer-grained compilation models for named modules
Am 21.11.2023 um 09:44 schrieb Chuanqi Xu via SG15:
It is a major concern that incremental build of modules may build too many files if the interface units or the included files get changed. To solve/mitigate the issue, I presented two solutions. See https://isocpp.org/files/papers/P3057R0.html <https://isocpp.org/files/papers/P3057R0.html > for details. 
The demo tools in the compiler (clang) side are presented. So the build tools are able to do some simple experiments.
If I understand you correctly, you speak about primary module interfaces that
 * /use/ headers from the global module fragment
 * /export/ using declarations to make names from said headers visible to importers of the module and also create a dependency to /parts/ of the contents of those headers as prescribed in [module.global.frag]/3, taking into account the discarding of entities that are determined as "unused" according to [module.global.frag]/4
 * /emit/ some of the surviving (the step before) declarations into the BMI, and the other survivors into the object file. The target artifact is determined individually by the properties of each declaration and possibly the way how an modules implementation works.
In general, each module interface has dependencies
 * on all headers that it directly or transitively #includes into the GMF
 * on all headers that it directly or transitively #includes into the purview
 * on all modules from inside or outside the module the interface pertains to
Those are the dependencies if you look at them on a file / TU basis. On a finer granularity level, you can look at /actual/ dependencies based on an analysis like the one in [module.global.frag] as described before, applied to /all/ of the file / TU dependencies above. Some of the dependencies in the set above might to turn out to be no real dependencies if you just look close enough.
What you propose is to go even farther: create a dependency on /all/ explicit or implicit importers of a given module interface, right? I just want to make sure that we are all on the same page.
You envision a mechanism to inform build systems about the /actual/ dependencies of a modules, based on the /uses/ in importers and the entities they reference in the module interface. Correct?
Thanks
 Dani

Received on 2023-11-22 02:22:02