On Sat, Feb 23, 2019 at 8:18 AM Ben Craig <ben.craig@ni.com> wrote:

I would like to find a way for users to decouple the upgrading of tools from the migration to modules.  I've got a half-baked suggestion on how to do so.  I think this has the potential to make the upgrade from C++17 to C++20 roughly the same cost to users as the upgrade from a C++14 to C++17.  This was discussed some in the impromptu tooling session on Friday at Kona 2019.

The no-build-system-upgrade constraint implies other constraints:
1. No up-front scanning of the source to find module name and dependency information, because a lot of current build systems don't currently have a scan step.
2. No dynamic dependencies between TUs.  Many current build systems assume that the .cpp -> .o[bj] transformation is trivially parallelizable.
3. No upgrade of build tool executables.  This has to work with versions of "make", "ninja", and "cmake" from 10+ years ago.
4. No drastically different file formats to parse (like binary module interfaces).
5. You _can_ add compiler / linker flags.

The scheme I have in mind would result in no build throughput improvements with the old bad build systems, but I think it would still provide the isolation benefits of modules and be conforming.  When the user is able to upgrade their build system, they can start getting the build throughput improvements.

The general idea is to treat the module interface file as a glorified header (Gaby has mentioned this possibility in various venues).  When the user passes --strawman-slow-modules to the compiler, the compiler does a textual inclusion of the module interface file (no BMI involved at all).  The textual inclusion would likely involve placing a #pragma strawman-module begin(name-of-module) directive, with a #pragma strawman-module end(name-of-module) directive at the end of the module text.  Each TU will duplicate this work.  If the compiler can emit this text file, then it can be distributed using existing technologies that are expecting preprocessed files.  This is similar in nature to clang's -frewrite-modueles (I think that's the right spelling)

So this requires that compilers support this textual modules approach.  It also requires that the compiler be able to find the module interface files without requiring the (dumb) build system to scan in advance.  The "easiest" (and slow) way to make this happen is to require that module names correspond to file names, and that compilers provide a search path.  I am well aware that this isn't fast, but this general scheme is intended for build system compatibility.  Vendors should also provide a faster thing that can be used by newer build systems.  Compilers can also provide a command line override to say where a creatively named module can be found.

Users would still need to build each module (as they have to build each .cpp) in order for all symbols to get defined.  This might disappoint some people that think that textual modules will provide behavior similar to "unity" / "blob" builds.  Non-inline function definitions in an imported module wouldn't have a strong linker definition (wrong words there, sorry) in importers... they would only be provided in the TU that defines that module.

All of this is intended to allow a fully conforming modules implementation.  It also does not preclude additional build options intended for new, smart, fast, build systems.  To the contrary, this is an area that I encourage investigation and research.

Let me know if there are holes in this plan, and if it sounds reasonable to implement.  Also let me know if this sounds like it won't help in keeping your existing tool or build system chugging along.

Ignoring distributed builds, this actually isn't really requiring anything past a mapping from module names to files, as implementations can still build the modules without ever producing that text.  I do however think that it would be useful for other non compiler tools and distributed builds to have this mode.

btw, the clang option is -frewrite-imports.

- Michael Spencer