Date: Fri, 1 Feb 2019 21:10:37 +0000
* In the header generation cases I have encountered, there tend to be a small number of source files in some domain specific language. I can generate all the headers that originate from that small set of source files. The small number of inputs makes the process tolerable.
I have seen from small, to medium, to large – Windows source code would be an example of large.
With modules, I expect the number of files need needing scanning for dependency to be small. Of course if you adopt a programming style that conflates modules with Java classes, where you get one module per class, you get what you ask for…
* If I apply that same practice to modules, would I need to scan the entirety of my project, boost, qt, and any other dependencies I have before compiling anything? My current understanding is that I would need to.
How so?
* With vigorous hand-waving
Which I am not doing, and won’t do.
* , you could say that each of those external projects could provide that dependency mapping for their project so that clients don’t need to. But which file format(s) do they choose to distribute to advertise this mapping?
That is one possible solution. Both Tom and I have suggested (independently) MANIFEST-style file. There are other proposals on the table by Isabelle and Richard.
* Would every library need to provide something akin to a CMakeLists.txt, Visual Studio project, bazel build file, etc… in order to be usable in those environments?
My view is that – regardless of modules – we (the C++ community) do need to have a common way to describe components (e.g. where installed build artefacts go, etc.)
I look at Maven, and say “yup, they got something right.”
-- Gaby
From: tooling-bounces_at_[hidden] <tooling-bounces_at_[hidden]> On Behalf Of Ben Craig
Sent: Friday, February 1, 2019 1:01 PM
To: WG21 Tooling Study Group SG15 <tooling_at_open-std.org>
Subject: Re: [Tooling] Modules
In the header generation cases I have encountered, there tend to be a small number of source files in some domain specific language. I can generate all the headers that originate from that small set of source files. The small number of inputs makes the process tolerable.
If I apply that same practice to modules, would I need to scan the entirety of my project, boost, qt, and any other dependencies I have before compiling anything? My current understanding is that I would need to.
With vigorous hand-waving, you could say that each of those external projects could provide that dependency mapping for their project so that clients don’t need to. But which file format(s) do they choose to distribute to advertise this mapping? Would every library need to provide something akin to a CMakeLists.txt, Visual Studio project, bazel build file, etc… in order to be usable in those environments?
From: tooling-bounces_at_[hidden]<mailto:tooling-bounces_at_[hidden]> <tooling-bounces_at_[hidden]<mailto:tooling-bounces_at_[hidden]>> On Behalf Of Gabriel Dos Reis
Sent: Friday, February 1, 2019 2:42 PM
To: WG21 Tooling Study Group SG15 <tooling_at_[hidden]<mailto:tooling_at_[hidden]>>
Subject: Re: [Tooling] Modules
* ** This isn’t 100% true, as generated headers from tools like SWIG, protobufs, thrift, and I think some QT things, all require those headers to be generated before you can compile your .cpp files. In my experience, this is usually handled by doing all the header generation early in the build process, before any .cpp files are built, and that approach doesn’t scale well to modules. build2 does this different and more correctly, and I suspect that Google’s internal build process is similar.
You can think of the metadata generation as generation of the headers, early in the build process.
From: tooling-bounces_at_open-std.org<mailto:tooling-bounces_at_[hidden]> <tooling-bounces_at_[hidden]<mailto:tooling-bounces_at_[hidden]>> On Behalf Of Ben Craig
Sent: Friday, February 1, 2019 7:47 AM
To: WG21 Tooling Study Group SG15 <tooling_at_[hidden]rg<mailto:tooling_at_[hidden]>>
Subject: Re: [Tooling] Modules
Current state of things: Within a library*, no dependency information** is needed in order to generate a clean build. I can build all my .cpp files in any order I want.
Future state of things: With a library, dependency information will need to be generated / provided in order to generate a clean build. I need to build my modules in dependency order. The correct order depends upon the contents of the modules.
* Across libraries, dependency order matters again because later linking steps depend on earlier linking steps.
** This isn’t 100% true, as generated headers from tools like SWIG, protobufs, thrift, and I think some QT things, all require those headers to be generated before you can compile your .cpp files. In my experience, this is usually handled by doing all the header generation early in the build process, before any .cpp files are built, and that approach doesn’t scale well to modules. build2 does this different and more correctly, and I suspect that Google’s internal build process is similar.
From: tooling-bounces_at_[hidden]g<mailto:tooling-bounces_at_[hidden]> <tooling-bounces_at_[hidden]<mailto:tooling-bounces_at_[hidden]>> On Behalf Of Titus Winters
Sent: Friday, February 1, 2019 8:25 AM
To: WG21 Tooling Study Group SG15 <tooling_at_[hidden]rg<mailto:tooling_at_[hidden]>>
Subject: Re: [Tooling] Modules
No, naturally not. The build system handles transitivity and making transitive includes available in each parallel build shard.
I'm just trying to figure out how much of this is fundamental to all build systems (I don't think it is) vs. "I just want globbing to be enough" (I also don't think it's that). There's some subtlety in here that I think is getting lost and conflated with "stuff works this way now". Of course if we change things, things will be different.
On Fri, Feb 1, 2019 at 9:20 AM Boris Kolpackov <boris_at_[hidden]<mailto:boris_at_[hidden]>> wrote:
Titus Winters <titus_at_google.com<mailto:titus_at_[hidden]>> writes:
> We've been doing explicit statements of the dependency chain for our
> codebase for almost 20 years, and I've literally never heard a new hire (or
> anyone else) say it is a "huge" burden.
The question is to what degree. I am sure you don't require new
hires to manually specify for each translation unit dependencies
on headers it includes, transitively?
But that would sure make for a nice hazing ritual.
_______________________________________________
Tooling mailing list
Tooling_at_[hidden]<mailto:Tooling_at_[hidden]>
http://www.open-std.org/mailman/listinfo/tooling<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__nam06.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttp-2D3A-5F-5Fwww.open-2D2Dstd.org-5Fmailman-5Flistinfo-5Ftooling-2526d-253DDwMFaQ-2526c-253DI-5F0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA-2526r-253Dy8mub81SfUi-2DUCZRX0Vl1g-2526m-253DeaeY1tSTEuCzTo9xdX44sV6O7OLiE17toll5vn7KdAI-2526s-253Dbctoohp0MRXTUcbLj9zhZjut-2DQJKdYHhKi2-5Fwff-5FSKk-2526e-253D-26data-3D02-257C01-257Cgdr-2540microsoft.com-257Cb88dea0d02ba44acd4a608d6885c9591-257C72f988bf86f141af91ab2d7cd011db47-257C1-257C0-257C636846328562989351-26sdata-3DvNZY4Lu2DOLksaX-252B2nSqtNMsA-252FHHdQZF1zIjh7AQ6eY-253D-26reserved-3D0%26d%3DDwMGaQ%26c%3DI_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA%26r%3Dy8mub81SfUi-UCZRX0Vl1g%26m%3DfLj8zoWsErmIt1ruqqviQZCc6zBJ6AoprP0fx17_NbA%26s%3D0krHG0s9fTdIoLSXm5K7faOhKLkktaDA2GEvf1O0XYY%26e%3D&data=02%7C01%7Cgdr%40microsoft.com%7C04e0b2f75993409024e208d68888607c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636846516657415294&sdata=Begukdi7WMpWjsFT8czjP%2Bax%2BlHUEVBHMuCQcJCGiWg%3D&reserved=0>
I have seen from small, to medium, to large – Windows source code would be an example of large.
With modules, I expect the number of files need needing scanning for dependency to be small. Of course if you adopt a programming style that conflates modules with Java classes, where you get one module per class, you get what you ask for…
* If I apply that same practice to modules, would I need to scan the entirety of my project, boost, qt, and any other dependencies I have before compiling anything? My current understanding is that I would need to.
How so?
* With vigorous hand-waving
Which I am not doing, and won’t do.
* , you could say that each of those external projects could provide that dependency mapping for their project so that clients don’t need to. But which file format(s) do they choose to distribute to advertise this mapping?
That is one possible solution. Both Tom and I have suggested (independently) MANIFEST-style file. There are other proposals on the table by Isabelle and Richard.
* Would every library need to provide something akin to a CMakeLists.txt, Visual Studio project, bazel build file, etc… in order to be usable in those environments?
My view is that – regardless of modules – we (the C++ community) do need to have a common way to describe components (e.g. where installed build artefacts go, etc.)
I look at Maven, and say “yup, they got something right.”
-- Gaby
From: tooling-bounces_at_[hidden] <tooling-bounces_at_[hidden]> On Behalf Of Ben Craig
Sent: Friday, February 1, 2019 1:01 PM
To: WG21 Tooling Study Group SG15 <tooling_at_open-std.org>
Subject: Re: [Tooling] Modules
In the header generation cases I have encountered, there tend to be a small number of source files in some domain specific language. I can generate all the headers that originate from that small set of source files. The small number of inputs makes the process tolerable.
If I apply that same practice to modules, would I need to scan the entirety of my project, boost, qt, and any other dependencies I have before compiling anything? My current understanding is that I would need to.
With vigorous hand-waving, you could say that each of those external projects could provide that dependency mapping for their project so that clients don’t need to. But which file format(s) do they choose to distribute to advertise this mapping? Would every library need to provide something akin to a CMakeLists.txt, Visual Studio project, bazel build file, etc… in order to be usable in those environments?
From: tooling-bounces_at_[hidden]<mailto:tooling-bounces_at_[hidden]> <tooling-bounces_at_[hidden]<mailto:tooling-bounces_at_[hidden]>> On Behalf Of Gabriel Dos Reis
Sent: Friday, February 1, 2019 2:42 PM
To: WG21 Tooling Study Group SG15 <tooling_at_[hidden]<mailto:tooling_at_[hidden]>>
Subject: Re: [Tooling] Modules
* ** This isn’t 100% true, as generated headers from tools like SWIG, protobufs, thrift, and I think some QT things, all require those headers to be generated before you can compile your .cpp files. In my experience, this is usually handled by doing all the header generation early in the build process, before any .cpp files are built, and that approach doesn’t scale well to modules. build2 does this different and more correctly, and I suspect that Google’s internal build process is similar.
You can think of the metadata generation as generation of the headers, early in the build process.
From: tooling-bounces_at_open-std.org<mailto:tooling-bounces_at_[hidden]> <tooling-bounces_at_[hidden]<mailto:tooling-bounces_at_[hidden]>> On Behalf Of Ben Craig
Sent: Friday, February 1, 2019 7:47 AM
To: WG21 Tooling Study Group SG15 <tooling_at_[hidden]rg<mailto:tooling_at_[hidden]>>
Subject: Re: [Tooling] Modules
Current state of things: Within a library*, no dependency information** is needed in order to generate a clean build. I can build all my .cpp files in any order I want.
Future state of things: With a library, dependency information will need to be generated / provided in order to generate a clean build. I need to build my modules in dependency order. The correct order depends upon the contents of the modules.
* Across libraries, dependency order matters again because later linking steps depend on earlier linking steps.
** This isn’t 100% true, as generated headers from tools like SWIG, protobufs, thrift, and I think some QT things, all require those headers to be generated before you can compile your .cpp files. In my experience, this is usually handled by doing all the header generation early in the build process, before any .cpp files are built, and that approach doesn’t scale well to modules. build2 does this different and more correctly, and I suspect that Google’s internal build process is similar.
From: tooling-bounces_at_[hidden]g<mailto:tooling-bounces_at_[hidden]> <tooling-bounces_at_[hidden]<mailto:tooling-bounces_at_[hidden]>> On Behalf Of Titus Winters
Sent: Friday, February 1, 2019 8:25 AM
To: WG21 Tooling Study Group SG15 <tooling_at_[hidden]rg<mailto:tooling_at_[hidden]>>
Subject: Re: [Tooling] Modules
No, naturally not. The build system handles transitivity and making transitive includes available in each parallel build shard.
I'm just trying to figure out how much of this is fundamental to all build systems (I don't think it is) vs. "I just want globbing to be enough" (I also don't think it's that). There's some subtlety in here that I think is getting lost and conflated with "stuff works this way now". Of course if we change things, things will be different.
On Fri, Feb 1, 2019 at 9:20 AM Boris Kolpackov <boris_at_[hidden]<mailto:boris_at_[hidden]>> wrote:
Titus Winters <titus_at_google.com<mailto:titus_at_[hidden]>> writes:
> We've been doing explicit statements of the dependency chain for our
> codebase for almost 20 years, and I've literally never heard a new hire (or
> anyone else) say it is a "huge" burden.
The question is to what degree. I am sure you don't require new
hires to manually specify for each translation unit dependencies
on headers it includes, transitively?
But that would sure make for a nice hazing ritual.
_______________________________________________
Tooling mailing list
Tooling_at_[hidden]<mailto:Tooling_at_[hidden]>
http://www.open-std.org/mailman/listinfo/tooling<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__nam06.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttp-2D3A-5F-5Fwww.open-2D2Dstd.org-5Fmailman-5Flistinfo-5Ftooling-2526d-253DDwMFaQ-2526c-253DI-5F0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA-2526r-253Dy8mub81SfUi-2DUCZRX0Vl1g-2526m-253DeaeY1tSTEuCzTo9xdX44sV6O7OLiE17toll5vn7KdAI-2526s-253Dbctoohp0MRXTUcbLj9zhZjut-2DQJKdYHhKi2-5Fwff-5FSKk-2526e-253D-26data-3D02-257C01-257Cgdr-2540microsoft.com-257Cb88dea0d02ba44acd4a608d6885c9591-257C72f988bf86f141af91ab2d7cd011db47-257C1-257C0-257C636846328562989351-26sdata-3DvNZY4Lu2DOLksaX-252B2nSqtNMsA-252FHHdQZF1zIjh7AQ6eY-253D-26reserved-3D0%26d%3DDwMGaQ%26c%3DI_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA%26r%3Dy8mub81SfUi-UCZRX0Vl1g%26m%3DfLj8zoWsErmIt1ruqqviQZCc6zBJ6AoprP0fx17_NbA%26s%3D0krHG0s9fTdIoLSXm5K7faOhKLkktaDA2GEvf1O0XYY%26e%3D&data=02%7C01%7Cgdr%40microsoft.com%7C04e0b2f75993409024e208d68888607c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636846516657415294&sdata=Begukdi7WMpWjsFT8czjP%2Bax%2BlHUEVBHMuCQcJCGiWg%3D&reserved=0>
Received on 2019-02-01 22:10:41