Date: Mon, 4 Feb 2019 00:11:23 +0100
For some reasons, I don't get Gaby's emails at all. (not even in spam)
I just noticed this
With modules, I expect the number of files need needing scanning for
dependency to be small. Of course if you adopt a programming style that
conflates modules with Java classes, where you get one module per class,
you get what you ask for…
I have few remarks/questions
First, 100% of files (including existing code) will need to be scanned for
modules.
This is because, in the merged proposal, all non-modular code can import
modules ( legacy or not), at global scope, anywhere.
Then what is the size of a module?
Some misunderstanding may come from there (and I asked the question before,
but apparently got a contradicting answer)
Assuming we want to stop relying on headers, we only have modules.
(As far as build systems are concerned, modules partitions behave like
modules)
So, it isn't that far fetch that modules would be small are people tend to
limit the size of individual
files thereby limiting the size of modules.
I would expect the number of modules to be similar to the number of source
files.
Which is a lot of files to scan :)
On Sun, 3 Feb 2019 at 19:54 Isabella Muerte <imuerte_at_[hidden]> wrote:
> Definitely in the camp of "Maven got it right" on my end. Part of how they
> got it right? A standard directory layout as the default. Want to do
> anything more complex? That's on you to explicitly state so. Perhaps that's
> what we need. That way we can have some kind of implicit lookup ;)
>
>
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Friday, February 1, 2019 1:10 PM, Gabriel Dos Reis <gdr_at_[hidden]>
> wrote:
>
>
> - In the header generation cases I have encountered, there tend to be
> a small number of source files in some domain specific language. I can
> generate all the headers that originate from that small set of source
> files. The small number of inputs makes the process tolerable.
>
>
>
> I have seen from small, to medium, to large – Windows source code would be
> an example of large.
>
>
>
> With modules, I expect the number of files need needing scanning for
> dependency to be small. Of course if you adopt a programming style that
> conflates modules with Java classes, where you get one module per class,
> you get what you ask for…
>
>
>
> - If I apply that same practice to modules, would I need to scan the
> entirety of my project, boost, qt, and any other dependencies I have before
> compiling anything? My current understanding is that I would need to.
>
> How so?
>
>
>
> - With vigorous hand-waving
>
> Which I am not doing, and won’t do.
>
>
>
> - , you could say that each of those external projects could provide
> that dependency mapping for their project so that clients don’t need to.
> But which file format(s) do they choose to distribute to advertise this
> mapping?
>
> That is one possible solution. Both Tom and I have suggested
> (independently) MANIFEST-style file. There are other proposals on the
> table by Isabelle and Richard.
>
>
>
> - Would every library need to provide something akin to a
> CMakeLists.txt, Visual Studio project, bazel build file, etc… in order to
> be usable in those environments?
>
>
>
> My view is that – regardless of modules – we (the C++ community) do need
> to have a common way to describe components (e.g. where installed build
> artefacts go, etc.)
>
>
>
> I look at Maven, and say “yup, they got something right.”
>
>
>
> -- Gaby
>
>
>
> *From:* tooling-bounces_at_[hidden] <tooling-bounces_at_[hidden]> *On
> Behalf Of *Ben Craig
>
> *Sent:* Friday, February 1, 2019 1:01 PM
>
> *To:* WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
> *Subject:* Re: [Tooling] Modules
>
>
>
> In the header generation cases I have encountered, there tend to be a
> small number of source files in some domain specific language. I can
> generate all the headers that originate from that small set of source
> files. The small number of inputs makes the process tolerable.
>
>
>
> If I apply that same practice to modules, would I need to scan the
> entirety of my project, boost, qt, and any other dependencies I have before
> compiling anything? My current understanding is that I would need to.
>
>
>
> With vigorous hand-waving, you could say that each of those external
> projects could provide that dependency mapping for their project so that
> clients don’t need to. But which file format(s) do they choose to
> distribute to advertise this mapping? Would every library need to provide
> something akin to a CMakeLists.txt, Visual Studio project, bazel build
> file, etc… in order to be usable in those environments?
>
>
>
> *From:* tooling-bounces_at_[hidden] <tooling-bounces_at_[hidden]> *On
> Behalf Of *Gabriel Dos Reis
> *Sent:* Friday, February 1, 2019 2:42 PM
> *To:* WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
> *Subject:* Re: [Tooling] Modules
>
>
>
> - ** This isn’t 100% true, as generated headers from tools like SWIG,
> protobufs, thrift, and I think some QT things, all require those headers to
> be generated before you can compile your .cpp files. In my experience,
> this is usually handled by doing all the header generation early in the
> build process, before any .cpp files are built, and that approach doesn’t
> scale well to modules. build2 does this different and more correctly, and
> I suspect that Google’s internal build process is similar.
>
>
>
> You can think of the metadata generation as generation of the headers,
> early in the build process.
>
>
>
>
>
> *From:* tooling-bounces_at_[hidden] <tooling-bounces_at_[hidden]> *On
> Behalf Of *Ben Craig
> *Sent:* Friday, February 1, 2019 7:47 AM
> *To:* WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
> *Subject:* Re: [Tooling] Modules
>
>
>
> Current state of things: Within a library*, no dependency information**
> is needed in order to generate a clean build. I can build all my .cpp
> files in any order I want.
>
> Future state of things: With a library, dependency information will need
> to be generated / provided in order to generate a clean build. I need to
> build my modules in dependency order. The correct order depends upon the
> contents of the modules.
>
>
>
> * Across libraries, dependency order matters again because later linking
> steps depend on earlier linking steps.
>
> ** This isn’t 100% true, as generated headers from tools like SWIG,
> protobufs, thrift, and I think some QT things, all require those headers to
> be generated before you can compile your .cpp files. In my experience,
> this is usually handled by doing all the header generation early in the
> build process, before any .cpp files are built, and that approach doesn’t
> scale well to modules. build2 does this different and more correctly, and
> I suspect that Google’s internal build process is similar.
>
>
>
> *From:* tooling-bounces_at_[hidden] <tooling-bounces_at_[hidden]> *On
> Behalf Of *Titus Winters
> *Sent:* Friday, February 1, 2019 8:25 AM
> *To:* WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
> *Subject:* Re: [Tooling] Modules
>
>
>
> No, naturally not. The build system handles transitivity and making
> transitive includes available in each parallel build shard.
>
>
>
> I'm just trying to figure out how much of this is fundamental to all build
> systems (I don't think it is) vs. "I just want globbing to be enough" (I
> also don't think it's that). There's some subtlety in here that I think is
> getting lost and conflated with "stuff works this way now". Of course if we
> change things, things will be different.
>
>
>
>
>
> On Fri, Feb 1, 2019 at 9:20 AM Boris Kolpackov <boris_at_[hidden]>
> wrote:
>
> Titus Winters <titus_at_[hidden]> writes:
>
> > We've been doing explicit statements of the dependency chain for our
> > codebase for almost 20 years, and I've literally never heard a new hire
> (or
> > anyone else) say it is a "huge" burden.
>
> The question is to what degree. I am sure you don't require new
> hires to manually specify for each translation unit dependencies
> on headers it includes, transitively?
>
> But that would sure make for a nice hazing ritual.
> _______________________________________________
> Tooling mailing list
> Tooling_at_[hidden]
> http://www.open-std.org/mailman/listinfo/tooling
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__nam06.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttp-2D3A-5F-5Fwww.open-2D2Dstd.org-5Fmailman-5Flistinfo-5Ftooling-2526d-253DDwMFaQ-2526c-253DI-5F0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA-2526r-253Dy8mub81SfUi-2DUCZRX0Vl1g-2526m-253DeaeY1tSTEuCzTo9xdX44sV6O7OLiE17toll5vn7KdAI-2526s-253Dbctoohp0MRXTUcbLj9zhZjut-2DQJKdYHhKi2-5Fwff-5FSKk-2526e-253D-26data-3D02-257C01-257Cgdr-2540microsoft.com-257Cb88dea0d02ba44acd4a608d6885c9591-257C72f988bf86f141af91ab2d7cd011db47-257C1-257C0-257C636846328562989351-26sdata-3DvNZY4Lu2DOLksaX-252B2nSqtNMsA-252FHHdQZF1zIjh7AQ6eY-253D-26reserved-3D0%26d%3DDwMGaQ%26c%3DI_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA%26r%3Dy8mub81SfUi-UCZRX0Vl1g%26m%3DfLj8zoWsErmIt1ruqqviQZCc6zBJ6AoprP0fx17_NbA%26s%3D0krHG0s9fTdIoLSXm5K7faOhKLkktaDA2GEvf1O0XYY%26e%3D&data=02%7C01%7Cgdr%40microsoft.com%7C04e0b2f75993409024e208d68888607c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636846516657415294&sdata=Begukdi7WMpWjsFT8czjP%2Bax%2BlHUEVBHMuCQcJCGiWg%3D&reserved=0>
>
>
> _______________________________________________
> Tooling mailing list
> Tooling_at_[hidden]
> http://www.open-std.org/mailman/listinfo/tooling
>
I just noticed this
With modules, I expect the number of files need needing scanning for
dependency to be small. Of course if you adopt a programming style that
conflates modules with Java classes, where you get one module per class,
you get what you ask for…
I have few remarks/questions
First, 100% of files (including existing code) will need to be scanned for
modules.
This is because, in the merged proposal, all non-modular code can import
modules ( legacy or not), at global scope, anywhere.
Then what is the size of a module?
Some misunderstanding may come from there (and I asked the question before,
but apparently got a contradicting answer)
Assuming we want to stop relying on headers, we only have modules.
(As far as build systems are concerned, modules partitions behave like
modules)
So, it isn't that far fetch that modules would be small are people tend to
limit the size of individual
files thereby limiting the size of modules.
I would expect the number of modules to be similar to the number of source
files.
Which is a lot of files to scan :)
On Sun, 3 Feb 2019 at 19:54 Isabella Muerte <imuerte_at_[hidden]> wrote:
> Definitely in the camp of "Maven got it right" on my end. Part of how they
> got it right? A standard directory layout as the default. Want to do
> anything more complex? That's on you to explicitly state so. Perhaps that's
> what we need. That way we can have some kind of implicit lookup ;)
>
>
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Friday, February 1, 2019 1:10 PM, Gabriel Dos Reis <gdr_at_[hidden]>
> wrote:
>
>
> - In the header generation cases I have encountered, there tend to be
> a small number of source files in some domain specific language. I can
> generate all the headers that originate from that small set of source
> files. The small number of inputs makes the process tolerable.
>
>
>
> I have seen from small, to medium, to large – Windows source code would be
> an example of large.
>
>
>
> With modules, I expect the number of files need needing scanning for
> dependency to be small. Of course if you adopt a programming style that
> conflates modules with Java classes, where you get one module per class,
> you get what you ask for…
>
>
>
> - If I apply that same practice to modules, would I need to scan the
> entirety of my project, boost, qt, and any other dependencies I have before
> compiling anything? My current understanding is that I would need to.
>
> How so?
>
>
>
> - With vigorous hand-waving
>
> Which I am not doing, and won’t do.
>
>
>
> - , you could say that each of those external projects could provide
> that dependency mapping for their project so that clients don’t need to.
> But which file format(s) do they choose to distribute to advertise this
> mapping?
>
> That is one possible solution. Both Tom and I have suggested
> (independently) MANIFEST-style file. There are other proposals on the
> table by Isabelle and Richard.
>
>
>
> - Would every library need to provide something akin to a
> CMakeLists.txt, Visual Studio project, bazel build file, etc… in order to
> be usable in those environments?
>
>
>
> My view is that – regardless of modules – we (the C++ community) do need
> to have a common way to describe components (e.g. where installed build
> artefacts go, etc.)
>
>
>
> I look at Maven, and say “yup, they got something right.”
>
>
>
> -- Gaby
>
>
>
> *From:* tooling-bounces_at_[hidden] <tooling-bounces_at_[hidden]> *On
> Behalf Of *Ben Craig
>
> *Sent:* Friday, February 1, 2019 1:01 PM
>
> *To:* WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
> *Subject:* Re: [Tooling] Modules
>
>
>
> In the header generation cases I have encountered, there tend to be a
> small number of source files in some domain specific language. I can
> generate all the headers that originate from that small set of source
> files. The small number of inputs makes the process tolerable.
>
>
>
> If I apply that same practice to modules, would I need to scan the
> entirety of my project, boost, qt, and any other dependencies I have before
> compiling anything? My current understanding is that I would need to.
>
>
>
> With vigorous hand-waving, you could say that each of those external
> projects could provide that dependency mapping for their project so that
> clients don’t need to. But which file format(s) do they choose to
> distribute to advertise this mapping? Would every library need to provide
> something akin to a CMakeLists.txt, Visual Studio project, bazel build
> file, etc… in order to be usable in those environments?
>
>
>
> *From:* tooling-bounces_at_[hidden] <tooling-bounces_at_[hidden]> *On
> Behalf Of *Gabriel Dos Reis
> *Sent:* Friday, February 1, 2019 2:42 PM
> *To:* WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
> *Subject:* Re: [Tooling] Modules
>
>
>
> - ** This isn’t 100% true, as generated headers from tools like SWIG,
> protobufs, thrift, and I think some QT things, all require those headers to
> be generated before you can compile your .cpp files. In my experience,
> this is usually handled by doing all the header generation early in the
> build process, before any .cpp files are built, and that approach doesn’t
> scale well to modules. build2 does this different and more correctly, and
> I suspect that Google’s internal build process is similar.
>
>
>
> You can think of the metadata generation as generation of the headers,
> early in the build process.
>
>
>
>
>
> *From:* tooling-bounces_at_[hidden] <tooling-bounces_at_[hidden]> *On
> Behalf Of *Ben Craig
> *Sent:* Friday, February 1, 2019 7:47 AM
> *To:* WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
> *Subject:* Re: [Tooling] Modules
>
>
>
> Current state of things: Within a library*, no dependency information**
> is needed in order to generate a clean build. I can build all my .cpp
> files in any order I want.
>
> Future state of things: With a library, dependency information will need
> to be generated / provided in order to generate a clean build. I need to
> build my modules in dependency order. The correct order depends upon the
> contents of the modules.
>
>
>
> * Across libraries, dependency order matters again because later linking
> steps depend on earlier linking steps.
>
> ** This isn’t 100% true, as generated headers from tools like SWIG,
> protobufs, thrift, and I think some QT things, all require those headers to
> be generated before you can compile your .cpp files. In my experience,
> this is usually handled by doing all the header generation early in the
> build process, before any .cpp files are built, and that approach doesn’t
> scale well to modules. build2 does this different and more correctly, and
> I suspect that Google’s internal build process is similar.
>
>
>
> *From:* tooling-bounces_at_[hidden] <tooling-bounces_at_[hidden]> *On
> Behalf Of *Titus Winters
> *Sent:* Friday, February 1, 2019 8:25 AM
> *To:* WG21 Tooling Study Group SG15 <tooling_at_[hidden]>
> *Subject:* Re: [Tooling] Modules
>
>
>
> No, naturally not. The build system handles transitivity and making
> transitive includes available in each parallel build shard.
>
>
>
> I'm just trying to figure out how much of this is fundamental to all build
> systems (I don't think it is) vs. "I just want globbing to be enough" (I
> also don't think it's that). There's some subtlety in here that I think is
> getting lost and conflated with "stuff works this way now". Of course if we
> change things, things will be different.
>
>
>
>
>
> On Fri, Feb 1, 2019 at 9:20 AM Boris Kolpackov <boris_at_[hidden]>
> wrote:
>
> Titus Winters <titus_at_[hidden]> writes:
>
> > We've been doing explicit statements of the dependency chain for our
> > codebase for almost 20 years, and I've literally never heard a new hire
> (or
> > anyone else) say it is a "huge" burden.
>
> The question is to what degree. I am sure you don't require new
> hires to manually specify for each translation unit dependencies
> on headers it includes, transitively?
>
> But that would sure make for a nice hazing ritual.
> _______________________________________________
> Tooling mailing list
> Tooling_at_[hidden]
> http://www.open-std.org/mailman/listinfo/tooling
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__nam06.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttp-2D3A-5F-5Fwww.open-2D2Dstd.org-5Fmailman-5Flistinfo-5Ftooling-2526d-253DDwMFaQ-2526c-253DI-5F0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA-2526r-253Dy8mub81SfUi-2DUCZRX0Vl1g-2526m-253DeaeY1tSTEuCzTo9xdX44sV6O7OLiE17toll5vn7KdAI-2526s-253Dbctoohp0MRXTUcbLj9zhZjut-2DQJKdYHhKi2-5Fwff-5FSKk-2526e-253D-26data-3D02-257C01-257Cgdr-2540microsoft.com-257Cb88dea0d02ba44acd4a608d6885c9591-257C72f988bf86f141af91ab2d7cd011db47-257C1-257C0-257C636846328562989351-26sdata-3DvNZY4Lu2DOLksaX-252B2nSqtNMsA-252FHHdQZF1zIjh7AQ6eY-253D-26reserved-3D0%26d%3DDwMGaQ%26c%3DI_0YwoKy7z5LMTVdyO6YCiE2uzI1jjZZuIPelcSjixA%26r%3Dy8mub81SfUi-UCZRX0Vl1g%26m%3DfLj8zoWsErmIt1ruqqviQZCc6zBJ6AoprP0fx17_NbA%26s%3D0krHG0s9fTdIoLSXm5K7faOhKLkktaDA2GEvf1O0XYY%26e%3D&data=02%7C01%7Cgdr%40microsoft.com%7C04e0b2f75993409024e208d68888607c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636846516657415294&sdata=Begukdi7WMpWjsFT8czjP%2Bax%2BlHUEVBHMuCQcJCGiWg%3D&reserved=0>
>
>
> _______________________________________________
> Tooling mailing list
> Tooling_at_[hidden]
> http://www.open-std.org/mailman/listinfo/tooling
>
Received on 2019-02-04 00:11:38