Date: Mon, 21 Jun 2021 14:56:28 -0400
It's worth noting that one if the big gaps in pkg-config is its lack of metadata about what language runtimes particular packages (libraries) require. It's basically assumed that the executable build process knows how to handle it. In the case of C++ executables, it's assumed you'll just compile with a C++ compiler.
This is because pkg-config is just a flag aggregator that is good at walking graphs. It doesn't know anything about whether flags are complete if even coherent. That is an exercise for the distribution maintainers. And, no, just shipping header only doesn't solve the problem.
If you want to ship a C++ library that implicitly depends on the C++ standard library but exposes a clean C API, there's not a portable way to express that in pkg-config. There's no standard 'language_runtimes' field or anything like that. You could add equivalent link flags, but those flags will be overly specific or even wrong.
CMake does provide a way to express language runtime requirements for prebuilt (IMPORTED) libraries. I don't recall offhand if CMake modules (Find or Config packages) are typically implemented that way. I don't believe they are. Bloomberg is working with Kitware to enhance CMake to support modeling and consuming language runtime requirements slightly better, for what it's worth.
Incidentally, I didn't see anything in CPS about language runtime requirements, but I might have missed it. That seems like a solvable problem if folks want to use CPS as a starting point instead of building on pkg-config.
Anyway, this is a lot of detail about a specific aspect of discovering dependencies' build requirements, mostly to provide color. Collaboration and consensus building on these problems would be welcomed enthusiastically on my part. Even if it were mostly informative or nonbinding to start with.
Bret Brown
On Mon, Jun 21, 2021, at 1:45 PM, Isabella Muerte via SG15 wrote:
>
>
> On Mon, Jun 21, 2021, at 10:30, Gabriel Dos Reis wrote:
>> * But in the end you're actually usually taking the MSVC route and (instead of using a pragma) using a #[link(...)] attribute at the file scope, though it can also be attached to modules IIRC
>>
>> Yes, that was the plan with attributes on module declarations and import-declarations. Not yet implemented in released MSVC.
>>
>
> The difference to note (for others who may not be aware) with attributes in rust and C++ however are that attributes are required to be implemented. The Rust Reference book states:
>
>>
>> Attributes are modeled on Attributes in ECMA-335 <https://www.ecma-international.org/publications/standards/Ecma-335.htm>, with the syntax coming from ECMA-334 <https://www.ecma-international.org/publications/standards/Ecma-334.htm> (C#)
>
> For those curious, you can see the builtin-attributes related to ABI, linking, symbols, and FFI here: https://doc.rust-lang.org/reference/attributes.html#built-in-attributes-index
>
>> -- Gaby
>>
>>
>> *From:* SG15 <sg15-bounces_at_[hidden]> *On Behalf Of *Isabella Muerte via SG15
>> *Sent:* Monday, June 21, 2021 9:34 AM
>> *To:* SG15 <sg15_at_[hidden]>
>> *Cc:* Isabella Muerte <imuerte_at_[hidden]>
>> *Subject:* Re: [SG15] [EXTERNAL] Re: Draft: Requirements for Usage of C++ Modules at Bloomberg
>>
>>
>>
>>
>> On Mon, Jun 21, 2021, at 09:25, Poliakoff, David Zoeller via SG15 wrote:
>>> This would be beautiful, but one worry I have is that CMake handles a lot of ugly "if a target does X in Fortran, and a target Y in C++ depends on it, you need to add -foo to the CXXFlags." It's pretty gross. Add in the CUDA as a "language" support and you multiply the grossness. That said, I'm 100% with you on this being desirable, one of the ugly parts of CMake is how it integrates with a build system that uses pkg-config, or does something gross with their CMake, if those CMake scripts could become five lines on top of "just read the CPS artifact and do what is says, forehead" I would not be opposed.
>>>
>>> Does anybody know how/if these newer languages handle linking in external source code? Like, if I have Golang (picking randomly) app that needs visibility into a C++ and HIP library's headers, what that looks like? Or perhaps the Golang flags need to change based on how that library was compiled?
>>
>> you *might* want to pick a better language than golang 😅. The approach there is to use special "pragma comments" that work with gcc (and only gcc!). There is nothing you can really do to get around this, though I discovered you can use //go:generate to generate another file that has the correct //go:build pragma comments. But then your build goes from go build ./... to go generate ./...; go build ./... and this is apparently *too much* for a lot of people. Golang is extremely hostile to FFI and this is by design.
>>
>> In the case of rust, rustc itself does not know how to link this code into a crate, but rustc is dependent on the same linker as C++ and in some cases just calls the C or C++ compiler directly (and in the case of the stable-x86_64-pc-windows-gnu toolchain, they just provide a moderately out of date implementation of gcc). Tools like cargo handle some of this for you, but the real 'magic' for rust is that one crate is one translation unit and this simplfies a lot of the linking. But in the end you're actually usually taking the MSVC route and (instead of using a pragma) using a #[link(...)] attribute at the file scope, though it can also be attached to modules IIRC.
>>
>>> Also I'm reading this, people feel free to say "that's for the Packaging WG" and I'll leave it off of this thread
>>>
>>> Best,
>>>
>>> David P
>>>
>>> On 6/21/21, 9:29 AM, "SG15 on behalf of Matthew Woehlke via SG15" <sg15-bounces_at_[hidden] <mailto:sg15-bounces%40lists.isocpp.org> on behalf of sg15_at_[hidden] <mailto:sg15%40lists.isocpp.org>> wrote:
>>>
>>> ... (Admittedly, I don't have the time to dig into it too
>>> much, and am not as involved with build systems now as I was a few years
>>> ago. What I want to see, however, is a concerted effort being made to
>>> replace CMake's native package scripts.)
>>>
>>> ...
>>>
>>> --
>>> Matthew
>>> _______________________________________________
>>> SG15 mailing list
>>> SG15_at_[hidden] <mailto:SG15%40lists.isocpp.org>
>>> https://lists.isocpp.org/mailman/listinfo.cgi/sg15 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isocpp.org%2Fmailman%2Flistinfo.cgi%2Fsg15&data=04%7C01%7Cgdr%40microsoft.com%7C262869adc4df4082b70608d934d2618f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637598900457518820%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PFGLfrLwHCZd19bW%2FE8Bs9zj5Ph3SB%2BBlCKgeB2W9Tc%3D&reserved=0>
>>>
>>> _______________________________________________
>>> SG15 mailing list
>>> SG15_at_[hidden] <mailto:SG15%40lists.isocpp.org>
>>> https://lists.isocpp.org/mailman/listinfo.cgi/sg15 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isocpp.org%2Fmailman%2Flistinfo.cgi%2Fsg15&data=04%7C01%7Cgdr%40microsoft.com%7C262869adc4df4082b70608d934d2618f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637598900457518820%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PFGLfrLwHCZd19bW%2FE8Bs9zj5Ph3SB%2BBlCKgeB2W9Tc%3D&reserved=0>
>>>
>>
>
> _______________________________________________
> SG15 mailing list
> SG15_at_[hidden] <mailto:SG15%40lists.isocpp.org>
> https://lists.isocpp.org/mailman/listinfo.cgi/sg15
>
This is because pkg-config is just a flag aggregator that is good at walking graphs. It doesn't know anything about whether flags are complete if even coherent. That is an exercise for the distribution maintainers. And, no, just shipping header only doesn't solve the problem.
If you want to ship a C++ library that implicitly depends on the C++ standard library but exposes a clean C API, there's not a portable way to express that in pkg-config. There's no standard 'language_runtimes' field or anything like that. You could add equivalent link flags, but those flags will be overly specific or even wrong.
CMake does provide a way to express language runtime requirements for prebuilt (IMPORTED) libraries. I don't recall offhand if CMake modules (Find or Config packages) are typically implemented that way. I don't believe they are. Bloomberg is working with Kitware to enhance CMake to support modeling and consuming language runtime requirements slightly better, for what it's worth.
Incidentally, I didn't see anything in CPS about language runtime requirements, but I might have missed it. That seems like a solvable problem if folks want to use CPS as a starting point instead of building on pkg-config.
Anyway, this is a lot of detail about a specific aspect of discovering dependencies' build requirements, mostly to provide color. Collaboration and consensus building on these problems would be welcomed enthusiastically on my part. Even if it were mostly informative or nonbinding to start with.
Bret Brown
On Mon, Jun 21, 2021, at 1:45 PM, Isabella Muerte via SG15 wrote:
>
>
> On Mon, Jun 21, 2021, at 10:30, Gabriel Dos Reis wrote:
>> * But in the end you're actually usually taking the MSVC route and (instead of using a pragma) using a #[link(...)] attribute at the file scope, though it can also be attached to modules IIRC
>>
>> Yes, that was the plan with attributes on module declarations and import-declarations. Not yet implemented in released MSVC.
>>
>
> The difference to note (for others who may not be aware) with attributes in rust and C++ however are that attributes are required to be implemented. The Rust Reference book states:
>
>>
>> Attributes are modeled on Attributes in ECMA-335 <https://www.ecma-international.org/publications/standards/Ecma-335.htm>, with the syntax coming from ECMA-334 <https://www.ecma-international.org/publications/standards/Ecma-334.htm> (C#)
>
> For those curious, you can see the builtin-attributes related to ABI, linking, symbols, and FFI here: https://doc.rust-lang.org/reference/attributes.html#built-in-attributes-index
>
>> -- Gaby
>>
>>
>> *From:* SG15 <sg15-bounces_at_[hidden]> *On Behalf Of *Isabella Muerte via SG15
>> *Sent:* Monday, June 21, 2021 9:34 AM
>> *To:* SG15 <sg15_at_[hidden]>
>> *Cc:* Isabella Muerte <imuerte_at_[hidden]>
>> *Subject:* Re: [SG15] [EXTERNAL] Re: Draft: Requirements for Usage of C++ Modules at Bloomberg
>>
>>
>>
>>
>> On Mon, Jun 21, 2021, at 09:25, Poliakoff, David Zoeller via SG15 wrote:
>>> This would be beautiful, but one worry I have is that CMake handles a lot of ugly "if a target does X in Fortran, and a target Y in C++ depends on it, you need to add -foo to the CXXFlags." It's pretty gross. Add in the CUDA as a "language" support and you multiply the grossness. That said, I'm 100% with you on this being desirable, one of the ugly parts of CMake is how it integrates with a build system that uses pkg-config, or does something gross with their CMake, if those CMake scripts could become five lines on top of "just read the CPS artifact and do what is says, forehead" I would not be opposed.
>>>
>>> Does anybody know how/if these newer languages handle linking in external source code? Like, if I have Golang (picking randomly) app that needs visibility into a C++ and HIP library's headers, what that looks like? Or perhaps the Golang flags need to change based on how that library was compiled?
>>
>> you *might* want to pick a better language than golang 😅. The approach there is to use special "pragma comments" that work with gcc (and only gcc!). There is nothing you can really do to get around this, though I discovered you can use //go:generate to generate another file that has the correct //go:build pragma comments. But then your build goes from go build ./... to go generate ./...; go build ./... and this is apparently *too much* for a lot of people. Golang is extremely hostile to FFI and this is by design.
>>
>> In the case of rust, rustc itself does not know how to link this code into a crate, but rustc is dependent on the same linker as C++ and in some cases just calls the C or C++ compiler directly (and in the case of the stable-x86_64-pc-windows-gnu toolchain, they just provide a moderately out of date implementation of gcc). Tools like cargo handle some of this for you, but the real 'magic' for rust is that one crate is one translation unit and this simplfies a lot of the linking. But in the end you're actually usually taking the MSVC route and (instead of using a pragma) using a #[link(...)] attribute at the file scope, though it can also be attached to modules IIRC.
>>
>>> Also I'm reading this, people feel free to say "that's for the Packaging WG" and I'll leave it off of this thread
>>>
>>> Best,
>>>
>>> David P
>>>
>>> On 6/21/21, 9:29 AM, "SG15 on behalf of Matthew Woehlke via SG15" <sg15-bounces_at_[hidden] <mailto:sg15-bounces%40lists.isocpp.org> on behalf of sg15_at_[hidden] <mailto:sg15%40lists.isocpp.org>> wrote:
>>>
>>> ... (Admittedly, I don't have the time to dig into it too
>>> much, and am not as involved with build systems now as I was a few years
>>> ago. What I want to see, however, is a concerted effort being made to
>>> replace CMake's native package scripts.)
>>>
>>> ...
>>>
>>> --
>>> Matthew
>>> _______________________________________________
>>> SG15 mailing list
>>> SG15_at_[hidden] <mailto:SG15%40lists.isocpp.org>
>>> https://lists.isocpp.org/mailman/listinfo.cgi/sg15 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isocpp.org%2Fmailman%2Flistinfo.cgi%2Fsg15&data=04%7C01%7Cgdr%40microsoft.com%7C262869adc4df4082b70608d934d2618f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637598900457518820%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PFGLfrLwHCZd19bW%2FE8Bs9zj5Ph3SB%2BBlCKgeB2W9Tc%3D&reserved=0>
>>>
>>> _______________________________________________
>>> SG15 mailing list
>>> SG15_at_[hidden] <mailto:SG15%40lists.isocpp.org>
>>> https://lists.isocpp.org/mailman/listinfo.cgi/sg15 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isocpp.org%2Fmailman%2Flistinfo.cgi%2Fsg15&data=04%7C01%7Cgdr%40microsoft.com%7C262869adc4df4082b70608d934d2618f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637598900457518820%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PFGLfrLwHCZd19bW%2FE8Bs9zj5Ph3SB%2BBlCKgeB2W9Tc%3D&reserved=0>
>>>
>>
>
> _______________________________________________
> SG15 mailing list
> SG15_at_[hidden] <mailto:SG15%40lists.isocpp.org>
> https://lists.isocpp.org/mailman/listinfo.cgi/sg15
>
Received on 2021-06-21 13:56:54