Date: Sun, 20 Oct 2024 11:31:23 -0700
On Sunday 20 October 2024 10:02:13 GMT-7 Federico Kircheis via Std-Discussion
wrote:
> > So if you include this header in two TUs and they both end up in your
> > executable at runtime, it's an ODR violation.
>
> The two TU have a global variable named instance of the same type at
> different addresses.
Indeed. You're quoting to me what can happen under ODR violations.
> is equivalent, and to the best of my knowledge not problematic.
> The two variables are declared independently, at two different locations.
The difference here is that int is a primitive, so nothing gets emitted in the
first place, unless its address is taken. In that case, it's emitted as a
global and there is a problem.
> > The problem is what happens when you load the application for runtime:
> > then
> > lib1 and lib2 are both in memory at the same time, working as if "they
> > were
> > another TU". Therefore, their both including lib0.cpp implies it's an ODR
> > violation.
>
> I'm not sure what you mean by working as if "they were another TU".
> There is no another TU in the source code, there is only one.
There are two copies of lib0.cpp. That's effectively as if you had two
identical files compiled into your executable. Try this sometime and see what
happens.
The discussion started with "on Unix, shared libraries behave as if they were
'just another TU'" and I'm showing to you that it is true, if you don't use
extensions. Your examples are actually showing that it is and the problems
you're having are exactly what would happen if they were, indeed other TUs.
> Previously it was claimed you need to do something strange or some
> something like attributes and functions like dlopen.
> I claimed it is not necessary, and that standard c++ code with globals
> does not always work as expected.
>
> I think we agree on this point?
> Or did I do something strange?
I am disagreeing with you. C++ with shared libraries works exactly as
expected, so long as you stay inside the Standard and use well-formed code
with defined behaviour. Your code is ill-formed, therefore the Standard makes
no claim about its behaviour.
The fact that you _shouldn't_ use just standard C++ with shared libraries is a
point. People must read Dynamic Shared Objects 101, must use hidden
visibility, must use non-weak virtual tables, and a few other things. That's
not optional if you want a quality library. I personally couldn't care less
that you can't describe this in Standard C++. You must do this; I live in the
real world where extensions do exist to be used.
But IMO, C++ should get a way to describe this in the Standard.
> > Yes, but if wishes were fishes don't count. They're not.
>
> I do not think I understand.
> I wanted to point out that taking exactly the same source code,
> unmodified, and packaging it differently, avoids the issue.
Yes. But you changed how you compile and how many times you link lib0.cpp into
your executable. If you fix your problem, then the problem is not there any
more.
> Let's ignore I mentioned static libs and pretend I wrote "If you remove
> the libraries and compile everything together".
Yes, I am reading this as "if I remove the problematic build solution and fix
the issue". I agree that in this case the problem is solved.
> > You chose to use lib1 and lib2. You can choose not to, if they don't react
> > to your bug reports. Vote with your feet and your wallet.
>
> Is the bug in lib0, lib1 and lib2?
> They all work correctly, I do not see any UB in them, even after
> everything we wrote, as long as you use them independently.
> They do not work when mixed together.
Agreed. Strictly speaking, it's the combination of lib1 and lib2 into one
executable. You can't do that.
This is no different than having two copies of lib0 in your source code in
different paths and only using one of the two of them at a time, which of
course works. But if you try to use both, then it doesn't.
IMO, you solve this by making lib0 a shared library and linking lib1 and lib2
to it.
> But the issue also happens if two libraries have no "lib0" as common
> dependency.
> What if two projects happen to write
>
> const std::string answer_of_life = "42";
>
> as implementation detail in their shared library?
Yes, if you make an ODR violation, you have ill-formed program. How hard is it
to understand that?
> You can obviously report a bug to those libraries to hide their symbols
> with compiler specifics tools, but according to the standard everything
> is fine.
Sure. They *chose* to have a global symbol here (there's no "static"). That
means they are claiming "answer_of_life" as part of their ABI. Two TUs can't
do that at the same time.
If they didn't mean to claim that, then there are solutions to avoid the
collision, like namespaces.
> What about libraries that are header only?
Header-only libraries are fine, but the same ODR violation rules apply. ALL
copies of the library in the executable must be the same version. How you go
about doing that is up to you and the vendors who used that library. This is
not an optional condition: it MUST be the same version, lest your code become
ill-formed.
My answer above stands: the best solution for this is to make lib0 a shared
library of its own.
> Yes, I know; thus it is the other way round, which is my point.
> To have globals that work in shared libraries, I might need to use
> something outside of the standard, I might need pragmas/attributes/...
No, you don't. The globals work just fine, but they must of course be the same
and you must have a single definition of them. The problem here is that your
code has been violating the standard and thus is ill-formed. You're claiming
the standard has a problem in shared libraries, but that's incorrect. Your
code does.
> But your comment also mentions that having a feature that merges objects
> of shared libraries together is against common practice.
> You normally want to hide the symbols to avoid issues.
Indeed. Like I said, Dynamic Shared Objects 101 is mandatory reading.
> It is a surprise for end-users, and makes writing code harder.
> It would be one thing to say "avoid globals in the API of the library",
> that's relatively easy to follow.
> It is another to say that "globals used as implementation details can be
> problematic", every project I worked on used in one way or another
> multiple globals.
If you're going to use globals, make sure that you properly namespace them so
you don't collide the name with something different in another library.
The problem in your case is the presence of lib0 inside of lib1 and lib2. Stop
propagating this horrible practice of bundling copies of third-party content.
> And to return to the original discussion, I'm still not sure why this
> feature should be treated differently from all other global symbols.
It isn't.
wrote:
> > So if you include this header in two TUs and they both end up in your
> > executable at runtime, it's an ODR violation.
>
> The two TU have a global variable named instance of the same type at
> different addresses.
Indeed. You're quoting to me what can happen under ODR violations.
> is equivalent, and to the best of my knowledge not problematic.
> The two variables are declared independently, at two different locations.
The difference here is that int is a primitive, so nothing gets emitted in the
first place, unless its address is taken. In that case, it's emitted as a
global and there is a problem.
> > The problem is what happens when you load the application for runtime:
> > then
> > lib1 and lib2 are both in memory at the same time, working as if "they
> > were
> > another TU". Therefore, their both including lib0.cpp implies it's an ODR
> > violation.
>
> I'm not sure what you mean by working as if "they were another TU".
> There is no another TU in the source code, there is only one.
There are two copies of lib0.cpp. That's effectively as if you had two
identical files compiled into your executable. Try this sometime and see what
happens.
The discussion started with "on Unix, shared libraries behave as if they were
'just another TU'" and I'm showing to you that it is true, if you don't use
extensions. Your examples are actually showing that it is and the problems
you're having are exactly what would happen if they were, indeed other TUs.
> Previously it was claimed you need to do something strange or some
> something like attributes and functions like dlopen.
> I claimed it is not necessary, and that standard c++ code with globals
> does not always work as expected.
>
> I think we agree on this point?
> Or did I do something strange?
I am disagreeing with you. C++ with shared libraries works exactly as
expected, so long as you stay inside the Standard and use well-formed code
with defined behaviour. Your code is ill-formed, therefore the Standard makes
no claim about its behaviour.
The fact that you _shouldn't_ use just standard C++ with shared libraries is a
point. People must read Dynamic Shared Objects 101, must use hidden
visibility, must use non-weak virtual tables, and a few other things. That's
not optional if you want a quality library. I personally couldn't care less
that you can't describe this in Standard C++. You must do this; I live in the
real world where extensions do exist to be used.
But IMO, C++ should get a way to describe this in the Standard.
> > Yes, but if wishes were fishes don't count. They're not.
>
> I do not think I understand.
> I wanted to point out that taking exactly the same source code,
> unmodified, and packaging it differently, avoids the issue.
Yes. But you changed how you compile and how many times you link lib0.cpp into
your executable. If you fix your problem, then the problem is not there any
more.
> Let's ignore I mentioned static libs and pretend I wrote "If you remove
> the libraries and compile everything together".
Yes, I am reading this as "if I remove the problematic build solution and fix
the issue". I agree that in this case the problem is solved.
> > You chose to use lib1 and lib2. You can choose not to, if they don't react
> > to your bug reports. Vote with your feet and your wallet.
>
> Is the bug in lib0, lib1 and lib2?
> They all work correctly, I do not see any UB in them, even after
> everything we wrote, as long as you use them independently.
> They do not work when mixed together.
Agreed. Strictly speaking, it's the combination of lib1 and lib2 into one
executable. You can't do that.
This is no different than having two copies of lib0 in your source code in
different paths and only using one of the two of them at a time, which of
course works. But if you try to use both, then it doesn't.
IMO, you solve this by making lib0 a shared library and linking lib1 and lib2
to it.
> But the issue also happens if two libraries have no "lib0" as common
> dependency.
> What if two projects happen to write
>
> const std::string answer_of_life = "42";
>
> as implementation detail in their shared library?
Yes, if you make an ODR violation, you have ill-formed program. How hard is it
to understand that?
> You can obviously report a bug to those libraries to hide their symbols
> with compiler specifics tools, but according to the standard everything
> is fine.
Sure. They *chose* to have a global symbol here (there's no "static"). That
means they are claiming "answer_of_life" as part of their ABI. Two TUs can't
do that at the same time.
If they didn't mean to claim that, then there are solutions to avoid the
collision, like namespaces.
> What about libraries that are header only?
Header-only libraries are fine, but the same ODR violation rules apply. ALL
copies of the library in the executable must be the same version. How you go
about doing that is up to you and the vendors who used that library. This is
not an optional condition: it MUST be the same version, lest your code become
ill-formed.
My answer above stands: the best solution for this is to make lib0 a shared
library of its own.
> Yes, I know; thus it is the other way round, which is my point.
> To have globals that work in shared libraries, I might need to use
> something outside of the standard, I might need pragmas/attributes/...
No, you don't. The globals work just fine, but they must of course be the same
and you must have a single definition of them. The problem here is that your
code has been violating the standard and thus is ill-formed. You're claiming
the standard has a problem in shared libraries, but that's incorrect. Your
code does.
> But your comment also mentions that having a feature that merges objects
> of shared libraries together is against common practice.
> You normally want to hide the symbols to avoid issues.
Indeed. Like I said, Dynamic Shared Objects 101 is mandatory reading.
> It is a surprise for end-users, and makes writing code harder.
> It would be one thing to say "avoid globals in the API of the library",
> that's relatively easy to follow.
> It is another to say that "globals used as implementation details can be
> problematic", every project I worked on used in one way or another
> multiple globals.
If you're going to use globals, make sure that you properly namespace them so
you don't collide the name with something different in another library.
The problem in your case is the presence of lib0 inside of lib1 and lib2. Stop
propagating this horrible practice of bundling copies of third-party content.
> And to return to the original discussion, I'm still not sure why this
> feature should be treated differently from all other global symbols.
It isn't.
-- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Principal Engineer - Intel DCAI Platform & System Engineering
Received on 2024-10-20 18:31:28