std-discussion

From: Federico Kircheis <federico_at_[hidden]> · Date: Sun, 20 Oct 2024 19:02:13 +0200

----
>file1.hpp
const int i = 1;
>file2.cpp
#include "file1.hpp"
>file3.cpp
#include "file1.hpp"
----
and
----
>file2.cpp
const int i = 1;
>file3.cpp
const int i = 1;
----
is equivalent, and to the best of my knowledge not problematic.
The two variables are declared independently, at two different locations.
The standard says
> No translation unit shall contain more than one definition of any definable item [...]
If something else claims that this is an ODR, a pointer to the relevant 
content is greatly appreciated.
>>>> Same class, slightly different context:
>>>>
>>>> https://godbolt.org/z/Thv8xhc6Y
>>>
>>> Also ODR violation for the same reason.
>>
>> In this cases, there is one definition and one declaration in one TU.
> 
> See below because it's the same case.
> 
>>>> Another example that shows the same behavior
>>>>
>>>> https://godbolt.org/z/ecPPq8vns
>>>
>>> This is the exact same code as the previous one: lib0.cpp is linked twice
>>> into your executable, therefore ODR violation.
>>
>> lib0.cpp is linked once in the dynamic library lib1, once in the dynamic
>> library in lib2, and 0 times in main (and main uses lib1 and lib2).
>> lib0 has only one declaration.
> 
> The problem is what happens when you load the application for runtime: then
> lib1 and lib2 are both in memory at the same time, working as if "they were
> another TU". Therefore, their both including lib0.cpp implies it's an ODR
> violation.
I'm not sure what you mean by working as if "they were another TU".
There is no another TU in the source code, there is only one.
Previously it was claimed you need to do something strange or some 
something like attributes and functions like dlopen.
I claimed it is not necessary, and that standard c++ code with globals 
does not always work as expected.
I think we agree on this point?
Or did I do something strange?
>> If lib1,lib2 where applications and main would communicate through IPC,
>> then there would be no issues.
> 
> Yes, but if wishes were fishes don't count. They're not.
I do not think I understand.
I wanted to point out that taking exactly the same source code, 
unmodified, and packaging it differently, avoids the issue.
>> If lib1 and lib2 where static libraries (or to put it another way, if
>> they where not separate libraries), there would be no issue, since there
>> is only one declaration in the whole code basis.
> 
> Again, if you change what is to something different, the problem changes too.
Yes, it is what I wrote.
> Static libs don't link. What you get is CMake tracking that lib1 and lib2 have
> a dependency on lib0, so when you link your final executable, your final
> executable links directly to lib0.
Yes, talking about static libs was a bad idea since they are 
implementation defined too.
Let's ignore I mentioned static libs and pretend I wrote "If you remove 
the libraries and compile everything together".
> If you want to reproduce this problem, you first need to change both lib1 and
> lib2 so they incorporate lib0 into their .a. That simulates what happens
> during dynamic linking (roughly enough, but works for this case) meaning the
> contents of lib0.a get emitted inside of lib1.a and lib2.a. Then, you must
> link both of those libs into your executable as -Wl,--whole-archive, because
> this mimics what happens at runtime: the contents of the entire dynamic
> library become visible.
> Doing this means you have two copies of lib0.cpp.o into the executable. This
> is an ODR violation and thus IFNDR. And in this case, a diagnostic *would* be
> emitted, because the static linker can tell that some symbols were defined
> twice.
> 
>> But even if it where an ODE, how do you fix those issue with dynamic
>> libraries?
> 
> Emit a single copy of "instance" in exactly one library. Which one you choose
> to do that is up to you.
> 
>> Assuming that the author of lib0 is not the same of lib1 and lib2, who
>> needs to change what?
> 
> You chose to use lib1 and lib2. You can choose not to, if they don't react to
> your bug reports.  Vote with your feet and your wallet.
Is the bug in lib0, lib1 and lib2?
They all work correctly, I do not see any UB in them, even after 
everything we wrote, as long as you use them independently.
They do not work when mixed together.
> Usually, the problem here is that of vendoring other content. It's fine to
> vendor once (for some definitions of "fine"). It's not fine for that to happen
> twice. So if two modules do it, you can't use them both in the same
> executable.
I do not think that it is the issue, especially since I'm not sure what 
vendoring means in this context.
Does it mean to ship a precompiled library?
The example I provided has nothing precompiled.
Does it mean not to ship a precompiled library?
The issue can be reproduced with precompiled libraries too, in that 
case, since lib1 and lib2 might hve been compiled with different 
versions of lib0, the issue is even worse.
(and lib0 is not part of the API of lib1 and lib2)
But the issue also happens if two libraries have no "lib0" as common 
dependency.
What if two projects happen to write
const std::string answer_of_life = "42";
as implementation detail in their shared library?
You can obviously report a bug to those libraries to hide their symbols 
with compiler specifics tools, but according to the standard everything 
is fine.
What about libraries that are header only?
> You can work around the problem here by hiding those symbols from lib0
> completely inside of each of the two shared libraries. This is where you must
> step outside of the Standard and use compiler and platform-specific extensions,
> like hidden visibility for the entirety of lib0.
Yes, I know; thus it is the other way round, which is my point.
To have globals that work in shared libraries, I might need to use 
something outside of the standard, I might need pragmas/attributes/...
But your comment also mentions that having a feature that merges objects 
of shared libraries together is against common practice.
You normally want to hide the symbols to avoid issues.
> The practice of vendoring must stop. At a minimum, vendored content must be
> unbundleable, if nothing else for fixing security issues.
What do you mean by that?
>> In all the examples, the global variable is an implementation detail, it
>> is not even used directly outside of lib0.
> 
> Neither the compiler nor the Standard care about that.
Of course, and this is what makes this issue so problematic.
It is a surprise for end-users, and makes writing code harder.
It would be one thing to say "avoid globals in the API of the library", 
that's relatively easy to follow.
It is another to say that "globals used as implementation details can be 
problematic", every project I worked on used in one way or another 
multiple globals.
And to return to the original discussion, I'm still not sure why this 
feature should be treated differently from all other global symbols.