Henry,

"Are you proposing that we define an ABI for C++ - at least for the stable
types?"

Yes. Although what I think of here in terms of ABI is primarily memory
layout. For instance a std::stable_vector must be have a memory layout
like this:
stable_vector<T> {
   T* arr;
   size_t size;
   size_t capacity;
}

So that you can explicitly know ahead of time what the
sizeof(stable_vector<T>) is without asking your favorite compiler and which
memory offset each of the members are in. This will go a long way in terms
of me passing a reference to my stable_vector<T> across a DLL/SO and for
the consuming code's assembly to refer to it correctly.

I had not read the kde document but I certainly will.

In terms of language agnostic ABI, yes you are right that the C++ committee
does not have the jurisdiction to dictate this, but neither does the C
standardization group and they effectively do dictate it because they are
"the" low level language that guarantees stability. My argument here is
that if a subset of c++ has these guarantees the community of other
languages may choose to adopt it as an alternative to C.

"Right now C++ doesn't have a concept of shared library (I could be wrong
here)."

Yes this is something I would like to see changed. Many of the concepts
(lower case c here) in c++ is based on the model of the abstract
machine (Stroustrup
2005 <http://www.stroustrup.com/abstraction-and-machine.pdf>) which does
not tell the world how to design a computer. I know the ship has sailed
when it comes to telling the world what a shared library should be, but we
could describe an "abstract binary unit" and write standards based on our
description of that.

"All along you will run into problems: C++ is sometimes used on weird
hardware"

This is an interesting point. I know for instance some hardware does not
have dynamically allocated memory so many of the standard library types
don't work in implementations of C++ on that hardware. But we do not take
these types out of the standard. I think the position of C++ on this, and
I could be wrong, is that implementations of c++ on any given hardware is
to provide as much as possible. On most hardware this is everything, on
exotic hardware this may be much less. But it doesn't constrain
standardization.

Mohamed Koubaa


On Mon, Nov 11, 2019 at 9:18 AM <std-proposals-request_at_[hidden]>
wrote:

> Date: Mon, 11 Nov 2019 09:17:54 -0600
> From: "Henry Miller" <hank_at_[hidden]>
> To: "Henry Miller via Std-Proposals" <std-proposals_at_[hidden]>
> Subject: Re: [std-proposals] Compatible API proposal
> Message-ID: <174dacf2-f8de-4b88-b69d-3bbe0dbbd4eb_at_[hidden]>
> Content-Type: text/plain; charset="utf-8"
>
> It sounds like a great idea at first, ABI is a problem many of share.
> However I'm not clear what you actually are proposing.
>
> Currently C++ doesn't define an ABI at all. Gcc has settled on the
> itanimum ABI spec. Microsoft uses their own, which has changed somewhat
> with different releases of MSVC (I'm not sure if this is historic of if
> they are still making changes). This list is not exhaustive: just the ones
> I can think of immediately. Are you proposing that we define an ABI for C++
> - at least for the stable types?
>
> The final end of the above (though we may not want to go that far) is
> build a "stable only" library on MSVC, and then link it into a linux
> program with gcc. I can see vendors liking this - they would only have to
> worry about CPUs and defining how the user provides an OS abstraction layer
> for the things they need from the OS.
>
> The next possibility is the stable types you propose will obey the rules
> of
> https://community.kde.org/Policies/Binary_Compatibility_Issues_With_C%2B%2B
> (we would rewrite them in standardize) so that if you only use stable types
> you don't have to worry about users of your library using a different
> version of the C++ standard. You still need to worry about the exact
> standard library - different implementations might have different
> incompatible implementations, but the vendor of the implementation is
> restricted from changing it again.
>
> The next possibility I can think of (and you hinted at it) is a language
> agnostic ABI. To make this work we need a new module (not to be confused
> with the current C++ module spec though closely related to it) that
> multiple languages read. When you use such a module there is a definition
> of how to load it (this sounds easy, but I can't figure out how it could
> work in practice!), so that I can implement the module in say Haskell,
> someone else uses it from a C++ module, and finally the end program is
> written in Go, and also uses modules from Python, Java C and Rust. This
> last is interesting, but I think it is outside scope of the C++ committee
> if it is to get anywhere it needs to be a separate committee.
>
> One issue of all the above is keeping non-stable types "internal". Right
> now C++ doesn't have a concept of shared library (I could be wrong here).
> We would need to add it so we can define my library using only "stable
> types" in the interface can use "unstable types" internally without runtime
> issues of mixing implementations.
>
> All along you will run into problems: C++ is sometimes used on weird
> hardware. The users of that hardware will object to things that seem
> obvious to everybody working on "normal" hardware. In short, good luck. I
> like the idea, but I don't know if you can make something useful because
> the problem is hard in places where you need it to be easy.
>
> --
> Henry Miller
> hank_at_[hidden]
>
>
>
> On Thu, Nov 7, 2019, at 4:00 PM, mohamed koubaa via Std-Proposals wrote:
> > Hello,
> >
> > My name is Mohamed Koubaa and I am a software developer that works on a
> large scale desktop application. More than half of what I write is in C++
> and I often think about physical layout and ABI. I hope you'll consider my
> idea! This is my first post here and look forward to hearing feedback from
> all of you.
> >
> > Reply-to: Mohamed Koubaa <koubaa.m at gmail ~dot~ com>
>
> > Introduction
>
> >
> > This paper discusses a set of extensions to C++ to standardize a
> compatibility ABI in some parts of the standard library and type system. It
> is not meant to propose an ABI for all types and language features
>
> >
> > Motivation and Scope
>
> > C ABI compatibility is often wrongly assumed and expected by C++
> developers. The language evolution often will introduce new types and
> features leaving the ABI unspecified. Over time, code begins to depend on
> this ABI and changes to it become infeasible or impossible. This situation
> is not inherently c++ic (using Herb?s term). It is simply a historical
> outcome based on where the language came from and its often shared runtime
> with C.
>
> > ABI is not a problem with most other popular languages and is really
> holding back the C++ ecosystem. The way other languages deal with (or don't
> have to) are:
>
> > * C - Compatibility is a primary concern and the C ABI is practically
> how other languages often achieve ABI
>
> > * Python - See above. Python extension modules (often written using
> C++!) are ABI compatible because the CPython interface is, well, C.
>
> > * JS/C#/Java - these languages use a virtual machine or interpreter so
> that the ABI design are outside of the domain of the everyday programmer
> and library author
>
> >
> > After using C++ for some years I and others have the understanding that
> in order to have truly portable C++ code, i*t must be distributed as source
> code and built by clients*. Clients need to understand the (often varied)
> build systems and tests in order to use just about any useful C++ library.
> This is a burden that is neither expected nor tolerated when using other
> compiled languages.
>
> >
> > Another concern with the current state of affairs is that for any
> individual to be a competent software architect of large scale software
> systems (perhaps excluding distributed systems and microservices) needs to
> master the sharp edges around ABI compatibility.
>
> >
> > One technique for building C++ code without dealing with ABI is called
> an hourglass pattern. This means building a C layer at the interface of
> binaries and writing c++ code to wrap that C interface in a user friendly
> C++ API. This is sometimes done using plain C and other times done using
> COM (on windows). Essentially, this is an admission that in order to have
> stable ABIs one must only use a small subset of C++ types and language
> features on the interface, and that subset is essentially C. In the narrow
> center of the hourglass free functions for allocating and freeing memory,
> as well as errno() are often used and converted back into constructors,
> destructors and exceptions.
>
> >
> > If a stable ABI is known which includes many important standard library
> types, language interop can become idiomatic. For instance, a language
> consuming an exported method which returns a stable_vector can wrap that in
> the most appropriate language semantics. I.e. IEnumerable/IList for C# or
> __len__ in python
>
> > Impact on the standard
>
> > This paper does *not* attempt to propose an ABI for the entirety of the
> c++ type system and library features. Instead, a portion of the standard
> library (some existing, some new) will be marked as having a stable ABI,
> and library authors can opt into using that portion at the binary interface
> of libraries so that ABI compatibility is no longer a concern for their
> clients. In spirit, this is no different than the hourglass pattern, except
> that the subset of C++ which can be considered to be ABI stable is larger
> than simply what is shared with C due to history. Examples of these types
> are the P0709 exception and ranges which are in c++20. Examples of library
> features are a new stable_allocator and a std::stable metaclass which can
> be used for compile time reflection on classes to ensure that they are
> indeed stable.
>
> >
> > Other language features are explicitly not expected to be stable and
> implementations of c++ can change their ABI at any time. In practice it
> will likely be difficult for an implementation to do that until using
> stable ABIs at binary interfaces becomes a common practice.
>
> > Technical Specifications
>
> > This proposal introduces the notion of *stable*, a property of a type or
> feature whose ABI is defined to never change. *Stable* can be thought of as
> a superset of is_standard_layout because ABI concerns extend beyond memory
> layout. All types and language features are considered unstable unless
> explicitly defined as stable. Following are the list of all stable types
> and language features (there are not many):
>
> >
> > * Standard layout
>
> > * P0709 exceptions
>
> > * Stable_allocator
>
> > * Fixed width integer types
>
> > * Range
>
> > * String_view
>
> > * Array
>
> > * Stable_vector
>
> > * Stable_string
>
> >
> > This set of types and features may grow slowly over time, much like
> constexpr did since c++11, except with less urgency and with no interest in
> someday covering the entire language.
>
> >
> > std::stable_allocator
>
> > This proposal defines a std::stable_allocator which controls the
> behavior of the new and delete functions and fixes them to a given version
> of malloc() and free() on a given platform. For instance, the windows
> platform has multiple C runtimes to choose from. Classes that have an
> allocator template argument and whose clients provide stable_allocator are
> such that any new and delete function that occurs in their member functions
> will use the stable allocator. This will allow such a class to be allocated
> in one binary unit and deleted in another without worrying about the
> runtime of each unit.
>
> >
> > std::stable metaclass
>
> > The std::stable metaclass can be used for types that which to declare
> themselves as ABI stable. The class will fail to compile if any unstable
> types or features are used at the interface of such a class.
>
> > Modules
>
> > This proposal does not intend to provide any relationship between
> modules and the notion of stability. Modules may include all language
> features which is larger in scope than stable ABI features. However, if a
> standard attribute is added for visible symbols/dll export (does this exist
> yet?), compilers may warn about any unstable types or features used in
> conjunction with that attribute.
>
> >
> >
> > Thanks,
> > Mohamed
> > --
> > Std-Proposals mailing list
> > Std-Proposals_at_[hidden]
> > https://lists.isocpp.org/mailman/listinfo.cgi/std-proposals
> >
> -------------- next part --------------
> HTML attachment scrubbed and removed
>
> ------------------------------
>
> Subject: Digest Footer
>
> Std-Proposals mailing list
> Std-Proposals_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/std-proposals
>
>
> ------------------------------
>
> End of Std-Proposals Digest, Vol 8, Issue 12
> ********************************************
>