Hi Tiago,

Thanks for your reply, your points are quite valid. I can imagine some of those issues! In fact, RTTI of exceptions across modules has stumped me so many times myself! (In particular, throwing boost::exceptions in python extension modules, or boost exceptions thrown by precompiled shared libraries, being caught in my code).

My thoughts on the issues you mentioned, and how this paper may be a bit different, are as follows:

1. RTTI is subtly different from just checking a contract between two modules. Instead of "are these the same type", we're just asking "can I use this incoming data and expect all the members etc that I care about, to be in the same place with the same types"? These two are mostly interchangeable, but point 2 explains this in some more detail.

2. For use-cases in software that I'm familiar with (low latency, distributed systems, python extension modules), having a "can I use this incoming data as usual" boolean can solve a majority of the pain points, without actually endeavoring to be a 100% precision AND 100% recall system. Basically, my claim is that if user defined types (and perhaps STL structures) can be covered in a relatively precise manner (100% precision, but NOT 100% recall), that would be valuable to a lot of users and would solve a number of problems in those systems that presently people have to jump through hoops to solve. And the proposed solution is not at all intrusive since it is purely a library.

3. With a reflection system as proposed by P2996, implementation is pretty straightforward for the most part. This is something that was not the case in the past, and may have added unnecessary friction to past attempts as far as I can think. A lot of the tasks the compiler may have had to do, kind of, now are offloaded to the reflection system to handle. So once we hash all members and their offsets recursively (among other things), presumably that is enough to introspect the compiler's internals in a way and compute a perfect comparison hash (assuming hash functions are perfect).

This also made me think of one more item. The "add a version integer into your struct and increment it manually" solution is not 100% precise because it does not cover that the two modules may have been compiled with different compilers and may have different layouts. This reflection based approach would be safer there.

Thanks again for your thoughts, this definitely adds to the conversation on this topic. Let me know what you think on the above. 

Regards,
Saksham

On Thu, Jan 18, 2024 at 3:52 AM Tiago Freire <tmiguelf@hotmail.com> wrote:

Hi Saksham,

I have previously done investigation about this topic to just be able to do some form of RTTI across modules, and I failed miserably at this.

I’m not trying to dash your hopes, but beware of what you are getting into.

The issue is the type information of a piece of memory layout is by design ephemeral, this is a fundamental feature of the language and not something you can get rid of.

Whatever solution I managed to come up with it either made it impossible to distinguish 2 objects that were supposed to be different but looked similar, or 2 objects that were supposed to be the same but because of duplications across modules they would look different, or worse couldn’t handle both.

This is a problem that exists right now in every single compiler when for example throwing an exception across modules, even if compiled in the exact same way I can trick them to do the wrong thing by exploiting certain properties of the data that a compiler can’t see across 2 different translation contexts. The compilers are not inherently bugged, the developers did a great job with the challenge that they have at hand. While there are certain things that can be done at link time to detect problems within the same module, if no information is shared I just don’t see how you would do it. While exceptions are a well-defined language feature, it is in practice UB when done across modules specifically because of incomplete reflection, and there’s no design that I know of that could solve this (or else it would have already been solved).

 

The conclusion that I arrived at the time is that this may not just be difficult to solve, it might just be unsolvable.

 

While you can make it work, with specifically selected types that you agree with a protocol to do it with a very specific way, with user designed types outside of the protocol support it’s a whole different story.

 

Best regards,

Tiago

 

 

From: Std-Proposals <std-proposals-bounces@lists.isocpp.org> On Behalf Of Saksham Sharma via Std-Proposals
Sent: Thursday, 18 January 2024 01:30
To: std-proposals@lists.isocpp.org
Cc: Saksham Sharma <saksham0808@gmail.com>
Subject: [std-proposals] ABI comparison with reflection : new proposal

 

Hi everyone,

 

This is my first post on this list. I have been working on a POC implementation to come up with an ABI comparison standard methodology using the latest P2996R1 reflection paper and its corresponding experimental compiler. Based on some discussions I've had with various people, it should be targeted towards SG-7 Reflection, and LEWG.

 

I have attached my proposed draft paper with this email. The idea is to solve a seemingly common problem of two processes or modules trying to compare if they are sharing a data object with the same memory layout or not. This is often due to the two parties being compiled at different times, potentially with different versions of the code, and makes for tricky and brittle solutions proliferating across systems.

 

I would love to hear feedback in general, but specifically on things I may have missed, or suggestions on reducing / broadening the scope. The part about virtual functions is still undocumented as of now but can be implemented, and the implementation is a little rough around the edges due to some limitations of the current experimental EDG compiler, but they should be fixed in some time.

 

Thank you.

 

Regards,

Saksham Sharma