One possible alternative reflection API that can be implemented on top of
the new compiler builtins is this;
Examples:
https://github.com/matus-chochlik/llvm-project/blob/reflection/_reflexpr/mirror01.cpp
https://github.com/matus-chochlik/llvm-project/blob/reflection/_reflexpr/mirror02.cpp

Implementation:
https://github.com/matus-chochlik/llvm-project/blob/reflection/libcxx/include/experimental/mirror

It is a mix of TMP/consteval/constexpr in the implementation but TMP is not
(very) visible when using it.



On Fri, Dec 3, 2021 at 9:00 AM Matus Chochlik <chochlik_at_[hidden]> wrote:

> One thing I forgot about are the extensions to the TS:
>
> - The Specifier concept (allows to reflect specifiers like static, etc.)
> - The get_id (get_id_v) operation, allows to get the metaobject value from
> the metaobject type.
> - The hide_private and hide_protected operations allow to hide private and
> protected elements of metaobject ranges that would otherwise iterate
> through all elements regardless of their accessibility.
> Also the contextual get_accessible_* operations are not implemented yet,
> I'll have to do more digging to find out how to do this,
>
> On Thu, Dec 2, 2021 at 6:06 PM Matus Chochlik <chochlik_at_[hidden]> wrote:
>
>> Hi Dave,
>>
>> On Thu, Dec 2, 2021 at 4:37 PM David Rector <davrec_at_[hidden]> wrote:
>>
>>> Hi Matus,
>>>
>>> Thanks for your efforts, this sort of project requires an extraordinary
>>> amount of work.
>>>
>> Thanks, that is very kind.
>>
>>>
>>> I have not played around sufficiently with the implementation, but let’s
>>> talk about its role vis a vis Andrew et al’s proposed reflection features,
>>> under the assumption that this is/can be easily made bug-free and properly
>>> implements the Reflection TS.
>>> (This assumption seems reasonable because, IIUC, this implementation
>>> expects the user to rely on existing template metaprogramming techniques to
>>> iterate over members, expressed as template argument packs, whereas
>>> `template for` etc. iteration techniques seems to be one of the most
>>> frustratingly difficult and buggy features to implement, particularly for
>>> type-based reflection. Correct me if I’m wrong, and describe any novel
>>> iteration features you introduce.)
>>>
>>> The language keywords you introduce are these:
>>> https://github.com/matus-chochlik/llvm-project/blob/reflection/clang/include/clang/Basic/TokenKinds.def#L425
>>> <https://github.com/matus-chochlik/llvm-project/blob/reflection/clang/include/clang/Basic/TokenKinds.def>.
>>>
>>> It seems all, with the exception of `reflexpr()` (which could be easily
>>> changed to `^`), are prefixed by double underscore.
>>> Do I understand correctly that this means the user does not *ever* need
>>> to directly use the double-underscored keywords — that they are private to
>>> the Reflection TS implementation accessed via `#include
>>> <experimental/reflect>`?
>>>
>>
>> The double underscore builtins are technically for implementation
>> purposes only and they are following the same patterns as the
>> implementation of the existing type traits (and other compiler "magic") in
>> clang. I say technically, because one of my goals was to play around with
>> other than TMP-based reflection APIs (and I invite other WG21/SG7 people to
>> do so as well) but they definitely should not be used in end-user code.
>>
>>
>>>
>>> If not, please list any other keywords beyond `reflexpr` with which the
>>> user is expected to interface directly.
>>>
>>
>> At the moment `reflexpr` is the only official keyword added by the TS, I
>> expect other ones to be added in the future for "unreflection".
>>
>> In particular please discuss any that differ fundamentally from the
>>> primitives introduced by Andrew et al.
>>> Also, if there are any components of <experimental/reflect> that go
>>> beyond what the ReflectionTS specifies, please list/motivate them.
>>>
>>
>> This TS implementation is intentionally value-based. The metaobjects are
>> at the fundamental level represented as constant sort-of-integral values
>> (you cannot do the usual integral operations on them, but they are integral
>> "keys" used to look things up in the compiler) wrapped in a very simple
>> class template to match what the TS says.
>>
>> The notable differences from the proposal and implementation by Andrew,
>> Daveed et al.:
>> - syntax: reflexpr(X) vs ^X
>> - the "unreflection" is more limited in the TS (no "universal" [: ... :]
>> operator, there are the get_reflected_type, get_pointer and get_constant
>> operations for the moment.
>> - There might be some differences in the metaobject concepts and
>> operations.
>>
>>
>>> It seems to me that, given the premises that
>>> 1. everyone agrees on a few basic primitives like ^ and whatever other
>>> basic interface this implementation requires,
>>> 2. we have a ReflectionTS that at some point was an agreed-upon
>>> reflection interface (even if not the most efficient/user-friendly),
>>> 3. Matus’s implementation of the Reflection TS is solid, and
>>> 4. the status and completion time for the official reflection features
>>> is very uncertain,
>>> this would be a very useful placeholder reflection implementation,
>>> worthy of being upstreamed to clang, to allow users to experiment with
>>> reflection and give viable feedback to guide the efforts of Andrew et al.
>>>
>>> The eventual official reflection implementation could simply wipe out
>>> the double-underscored language keywords and implement
>>> <experimental/reflect> in terms of the official language keywords.
>>>
>> The <experimental/reflection> header is already present and mostly
>> conforming to the TS. It is not yet finished and debugged, but I try to
>> make at least some small progress every day.
>>
>>>
>>> Perhaps we can all first discuss this a bit and clear up any confusions.
>>>
>>
>> I wrote some notes on my motivation for this implementation below.
>>
>>>
>>> Then, Matus if you believe your implementation is indeed ready for prime
>>> time or nearly so, I think the next step would be to a) ask Andrew et al to
>>> weigh in on whether such a "placeholder implementation" would indeed help
>>> their efforts (and if not, why), b) cc Richard to get his thoughts on the
>>> viability of upstreaming this to clang.
>>>
>>
>> Richard had a brief look at the implementation couple of weeks ago and
>> said it is generally going in the right direction, but I think these things
>> need to be finished before it can go for review:
>> - Currently the metaobjects are just reinterpreted pointers (which change
>> between compiler executions even for the same TU), it will probably be
>> necessary to turn them into something more stable.
>> - There are FIXMEs in name mangling, code-generation, AST serialization
>> and tooling.
>> - Some metaobject operations mostly related to functions and lambdas are
>> missing.
>> - Unit tests are missing (and will be necessary for ensuring conformance
>> with the TS).
>>
>>
>> Background and motivation for the TS implementation:
>> People have expressed concerns that having a TMP based API will be slow
>> and that we should not be basing it on TMP because it is as a
>> metaprogramming paradigm working "only by accident". I understand and
>> generally agree with these arguments.
>> However, at this moment TMP simply is the most powerful metaprogramming
>> paradigm that we have in C++ and we have ~20 years of experience with the
>> good, the bad and the ugly in TMP and we understand it.
>> This implementation gives people the opportunity to play with the
>> reflection API in various use-cases, they can give feedback about which
>> meta-information is missing and given a larger body of code it will be
>> easier to spot the patterns and places where things get repetitive (because
>> of the shortcomings of TMP) and would be a good target for some
>> syntax-sugar language expressions (for example code splicing).
>> Being value-based it also provides an opportunity to write value-based
>> consteval APIs around the builtins and determine what is missing in
>> consteval.
>> Personally I'd say that we need at the very least the following two
>> things to work in consteval in order to be able to replace TMP:
>>
>> 1) Consteval arguments (at least boolean/integer) to have the same
>> "constantness" as NTTPs:
>> consteval auto foo(int i) {
>> return integral_constant<int, i>{};
>> }
>>
>> 2) Something like consteval std::conditional_t = the ability to switch
>> consteval function return type based on value of (integral) argument.
>>
>> Without these many useful things that are now trivial in TMP cannot be
>> done in consteval metaprogramming libraries.
>>
>> --Matus
>>
>