Hi Dave,

On Thu, Dec 2, 2021 at 4:37 PM David Rector <davrec_at_[hidden]> wrote:

> Hi Matus,
>
> Thanks for your efforts, this sort of project requires an extraordinary
> amount of work.
>
Thanks, that is very kind.

>
> I have not played around sufficiently with the implementation, but let’s
> talk about its role vis a vis Andrew et al’s proposed reflection features,
> under the assumption that this is/can be easily made bug-free and properly
> implements the Reflection TS.
> (This assumption seems reasonable because, IIUC, this implementation
> expects the user to rely on existing template metaprogramming techniques to
> iterate over members, expressed as template argument packs, whereas
> `template for` etc. iteration techniques seems to be one of the most
> frustratingly difficult and buggy features to implement, particularly for
> type-based reflection. Correct me if I’m wrong, and describe any novel
> iteration features you introduce.)
>
> The language keywords you introduce are these:
> https://github.com/matus-chochlik/llvm-project/blob/reflection/clang/include/clang/Basic/TokenKinds.def#L425
> <https://github.com/matus-chochlik/llvm-project/blob/reflection/clang/include/clang/Basic/TokenKinds.def>.
>
> It seems all, with the exception of `reflexpr()` (which could be easily
> changed to `^`), are prefixed by double underscore.
> Do I understand correctly that this means the user does not *ever* need to
> directly use the double-underscored keywords — that they are private to the
> Reflection TS implementation accessed via `#include
> <experimental/reflect>`?
>

The double underscore builtins are technically for implementation purposes
only and they are following the same patterns as the implementation of the
existing type traits (and other compiler "magic") in clang. I say
technically, because one of my goals was to play around with other than
TMP-based reflection APIs (and I invite other WG21/SG7 people to do so as
well) but they definitely should not be used in end-user code.


>
> If not, please list any other keywords beyond `reflexpr` with which the
> user is expected to interface directly.
>

At the moment `reflexpr` is the only official keyword added by the TS, I
expect other ones to be added in the future for "unreflection".

In particular please discuss any that differ fundamentally from the
> primitives introduced by Andrew et al.
> Also, if there are any components of <experimental/reflect> that go beyond
> what the ReflectionTS specifies, please list/motivate them.
>

This TS implementation is intentionally value-based. The metaobjects are at
the fundamental level represented as constant sort-of-integral values (you
cannot do the usual integral operations on them, but they are integral
"keys" used to look things up in the compiler) wrapped in a very simple
class template to match what the TS says.

The notable differences from the proposal and implementation by Andrew,
Daveed et al.:
- syntax: reflexpr(X) vs ^X
- the "unreflection" is more limited in the TS (no "universal" [: ... :]
operator, there are the get_reflected_type, get_pointer and get_constant
operations for the moment.
- There might be some differences in the metaobject concepts and operations.


> It seems to me that, given the premises that
> 1. everyone agrees on a few basic primitives like ^ and whatever other
> basic interface this implementation requires,
> 2. we have a ReflectionTS that at some point was an agreed-upon reflection
> interface (even if not the most efficient/user-friendly),
> 3. Matus’s implementation of the Reflection TS is solid, and
> 4. the status and completion time for the official reflection features is
> very uncertain,
> this would be a very useful placeholder reflection implementation, worthy
> of being upstreamed to clang, to allow users to experiment with reflection
> and give viable feedback to guide the efforts of Andrew et al.
>
> The eventual official reflection implementation could simply wipe out the
> double-underscored language keywords and implement <experimental/reflect>
> in terms of the official language keywords.
>
The <experimental/reflection> header is already present and mostly
conforming to the TS. It is not yet finished and debugged, but I try to
make at least some small progress every day.

>
> Perhaps we can all first discuss this a bit and clear up any confusions.
>

I wrote some notes on my motivation for this implementation below.

>
> Then, Matus if you believe your implementation is indeed ready for prime
> time or nearly so, I think the next step would be to a) ask Andrew et al to
> weigh in on whether such a "placeholder implementation" would indeed help
> their efforts (and if not, why), b) cc Richard to get his thoughts on the
> viability of upstreaming this to clang.
>

Richard had a brief look at the implementation couple of weeks ago and said
it is generally going in the right direction, but I think these things need
to be finished before it can go for review:
- Currently the metaobjects are just reinterpreted pointers (which change
between compiler executions even for the same TU), it will probably be
necessary to turn them into something more stable.
- There are FIXMEs in name mangling, code-generation, AST serialization and
tooling.
- Some metaobject operations mostly related to functions and lambdas are
missing.
- Unit tests are missing (and will be necessary for ensuring conformance
with the TS).


Background and motivation for the TS implementation:
People have expressed concerns that having a TMP based API will be slow and
that we should not be basing it on TMP because it is as a metaprogramming
paradigm working "only by accident". I understand and generally agree with
these arguments.
However, at this moment TMP simply is the most powerful metaprogramming
paradigm that we have in C++ and we have ~20 years of experience with the
good, the bad and the ugly in TMP and we understand it.
This implementation gives people the opportunity to play with the
reflection API in various use-cases, they can give feedback about which
meta-information is missing and given a larger body of code it will be
easier to spot the patterns and places where things get repetitive (because
of the shortcomings of TMP) and would be a good target for some
syntax-sugar language expressions (for example code splicing).
Being value-based it also provides an opportunity to write value-based
consteval APIs around the builtins and determine what is missing in
consteval.
Personally I'd say that we need at the very least the following two things
to work in consteval in order to be able to replace TMP:

1) Consteval arguments (at least boolean/integer) to have the same
"constantness" as NTTPs:
consteval auto foo(int i) {
  return integral_constant<int, i>{};
}

2) Something like consteval std::conditional_t = the ability to switch
consteval function return type based on value of (integral) argument.

Without these many useful things that are now trivial in TMP cannot be done
in consteval metaprogramming libraries.

--Matus