Date: Mon, 11 Nov 2024 11:32:29 +0200
On Mon, 11 Nov 2024 at 11:12, Jeremy Ong via SG7 <sg7_at_[hidden]> wrote:
>
> > The standard needs to show discipline because users won't.
>
> I've worked extensively with game engine technologies where we've explored reflection done in user space for well over a decade now. While I can't speak to all game engines, I've used several large commercial ones, and a couple large internal engines. For the most part reflection must encompass public, private, and protected members, and failure to do so would break tooling and cause huge surprises to users when member field visibility changes. Reflection is typically built on custom preprocessors, and to a lesser extent, compiler-assisted tooling like clang. Reflection is used to surface editable properties to artists and content creators in tools, serialize payloads to disk, serialize networked properties over the network interface, and other such use cases. The visibility of a property/field or even member function to other C++ interfaces is an orthogonal concern to visibility in these other externally-facing use cases. I actually can't think of a single moment where I cared to iterate over specifically public fields of a class. If we needed to control visibility, this was typically done with user-defined attributes we'd attach to the properties themselves.
>
> In short, I don't think the desire or need to bypass the C++ access specifiers has anything to do with a lack of "discipline." They are not the same problem, and visibility in the C++ domain is not to be conflated with visibility in the exterior domain (which is what reflection is intended to tie us to). Here is a page that lists a number of property specifiers used in Unreal to annotate how fields are exposed to other bits of tooling: https://benui.ca/unreal/uproperty/. The official documentation is here (https://dev.epicgames.com/documentation/en-us/unreal-engine/unreal-engine-uproperties#propertyspecifiers) but Ben's page is a bit more illustrative of the end effect.
>
> I understand that game engines are but one slice of the pie among a variety of C++ users in our community, but thought I'd weigh in briefly because I believe game developers likely have clocked more hours with reflection in general, with perhaps more intensive needs (not as a general rule, this is a personal observation, I'm sure there are exceptions).
Right. For other anecdotes, I do iteration specifically over the
public entities of a class on a daily basis, when generating API
wrappers from
programming language X to programming language Y. I do iteration
specifically over the accessible members slightly less often than
that,
to wrap the non-public interfaces.
And it *is* a question of lack of discipline. Every one of those
reflection use cases can be done(*) with an API that you hook into in
the classes that need
to hook into it, and then you sometimes generate those
hook-ins/opt-ins automatically, sometimes not, by writing certain
opt-ins manually because they
don't fit into a generic form, and generating the opt-ins
automatically for classes that do fit into a generic form. In
some/many cases, that generic
form is "has a certain kind of non-private API", not "has the right
kind of data members".
(*) Technically. I understand perfectly well that the shape of
existing code, budgets, and deadlines may mean that such disciplined
APIs and
opt-ins are infeasible.
There's splendid amounts of cases where memberwise hashing and
serialization gives you the wrong outcome. You have pointer members
that
point to things that are not a part of any salient property of your
object, and then you have pointer members that point to things that
are. There's no
shortage of types where the author doesn't want *anything*, including
reflection, to muck with the private members directly, because there's
a public
API that does that mucking correctly, maintaining all sorts of invariants.
If your use cases require access-bypassing, by all means do that. I'm
not asking to remove your ability to do so. I'm asking for an API that
let's me program in a disciplined manner, with facilities that are
guaranteed to obey access controls, so that I don't have to read every
detail of every filter to figure out whether the programmer was able
to recreate the access controls of C++ correctly.
And by the way, there are users who want a change in a visibility of a
member to cause such a huge surprise to the users of that class
that the visibility change breaks the existing users *loudly*.
Including users who reflect on the class. And the last thing some such
users
want is the reflection code continuing to (attempt to) do what it did
before, because that means the program and its reflection results
don't do the
same thing any more than they did before. There are various cases
where a member is made less accessible than before because it's now
protected by stronger invariants.
>
> > The standard needs to show discipline because users won't.
>
> I've worked extensively with game engine technologies where we've explored reflection done in user space for well over a decade now. While I can't speak to all game engines, I've used several large commercial ones, and a couple large internal engines. For the most part reflection must encompass public, private, and protected members, and failure to do so would break tooling and cause huge surprises to users when member field visibility changes. Reflection is typically built on custom preprocessors, and to a lesser extent, compiler-assisted tooling like clang. Reflection is used to surface editable properties to artists and content creators in tools, serialize payloads to disk, serialize networked properties over the network interface, and other such use cases. The visibility of a property/field or even member function to other C++ interfaces is an orthogonal concern to visibility in these other externally-facing use cases. I actually can't think of a single moment where I cared to iterate over specifically public fields of a class. If we needed to control visibility, this was typically done with user-defined attributes we'd attach to the properties themselves.
>
> In short, I don't think the desire or need to bypass the C++ access specifiers has anything to do with a lack of "discipline." They are not the same problem, and visibility in the C++ domain is not to be conflated with visibility in the exterior domain (which is what reflection is intended to tie us to). Here is a page that lists a number of property specifiers used in Unreal to annotate how fields are exposed to other bits of tooling: https://benui.ca/unreal/uproperty/. The official documentation is here (https://dev.epicgames.com/documentation/en-us/unreal-engine/unreal-engine-uproperties#propertyspecifiers) but Ben's page is a bit more illustrative of the end effect.
>
> I understand that game engines are but one slice of the pie among a variety of C++ users in our community, but thought I'd weigh in briefly because I believe game developers likely have clocked more hours with reflection in general, with perhaps more intensive needs (not as a general rule, this is a personal observation, I'm sure there are exceptions).
Right. For other anecdotes, I do iteration specifically over the
public entities of a class on a daily basis, when generating API
wrappers from
programming language X to programming language Y. I do iteration
specifically over the accessible members slightly less often than
that,
to wrap the non-public interfaces.
And it *is* a question of lack of discipline. Every one of those
reflection use cases can be done(*) with an API that you hook into in
the classes that need
to hook into it, and then you sometimes generate those
hook-ins/opt-ins automatically, sometimes not, by writing certain
opt-ins manually because they
don't fit into a generic form, and generating the opt-ins
automatically for classes that do fit into a generic form. In
some/many cases, that generic
form is "has a certain kind of non-private API", not "has the right
kind of data members".
(*) Technically. I understand perfectly well that the shape of
existing code, budgets, and deadlines may mean that such disciplined
APIs and
opt-ins are infeasible.
There's splendid amounts of cases where memberwise hashing and
serialization gives you the wrong outcome. You have pointer members
that
point to things that are not a part of any salient property of your
object, and then you have pointer members that point to things that
are. There's no
shortage of types where the author doesn't want *anything*, including
reflection, to muck with the private members directly, because there's
a public
API that does that mucking correctly, maintaining all sorts of invariants.
If your use cases require access-bypassing, by all means do that. I'm
not asking to remove your ability to do so. I'm asking for an API that
let's me program in a disciplined manner, with facilities that are
guaranteed to obey access controls, so that I don't have to read every
detail of every filter to figure out whether the programmer was able
to recreate the access controls of C++ correctly.
And by the way, there are users who want a change in a visibility of a
member to cause such a huge surprise to the users of that class
that the visibility change breaks the existing users *loudly*.
Including users who reflect on the class. And the last thing some such
users
want is the reflection code continuing to (attempt to) do what it did
before, because that means the program and its reflection results
don't do the
same thing any more than they did before. There are various cases
where a member is made less accessible than before because it's now
protected by stronger invariants.
Received on 2024-11-11 09:32:44