On a different topic, here’s one more… I’d defer this to after Prague, but there’s at least one paper in the pre-Prague mailing about optional<T&>, and in the past couple of weeks we’ve had two different Reddit threads about optional<T&>.
So I wrote the following about references and specifically optional<T&> – like the “move” one, it’s designed both for broad teaching of C++ users, and also for committee members as input for Prague if people think it’s a helpful contribution.
I’d appreciate any feedback. I plan to post this publicly as well, but for now I hope it may be a useful contribution for Prague if optional<T&> is discussed.
For
P2070’s design option of optional<T&>, the basic message below is “please pursue the other option(s) instead.”
Thanks,
Herb
---
C++ references are useful for passing information to/from functions and range-for
,
and sometimes as local variables. Attempts to use references anywhere else in the language typically leads to endless design debates. This post is an attempt to shed light on that situation, and perhaps reduce some of the time spent on unresolved ongoing design
debates in the C++ community.
The first 5% of this post teaches everything I think is important about how to use references (not including the usual parameter passing guidance, covered elsewhere). The remaining
95% is a FAQ. Disclaimer: This is an opinionated post. Other experts, including reviewers, may disagree (especially where noted).
Thank you to the following for their feedback on drafts of this material: Howard Hinnant, Richard Smith, Bjarne Stroustrup.
In C++, a C&
or C&&
reference
is an indirect way to refer to an existing object. It is implemented under the covers as a pointer, but semantically acts like an alias because it's automatically dereferenced when you use its name. (Other details are not covered here, including that C&&
has
a different meaning depending on whether C
is a concrete type or a template parameter type.)
C++ references were invented to be used as function parameter/return types, and that's what they're still primarily useful for. Since C++11, that includes the range-for
loop
which conceptually works like a function call (see Q&A).
A reference can also be useful as a local variable, though in modern C++ a pointer or structured binding is nearly always as good or better (see Q&A).
That's it. All other uses of references should be avoided.
Please see the Q&A for const&
lifetime
extension, pair<T&, U&>
, and especially optional<T&>
.
Here is a summary, but for more detail please see The Design and Evolution of C++ (D&E) section 3.7, which begins: "References
were introduced primarily to support operator overloading..."
In C, to pass/return objects to/from functions you have two choices: either pass/return a copy, or take their address and pass/return a pointer which lets you refer to an existing
object.
Neither is desirable for overloaded operators. There are two motivating use cases, both described in D&E:
a - b
, which is natural and consistent with built-in types' operators. If we had to write &a
- &b
to pass by pointer, that would be inconvenient, inconsistent with how we use the built-in operators, and a conflict when that operator already has a different meaning for raw
pointers (as it does in this example).*
and []
.
Passing by reference lets calling code write str[0] = 'a';
which is natural and consistent with built-in arrays
and operators. If we had to write *str[0] = 'a';
to return by pointer, that would be a little inconvenient and also
inconsistent with built-in operators, but not the end of the world and so this one is only a secondary motivating case.
Those are the only uses of references discussed in D&E (including in the section on smart references and operator.
),
and the only places where references are really needed, where using a pointer is not as good.
for
being
like a function call?
The C++11 range-for
loop
is semantically like function parameter passing: We pass a range to the loop which takes it as if by an auto&&
parameter,
and then the loop passes each element in turn to each loop iteration and the loop body takes the element in the way it declares the loop element variable. For example, this loop body takes its element parameter by const
auto&
:
// Using range-for: The loop variable is a parameter to
// the loop body, which is called once per loop iteration
for (const auto& x : rng) { ... }
If we were instead using the std::for_each
algorithm
with the loop body in a lambda, the parameter passing is more obvious: for_each
takes the
range via an iterator pair of parameters, and then calls the loop body lambda passing each element as an argument to the lambda's parameter:
// Using std::for_each: Basically equivalent
for_each (begin(rng), end(rng), [&](const auto& x) { ... });
Yes, but they're brittle and basically unnecessary as of C++17.
After references were first added in the 1980s, C++ later added a special case where binding a temporary object to a local variable of type const&
and
still later auto&&
(but not generally other kinds of local references) was "made useful"
by imbuing only those references with the special power of extending the lifetime of a temporary object, just because we could (and because there were use cases where it was important for performance, before C++17 guaranteed copy elision). However, these cases
have always been:
const T& t = f();
and const
T& t = f().x;
and struct X { const T& r; } x = { f() };
extend
the lifetime of an object returned by value from f()
, but const
T& t = f().g();
does not);T& t = f();
is
ill-formed, whereas const T& t = f();
and T t =
f();
still uniformly work); andT
t = f();
and the meaning is both obvious and correct, as well as way easier to teach and learn and use).
Yes, but C++17 structured bindings are strictly better.
For example, given a set<int> s
and
calling an insert
function that returns a pair<iterator,
bool>
, just accessing the members of the pair directly means putting up with hard-to-read names:
// accessing the members of a pair directly
auto value = s.insert(4);
if (value.second) {
do_something_with(value.first);
}
Structured bindings lets us directly name the members -- note that this just invents names for them, it does not create any actual pointer indirection:
// using structured bindings (just as easy, and more readable)
auto [position, succeeded] = s.insert(4);
if (succeeded) {
do_something_with(position);
}
In the olden days before structured bindings, some people like to use references to indirectly name the members -- which like the above gives them readable names, but unlike the above
does create new pointer-equivalent indirect variables and follows those pointers which can incur a little space and time overhead (and also isn't as readable)...
// using references (cumbersome, don't do this anymore)
auto value = s.insert(4);
auto& position = value.first; // equivalent to pointers
auto& succeeded = value.second;
if (succeeded) { // invisible dereference
do_something_with(position); // invisible dereference
}
// or this (ditto)
... but even in the olden days, references were never significantly better than using pointers since the code is basically identical either way, and using pointers makes it clearer
that we're actually following an indirection:
// using pointers (equivalent to references)
auto value = s.insert(4);
auto position = &value.first; // clearly pointers
auto succeeded = &value.second;
if (*succeeded) { // visible dereference
do_something_with(*position); // visible dereference
}
Before C++17, using references could be a reasonable style choice, though even then pointers were arguably just about as good. Since C++17, we should prefer structured bindings.
Yes. Pointers can do it equivalently, it's a style choice.
For example, this local reference is useful:
int& r = a[f(i)];
// ... then use r repeatedly ...
Or you can equivalently use a pointer:
int* p = a[f(i)];
// ... then use *p repeatedly ...
T&
convenient
for easily expressing a pointer than can't be rebound to another object?
Yes, and this is mainly useful as a local variable. (See also previous answer.)
So does T* const
,
it's a style choice.
T&
convenient
for easily expressing a pointer that is not null?
Sort of -- T&
lets
you express a pointer that's not-null and that can't be rebound.
You can also express not-null by using gsl::not_null<>
(see
for example the Microsoft GSL implementation),
and one advantage of doing it this way is that it also lets you independently specify whether the pointer can be rebound or not -- if you want it not to be rebindable, just add const
as
usual.
[&]
capture?
[&]
is the right default for a lambda that's passed to a function that will just use it and then
return (aka structured lifetime), and doesn't store it somewhere where it will outlive the function call. Those structured uses fall under the umbrella of using references as parameter/return types. For non-parameter/return uses, prefer using pointers.
pair<T&, U&>
and tuple<T&,
U&>
and struct { T& t; U& u; }
?
I've mainly seen these come up as parameter and return types, where for the latter the most common motivation is that C++ doesn't (yet) support multiple return values, or as handwritten
equivalents of what lambda [&]
capture does. For those uses, they fall under the umbrella
of using references as parameter/return types. For non-parameter/return uses, prefer using pointers.
Yes -- it is either or both, depending on what you're doing at the moment.
This dual nature is the main problem of trying to use a reference as a general concept: Sometimes the language treats a reference as a pointer (one level of indirection), and sometimes
it treats it as an alias for the referenced object (no level of indirection, as if it were an implicitly dereferenced pointer), but those are not the same thing and references make those things visually ambiguous.
When passing/returning an object, this isn't a problem because we know we're always passing by pointer under the covers and when we use the name we're always referring to the existing
object by alias. That's clear, and references are well designed for use as function parameters and return values.
But when trying to use references elsewhere in the language, we have to know which aspect (and level of indirection!) we're dealing with at any given time, which leads to confusion
and woe. References have never been a good fit for non-function uses.
Don't. WOPR said it best, describing something like the game of trying to answer this class of question: "A
strange game. The only winning move is not to play."
Don't let yourself be baited into trying to answer this question. For example, if you're writing a class template, just assume (or document) that it can't be instantiated with reference
types. The question is a will o' the wisp, and to even try to answer it is to enter a swamp, because there won't be a general reasonable answer.
(Disclaimer: You, dear reader, may at this very moment be thinking of an ((other use)) for which you think you have a reasonable and correct answer. Whatever it is, it's virtually
certain that a significant fraction of other experts are at this very moment reading this and thinking of that ((other use)) with a different answer, and that you can each present technical arguments why the other is wrong. See optional<T&>
below.)
For the specific case of pair<T&, U&>
and tuple<T&,
U&>
and struct { T& t; U& u; }
,
see the previous answer regarding those. Otherwise:
No, see previous. People keep trying this, and we keep having to teach them not to try because it makes classes work in weird and/or unintended ways.
Pop quiz: Is struct X { int& i; };
copyable?
If not, why not? If so, what does it do?
Basic answer: X
is
not copy assignable, because i
cannot be modified to point at something else. But X
is
copy constructible, where i
behaves just as if it were a pointer.
Better answer: X
behaves
the same as if the member were int* const i;
-- so why not just write that if that's what's
wanted? Writing a pointer is arguably simpler and clearer.
No, see above. Don't be drawn into trying to answer when this could be valid or useful.
Explicitly jamming a reference type into a template that didn't deduce it and isn't expecting it, such as calling std::some_algorithm<std::vector<int>::iterator&>(vec.begin(),
vec.end());
, will be either very confusing or a compile-time error (or both, a very confusing compile-time error).
No, see above. Don't be drawn into trying to answer when this could be valid or useful.
optional<T&>
?
No, see above. Especially for this question, don't be drawn into trying to answer when this could be valid or useful.
An astonishing amount of ink has been spilled on this particular question for years, and it's not slowing down -- the pre-Prague mailing has another paper mentioning this, and we've
had multiple Reddit'd posts about it in the past week or two as I write this (example, example).
Those posts are what prompted me to write this post, expanding on private email to one of the authors.
Merely knowing that the discussion has continued for so many years with no consensus is a big red flag that the question itself is flawed. And if you're reading this and think you
have answer, ask yourself whether in your answer optional<T&>
really IS-AN optional<T>
--
template specializations should be substitutable (ask vector<bool>
) and the proposed answers
I've seen for optional<T&>
are not substitutable semantically (you can't write generic code
that uses an optional<T>
and works for that optional<T&>
),
including that some of them go so far as actually removing common functions that are available on optional<T>
.
Knowing that references are only for parameter/return/local values will warn us away from even trying to answer "what should optional<T&>
do?"
as a design trap, and we won't fall into it. Don't let yourself be baited into trying to play the game of answering what it should mean. "The only winning
move is not to play."
Use optional<T>
for
values, and optional<T*>
or optional<not_null<T*>>
for
pointers.
If you believe you have a clear answer to what optional<T&>
can
mean that:
·
cannot be represented about equally well by optional<not_null<T*>>
;
and
·
does not already have published technical arguments against it showing problems with the approach;
then please feel free to post a link below to a paper that describes that answer in detail.
.end.