sg12

From: Herb Sutter <hsutter_at_[hidden]> · Date: Fri, 7 Feb 2020 07:05:23 +0000

---
The Simple Truth About References
C++ references are useful for passing information to/from functions and range-for, and sometimes as local variables. Attempts to use references anywhere else in the language typically leads to endless design debates. This post is an attempt to shed light on that situation, and perhaps reduce some of the time spent on unresolved ongoing design debates in the C++ community.
The first 5% of this post teaches everything I think is important about how to use references (not including the usual parameter passing guidance, covered elsewhere). The remaining 95% is a FAQ. Disclaimer: This is an opinionated post. Other experts, including reviewers, may disagree (especially where noted).
Thank you to the following for their feedback on drafts of this material: Howard Hinnant, Richard Smith, Bjarne Stroustrup.
________________________________
References
What references are and how to use them
In C++, a C& or C&& reference is an indirect way to refer to an existing object. It is implemented under the covers as a pointer, but semantically acts like an alias because it's automatically dereferenced when you use its name. (Other details are not covered here, including that C&& has a different meaning depending on whether C is a concrete type or a template parameter type.)
C++ references were invented to be used as function parameter/return types, and that's what they're still primarily useful for. Since C++11, that includes the range-for loop which conceptually works like a function call (see Q&A).
A reference can also be useful as a local variable, though in modern C++ a pointer or structured binding is nearly always as good or better (see Q&A).
That's it. All other uses of references should be avoided.
Advanced note for experts
Please see the Q&A for const& lifetime extension, pair<T&, U&>, and especially optional<T&>.
________________________________
Q&A
Can you elaborate a little more on why references were invented for function parameter/return types?
Here is a summary, but for more detail please see The Design and Evolution of C++ (D&E) section 3.7, which begins: "References were introduced primarily to support operator overloading..."
In C, to pass/return objects to/from functions you have two choices: either pass/return a copy, or take their address and pass/return a pointer which lets you refer to an existing object.
Neither is desirable for overloaded operators. There are two motivating use cases, both described in D&E:
  *   The primary use case is that we want to pass an existing object to an operator without copying it. Passing by reference lets calling code write just a - b, which is natural and consistent with built-in types' operators. If we had to write &a - &b to pass by pointer, that would be inconvenient, inconsistent with how we use the built-in operators, and a conflict when that operator already has a different meaning for raw pointers (as it does in this example).
  *   Secondarily, we want to return an existing object without copying it especially from operators like unary * and []. Passing by reference lets calling code write str[0] = 'a'; which is natural and consistent with built-in arrays and operators. If we had to write *str[0] = 'a'; to return by pointer, that would be a little inconvenient and also inconsistent with built-in operators, but not the end of the world and so this one is only a secondary motivating case.
Those are the only uses of references discussed in D&E (including in the section on smart references and operator.), and the only places where references are really needed, where using a pointer is not as good.
What was that about range-for being like a function call?
The C++11 range-for loop is semantically like function parameter passing: We pass a range to the loop which takes it as if by an auto&& parameter, and then the loop passes each element in turn to each loop iteration and the loop body takes the element in the way it declares the loop element variable. For example, this loop body takes its element parameter by const auto&:
// Using range-for: The loop variable is a parameter to
// the loop body, which is called once per loop iteration
for (const auto& x : rng) { ... }
If we were instead using the std::for_each algorithm with the loop body in a lambda, the parameter passing is more obvious: for_each takes the range via an iterator pair of parameters, and then calls the loop body lambda passing each element as an argument to the lambda's parameter:
// Using std::for_each: Basically equivalent
for_each (begin(rng), end(rng), [&](const auto& x) { ... });
Aren't local references useful because of lifetime extension?
Yes, but they're brittle and basically unnecessary as of C++17.
After references were first added in the 1980s, C++ later added a special case where binding a temporary object to a local variable of type const& and still later auto&& (but not generally other kinds of local references) was "made useful" by imbuing only those references with the special power of extending the lifetime of a temporary object, just because we could (and because there were use cases where it was important for performance, before C++17 guaranteed copy elision). However, these cases have always been:
  *   brittle and inconsistent (e.g., const T& t = f(); and const T& t = f().x; and struct X { const T& r; } x = { f() }; extend the lifetime of an object returned by value from f(), but const T& t = f().g(); does not);
  *   irregular (e.g., T& t = f(); is ill-formed, whereas const T& t = f(); and T t = f(); still uniformly work); and
  *   unnecessary now that C++17 has guaranteed copy elision (e.g., just write T t = f(); and the meaning is both obvious and correct, as well as way easier to teach and learn and use).
Aren't local references useful to get names for parts of an object returned from a function?
Yes, but C++17 structured bindings are strictly better.
For example, given a set<int> s and calling an insert function that returns a pair<iterator, bool>, just accessing the members of the pair directly means putting up with hard-to-read names:
// accessing the members of a pair directly
auto value = s.insert(4);
if (value.second) {
    do_something_with(value.first);
}
Structured bindings lets us directly name the members -- note that this just invents names for them, it does not create any actual pointer indirection:
// using structured bindings (just as easy, and more readable)
auto [position, succeeded] = s.insert(4);
if (succeeded) {
    do_something_with(position);
}
In the olden days before structured bindings, some people like to use references to indirectly name the members -- which like the above gives them readable names, but unlike the above does create new pointer-equivalent indirect variables and follows those pointers which can incur a little space and time overhead (and also isn't as readable)...
// using references (cumbersome, don't do this anymore)
auto value      = s.insert(4);
auto& position  = value.first;          // equivalent to pointers
auto& succeeded = value.second;
if (succeeded) {                        // invisible dereference
    do_something_with(position);        // invisible dereference
}
// or this (ditto)
... but even in the olden days, references were never significantly better than using pointers since the code is basically identical either way, and using pointers makes it clearer that we're actually following an indirection:
// using pointers (equivalent to references)
auto value     = s.insert(4);
auto position  = &value.first;          // clearly pointers
auto succeeded = &value.second;
if (*succeeded) {                       // visible dereference
    do_something_with(*position);       // visible dereference
}
Before C++17, using references could be a reasonable style choice, though even then pointers were arguably just about as good. Since C++17, we should prefer structured bindings.
Aren't local references useful to express aliases, for example to a member of an array or container?
Yes. Pointers can do it equivalently, it's a style choice.
For example, this local reference is useful:
int& r = a[f(i)];
// ... then use r repeatedly ...
Or you can equivalently use a pointer:
int* p = a[f(i)];
// ... then use *p repeatedly ...
Isn't T& convenient for easily expressing a pointer than can't be rebound to another object?
Yes, and this is mainly useful as a local variable. (See also previous answer.)
So does T* const, it's a style choice.
Isn't T& convenient for easily expressing a pointer that is not null?
Sort of -- T& lets you express a pointer that's not-null and that can't be rebound.
You can also express not-null by using gsl::not_null<><http://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#SS-views> (see for example the Microsoft GSL implementation<https://github.com/microsoft/GSL/blob/b4dd39615aebb52dd73b6b1dfabd41b5e96c6962/include/gsl/pointers>), and one advantage of doing it this way is that it also lets you independently specify whether the pointer can be rebound or not -- if you want it not to be rebindable, just add const as usual.
What about lambda [&] capture?
[&] is the right default for a lambda that's passed to a function that will just use it and then return (aka structured lifetime), and doesn't store it somewhere where it will outlive the function call. Those structured uses fall under the umbrella of using references as parameter/return types. For non-parameter/return uses, prefer using pointers.
What about pair<T&, U&> and tuple<T&, U&> and struct { T& t; U& u; }?
I've mainly seen these come up as parameter and return types, where for the latter the most common motivation is that C++ doesn't (yet) support multiple return values, or as handwritten equivalents of what lambda [&] capture does. For those uses, they fall under the umbrella of using references as parameter/return types. For non-parameter/return uses, prefer using pointers.
Is a reference a pointer to an object, or an alternate name for the object?
Yes -- it is either or both, depending on what you're doing at the moment.
This dual nature is the main problem of trying to use a reference as a general concept: Sometimes the language treats a reference as a pointer (one level of indirection), and sometimes it treats it as an alias for the referenced object (no level of indirection, as if it were an implicitly dereferenced pointer), but those are not the same thing and references make those things visually ambiguous.
When passing/returning an object, this isn't a problem because we know we're always passing by pointer under the covers and when we use the name we're always referring to the existing object by alias. That's clear, and references are well designed for use as function parameters and return values.
But when trying to use references elsewhere in the language, we have to know which aspect (and level of indirection!) we're dealing with at any given time, which leads to confusion and woe. References have never been a good fit for non-function uses.
[GENERAL UMBRELLA QUESTION] But what about using a reference for ((other use not as a parameter or return type or local variable))?
Don't. WOPR said it best, describing something like the game of trying to answer this class of question: "A strange game. The only winning move is not to play."<https://www.youtube.com/watch?v=uOoXwxqeVzg>
Don't let yourself be baited into trying to answer this question. For example, if you're writing a class template, just assume (or document) that it can't be instantiated with reference types. The question is a will o' the wisp, and to even try to answer it is to enter a swamp, because there won't be a general reasonable answer.
(Disclaimer: You, dear reader, may at this very moment be thinking of an ((other use)) for which you think you have a reasonable and correct answer. Whatever it is, it's virtually certain that a significant fraction of other experts are at this very moment reading this and thinking of that ((other use)) with a different answer, and that you can each present technical arguments why the other is wrong. See optional<T&> below.)
But what about using a reference type as a class data member?
For the specific case of pair<T&, U&> and tuple<T&, U&> and struct { T& t; U& u; }, see the previous answer regarding those. Otherwise:
No, see previous. People keep trying this, and we keep having to teach them not to try because it makes classes work in weird and/or unintended ways.
Pop quiz: Is struct X { int& i; }; copyable? If not, why not? If so, what does it do?
Basic answer: X is not copy assignable, because i cannot be modified to point at something else. But X is copy constructible, where i behaves just as if it were a pointer.
Better answer: X behaves the same as if the member were int* const i; -- so why not just write that if that's what's wanted? Writing a pointer is arguably simpler and clearer.
But what about using a reference type as an explicit template argument?
No, see above. Don't be drawn into trying to answer when this could be valid or useful.
Explicitly jamming a reference type into a template that didn't deduce it and isn't expecting it, such as calling std::some_algorithm<std::vector<int>::iterator&>(vec.begin(), vec.end());, will be either very confusing or a compile-time error (or both, a very confusing compile-time error).
But what about using a reference type for a class template specialization?
No, see above. Don't be drawn into trying to answer when this could be valid or useful.
But wait, not even optional<T&>?
No, see above. Especially for this question, don't be drawn into trying to answer when this could be valid or useful.
An astonishing amount of ink has been spilled on this particular question for years, and it's not slowing down -- the pre-Prague mailing has another paper mentioning this, and we've had multiple Reddit'd posts about it in the past week or two as I write this (example<https://www.reddit.com/r/cpp/comments/et6eht/to_bind_and_loose_a_reference/>, example<https://www.reddit.com/r/cpp/comments/eu5j1s/optional_references_assignthrough_vs_rebinding/>). Those posts are what prompted me to write this post, expanding on private email to one of the authors.
Merely knowing that the discussion has continued for so many years with no consensus is a big red flag that the question itself is flawed. And if you're reading this and think you have answer, ask yourself whether in your answer optional<T&> really IS-AN optional<T> -- template specializations should be substitutable (ask vector<bool>) and the proposed answers I've seen for optional<T&> are not substitutable semantically (you can't write generic code that uses an optional<T> and works for that optional<T&>), including that some of them go so far as actually removing common functions that are available on optional<T>.
Knowing that references are only for parameter/return/local values will warn us away from even trying to answer "what should optional<T&> do?" as a design trap, and we won't fall into it. Don't let yourself be baited into trying to play the game of answering what it should mean. "The only winning move is not to play."<https://www.youtube.com/watch?v=uOoXwxqeVzg>
Use optional<T> for values, and optional<T*> or optional<not_null<T*>> for pointers.
If you believe you have a clear answer to what optional<T&> can mean that:
*         cannot be represented about equally well by optional<not_null<T*>>; and
*         does not already have published technical arguments against it showing problems with the approach;
then please feel free to post a link below to a paper that describes that answer in detail.
.end.