D2226R0
A proposal for a function template to move from an object and reset it to its default constructed state

Draft Proposal,

Issue Tracking:
Inline In Spec
Author:
Audience:
SG18, SG1, SG20
Project:
ISO/IEC JTC1/SC22/WG21 14882: Programming Language — C++

Abstract

This paper proposes a new utility function template, take, which resets an object to its default constructed state, and returns the old value of that object. Effectively, take(obj) becomes the idiomatic form for C++14’s exchange(obj, {}). Formally, the new value assigned is built through value initialization, not default initialization (e.g. take called on an object of pointer type will reset it to nullptr; cf. [dcl.init.general]). In the paper we are however going to liberally talk about "default constructed" state or value, implying the state (resp. value) reached through value initialization.

1. Changelog

2. Tony Tables

Before After
class C {
    Data *data;
public:
    // idiomatic, C++14
    C(C&& other) noexcept : data(std::exchange(other.data, {})) {}
};
class C {
    Data *data;
public:
    // idiomatic, C++2?
    C(C&& other) noexcept : data(std::take(other.data)) {}
};
template <typename K,
          typename V,
          template <typename...> typename C = std::vector>
class flat_map {
    C<K> m_keys;
    C<V> m_values;

public:
    /*
      If flat_map wants to ensure a "valid but not specified"
      state after a move, then it cannot default its move
      operations: C’s own move operations may leave flat_map
      in an invalid state, e.g. with m_keys.size() != m_values.size()
      (breaking a more than reasonable class invariant for flat_map).
      In other words, "valid but not specified" states do not compose;
      we need to reset them to a fully specified state to restore
      this class' invariants.
    */
    flat_map(flat_map&& other) noexcept(~~~)
        : m_keys(std::exchange(other.m_keys, {})),
          m_values(std::exchange(other.m_values, {}))
    {}
};
template <typename K,
          typename V,
          template <typename...> typename C = std::vector>
class flat_map {
    C<K> m_keys;
    C<V> m_values;

public:









    // same, idiomatic
    flat_map(flat_map&& other) noexcept(~~~)
        : m_keys(std::take(other.m_keys)),
          m_values(std::take(other.m_values))
    {}
};
void Engine::processAll()
{
    // process only the data available at this point
    for (auto& value : std::exchange(m_data, {})) {
        // may end up writing into m_data; we will not process it,
        // and it will not invalidate our iterators
        processOne(value);
    }
}
void Engine::processAll()
{
    // same, idiomatic
    for (auto& value : std::take(m_data)) {


        processOne(value);
    }
}
void ConsumerThread::process()
{
    // grab the pending data under mutex protection,
    // so this thread can then process it
    Data pendingData = [&]() {
        std::scoped_lock lock(m_mutex);
        return std::exchange(m_data, {});
    }();

    for (auto& value : pendingData)
        process(value);
}
void ConsumerThread::process()
{


    Data pendingData = [&]() {
        std::scoped_lock lock(m_mutex);
        return std::take(m_data);
    }();

    for (auto& value : pendingData)
        process(value);
}
void Engine::maybeRunOnce()
{
    if (std::exchange(m_shouldRun, false))
        run();
}
void Engine::maybeRunOnce()
{
    if (std::take(m_shouldRun))
        run();
}
struct S {
    // unconditional sinks
    // overloaded for efficiency following F.15
    void set_data(const Data& d);
    void set_data(Data&& d);
} s;

Data d = ~~~;

// sink "d", but leave it in a *specified* valid state
s.set_data(std::exchange(d, {}));

assert(d == Data());
struct S {


    void set_data(const Data& d);
    void set_data(Data&& d);
} s;

Data d = ~~~;

// same, but idiomatic
s.set_data(std::take(d));

assert(d == Data());

In the example we are referring to the C++ Core Guideline F.15.

3. Motivation and Scope

C++14, with the adoption of [N3668], introduced the exchange utility function template. exchange is commonly found in the implementation of move operations, in algorithms, and in other similar scenarios. Its intent is to streamline multiple operations in one function call, making them less error prone, and ultimately creating an idiom:

struct MyPtr {
    Data *d;

    // BAD, the split makes it possible to forget to reset other.d to nullptr
    MyPtr(MyPtr&& other) : d(other.d) { other.d = nullptr; }

    // BETTER, use std::exchange
    MyPtr(MyPtr&& other) : d(std::exchange(other.d, nullptr)) {}

    // GOOD, idiomatic: use std::exchange, generalizing
    MyPtr(MyPtr&& other) : d(std::exchange(other.d, {})) {}


    void reset(Data *newData = nullptr)
    {
        // BAD, poor readability
        swap(d, newData);
        if (newData)
            dispose(newData);

        // BETTER, readable
        Data *old = d;
        d = newData;
        if (old)
            dispose(old);

        // GOOD, streamlined
        if (Data *old = std::exchange(d, newData))
            dispose(old);
    }
};

By surveying various code bases, we noticed a common pattern: a significant amount (50%-90%) of calls to exchange uses a default constructed value as the second parameter. The typical call has the idiomatic form exchange(obj, {}), or it has some other form that could still be rewritten into that one (like exchange(pointer, nullptr) or exchange(boolean_flag, false)).

For instance, here’s some results form very popular C++ projects:

Project Number of calls to exchange Number of calls to exchange(obj, {}) or equivalent (i.e. calls that could be replaced by take) Percentage Notes
Boost 1.74.0 121 97 80% Incl. calls to boost::exchange, as well as boost::exchange's own autotests.
Qt (qtbase and qtdeclarative repositories, dev branch) 37 33 89% Incl. calls to qExchange. Of the 4 calls that do not use a default constructed second argument, 2 are actually workaround for broken/legacy APIs and may get removed in the future.
Absl (master branch) 10 9 90% Incl. calls to absl::exchange; the 1 call that cannot be replaced comes from absl::exchange's own autotests.
Firefox (mozilla-central repository) 14 10 71% Incl. calls to skstd::exchange.
Chromium (master branch) 38 30 79%

Note: it is interesting that, one way or another, several projects introduced their own version of exchange in order to be able to use it without a C++14 toolchain.

The observation of such a widespread pattern led to the present proposal. Obviously, the figures above do not include any code path where the semantic equivalent of exchange is "unrolled" in those codebases; nonetheless, we claim that there is positive value for the C++ community if the pattern of "move out the old value, set a new default constructed one" could be given a name, and become an idiom on its own. If the chosen name for such a pattern is clear and sufficiently concise, it would improve the usage of exchange(obj, {}) (which is heavy on the eyes and somehow distracts from the actual intentions).

We propose to call this idiom take.

3.1. About move semantics

We also believe that such an idiom would become an useful tool in understanding and using move semantics. The function template take, as presented, can be used as a drop-in replacement for move (under the reasonable assumption that movable types are also, typically, default constructible). Unlike move, it would leave the source object in a well-defined state — its default constructed state:

f(std::move(obj)); // obj’s state is unknown
                   // - could be valid but unspecified
                   // - could be not valid / moved from
                   // - potentially, could be not moved-from at all...

// VERSUS

f(std::take(obj)); // obj has specified state

Using Sean Parent's definitions, take would constitute a safe operation, and be the counterpart of move (an unsafe operation).

As for some other differences between the two:

move take
Does auto new_obj = xxxx(old_obj); throw exceptions? Usually no: move constructors are commonly noexcept Depends: generally no assumptions can be made regarding the default constructor
What is the state of old_obj after the call above? Usually unspecified; depending on the class, it’s valid or not valid/partially formed Specified: default constructed state
If a type is cheap to move, is the above call cheap? Yes (tautological) Depends: generally no assumptions can be made regarding the default constructor
What does obj = xxxx(obj); do? Leaves obj in its moved-from state, or leaves obj in its original state, depending on the implementation Leaves obj in its original state, assuming no exceptions occur; it’s, however, expensive and not a no-op!
If I no longer need old_obj, is new_obj = xxxx(old_obj); a good idea? Yes No

In conclusion: we believe the Standard Library should offer both move and take to users; each one has its merits and valid use cases.

4. Impact On The Standard

This proposal is a pure library extension.

It proposes changes to an existing header, <utility>, but it does not require changes to any standard classes or functions and it does not require changes to any of the standard requirement tables.

This proposal does not require any changes in the core language, and it has been implemented in standard C++.

This proposal does not depend on any other library extensions.

5. Design Decisions

The most natural place to add the function template presented by this proposal is the already existing <utility> header, following the precedent of move and exchange.

5.1. Bikeshedding: naming

We foresee that finding the right name for the proposed function template is going to be a huge contention point. Therefore, we want to kickstart a possible discussion right away. In R0, we are proposing take, inspired by Rust’s own std::mem::take function, which has comparable semantics to the ones defined here.

Other possible alternatives include:

We strongly believe that this idiom needs a concise name in order to be useful; therefore we are not proposing something like exchange_with_default_constructed.

Submit a poll, seeking ideas and consensus for a name.

5.2. Why not simply defaulting the second parameter of exchange to T()?

[N3668] mentions this idea, but indeed rejects it because it makes the name exchange much less clear:

another_obj = std::exchange(obj); // exchange obj... with what? with another_obj? wait, is this a swap?
We agree with that reasoning, so we are not proposing to change exchange.

5.3. Should there be an atomic_take?

The rationale of adding exchange (with that specific name) in the Standard Library was generalizing the already existing semantics of atomic_exchange, extending them to non-atomic, movable types. By having take(obj) as a "shortcut" for exchange(obj, {}), one may indeed wonder if also atomic_take(obj) should be added, as a shortcut for atomic_exchange(obj, {}) (mut. mut. for atomic<T>::take).

We are not fully convinced by the usefulness of atomic_take and therefore we are not proposing it here. First and foremost, unlike exchange, atomic_exchange has many more uses where the second argument is not a default constructed value. Second, the overwhelming majority of usage of atomic types consist of atomic integral types and atomic pointer types. We do not believe that substituting atomic_exchange(val, 0) or atomic_exchange(val, nullptr) with atomic_take(val) would convey the same "meaning" when used in the context of atomic programming.

We would be happy to be convinced otherwise.

Ask SG1 for their opinion regarding atomic_take.

5.4. Should we promote the usage of take over move (in education materials, coding guidelines, etc.)?

No. We believe we need instead to educate users about the availability of both options in the Standard Library, make them understand the implications of each one, and let the users choose the right tool for the job at hand. To give an example: it sounds extremely unlikely that one should be using take inside a shuffle-based algorithm (the impact in performance and correctness would be disastrous when compared to using move instead).

There is however an important point to be made: the state of a moved-from object has being actively disputed in the C++ community pretty much since the very introduction of move semantics. The usual position is one between these two:

  1. object is valid but unspecified (e.g. [lib.types.movedfrom], [Sutter], [C.64]), and therefore it can be used in any way that does not have preconditions (or similarly it would still be possible to check such preconditions before usage); or

  2. object is partially formed / not valid, (e.g. [P2027R0]), and therefore the only operations allowed on such an object are assignment and destruction.

This debate has not yet reached a conclusion. In the meanwhile, what should a user handling a generic object of type T do, if they want to keep using it after it has been moved? The simplest solution would be to reset the object to a well known state. If there is already such a well known state readily available for the user, then the combination of move + "reset to state X" (however that would be expressed in code) makes perfect sense and it’s certainly the way to go. Otherwise, the easiest state to reason about objects of type T is their default constructed state; therefore one may want to reset their moved-from object to such default constructed state, and then keep using the object. Moving plus resetting to default constructed state is precisely what take does.

We would like to clarify that we do not have any sort of "hidden agenda" that wants to settle the state of a moved-from object (in the Standard Library, or in general). And, we absolutely do not claim that moved-from objects should always be reset to their default constructed state (in their move operations, or by always using take instead of move in one’s codebase). The availability of take for users should simply constitute another tool in their toolbox, allowing them to choose which kind of operation (safe/unsafe) makes most sense for their programs at any given point.

5.5. take guarantees a move and resets the source object; on the other hand, move does not even guarantee a move. Should there be another function that guarantees a move, but does not reset?

In other words, should there be some sort of "intermediate" function between move and take, that simply guarantees that the input object will be moved from? For instance:

template <class T>
constexpr T really_move(T& obj) requires (!is_const_v<T>) /* or equivalent */
{
    T moved = move(obj);
    return moved;
}

A possible use case would be to "sink" an object, therefore deterministically releasing its resources from the caller, even if the called function does not actually move from it:

template <class Fun>
void really_sink(Fun f)
{
    Object obj = ~~~;

    f(std::move(obj)); // if f doesn’t actually move,
                       // we still have obj’s resources here in the caller

    // VERSUS

    f(std::really_move(obj)); // obj is moved from, always.
                              // using take would be an overkill
                              // (for instance, obj is not used afterwards,
                              // so why resetting it?)
}

Now, having functions which have parameters of type rvalue reference (or forwarding reference), and then do not move / forward them unconditionally in all code paths, is generally frowned upon (cf. [F.18] and [F.19], and especially the "Enforcement" sections). The possibility for move to actually not move seems more a theoretical exercise than an issue commonly found in practice.

In any case, we do not have enough data to claim that there is a "widespread need" for such a function in the Standard Library; surveying the same projects listed in § 3 Motivation and Scope gives inconcludent results (it seems that such a function is not defined in any of them, for their own internal purposes).

Therefore, we are not going to propose such a really_move function here.

We do not think that take does impede in any way the addition of such a function anyhow, via a separate proposal. One might even argue that the addition of such a really_move function should be not tied to the addition of take, as it solves a different problem: ensuring that an object gets always moved from.

Poll for more opinions.

6. Technical Specifications

6.1. Implementation

We think the implementation should be straightforward and do not foresee any particular challenge. A possible implementation is:

// in namespace std

template <class T>
constexpr T take(T& obj)
{
    T old_val = move(obj);
    obj = T();
    return old_val;
}

6.2. Feature testing macro

6.3. Proposed wording

All changes are relative to [N4861].

TBD; matches the wording of exchange pretty closely.

7. Acknowledgements

Thanks to KDAB for supporting this work.

Thanks to Marc Mutz for reviewing this proposal, and pointing me to Sean Parent's blog post. His educational work regarding move semantics has been inspirational. He originally proposed the idea of an idiomatic form for std::exchange(obj, {}) on the std-proposals mailing list.

References

Informative References

[C.64]
Bjarne Stroustrup; Herb Sutter. C++ Core Guidelines, C.64: A move operation should move and leave its source in a valid state. URL: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rc-move-semantic
[F.15]
Bjarne Stroustrup; Herb Sutter. C++ Core Guidelines, F.15: Prefer simple and conventional ways of passing information. URL: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rf-conventional
[F.18]
Bjarne Stroustrup; Herb Sutter. C++ Core Guidelines, F.18: For “will-move-from” parameters, pass by X&& and std::move the parameter. URL: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#f18-for-will-move-from-parameters-pass-by-x-and-stdmove-the-parameter
[F.19]
Bjarne Stroustrup; Herb Sutter. C++ Core Guidelines, F.19: For “forward” parameters, pass by TP&& and only std::forward the parameter. URL: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#f19-for-forward-parameters-pass-by-tp-and-only-stdforward-the-parameter
[MarcMutz]
Marc Mutz. Is `std::exchange(obj, {})` common enough to be spelled `std::move_and_reset(obj)`?. URL: https://groups.google.com/a/isocpp.org/forum/#!topic/std-proposals/qDB0BG-GQqQ/discussion
[N3668]
Jeffrey Yasskin. exchange() utility function, revision 3. URL: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3668.html
[N4861]
Richard Smith, Thomas Koeppe, Jens Maurer, Dawn Perchik. Working Draft, Standard for Programming Language C++. URL: http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2020/n4861.pdf
[P2027R0]
Geoff Romer. Moved-from objects need not be valid. URL: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p2027r0.pdf
[SeanParent]
Sean Parent. About Move. URL: https://sean-parent.stlab.cc/2014/05/30/about-move.html
[StdMemTake]
Rust. Function std::mem::take. URL: https://doc.rust-lang.org/std/mem/fn.take.html
[Sutter]
Herb Sutter. Move, simply. URL: https://herbsutter.com/2020/02/17/move-simply/

Issues Index

Submit a poll, seeking ideas and consensus for a name.
Ask SG1 for their opinion regarding atomic_take.
Poll for more opinions.