Just to update this: it looks like shortly after I posted this last year, there was a paper P3330 by Gonzalo Brito Gadeschi and Damien Lebrun-Grandie (https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p3330r0.html) proposing just such a feature, even with the same name `fetch_update`.

It attempts to address some of the issues identified in our discussion by placing tight restrictions on what the function f is allowed to do.

> On May 7, 2024, at 23:19, Nate Eldredge via Std-Discussion <std-discussion_at_[hidden]> wrote:
>
> Along the lines of std::atomic<T>::fetch_add() and friends, I have been wondering about the possibility of an interface that would let you atomically apply an *arbitrary* function to an atomic object. I'm wondering if such an interface has ever been considered for inclusion in C++.
>
> To explain what I mean, call it fetch_update() by analogy with Java's getAndUpdate(). I'm envisioning that something like
>
> std::atomic<int> x;
> int f(int);
> x.fetch_update(f);
>
> would atomically perform the equivalent of
>
> x.store(f(x.load());
[...]