I think this still is a really simple solution. I'll see if this gets more traction.
You need to provide more motivation.
"What if I have an empty struct type that is trivially copyable and assignable but for some bizarre unexplained reason I want it to be non-swappable" is simply no motivation at all.
–Arthur