How would you know that the container specialization is different?

Or in other words they are always different as it is a template.

At least each container would have to support such a facility to remove the type by itself. Otherwise the implementation details could hide incompatibilities.

std::vector could support a move_to member function:

std::vector<my_distance_type> d1;

// creates a new d2 with new metadata, but keeps the convertible actual data without copying

// d1 is empty afterwards

std::vector<double> d2 = d1.move_to<double>();