> It is possible to declare a constant variable in a header file, and
then have it constexpr in the translation unit?

Yes; the "constexpr" applies to the full-expression of the
initialization (https://eel.is/c++draft/dcl.constexpr#6), not to the
object. It only makes the object "const". "constexpr" makes functions
constexpr functions, so it cannot be applied only in one place and not
the other because that would be an ODR violation.

See also https://stackoverflow.com/q/30208685/5740428