Canonicalize NaNs when casting between float and double

[stevenfontanella]

Fixes #8626. Also part of #8261.

The bug is due to the fact that C++ doesn't guarantee a bitwise representation for NaNs that are casted from double -> float or vice versa. From https://en.cppreference.com/cpp/language/implicit_conversion:

Floating-point promotion
A prvalue of type float can be converted to a prvalue of type double. The value does not change.

In the case of NaNs, the value (NaN) is preserved regardless of the particular NaN that is picked, so we have no guarantee of the bitwise representation. And for double -> float:

If the conversion is listed under floating-point promotions, it is a promotion and not a conversion.

• If the source value can be represented exactly in the destination type, it does not change.
• If the source value is between two representable values of the destination type, the result is one of those two values (it is implementation-defined which one, although if IEEE arithmetic is supported, rounding defaults to nearest).
• Otherwise, the behavior is undefined.

I assume NaN conversions again fall under case 1 meaning that the "value" (NaN) is preserved but not the bitwise representation.

Canonicalize NaNs after promotion or demotion to ensure that the quiet bit remains set.

[stevenfontanella]

@rathann, would you be able to try to repro your test from this branch? I'm not aware of a good way to run this architecture myself locally.

[rathann]

Thanks for the PR. I'll try to test it over the weekend.

[tlively]

LGTM if this fixes the problem

[stevenfontanella]

Thanks! I'll wait for the test to confirm.