Stop the log and rsqrt epsilon defaults underflowing to zero in fp16

[NishchayMahor]

Fixes #2886.

What was wrong

log and rsqrt cast their default epsilon to the input dtype:

epsilon=nptype_from_builtin(self.x.dtype)(1e-45)   # log
epsilon=nptype_from_builtin(self.x.dtype)(1e-12)   # rsqrt

np.float16(1e-45) and np.float16(1e-12) are both exactly 0.0. Both docstrings describe epsilon as "a small constant that is added to the input ... for stability", and in fp16 it was not being added at all:

                epsilon                  folded value for x = [0.0, 1e-8]
rsqrt  fp16     0.0                      [inf inf]
rsqrt  fp32     1e-12                    [1.0000e+06 9.9995e+03]
log    fp16     0.0                      [-inf -inf]
log    fp32     1e-45                    [-103.27893  -18.420681]

What I changed

_epsilon_for_dtype(builtin_dtype, value) casts as before, then floors the result at the smallest positive subnormal for that dtype so it can never land on zero. log and rsqrt call it instead of casting directly.

I used np.nextafter(dtype(0), dtype(1)) rather than np.finfo(...).smallest_subnormal because smallest_subnormal only arrived in numpy 1.22 and setup.py still allows numpy >= 1.14.5.

fp32 is untouched. Both defaults are representable there and come back bit-identical; there is a test pinning that, because the alternative — picking a new epsilon magnitude for every dtype — would silently change existing fp32 models. fp16 gets 6e-08 instead of 0.0, which is the smallest change that restores the documented behaviour.

inverse casts its epsilon the same way but defaults to 1e-4, which is representable in fp16, so I left it alone rather than touch all three for symmetry.

How this helps

Any fp16 model where an rsqrt or log input can reach zero at convert time was folding to inf / -inf, which then propagates as NaN through whatever consumes it. rsqrt is the core of RMSNorm, so a zero-variance row is a realistic way to hit it. After this change those fold to finite values in fp16 as they already did in fp32.

There is a second effect I did not plan for and think is worth flagging: fp16 rsqrt(0) now folds to 4096.0, which is exactly what the Core ML runtime returns for that input. So for fp16, constant folding and the runtime now agree where they previously did not. The fp32 disagreement described in #2886 is unaffected by this change and still open.

Tests

Two classes in test_elementwise_unary.py:

• TestEpsilonForDtype — unit tests on the helper: never zero for fp16 or fp32, fp32 returns the value unchanged, fp16 falls back to the smallest subnormal.
• TestFp16EpsilonKeepsFoldedValuesFinite — the behavioural consequence, written so it does not depend on the helper existing. On current main only the two fp16 cases fail and both fp32 cases pass, so it discriminates rather than failing everywhere:

FAILED ...test_folded_value_is_finite_at_zero[op_name=rsqrt-np_dtype=<class 'numpy.float16'>]
FAILED ...test_folded_value_is_finite_at_zero[op_name=log-np_dtype=<class 'numpy.float16'>]

With the change, tests/iOS15/test_elementwise_unary.py is 20 passed.

What I did not verify

tests/iOS15/ as a whole has two pre-existing failures on this machine — TestCropResize::test_builder_to_backend_smoke and TestReshape::test_reshape_with_zero, both on the fp16 mlprogram backend. They fail identically on clean main, so they are not from this change, but I have not looked into them.

I also have not measured the effect on a real converted model. The evidence here is at the op and folding level.