MATLAB DISCUSSION

How can MATLAB reveal precision problems before converting an algorithm to float32?

Started by sarah MATLAB precisionfloat32numerical stabilityembedded algorithmserror analysis
3 replies 248 views 4 participants
Latest activity · 27 Sep 2026

How can MATLAB reveal precision problems before converting an algorithm to float32?

sarah MATLAB Forum
#1

What tests should accompany conversion of a MATLAB algorithm from double precision to single precision for an embedded target? A few ordinary inputs may produce nearly identical outputs while edge cases fail.

For a running sum or a calculation involving the subtraction of similar values, how can the test set expose a numerically fragile formulation rather than just compare rounded final results?

Community replies 3

Re: How can MATLAB reveal precision problems before converting an algorithm to float32?

#2

Include the expected input range, long sequences and cases with large dynamic range. Compare absolute error and relative error where the reference is not close to zero. Near zero, relative error can become misleading, so define an absolute tolerance as well. Keep a higher-precision reference, but remember that an inaccurate algorithm can produce similar-looking errors in both numeric types.

Re: How can MATLAB reveal precision problems before converting an algorithm to float32?

#3

In a long sum, adding a very small value to a much larger accumulator can stop changing a float32 result. Vary the order and scale of the inputs and examine the error as the sequence grows. Depending on the algorithm, rescaling, pairwise summation or compensated accumulation may help. Measure their memory and execution costs before choosing a target implementation.

Re: How can MATLAB reveal precision problems before converting an algorithm to float32?

#4

For subtraction of nearly equal values, look for an algebraically equivalent form that avoids cancellation. Also check intermediate values for overflow or underflow, not only the final output. Run the actual target implementation on the same vectors, because compiler settings and library functions can affect results. Choose tolerances from the application's error budget rather than expecting bit-for-bit equality with desktop double precision.

TEP COMMUNITY