The derivative of `ArcTan(X, Sqrt(1-X*X))`

with respect to X is `1/Sqrt(1-X*X)`

. This goes to infinity as |X| goes to 1. Therefore, when X is near 1 or -1, any error in evaluation has a huge effect on the result. Thus, it is critical that evaluation minimize error in these cases.

When X is near 1, evaluation of `1-X`

has no error (in IEEE 754 or any good floating-point system, because the scale of the result is such that its least significant bit is at least as low as the least significant bit in 1 or X, so the exact mathematical result has no bits outside the available significand bits). Since `1-X`

is exact, consider the effect of the error in `1+X`

by considering the derivative of `ArcTan(X, Sqrt((1-X)*(1+X+e))`

with respect to e, where e is the error introduced in the `1+X`

operation. The derivative is, when X is near 1 and e is small, approximately -1/10. (Taking the derivative with Maple and substituting 1 for x yields `-1/(sqrt(4+2e)*(5+2e))`

. Then substituting 0 for e yields -1/10.) Thus, the error in `1+X`

is not critical.

Therefore, evaluating the expression as `ArcTan(X, Sqrt((1-X)*(1+X))`

is a good way to evaluate it.

The situation is symmetric for X near -1. (`1+X`

has no error, and `1-X`

is not critical.)

Conversely, if we consider the error in `X*X`

, the derivative of `ArcTan(X, Sqrt(1-X*X+e))`

with respect to e is, when X is near 1, approximately -1/(2*sqrt(e)*(1+e)), so it is large when e is small. So a small error in evaluating `X*X`

will cause a large error in the result, when X is near 1.

Ask Pascal Cuoq points out, when evaluating a function f(x), we are generally interested in minimizing the relative error in the final result. And, as I have pointed out, the errors that occur during calculation are generally relative errors in intermediate results due to floating-point rounding. I was able to ignore this in the above because I was considering the function when X is near 1, so both the intermediate values under consideration (1+X and X*X) and the final value had magnitudes near 1, so dividing the values by those magnitudes would not change anything significantly.

However, for completeness, I examined the situation more closely. In Maple, I wrote `g := arctan(x, sqrt((1-x*x*(1+e0))*(1+e1))*(1+e2))`

, thus allowing for relative errors e0, e1, and e2 in the calculations of `x*x`

, `1-x*x`

, and the `sqrt`

, respectively, and I wrote `h:= arctan(x, sqrt((1-x)*(1+x)*(1+e0))*(1+e2))`

for the alternative. Note that e0 in this case combines the three errors in `1-x`

, `1+x`

, and the multiplication of them; the full error term could be `(1+ea)*(1+eb)*(1+ec)`

, but this is effectively `1+e0`

with a larger possible range for e0.

Then I examined the derivatives of these functions with respect to (one at a time) e0, e1, and e2 divided by abs(f(x)), where `f`

was the ideal function, `arctan(x, sqrt(1-x*x))`

. E.g., in Maple, I examined `diff(g, e0) / abs(f(x))`

. I did not perform a full analytic evaluation of these; I examined the values for some values of x near 0 and near 1 and for values of e0, e1, and e2 at one of their limits, -2^{-54}.

For x near 0, the values were all of magnitude about 1 or less. That is, any relative error in calculation resulted in a similar relative error in the result, or less.

For x near 1, the values with the derivatives of e1 and e2 were tiny, about 10^{-8} or less. However, the values with the derivatives of e0 were hugely different for the two methods. For the `1-x*x`

method, the value was about 2•10^{7} (using x = 1-2^{-53}). For the `(1-x)*(1+x)`

method, the value was about 5•10^{-9}.

In summary, the two methods do not differ much near x = 0, but the `(1-x)*(1+x)`

method is significantly better near x = 1.