It is true. It is an inherent limitation of how floating point values are represented in memory in a finite number of bits. This program, for instance, prints “false”: public class Main { public static void main(String[] args) { double a = 0.7; double b = 0.9; double x = a + 0.1; double y … Read more
You sort of answered your own question – build your format string dynamically… valid format strings follow the conventions outlined here: http://java.sun.com/j2se/1.5.0/docs/api/java/util/Formatter.html#syntax. If you want a formatted decimal that occupies 8 total characters (including the decimal point) and you wanted 4 digits after the decimal point, your format string should look like “%8.4f”… To my … Read more
There is a no exact representation of 0.1 as a float or double. Because of this representation error the results are slightly different from what you expected. A couple of approaches you can use: When using the double type, only display as many digits as you need. When checking for equality allow for a small … Read more
Short version: The reason it’s different is because pandas uses bottleneck (if it’s installed) when calling the mean operation, as opposed to just relying on numpy. bottleneck is presumably used since it appears to be faster than numpy (at least on my machine), but at the cost of precision. They happen to match for the … Read more
The reason the quotients in your test case are not equal is that in the math.floor(a/b) case, the result is calculated with floating point arithmetic (IEEE-754 64-bit), which means there is a maximum precision. The quotient you have there is larger than the 253 limit above which floating point is no longer accurate up to … Read more
I am going to assume IEEE 754 binary floating point arithmetic, with float 32 bit and double 64 bit. In general, there is no advantage to doing the calculation in double, and in some cases it may make things worse through doing two rounding steps. Conversion from float to double is exact. For the infinite, … Read more
Yes! It really is more numerically stable. For the case that you’re looking at, the numbers [0.0, 0.1, …, 0.9], note that under round-ties-to-away, only four of those numbers are rounding down (0.1 through 0.4), five are rounded up, and one (0.0) is unchanged by the rounding operation, and then of course that pattern repeats … Read more
You can use truncatingRemainder and 1 as the divider. Returns the remainder of this value divided by the given value using truncating division. Apple doc Example: let myDouble1: Double = 12.25 let myDouble2: Double = 12.5 let myDouble3: Double = 12.75 let remainder1 = myDouble1.truncatingRemainder(dividingBy: 1) let remainder2 = myDouble2.truncatingRemainder(dividingBy: 1) let remainder3 = myDouble3.truncatingRemainder(dividingBy: … Read more
The answer is in the documentation you quoted Computes the square root of the sum of the squares of x and y, without undue overflow or underflow at intermediate stages of the computation. If x*x + y*y overflows, then if you carry out the calculation manually, you’ll get the wrong answer. If you use std::hypot, … Read more
Picture is worth a thousand words – try to draw equation f(k) : and you will get such XY graph (X and Y are in logarithmic scale). If computer could represent 32-bit floats without rounding error then for every k we should get zero. But instead error increases with bigger values of k because of … Read more