Floating point is HARD!
Wednesday, 26 November 2003 14:06Co-worker trying to convince me that "0.5 is not exactly representable in [binary] floating-point".
Hello? It's exactly 2-1. Of course it's exactly representable!
Maybe you were thinking of 0.1 or something, which isn't representable exactly in binary. But not 0.5.
A little knowledge is a dangerous thing.
no subject
Date: Wednesday, 26 November 2003 09:09 (UTC)no subject
Date: Wednesday, 26 November 2003 09:14 (UTC)Yes, it should be. 0.510 = 2-1, so yeah, 0.12 sounds right.
0.110, on the other hand, is 0.110011001100…2—a repeating decimal.
(Hm, I wonder whether this works: 0.11002.)
no subject
Date: Wednesday, 26 November 2003 09:27 (UTC)no subject
Date: Wednesday, 26 November 2003 09:53 (UTC)(I think it doesn't work in the bio, though.)
no subject
Date: Wednesday, 26 November 2003 09:57 (UTC)no subject
Date: Wednesday, 26 November 2003 17:47 (UTC)... you mean ... a repeating binary? :-)
no subject
Date: Wednesday, 26 November 2003 23:15 (UTC)