numpy 浮点数的精度限制?

Precision limits of numpy floats?

提问人:sten 提问时间:9/16/2021 最后编辑:sten 更新时间:9/16/2021 访问量:1046

问:

给定序列

1/1, 1/2, 1/3, ... , 1/n

如果我使用 numpy.float16,我如何计算在哪个点上我无法以精度 E 区分两个连续元素 1/i 和 1/i+1?即什么是“我”?

其他 np-floats 呢?

最小的E是多少?以及如何计算它的“i”?

例如,如果 E = 0.01,我可以区分 1/9 和 1/10,但不能区分 1/10 和 1/11,因为:

1/9 = 0.111
1/10 = 0.100
1/11 = 0.091

0.111 - 0.100 = 0.01  >= E
0.100 - 0.091 = 0.009 < E

i = 10

以更抽象的方式,给定 f(i) np.floatXX 中可表示的最大 'i' 是多少?


有趣的是,实际中的精度比计算的要差: /逻辑中断的地方/

for i in range(int(1e3),int(12e6)) : 
   if not np.floatXX(1/i) > np.floatXX(1/(i+1)) :
       print(i); break

float32: 11864338
float16: 1464
python numpy 类型 浮动精度

评论

1赞 Mad Physicist 9/16/2021
不得不发布第二个答案,因为我终于明白了你问题的确切要点。它可以从我最初写的内容中衍生出来,但绝对不是微不足道的。

答:

2赞 Mad Physicist 9/16/2021 #1

不要加 1,而是将分母加倍。您可以放心地假设它是某个二进制数。这里有一个简单的方法:

one = np.float64(1.0)
two = np.float64(2.0)
n = one
bits = 0
while one + n != one:
    bits += 1
    n /= two

你从开始,否则你会得到带你超过分辨率的位数。bits = 0

最后,你得到 ,这是 IEEE-754 编码的 64 位浮点数中的位数。bits = 53

这意味着,对于任何以有效的二进制科学记数法编码的数字,ULP(最小精度单位)约为 。具体来说,其中数字四舍五入到其最高位。您将无法解析浮点数中较小的相对变化。n * 2**-53n

奖励:为其他浮点类型运行上述代码可提供:

float16 (half):   11 bits
float32 (single): 24 bits
float64 (double): 53 bits
float96 (sometimes longdouble): 80 bits
float128 (when available): 113 bits

您可以修改上面的代码以适用于任何目标号码:

target = np.float16(0.0004883)
one = np.float16(1.0)
two = np.float16(2.0)
n = two**(np.floor(np.log2(target)) - one)
bits = 0
while target + n != target:
    bits += 1
    n /= two

结果 (ULP) 由下式给出,因为循环在失去分辨率后停止。这与我们开始的原因相同。在这种情况下:n * 2bits = 0

>>> n * two
5e-07

如果您预先知道尾数中的位数,则可以完全缩短计算。所以对于 , where , 你可以做float16bits = 11

>>> two**(np.floor(np.log2(target)) - np.float16(bits))
5e-07

在这里阅读更多内容:

评论

0赞 sten 9/16/2021
所以对于 F16,n=0.0004883,bits=11,所以 max-i 是 ??
0赞 Mad Physicist 9/16/2021
@sten。更新了示例。
0赞 Mad Physicist 9/16/2021
@sten 2.4E-07.如果插入所有正确的类型
0赞 sten 9/16/2021
谢谢,但我很困惑 E <=> n 和 i <=> 2^位吗?
0赞 Mad Physicist 9/16/2021
@sten。位是可以存储在尾数中的精度位数。 是 2 的幂的最大数字,不能解析为与目标数字的差值,即 1 位比目标最顶端低 12 个二进制位。 是可以解析的最小数字。这将导致目标的 ULP 恰好变化 1。nnn * two
-1赞 Mad Physicist 9/16/2021 #2

我的另一个答案提供了你实际提出的问题背后的理论,但需要一些不平凡的解释。这是缺少的步骤:

给定一些整数,你可以写成i

1 / i - 1 / (i + 1) =
(i + 1 - i) / (i * (i + 1)) =
1 / (i * (i + 1)) =
1 / (i**2 + i)

要找到在某些二进制表示中低于 ULP 的此类方法,您可以直接使用我的另一个答案。i1 / (i**2 + i)1 / i

的 ULP 由下式给出1 / i

ulp = 2**(floor(log2(1 / i)) - (bits + 1))

你可以尝试找到一个这样的i

1 / (i**2 + i) < 2**(floor(log2(1 / i)) - (bits + 1))
1 / (i**2 + i) < 2**floor(log2(1 / i)) / 2**(bits + 1)
2**(bits + 1) < (i**2 + i) * 2**floor(log2(1 / i))

It's not trivial as written because of the operation, and Wolfram Alpha runs out of time. Since I am cheap and don't want to buy Mathematica, let's just approximate:floor

2**(bits + 1) < (i**2 + i) * 2**floor(log2(1 / i))
2**(bits + 1) < (i**2 + i) / i
2**(bits + 1) < i + 1

You may be off by one or so, but you should see that around , the difference stops being resolvable. And indeed for the 11 bit mantissa of , we see:i = 2**(bits + 1) - 1float16

>>> np.float16(1 / (2**12 - 1)) - np.float16(1 / (2**12))
0.0

The actual number is a tiny bit less here (remember that approximation where we took away the ):floor

>>> np.float16(1 / (2**12 - 5)) - np.float16(1 / (2**12 - 4))
0.0
>>> np.float16(1 / (2**12 - 6)) - np.float16(1 / (2**12 - 5))
2.4e-07

As you noted in your comments, isi

>>> 2**12 - 6
4090

You can compute the exact values for all the other floating point types in a similar manner. But that is indeed left as an exercise for the reader.

评论

0赞 Eric Postpischil 9/16/2021
“The ULP of is given by ” is incorrect. Take the case of i=1, for which 1/i = 1, and its ULP is 2^−10. From your other answer, is 11 for . Then 2**(floor(log2(1 / i)) - (bits + 1)) = 2**(floor(log2(1)) - (11 + 1)) = 2**(floor(0) - 12) = 2**(0-12) = 2**-12.1 / iulp = 2**(floor(log2(1 / i)) - (bits + 1))float16bitsfloat16
1赞 Eric Postpischil 9/16/2021
The criterion given, , even when corrected to , determines when the difference between two successive terms falls below the ULP of the earlier term. However, that is not generally where the first failure to distinguish successive terms appears. Two terms that are less than an ULP apart are still distinguishable if they do not round to the same representable number.1 / (i**2 + i) < 2**(floor(log2(1 / i)) - (bits + 1))1 / (i**2 + i) < 2**(floor(log2(1 / i)) - (bits - 1))
1赞 Eric Postpischil 9/16/2021
Observe the points reported by the OP, 11864338 and 1464, both have significands near sqrt(2), 1.414339… and 1.4296875. I suspect that is not merely a coincidence.
0赞 Eric Postpischil 9/16/2021
Indeed, testing significand widths of 11 and up shows the significands of i for the last 1/i and 1/(i+1) that are distinguishable are 1.42969, 1.43896, 1.42358, 1.41626, 1.42108, 1.41928, 1.41809, 1.41733, 1.41549, 1.41473, 1.41518, 1.41499, 1.41453, 1.41434, 1.41445, 1.41441, 1.41429, 1.41422, 1.41427, 1.41425, 1.41424,… However, it drops below sqrt(2) at 37 bits and keeps going.