Spark Java sum 给出的值不正确-解网

问：

Spark Java sum 给出的值不正确

Java 示例代码如下

List<Double> points = Arrays.asList(-6221.4, 6380.46);
Dataset<Row> dt  = spark.createDataset(points, Encoders.DOUBLE()).toDF("double_vals");
dt.createOrReplaceTempView("dual_table");
spark.sql("select sum(double_vals) from dual_table").show(false);

预期结果是 159.06，但我得到的结果如下

+-----------------+
|sum(double_vals) |
+-----------------+
|159.0600000000004|
+-----------------+

我做错了什么吗？

扩展示例，如果小数在求和后是动态的，如下所示，我将无法将其限制为小数点后 2 位，有什么解决方案吗？

Tuple3<String,String,Double> val1 = new Tuple3<>("Day1","Ram", -6221.4);
Tuple3<String,String,Double> val2 = new Tuple3<>("Day2","Ram", 6380.46);
Tuple3<String,String,Double> val3 = new Tuple3<>("Day1","Sam", 380.46);
Tuple3<String,String,Double> val4 = new Tuple3<>("Day2","Sam", 6380.462);
List<Tuple3<String,String,Double>> points = Arrays.asList(val1,val2,val3,val4);

Dataset<Row> dt  = spark.createDataset(points, Encoders.tuple(Encoders.STRING(),Encoders.STRING(),Encoders.DOUBLE())).toDF("day","name","profit");

dt.createOrReplaceTempView("dual_table");

Dataset<Row> newDs = spark.sql("select NAME, sum(profit) sum_val from dual_table group by name");

newDs.show();

结果将是

+----+------------------+
|NAME|           sum_val|
+----+------------------+
| Ram| 159.0600000000004| This needs 2 decimal points 
| Sam|6760.9220000000005| This needs 3 decimal points
+----+------------------+

apache-spark apache-spark-sql spark-java

欢迎来到浮点运算的广阔世界，其中的表达式是假的（在 Python、Java 等中尝试一下）。简而言之，如果一个数字存储在 64 位上，则最多可以表示 2^{64} 个不同的数字。然而，数学中有无限数量的实数，其中许多被表示为近似值，这会导致不准确的错误，例如您发现的错误。查看 en.wikipedia.org/wiki/Floating-point_arithmetic 以获取该主题的介绍0.1 + 0.2 == 0.3

答：

1赞 Srinivas 8/16/2023 #1

使用函数截断精度round

spark.sql("select round(sum(double_vals), 2) as sum_value from dual_table").show(false)

+---------+
|sum_value|
+---------+
|159.06   |
+---------+

用cast( sum(<column name>) AS decimal(10, 2))

spark.sql("select cast(sum(double_vals) as decimal(10, 2)) as sum_value from dual_table").show(false)

+---------+
|sum_value|
+---------+
|159.06   |
+---------+

Spark Java sum 给出的值不正确

Spark Java sum is giving incorrect value

评论

评论