问题描述
当我尝试对Hive使用Spark-Sql时，将引发以下错误.
When I tried to use Spark-Sql against Hive, the error like below is thrown.
Exception in thread "main" java.lang.NoSuchFieldError: HIVE_STATS_JDBC_TIMEOUT
at org.apache.spark.sql.hive.HiveUtils$.formatTimeVarsForHiveClient(HiveUtils.scala:204)
at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:90)
at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
根据SO线程 hive-stats-jdbc-timeout -for-hive-queries-spark 和
As per SO thread hive-stats-jdbc-timeout-for-hive-queries-in-spark andspark-on-hive-sql-query-error-nosuchfielderror-hive-stats-jdbc-timeout, this issue occurs when you're using specific version of Spark and Hive, actually, if you want to use latest version spark like 2.4.3 and latest Hive like 3.1.1, it can't be skipped.
我们可以检查此社区线程以获取详细信息， https://issues.apache. org/jira/browse/SPARK-13446 ，自2019年2月以来没有更新.
We can check this community thread for details, https://issues.apache.org/jira/browse/SPARK-13446, no update since Feb.2019.
所以您知道有关此问题的任何更新吗?如果我们想在源代码级别上自己跳过它，那么有关如何制作它的任何线索?
So so you know any update about this issue? If we want to skip it ourselves in source level, any clue about how to make it?
非常感谢您的帮助.
推荐答案
仅在Spark 3.0.0(尚未发布)中提供对使用Hive 3.1.1的支持.吉拉- https://jira.apache.org/jira/browse/SPARK-24360
Support for using Hive 3.1.1 will be available only from Spark 3.0.0(Yet to be released).
Jira - https://jira.apache.org/jira/browse/SPARK-24360
这篇关于关于HIVE_STATS_JDBC_TIMEOUT以及在源代码级别如何跳过的任何更新的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！