在 Kubernetes 中将 Superset 连接到 Dremio 时出现问题

Issue connecting Superset to Dremio in Kubernetes

提问人:ilmar 提问时间:10/13/2023 更新时间:10/16/2023 访问量:69

问:

我正在尝试在 Apache SupersetDremio 之间建立连接,两者都部署在 Kubernetes 集群中。虽然设置与本文中描述的问题非常相似,但我认为与 Kubernetes 相关的一些细微差别可能会导致问题。

设置:

Apache Superset: Deployed on Kubernetes using the official helm chart (3.0.0).
Dremio: Also deployed on Kubernetes using its respective helm chart (24.1.0).
Python: v3.9.18
sqlachemy-dremio: v3.0.3

使用的连接字符串:

dremio+flight://dremioUser:dremioPwd@<dremio-service>.<namespace>.svc.cluster.local:32010/dremio

遇到的错误:在Superset中按下“测试连接”按钮时,我遇到了以下错误:

The above exception was the direct cause of the following exception:

Traceback (most recent call last):   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)   File "/usr/local/lib/python3.9/site-packages/flask_appbuilder/security/decorators.py", line 95, in wraps
    return f(self, *args, **kwargs)   File "/app/superset/views/base_api.py", line 127, in wraps
    raise ex   File "/app/superset/views/base_api.py", line 121, in wraps
    duration, response = time_function(f, self, *args, **kwargs)   File "/app/superset/utils/core.py", line 1518, in time_function
    response = func(*args, **kwargs)   File "/app/superset/utils/log.py", line 255, in wrapper
    value = f(*args, **kwargs)   File "/app/superset/views/base_api.py", line 93, in wraps
    return f(self, *args, **kwargs)   File "/app/superset/databases/api.py", line 905, in test_connection
    TestConnectionDatabaseCommand(item).run()   File "/app/superset/databases/commands/test_connection.py", line 192, in run
    raise SupersetErrorsException(errors) from ex superset.exceptions.SupersetErrorsException: [SupersetError(message='(builtins.NoneType) None\n[SQL: Flight returned unavailable error, with message: failed to connect to all addresses; last error: UNKNOWN: ipv4:10.1.2.3:32010: Ssl handshake failed: SSL_ERROR_SSL: error:0A00010B:SSL routines::wrong version number]\n(Background on this error at: https://sqlalche.me/e/14/dbapi)', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Dremio', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})] 2023-10-13 12:09:50,495:WARNING:superset.views.base:SupersetErrorsException Traceback (most recent call last):   File "/app/superset/databases/commands/test_connection.py", line 163, in run
    raise DBAPIError(ex_str or None, None, None) sqlalchemy.exc.DBAPIError: (builtins.NoneType) None [SQL: Flight returned unavailable error, with message: failed to connect to all addresses; last error: UNKNOWN: ipv4:10.1.2.3:32010: Ssl handshake failed: SSL_ERROR_SSL: error:0A00010B:SSL routines::wrong version number] (Background on this error at: https://sqlalche.me/e/14/dbapi)

采取的步骤:

  1. 已从 Superset pod 检查 Dremio 服务的可访问性。未检测到任何问题。
  2. 已验证 sqlalchemy-dremio 是否已正确安装在 Superset pod 中。
  3. 使用 Python 脚本测试从 Superset pod 中到 Dremio 的连接。此连接成功,表示没有明显的网络或服务问题。

但是,通过 Superset UI 进行连接仍会引发错误。

我已经浏览了上一篇文章中的建议,但没有任何运气。鉴于这两项服务都在 Kubernetes 上,我想知道是否有任何我可能忽略的特定配置。

如果有人在 Kubernetes 环境中遇到过类似的问题,或者对潜在的修复方法有深入的了解,我们将不胜感激。

python kubernetes apache-superset dremio

评论


答:

1赞 Subhasmita Swain 10/16/2023 #1

尝试将 dremio 应用程序公开给公共服务,例如负载均衡器,并获取公共 IP 地址。然后,您可以将该 IP 用于您的连接 uri,例如:

dremio+flight://dremioUser:dremioPwd@<dremio-public-ip>:32010/dremio?%3Coption%3E=%3Cvalue%3E%E2%80%9D&UseEncryption=false