Azure Databricks 不会从 RBAC 访问密钥保管库权限错误

Azure Databricks won't access key vault permissions error from RBAC

提问人:geds133 提问时间:11/15/2023 更新时间:11/15/2023 访问量:98

问:

我正在尝试遵循本教程:https://learn.microsoft.com/en-us/azure/databricks/getting-started/connect-to-azure-storage

我已经按照它进行了操作,但是当我到达步骤 6:使用 python 连接到 Azure Data Lake Storage Gen2 时,我在尝试访问密钥保管库时遇到错误。

我的错误是这样的:

com.databricks.common.client.DatabricksServiceHttpClientException: PERMISSION_DENIED: Invalid permissions on the specified KeyVault https:X. Wrapped Message: Status code 403, "{"error":{"code":"Forbidden","message":"Caller is not authorized to perform action on resource.\r\nIf role assignments, deny assignments or role definitions were changed recently, please observe propagation time.\r\nCaller: name=AzureDatabricksnDecisionReason: 'DeniedWithNoValidRBAC' \r\nVault: KV;location=uksouth\r\n","innererror":{"code":"ForbiddenByRbac"}}}"

我认为这是某个角色,所以添加了 Key Vault 管理员 Key Vault 证书官 Key Vault Secrets User 尝试解决问题,但仍然出现相同的错误。我是新手,无法弄清楚为什么这不起作用。

任何建议将不胜感激。

azure-databricks azure-keyvault

评论

0赞 Anupam Chand 11/15/2023
设置 Databricks 以与 RBAC Key Vault 连接时存在一些问题。使用RBAC keyvault时,请使用访问策略密钥保管库,或参阅此详细文章 medium.com/@scott.fountain/...。这种方法的缺陷在于,你将授予对整个密钥保管库的完全访问权限,而不是对特定机密/密钥的访问权限。

答:

1赞 Sridevi 11/15/2023 #1

如果错过了将适当的 RBAC 角色分配给密钥保管库下的 AzureDatabricks 服务主体,通常会发生此错误。

当我在我的环境中尝试相同的操作而没有为应用程序分配角色时,我遇到了相同的错误,如下所示:Azure Databricks

service_credential = dbutils.secrets.get(scope="<scope>",key="<service-credential-key>")

spark.conf.set("fs.azure.account.auth.type.<storage-account>.dfs.core.windows.net", "OAuth")
spark.conf.set("fs.azure.account.oauth.provider.type.<storage-account>.dfs.core.windows.net", "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider")
spark.conf.set("fs.azure.account.oauth2.client.id.<storage-account>.dfs.core.windows.net", "<application-id>")
spark.conf.set("fs.azure.account.oauth2.client.secret.<storage-account>.dfs.core.windows.net", service_credential)
spark.conf.set("fs.azure.account.oauth2.client.endpoint.<storage-account>.dfs.core.windows.net", "https://login.microsoftonline.com/<directory-id>/oauth2/token")

响应:

com.databricks.common.client.DatabricksServiceHttpClientException: PERMISSION_DENIED: Invalid permissions on the specified KeyVault https://srikv151.vault.azure.net/. Wrapped Message: Status code 403, "{"error":{"code":"Forbidden","message":"Caller is not authorized to perform action on resource.\r\nIf role assignments, deny assignments or role definitions were changed recently, please observe propagation time.\r\nCaller: name=AzureDatabricks;appid=2ff814a6-3304-4ab8-85cb-xxxxxx;oid=fe597bb2-377c-44f1-8515-xxxxxxx;iss=https://sts.windows.net/72f988bf-86f1-41af-91ab-xxxxxxx/\r\nAction: 'Microsoft.KeyVault/vaults/secrets/getSecret/action'\r\nResource: '/subscriptions/b83c1ed3-c5b6-44fb-b5ba-xxxxx/resourcegroups/v-sridevim/providers/microsoft.keyvault/vaults/srikv151/secrets/secret'\r\nAssignment: (not found)\r\nDenyAssignmentId: null\r\nDecisionReason: 'DeniedWithNoValidRBAC' \r\nVault: srikv151;location=eastus\r\n","innererror":{"code":"ForbiddenByRbac"}}}"

enter image description here

若要解决此错误,请确保将 Key Vault 管理员角色分配给 Key Vault 下的 AzureDatabricks 服务主体。

可以使用以下 CLI 命令将角色分配给密钥保管库下的 AzureDatabricks 服务主体:

az role assignment create --assignee-object-id <oid_from_error> --role "Key Vault Administrator" --scope "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.KeyVault/vaults/{keyVaultName}"

响应:

enter image description here

当我在门户中检查相同的内容时,已成功将角色分配给密钥保管库下的 AzureDatabricks 服务主体:

enter image description here

现在可以使用 python 从 Azure Databricks 工作区访问 Azure Data Lake Storage Gen2 帐户。