Hi all, I'm currently updating the dependencies in my pulumi project for Azure Databricks resources. However, with the version bump

v1.47.0 -> v1.48.0

for

pulumi-databricks

, I obtain an error in both

pulumi preview

and

pulumi up

:

error: Program failed with an unhandled exception:
    Traceback (most recent call last):
      File "/home/ffber/.pulumi/bin/pulumi-language-python-exec", line 192, in <module>
        loop.run_until_complete(coro)
      File "/usr/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
        return future.result()
               ^^^^^^^^^^^^^^^
      File "/home/ffber/source/PlatformPmbi/pulumi/venv/lib/python3.11/site-packages/pulumi/runtime/stack.py", line 142, in run_in_stack
        await run_pulumi_func(run)
      File "/home/ffber/source/PlatformPmbi/pulumi/venv/lib/python3.11/site-packages/pulumi/runtime/stack.py", line 56, in run_pulumi_func
        await wait_for_rpcs()
      File "/home/ffber/source/PlatformPmbi/pulumi/venv/lib/python3.11/site-packages/pulumi/runtime/stack.py", line 118, in wait_for_rpcs
        await task
    Exception: invoke of databricks:index/getGroup:getGroup failed: invocation of databricks:index/getGroup:getGroup returned an error: invoking databricks:index/getGroup:getGroup: 1 error occurred:
        * cannot read group: io.jsonwebtoken.IncorrectClaimException: Expected iss claim to be: <https://sts.windows.net/><correct-azure-tenant-id>/, but was: <https://sts.windows.net/><incorrect-azure-tenant-id>/

So the issue is that there's a mismatch in Azure tenant IDs. However, I never changed the pulumi configuration in any way for the tenant settings and also verified that the Azure tenant IDs are still correct. On top of that, all of

pulumi preview

and

pulumi up

worked fine for every

pulumi-databricks<=1.47.0

. Reviewed the release notes as well and no indication that there are breaking changes coming in this regard. Did anyone else experience some similar issues at some point or is there anyone that might point me in the right direction as to where the issue is originating from? Thanks 🙂.

• Are you explicitly defining a

pulumi_databricks.Provider

or using the default provider? ◦ If explicit, are you setting the

azure_tenant_id

parameter? ◦ If implicit, are you setting

databricks:azure_tenant_id

in your stack config? • How are you authenticating against azure (cli, managed identity, service principal)? • If you're not setting any explicit values (would not recommend), what does

az account show

tell you? ◦ You can set the correct account by running

az account set --subscription <subscription_name or id>

• I'm defining the provider like this prior to creating resources:

import pulumi_databricks as db

account_provider = db.Provider("databricks-account-provider",
                               account_id=<my-databricks-account-id>,
                               host='<https://accounts.azuredatabricks.net/>')

• For local development I'm authenticating via the cli, and in the CI/CD I'm using the built in pulumi task in Azure DevOps (see here). • Output of

az account show

:

{
  "environmentName": "AzureCloud",
  "homeTenantId": <expected-tenant-id>,
  "id": "0ca0eac1-5fe4-42dc-ba8f-f5849e0b65xx",
  "isDefault": true,
  "managedByTenants": [
    {
      "tenantId": ...
    }
  ],
  "name": "<subsription-name>",
  "state": "Enabled",
  "tenantId": "<expected-tenant-id>",
  "user": {
    "name": "<my-user-name>",
    "type": "user"
  }
}

• I set the correct subscription interactively when using

az login

prior to my pulumi runs. Can you elaborate on what you mean with "setting explicit values"?

You can configure the tenant ID by passing it to the Provider, see docs for list of other available azure related parameters:

account_provider = db.Provider(
  "databricks-account-provider",
  account_id=<my-databricks-account-id>,
  host='<https://accounts.azuredatabricks.net/>',
  azure_tenant_id='<your tenant_id>', # <== Explicitly setting the Azure tenant ID
)

Also since you are defining an explicit provider (as opposed the default provider configured via pulumi stack config) make sure you are setting that provider in resource options for other resources:

opts*=*pulumi*.*ResourceOptions(provider*=account_provider*)

Alternatively you can use the default provider and set those configuration values via

pulumi config set databricks:<parameter> <value>

There's also an

auth_type

parameter which you can set to

azure-msi

,

azure-cli

&

github-oidc-azure

. And since the databricks provider is based on the Terraform provider the documentation there might be helpful: https://registry.terraform.io/providers/databricks/databricks/latest/docs#special-configurations-for-azure

Nice, thank you! I'll give it a try and see whether it'll solve my issue 🙂.