https://pulumi.com logo
#aws
Title
# aws
s

sparse-optician-70334

09/12/2023, 8:36 AM
I am trying to create a databricks workspace from pulumi. The steps described here https://docs.databricks.com/en/administration-guide/account-settings-e2/credentials.html#step-3-create-a-credential-configuration-in-databricks have already been ported to pulumi: - create cross account policy - create policy When trying to create the MwsCredentials or MwsStorageConfigurations I run into issues such as: cannot create mws credentials: default auth: cannot configure default credentials This issue also seems to be something for terraform: https://registry.terraform.io/providers/databricks/databricks/latest/docs/data-sources/mws_workspaces If you have a fully automated setup with workspaces created by databricks_mws_workspaces or azurerm_databricks_workspace, please make sure to add depends_on attribute in order to prevent default auth: cannot configure default credentials errors. How can this be solved with Pulumi?
Copy code
import pulumi
from pulumi_aws_native import s3, iam
import json
from pulumi_databricks import MwsWorkspaces, MwsCredentials, MwsStorageConfigurations, MwsNetworks, get_aws_assume_role_policy, get_aws_cross_account_policy, ServicePrincipal, ServicePrincipalSecret
import pulumi_databricks as databricks
from jinja2 import Environment, FileSystemLoader
from pathlib import Path

db_account_id = config.require('db_account_id')

creds = MwsCredentials("my-db-credentials",  credentials_name="my-db-credentials",
# ARN of <https://docs.databricks.com/en/administration-guide/account-settings-e2/credentials.html#step-1-create-a-cross-account-iam-role> (already craeted)
                        role_arn=cross_account_role.arn, account_id=f"{db_account_id}")
storage_config = MwsStorageConfigurations("my-storage-config",
# bucket_databricks_root -> already created
                                           bucket_name=bucket_databricks_root.id,
                                           storage_configuration_name="my-storage-config", 
                                           account_id=f"{db_account_id}")

workspace = MwsWorkspaces("my-workspace", 
     account_id=f"{db_account_id}",
     aws_region=region,
     credentials_id=creds.credentials_id,
     storage_configuration_id=storage_config.storage_configuration_id,
     workspace_name="myname",)
Also https://www.pulumi.com/registry/packages/databricks/api-docs/getmwscredentials/ seems to list this If you have a fully automated setup with workspaces created by databricks_mws_workspaces, please make sure to add depends_on attribute in order to prevent default auth: cannot configure default credentials errors.
How can the: depends_on be set? Exploring: opts=pulumi.ResourceOptions(depends_on=[creds, storage_config]), does not solve the issue?
It looks like databricks cli must be installed and configured for this to work. Use: databricks configure with host https://accounts.cloud.databricks.com
h

hallowed-fireman-90476

11/20/2023, 9:01 PM
@sparse-optician-70334 i am running into this issue now, i even created own Databricks provider by passing the account credentials still no luck, i can understand the default provider is picking up some authentication type by default. Can you share how you overcome this issue
NVM, figured it i changed the auth type to basic instead of its looking for my local databricks-cli config