Hi everyone, new here! At a big bank in the Netherlands we’re starting to adopt Pulumi, but in our first PoC we’re running into some strange issues that we cannot explain. Here it goes: We deploy a small stack that contains storage accounts, databricks, vnets, subnets, log analytics and some diagnostic settings. The latter, the diagnostic settings, prove a bit… buggy. While deploying everything is deployed correctly in Azure, and all the stack information is persisted. There’s one exception: the diagnostic settings. We create 2 (with unique names) of them, but after the first one

pulumi up

fails with the message:

cannot create already existing resource

Note: we start with an empty state, and an empty resources group. The diagnostic settings are created successfully in Azure but are not written to the state. And it fails complaining it already exists. We use the latest release of pulumi (3.1.0) and the azure-native provider (1.3.0). We’ve tried with a storage container as backend, as well as local storage. We also use KeyVault as encryption provider and run everything in Azure Pipelines. What could be the cause of that? We noticed it specifically in the diagnostic settings, nowhere else….

Hi, can you share the code you're using for the diagnostic settings?

Sure:

Sure:
law = azure_native.operationalinsights.Workspace(
    resource_name("log-workspace"),
    resource_group_name=resource_group.name,
    retention_in_days=365,
    sku=azure_native.operationalinsights.WorkspaceSkuArgs(
        name=azure_native.operationalinsights.WorkspaceSkuNameEnum.PER_GB2018
    ),
    workspace_name=resource_name("log-workspace")
)

kv_diag_settings = azure_native.insights.DiagnosticSetting(
    "kvDiagnosticSetting",
    resource_uri=kv.id,
    log_analytics_destination_type="Dedicated",
    logs=[azure_native.insights.LogSettingsArgs(
        category="AuditEvent",
        enabled=True,
        retention_policy=azure_native.insights.RetentionPolicyArgs(
            days=365,
            enabled=True
        ),
    )],
    metrics=[azure_native.insights.MetricSettingsArgs(
        category="AllMetrics",
        enabled=True,
        retention_policy=azure_native.insights.RetentionPolicyArgs(
            days=365,
            enabled=True
        ),
    )],
    name="kvDiagnosticSetting",
    workspace_id=law.id
)

adf_diag_settings = azure_native.insights.DiagnosticSetting(
    "adfDiagnosticSetting",
    resource_uri=adf.id,
    log_analytics_destination_type="Dedicated",
    logs=[azure_native.insights.LogSettingsArgs(
        category="PipelineRuns",
        enabled=True,
        retention_policy=azure_native.insights.RetentionPolicyArgs(
            days=365,
            enabled=True
        ),
    ),
        azure_native.insights.LogSettingsArgs(
            category="TriggerRuns",
            enabled=True,
            retention_policy=azure_native.insights.RetentionPolicyArgs(
                days=365,
                enabled=True
            ),
        ),
        azure_native.insights.LogSettingsArgs(
            category="ActivityRuns",
            enabled=True,
            retention_policy=azure_native.insights.RetentionPolicyArgs(
                days=365,
                enabled=True
            ),
        )],
    metrics=[azure_native.insights.MetricSettingsArgs(
        category="AllMetrics",
        enabled=True,
        retention_policy=azure_native.insights.RetentionPolicyArgs(
            days=365,
            enabled=True
        ),
    )],
    name="adfDiagnosticSetting",
    workspace_id=law.id
)

adf

,

kv

and

resource_group

are Pulumi resources, Data Factory, KeyVault and a Resource Group respectively.

Is it each of the

DiagnosticSetting

resources that fails or just one of them?

The first one fails and never continues with the second one.

Hehe, got it. I'm looking into this and will get back to your shortly.

Thanks!

Local storage as in

pulumi login --local

?

Yes

Hi Tim, Cameron asked me to look at this problem this morning. Just to clarify, do you think that you'd deleted a resource earlier that left the settings dangling and then re-created the new resource? Did you do this with pulumi or the Azure portal?

Hi Piers, The dangling resource happened when we manually delete all or some of the resources. We’ve now checked with

pulumi destroy

as well and it appears Pulumi does it correctly and removes everything, including the diagnostic! (another reason to use Pulumi 😉 ) So this is purely an Azure (maybe Portal only, we didn’t check the CLI) thing.

That's great news! Let me or Cameron know if you have any more questions 😄

Will do, thanks for helping out!