https://pulumi.com logo
Title
a

astonishing-intern-99200

05/04/2021, 3:20 PM
Hi everyone, new here! At a big bank in the Netherlands we’re starting to adopt Pulumi, but in our first PoC we’re running into some strange issues that we cannot explain. Here it goes: We deploy a small stack that contains storage accounts, databricks, vnets, subnets, log analytics and some diagnostic settings. The latter, the diagnostic settings, prove a bit… buggy. While deploying everything is deployed correctly in Azure, and all the stack information is persisted. There’s one exception: the diagnostic settings. We create 2 (with unique names) of them, but after the first one
pulumi up
fails with the message:
cannot create already existing resource
Note: we start with an empty state, and an empty resources group. The diagnostic settings are created successfully in Azure but are not written to the state. And it fails complaining it already exists. We use the latest release of pulumi (3.1.0) and the azure-native provider (1.3.0). We’ve tried with a storage container as backend, as well as local storage. We also use KeyVault as encryption provider and run everything in Azure Pipelines. What could be the cause of that? We noticed it specifically in the diagnostic settings, nowhere else….
g

gentle-diamond-70147

05/04/2021, 3:26 PM
Hi, can you share the code you're using for the diagnostic settings?
a

astonishing-intern-99200

05/04/2021, 3:40 PM
Sure:
Sure:
law = azure_native.operationalinsights.Workspace(
    resource_name("log-workspace"),
    resource_group_name=resource_group.name,
    retention_in_days=365,
    sku=azure_native.operationalinsights.WorkspaceSkuArgs(
        name=azure_native.operationalinsights.WorkspaceSkuNameEnum.PER_GB2018
    ),
    workspace_name=resource_name("log-workspace")
)

kv_diag_settings = azure_native.insights.DiagnosticSetting(
    "kvDiagnosticSetting",
    resource_uri=kv.id,
    log_analytics_destination_type="Dedicated",
    logs=[azure_native.insights.LogSettingsArgs(
        category="AuditEvent",
        enabled=True,
        retention_policy=azure_native.insights.RetentionPolicyArgs(
            days=365,
            enabled=True
        ),
    )],
    metrics=[azure_native.insights.MetricSettingsArgs(
        category="AllMetrics",
        enabled=True,
        retention_policy=azure_native.insights.RetentionPolicyArgs(
            days=365,
            enabled=True
        ),
    )],
    name="kvDiagnosticSetting",
    workspace_id=law.id
)

adf_diag_settings = azure_native.insights.DiagnosticSetting(
    "adfDiagnosticSetting",
    resource_uri=adf.id,
    log_analytics_destination_type="Dedicated",
    logs=[azure_native.insights.LogSettingsArgs(
        category="PipelineRuns",
        enabled=True,
        retention_policy=azure_native.insights.RetentionPolicyArgs(
            days=365,
            enabled=True
        ),
    ),
        azure_native.insights.LogSettingsArgs(
            category="TriggerRuns",
            enabled=True,
            retention_policy=azure_native.insights.RetentionPolicyArgs(
                days=365,
                enabled=True
            ),
        ),
        azure_native.insights.LogSettingsArgs(
            category="ActivityRuns",
            enabled=True,
            retention_policy=azure_native.insights.RetentionPolicyArgs(
                days=365,
                enabled=True
            ),
        )],
    metrics=[azure_native.insights.MetricSettingsArgs(
        category="AllMetrics",
        enabled=True,
        retention_policy=azure_native.insights.RetentionPolicyArgs(
            days=365,
            enabled=True
        ),
    )],
    name="adfDiagnosticSetting",
    workspace_id=law.id
)
adf
,
kv
and
resource_group
are Pulumi resources, Data Factory, KeyVault and a Resource Group respectively.
g

gentle-diamond-70147

05/04/2021, 3:57 PM
Is it each of the
DiagnosticSetting
resources that fails or just one of them?
a

astonishing-intern-99200

05/04/2021, 4:01 PM
The first one fails and never continues with the second one.
Well, the resource is created, just not saved to the state and then throws an error it already exists. Like Schrödinger's cat: It exists, but doesn't exist 😅
g

gentle-diamond-70147

05/04/2021, 5:07 PM
Hehe, got it. I'm looking into this and will get back to your shortly.
a

astonishing-intern-99200

05/04/2021, 5:59 PM
Thanks!
Maybe a small note: If we use a local storage during our CI pipeline both diagnostic settings are created, but neither end up in the state and the error about the resources (one of them) already existing remains. The fact that a storage container backend gives different results than using a local storage might help 🙂.
g

gentle-diamond-70147

05/04/2021, 6:14 PM
Local storage as in
pulumi login --local
?
a

astonishing-intern-99200

05/05/2021, 5:16 AM
Yes
So, we’ve did some more investigation and it appears to be a problem in Azure. It appears that if you delete a resource that has a diagnostic settings attached to it the diagnostic settings is left dangling in Azure. Even if you delete the Log Analytics workspace AND the entire resource group. If you later on recreate the resources with the same name the diagnostic setting magically re-attaches without being re-created. If you explicitly delete the setting first, the problem goes away. We’re now going to check if
pulumi destroy
will clean it up properly.
b

brave-planet-10645

05/05/2021, 9:11 AM
Hi Tim, Cameron asked me to look at this problem this morning. Just to clarify, do you think that you'd deleted a resource earlier that left the settings dangling and then re-created the new resource? Did you do this with pulumi or the Azure portal?
a

astonishing-intern-99200

05/05/2021, 10:59 AM
Hi Piers, The dangling resource happened when we manually delete all or some of the resources. We’ve now checked with
pulumi destroy
as well and it appears Pulumi does it correctly and removes everything, including the diagnostic! (another reason to use Pulumi 😉 ) So this is purely an Azure (maybe Portal only, we didn’t check the CLI) thing.
b

brave-planet-10645

05/05/2021, 11:26 AM
That's great news! Let me or Cameron know if you have any more questions 😄
a

astonishing-intern-99200

05/05/2021, 11:45 AM
Will do, thanks for helping out!