Hi, we are importing quite extensive infrastructur...
# general
t
Hi, we are importing quite extensive infrastructure build on azure inside pulumi using the azure-native provider. Our workflow is: 1. using a script to prepare
import.json
file from terraform state. 2. import the resources
pulumi import --file .\import.json --protect=false
into the stack state 3. get the pulumi program to match the state 4. todo: profit 🚀 However, now we are troubleshooting potential issue. After the import, the state of the resource is populated with values for all properties. - kind of related https://github.com/pulumi/pulumi/issues/6146 If we specify all the resource properties in the code we will get to no changes during pulumi up and one could call the job done. Nonetheless, I suppose you will agree that having all properties specified in the code (for possibly hundreds of resources) is not a convenient way to manage your infrastructure. We want to keep in the code only properties that are relevant/different from the defaults. How can we achieve this? I have found one potential approach that I want to verify with you: If we delete resource property from pulumi program and then run
pulumi up
, it will show the property being deleted (from the state
inputs
), however as I understand it, the deleted properties will also be missing from the PATCH request hitting azure. So in the end nothing will get changed in azure and in state
outputs
. At first glance this looks like a way to get rid of the unnecessary definitions in pulumi program, however, it feels a bit hacky and I tested it only on one combination of resource and property. What is the proper way (or your way) of overcoming the necessity of specifying every resource property after import? What would you recommend?
t
> as I understand it, the deleted properties will also be missing from the PATCH request hitting azure This is not correct. We use PUT endpoints for (almost?) all resources, so we always send all the specified property values to it. At the same time, we don't send the deleted properties. Most services interpret this as no value and reset it to default. There may be some odd resources that interpret that as "no change, keep the old value" but that is wrong service behavior according to Azure API design.
So in the end nothing will get changed in azure and in state
outputs
As I said above, this should result in a change in Azure. The outputs are defined by what we read from Azure after the resource update is complete, so that what Azure returns to us. Usually, it's updated too.
After the import, the state of the resource is populated with values for all properties.
This is slightly surprising. As a quick test, I ran
Copy code
pulumi import azure-native:resources:ResourceGroup my-resource-group /subscriptions/<redacted>/resourceGroups/Azure-Account-Foo
the generated code is
Copy code
const my_resource_group = new azure_native.resources.ResourceGroup("my-resource-group", {
    location: "westus",
    resourceGroupName: "Azure-Account-Foo",
    tags: {
        Owner: "Buzz",
    },
});
and the state inputs are
Copy code
"inputs": {
                    "location": "westus",
                    "resourceGroupName": "Azure-Account-Foo",
                    "tags": {
                        "Owner": "Buzz",
                    }
                },
which doesn't look like all properties? Can you give an example of a resource that you are importing?
t
thank you for the reply. one example that comes to mind is
azure-native:documentdb:DatabaseAccount
the generated code is
Copy code
const db_account_name = new azure_native.documentdb.DatabaseAccount("db-account-name", {
    accountName: "Db-Account-Name",
    backupPolicy: {
        continuousModeProperties: {
            tier: azure_native.documentdb.ContinuousTier.Continuous30Days,
        },
        type: "Continuous",
    },
    consistencyPolicy: {
        defaultConsistencyLevel: azure_native.documentdb.DefaultConsistencyLevel.Strong,
        maxIntervalInSeconds: 5,
        maxStalenessPrefix: 100,
    },
    createMode: azure_native.documentdb.CreateMode.Default,
    databaseAccountOfferType: azure_native.documentdb.DatabaseAccountOfferType.Standard,
    defaultIdentity: "",
    disableKeyBasedMetadataWriteAccess: false,
    disableLocalAuth: false,
    enableAnalyticalStorage: false,
    enableAutomaticFailover: false,
    enableFreeTier: false,
    enableMultipleWriteLocations: false,
    enablePartitionMerge: false,
    identity: {
        type: azure_native.documentdb.ResourceIdentityType.None,
    },
    isVirtualNetworkFilterEnabled: false,
    kind: azure_native.documentdb.DatabaseAccountKind.GlobalDocumentDB,
    location: "East US",
    locations: [{
        failoverPriority: 0,
        isZoneRedundant: false,
        locationName: "East US",
    }],
    minimalTlsVersion: azure_native.documentdb.MinimalTlsVersion.Tls12,
    networkAclBypass: azure_native.documentdb.NetworkAclBypass.None,
    publicNetworkAccess: azure_native.documentdb.PublicNetworkAccess.Enabled,
    resourceGroupName: "Rg-Name",
}, {});
in comparison the terraform definition was something along this
Copy code
resource "azurerm_cosmosdb_account" "db_account_name" {
  name                = var.name
  location            = var.resource_group.location
  resource_group_name = var.resource_group.name


  offer_type = "Standard"
  kind       = "GlobalDocumentDB"

  consistency_policy {
    consistency_level = "Strong"
  }

  geo_location {
    location          = var.resource_group.location
    failover_priority = 0
  }

  backup {
    type = "Continuous"
  }
}
As you can see, there are so many properties that feel like default values (at least in direct comparison with terraform), that we don't want to specify in code. So we are looking for the cleanest/safest way how to remove them from the program definition.
t
Yeah I can see how this is sub-optimal. I tried searching for an existing Azure Native issue tracking this but failed to find one. Could you please open a new one? Meanwhile, removing those default values and running
pulumi up
sounds like a safe thing to do.
t
There are a few issues touching the topic from a different perspective, but none of them address it directly, i will open a new one. https://github.com/pulumi/pulumi/issues/5878 https://github.com/pulumi/pulumi-azure-native/issues/794 https://github.com/pulumi/pulumi/issues/6146
t
I believe all these issues are for the
importId
resource option, not for the
pulumi import
command.
t
Yes, their issues were that you can't import the services without specifying all (default) properties. As cli
pulumi import
does not block you in that regard, you can go through the import. nonetheless after that you still need to specify everything in code, otherwise you will see properties being deleted during the next
pulumi up
. but as far as I understand, as long as the properties scheduled for deletion are the "default" ones, you can confirm - say
yes
on the deletion. This will clear the properties from
inputs
in the state file and if they indeed were the defaults, the summary after
pulumi up
is completed will show no changes.
Thanks @tall-librarian-49374 for the participation, i have created the issue https://github.com/pulumi/pulumi-azure-native/issues/3621