https://pulumi.com logo
Title
g

glamorous-helmet-50600

05/25/2021, 9:46 PM
Hi there! I'm setting up a CosmosDB database with SQL Containers using AzureNative and it looks like the indexing policy specified is not being applied.
IndexingPolicy = new AzureNative.DocumentDB.Inputs.IndexingPolicyArgs
                {
                    Automatic = true,
                    ExcludedPaths = {},
                    IncludedPaths = 
                    {
                        new AzureNative.DocumentDB.Inputs.IncludedPathArgs
                        {
                            Indexes = 
                            {
                                new AzureNative.DocumentDB.Inputs.IndexesArgs
                                {
                                    DataType = "String",
                                    Kind = "Range",
                                    Precision = -1,
                                },
                                new AzureNative.DocumentDB.Inputs.IndexesArgs
                                {
                                    DataType = "Number",
                                    Kind = "Range",
                                    Precision = -1,
                                },
                                new AzureNative.DocumentDB.Inputs.IndexesArgs {
                                    Kind = "Spatial",
                                    DataType = "Point"
                                }
                            },
                            Path = "/*",
                        },
                    },
                    IndexingMode = "consistent",
                },
It's as per documentation, but when I check the container via the portal this is the indexing policy:
{
  "indexingMode": "consistent",
  "automatic": true,
  "includedPaths": [
    {
      "path": "/*"
    }
  ],
  "excludedPaths": [
    {
      "path": "/\"_etag\"/?"
    }
  ],
  "spatialIndexes": [
    {
      "path": "/*",
      "types": [
        "Point",
        "LineString",
        "Polygon",
        "MultiPolygon"
      ]
    }
  ]
}
Any clues?
t

tall-librarian-49374

05/25/2021, 9:50 PM
Could you change something in that resource definition, run the update with debug logs (
pulumi up --skip-preview --yes -v=9 --debug --logflow --logtostderr
), find the HTTP request that we are sending and make sure it looks valid?
g

glamorous-helmet-50600

05/25/2021, 9:51 PM
Sure, trying it now.
Created a new container with the same settings, the request looks okay to me, this is the body of the request:
{
  "location": "uksouth",
  "properties": {
    "options": {},
    "resource": {
      "conflictResolutionPolicy": {
        "conflictResolutionPath": "/_ts",
        "mode": "LastWriterWins"
      },
      "id": "test-conta",
      "indexingPolicy": {
        "automatic": true,
        "includedPaths": [
          {
            "indexes": [
              {
                "dataType": "String",
                "kind": "Range",
                "precision": -1
              },
              {
                "dataType": "Number",
                "kind": "Range",
                "precision": -1
              },
              {
                "dataType": "Point",
                "kind": "Spatial"
              }
            ],
            "path": "/*"
          }
        ],
        "indexingMode": "consistent"
      },
      "partitionKey": {
        "kind": "Hash",
        "paths": [
          "/pid"
        ]
      }
    }
  }
}
(still it's not created as supposed..)
the "indexes" never makes it through the actual indexing policy..I wonder if I'm doing something wrong or if this is a genuine bug
t

tall-librarian-49374

05/26/2021, 9:19 AM
Yes, it looks fine to my eye… Another thing to try - are you able to set it up via the portal and then either
pulumi refresh
if you changed the existing resource and/or export the ARM template and see if it’s different?
g

glamorous-helmet-50600

05/26/2021, 10:27 AM
I created the containers based on existing containers on a different db account so I double-checked the generated ARM template before replicating it via pulumi. Looking at the arm template from the resources created by Pulumi I can see that the indexing policy is indeed different and not what I configured pulumi to do.
In fact, during the pulumi run after it does the PUT request to create the container it will also do a GET to fetch it and the result from it shows that the "indexes" property (that should be within the IncludedPaths property) was not even there
I suspect this could be the reason why my functions are not picking up changes on the containers
It's also very easy to replicate by simply doing what on this article: https://www.pulumi.com/docs/reference/pkg/azure-native/documentdb/sqlresourcesqlcontainer/
The missing part on the resulting resource is going to be what's specified on the "Indexes" property of
AzureNative.DocumentDB.Inputs.IncludedPathArgs
t

tall-librarian-49374

05/26/2021, 10:59 AM
The question is why the API ignores those properties.
g

glamorous-helmet-50600

05/26/2021, 11:06 AM
Yes, if Pulumi is making the correct request (which seems to be..) then I'm not sure what to do from here. I can try the other provider to see if it does better
I'm using latest AzureNative (1.8) so I don't think it could be some mismatch on api versions
t

tall-librarian-49374

05/26/2021, 11:19 AM
Yeah, it would be nice to find something that works (the classic provider, or an ARM template)
g

glamorous-helmet-50600

05/26/2021, 1:14 PM
Looks like the classic provider doesn't offer the possibility of setting the Indexes as part of the IncludedPaths
Tested another thing (using azurenative) which was just setting the Automatic flag (Part of IndexingPolicyArgs) to false and it's also being ignored apparently.
Just to add a bit more info, trying to manually update the indexing policy using the portal also doesn't seem to work as expected, if I paste this:
{
  "indexingMode": "consistent",
  "automatic": true,
  "includedPaths": [
    {
      "path": "/*",
      "indexes": [
        {
          "kind": "Range",
          "dataType": "Number",
          "precision": -1
        },
        {
          "kind": "Range",
          "dataType": "String",
          "precision": -1
        },
        {
          "kind": "Spatial",
          "dataType": "Point"
        }
      ]
    }
  ],
  "excludedPaths": [
    {
      "path": "/\"_etag\"/?"
    }
  ]
}
it saves without errors but if I change tab and go back to the IndexingPolicy tab I get this:
{
  "indexingMode": "consistent",
  "automatic": true,
  "includedPaths": [
    {
      "path": "/*"
    }
  ],
  "excludedPaths": [
    {
      "path": "/\"_etag\"/?"
    }
  ],
  "spatialIndexes": [
    {
      "path": "/*",
      "types": [
        "Point",
        "LineString",
        "Polygon",
        "MultiPolygon"
      ]
    }
  ]
}
Definitely nothing wrong with what Pulumi is trying to do, it's just Azure being a bit shitty I'd say. Perhaps both versions are equivalent.
t

tall-librarian-49374

05/26/2021, 7:57 PM
Yeah it sounds like you should ask Azure folks why it works like that
👍 1