https://pulumi.com logo
Join Slack
Powered by
Is there any guarantee to how my pulumi objects ge...
# python
s
Is there any guarantee to how my pulumi objects get mapped to values of a dictionary during creation? This seems to be consistent during development but when purging python cache i notice already created resource objects are re-ordered in the dictionary randomly sometimes causing resource changes. Using Python > 3.9
e
dictionary order is not guaranteed to be consistent
but this is only a problem if your changing from dictionaries (unordered) to lists (ordered) in your program
c
Dictionary ordering in python is now guaranteed, though it's not a great idea to rely on the order of those keys (even though it's been a language feature since 3.7). I'd love to see what your program looks like to determine where this reordering is occurring.
s
This is something I didn’t noticed until i ran the stack on another system… venv and pycache were purged / rebuilt. Since then i’ve been able to reproduce it after purging these dirs… which makes me think the order is being maintained in the cache somehow… I’m building a dictionary in a loop with values which are Pulumi resources…
@calm-hairdresser-54232
c
Aha, that makes sense - I would not expect the resources themselves to come back in a deterministic order.
s
What is the right way to do this?
c
I'd have to see an example of the code which is causing problems to know for sure, but I wouldn't depend on the order of asynchronous cloud operations when constructing objects / dictionaries.
s
How would we build a dictionary with x resources assigned as values? And when re-running, get the same resources re-assigned to the same place in the dictionary?
basically i need to create x resources and store those objects someplace to operate on them later…
c
Are you operating on them within the same stack, or in a different one?
s
same stack
c
It depends on the operations and the resources, but could you not just operate on the resources themselves rather than storing them in a dictionary? Can you share a code snippet to demonstrate what you're trying to accomplish?
s
Copy code
for subnet in environment['subnets']:
        az = environment['subnets'][subnet]['az']
        if environment['subnets'][subnet]['type'] == "public":

            public_subnets[az] = myclass.Subnet(
                az_id=az,
                type="Public",
                vpc_id=vpc.id,
                cidr_block=subnet,
            )
then later on, i’ll use the public_subnets[“az-code”] to deploy other resources
c
and 
environment['subnets']
is created dynamically somewhere above in a loop?
s
Yes that is statically defined and is always the same
c
I'm curious what your 
pulumi preview
output looks like in this case - could you post that as well?
s
It’s different every time… but nothing happens while cache is maintained. Later in the program i’m creating resolver endpoints and something like this occurs
Copy code
~   └─ aws:route53:ResolverEndpoint  thing_ResolverEndpoint  update     [diff: ~ipAddresses]
c
Ah, and the order of the ipAddresses is triggering this diff?
s
yes… it changes sometimes because i’m selecting 2x keys in the dict for each ip
c
I think I would need to see the actual code to reproduce, then. It's hard to say with the information here so far.
s
I wish i could send you the entire codebase haha
But yeah this makes complete sense… I’m re-building the dictionary every time and the Pulumi resource is going to return a potentially different resource on each subsequent execution… (unless we keep that dict cached, which is not a reliable thing)
e
If your building lists based on iterating dicts I'd suggest you sort the lists after building them and before using them
s
something very weird is going on… and it’s gonna take me a while to get to the bottom of it… but i’ll check back here when i have more info. Thanks for your help!
But for now…I’m just terribly confused with update diff’s which are actually not different
Copy code
~ aws:route53/resolverEndpoint:ResolverEndpoint: (update)
        [id=rslvr-out-92d9]
        [urn=urn:pulumi:ResolverEndpoint]
        [provider=urn:pulumi:cfa6a9e0043f]
      ~ ipAddresses: [
          ~ [0]: {
                  ~ subnetId: "subnet-1725d" => "subnet-1725d"
                }
          ~ [1]: {
                  ~ subnetId: "subnet-98b2b" => "subnet-98b2b"
                }
        ]
I’m quite sure now that this is a pulumi_aws bug which was introduced sometime between
Copy code
pulumi==3.40.1
pulumi-aws==5.16.0
and now…
e
Copy code
~ ipAddresses: [
          ~ [0]: {
                  ~ subnetId: "subnet-1725d" => "subnet-1725d"
                }
          ~ [1]: {
                  ~ subnetId: "subnet-98b2b" => "subnet-98b2b"
                }
        ]
Probably worth raising this as an aws issue at https://github.com/pulumi/pulumi-aws. Spurious diffs like that that are generally the provider doing odd things.
s
OK I’ll raise an issue and link it here
Looks like this has been known about for a while and still not fixed https://github.com/pulumi/pulumi-aws/issues/2294
Seems like another upstream terraform provider issue. pulumi-aws maintainers claim to have their hands tied. These seem to happen often. I guess this will drive us toward using pulumi native libraries (when they are mature enough)
e
aws-native and aws legacy should play nicely together, I've seen the providers team suggest a few times to switch over to aws-native for just one or two resources that users are having issues with in the terraform based provider
s
They do work together? I’d heard otherwise but i’ll give that a go.
aws-native does not support resolverEndpoints…
I will need to version lock to 
pulumi-aws==5.21.1
to workaround this until something changes
Open in Slack
PreviousNext
Powered by