https://pulumi.com logo
Powered by
Title
s

strong-helmet-83704

01/27/2023, 2:41 AM
Is there any guarantee to how my pulumi objects get mapped to values of a dictionary during creation? This seems to be consistent during development but when purging python cache i notice already created resource objects are re-ordered in the dictionary randomly sometimes causing resource changes. Using Python > 3.9
e

echoing-dinner-19531

01/27/2023, 4:46 AM
dictionary order is not guaranteed to be consistent
but this is only a problem if your changing from dictionaries (unordered) to lists (ordered) in your program
c

creamy-agency-52831

01/27/2023, 1:37 PM
Dictionary ordering in python is now guaranteed, though it's not a great idea to rely on the order of those keys (even though it's been a language feature since 3.7). I'd love to see what your program looks like to determine where this reordering is occurring.
s

strong-helmet-83704

01/27/2023, 7:04 PM
This is something I didn’t noticed until i ran the stack on another system… venv and pycache were purged / rebuilt. Since then i’ve been able to reproduce it after purging these dirs… which makes me think the order is being maintained in the cache somehow… I’m building a dictionary in a loop with values which are Pulumi resources…
@calm-hairdresser-54232
c

creamy-agency-52831

01/27/2023, 7:36 PM
Aha, that makes sense - I would not expect the resources themselves to come back in a deterministic order.
s

strong-helmet-83704

01/27/2023, 7:42 PM
What is the right way to do this?
c

creamy-agency-52831

01/27/2023, 7:43 PM
I'd have to see an example of the code which is causing problems to know for sure, but I wouldn't depend on the order of asynchronous cloud operations when constructing objects / dictionaries.
s

strong-helmet-83704

01/27/2023, 7:46 PM
How would we build a dictionary with x resources assigned as values? And when re-running, get the same resources re-assigned to the same place in the dictionary?
basically i need to create x resources and store those objects someplace to operate on them later…
c

creamy-agency-52831

01/27/2023, 7:47 PM
Are you operating on them within the same stack, or in a different one?
s

strong-helmet-83704

01/27/2023, 7:47 PM
same stack
c

creamy-agency-52831

01/27/2023, 7:49 PM
It depends on the operations and the resources, but could you not just operate on the resources themselves rather than storing them in a dictionary? Can you share a code snippet to demonstrate what you're trying to accomplish?
s

strong-helmet-83704

01/27/2023, 7:50 PM
for subnet in environment['subnets']:
        az = environment['subnets'][subnet]['az']
        if environment['subnets'][subnet]['type'] == "public":

            public_subnets[az] = myclass.Subnet(
                az_id=az,
                type="Public",
                vpc_id=vpc.id,
                cidr_block=subnet,
            )
then later on, i’ll use the public_subnets[“az-code”] to deploy other resources
c

creamy-agency-52831

01/27/2023, 7:52 PM
and 
environment['subnets']
is created dynamically somewhere above in a loop?
s

strong-helmet-83704

01/27/2023, 7:52 PM
Yes that is statically defined and is always the same
c

creamy-agency-52831

01/27/2023, 7:59 PM
I'm curious what your 
pulumi preview
output looks like in this case - could you post that as well?
s

strong-helmet-83704

01/27/2023, 8:12 PM
It’s different every time… but nothing happens while cache is maintained. Later in the program i’m creating resolver endpoints and something like this occurs 
~   └─ aws:route53:ResolverEndpoint  thing_ResolverEndpoint  update     [diff: ~ipAddresses]
c

creamy-agency-52831

01/27/2023, 8:13 PM
Ah, and the order of the ipAddresses is triggering this diff?
s

strong-helmet-83704

01/27/2023, 8:13 PM
yes… it changes sometimes because i’m selecting 2x keys in the dict for each ip
c

creamy-agency-52831

01/27/2023, 8:14 PM
I think I would need to see the actual code to reproduce, then. It's hard to say with the information here so far.
s

strong-helmet-83704

01/27/2023, 8:15 PM
I wish i could send you the entire codebase haha
But yeah this makes complete sense… I’m re-building the dictionary every time and the Pulumi resource is going to return a potentially different resource on each subsequent execution… (unless we keep that dict cached, which is not a reliable thing)
e

echoing-dinner-19531

01/27/2023, 8:55 PM
If your building lists based on iterating dicts I'd suggest you sort the lists after building them and before using them
s

strong-helmet-83704

01/27/2023, 11:16 PM
something very weird is going on… and it’s gonna take me a while to get to the bottom of it… but i’ll check back here when i have more info. Thanks for your help!
But for now…I’m just terribly confused with update diff’s which are actually not different 
~ aws:route53/resolverEndpoint:ResolverEndpoint: (update)
        [id=rslvr-out-92d9]
        [urn=urn:pulumi:ResolverEndpoint]
        [provider=urn:pulumi:cfa6a9e0043f]
      ~ ipAddresses: [
          ~ [0]: {
                  ~ subnetId: "subnet-1725d" => "subnet-1725d"
                }
          ~ [1]: {
                  ~ subnetId: "subnet-98b2b" => "subnet-98b2b"
                }
        ]
I’m quite sure now that this is a pulumi_aws bug which was introduced sometime between 
pulumi==3.40.1
pulumi-aws==5.16.0
and now…
e

echoing-dinner-19531

01/28/2023, 10:02 AM
~ ipAddresses: [
          ~ [0]: {
                  ~ subnetId: "subnet-1725d" => "subnet-1725d"
                }
          ~ [1]: {
                  ~ subnetId: "subnet-98b2b" => "subnet-98b2b"
                }
        ]
Probably worth raising this as an aws issue at https://github.com/pulumi/pulumi-aws. Spurious diffs like that that are generally the provider doing odd things.
s

strong-helmet-83704

01/30/2023, 7:31 PM
OK I’ll raise an issue and link it here
Looks like this has been known about for a while and still not fixed https://github.com/pulumi/pulumi-aws/issues/2294
Seems like another upstream terraform provider issue. pulumi-aws maintainers claim to have their hands tied. These seem to happen often. I guess this will drive us toward using pulumi native libraries (when they are mature enough)
e

echoing-dinner-19531

01/31/2023, 9:38 PM
aws-native and aws legacy should play nicely together, I've seen the providers team suggest a few times to switch over to aws-native for just one or two resources that users are having issues with in the terraform based provider
s

strong-helmet-83704

01/31/2023, 9:42 PM
They do work together? I’d heard otherwise but i’ll give that a go.
aws-native does not support resolverEndpoints…
I will need to version lock to 
pulumi-aws==5.21.1
to workaround this until something changes
#pythonJoin Slack
Powered by