https://pulumi.com logo
#aws
Title
# aws
q

quick-garden-9190

03/23/2021, 6:30 PM
I’m in the process of implementing Managed Node Groups and I’m having trouble getting updates to work the way I’d like. With regard to the scaling config, I’m setting a min of 3 hosts and a max of 100. I’d like the cluster-autoscaler to do it’s thing and be able to scale way down if we don’t need the capcity. This becomes a problem considering
desiredSize
is a required value. This size will clobber whatever the cluster-autoscaler has landed on. I’ve tried using
ignoreChanges: ['scalingConfig.desiredSize']
but this option seems to be completely ignored. I’ve tried searching through the docs, it seems like this example is missing some key functionality, unless I’m not seeing something obvious.
g

gentle-diamond-70147

03/23/2021, 7:30 PM
Can you elaborate on the behavior you're seeing and the steps to see it? I was doing
ignoreChanges
for this same use case yesterday and it worked as expected for me.
q

quick-garden-9190

03/23/2021, 8:26 PM
Sure. For example, I’ll instantiate a node group with the following
scalingConfig
Copy code
scalingConfig: {
                maxSize: 100,
                minSize: 4,
                desiredSize: 5
            },
Then lets say some time passes and workloads are added to my cluster, the autoscaler will add nodes to satisfy the new requirements. If I were to run another update, even with
ignoreChanges: ['scalingConfig.desiredSize']
present, the node group will revert back to whatever is defined in
desiredSize
g

gentle-diamond-70147

03/23/2021, 8:30 PM
Can you share the output from
preview --diff
that shows this?
q

quick-garden-9190

03/23/2021, 8:38 PM
🤔 Is the purpose of
ignoreChanges
to ignore any drift that the resources themselves might have made over time? That is, if I were to manually modify the
desiredSize
even with
ignoreChanges
in place. Would pulumi update the value because I modified what is declared?
Does that question make sense?
Reason I ask, I’m seeing changes in behavior
Based on the docs I would expect any change I make to
desiredSize
to be ignored, which is not what I’m seeing
Maybe my tests weren’t properly fleshed out. Let’s say I deploy a node group with a desiredSize of 8 instances. Then I go delete 2 nodes from my cluster. If I run pulumi again it doesn’t attempt to add the 2 nodes back, regardless of whether or not I’m using
ignoreChanges
This seems to be behavior I haven’t accounted for previously.
Ok, putting that aside for a moment. I have more tests to conduct with desiredSize and the conditions for which it should update. Another scenario I’m trying to account for is what happens when Pulumi needs to delete-replace my node group. Which would happen if I were to change my
instanceTypes
for example. In that case a new node group would be spun up and I believe
ignoreChanges
would itself be ignored since it is instantiating the resource. Is it possible for me to lookup how many instances are currently running in an ASG so that I can pass that value as
desiredSize
?
g

gentle-diamond-70147

03/23/2021, 9:44 PM
hehe, let me catch up...
If values change outside of Pulumi, e.g. through the AWS Console, these will not be reflected in Pulumi's "view of the world" (state file) until you run
pulumi refresh
or run preview or up with the
--refresh
flag. This will update Pulumi's state. This would pick up any manual changes you make to
desiredSize
.
If you set
ignoreChanges: ["scalingConfig.desiredSize", "scalingConfig.maxSize"]
in your code, then on
preview
and
up
, Pulumi will ignore any changes you've made to your code or configuration that is driving these values.
This means if both things happen (you change a value in the console and have
ignoreChanges
set), your next
pre
or
up
should not change these values.
q

quick-garden-9190

03/23/2021, 9:50 PM
I see, thanks for the clarification. with
scalingConfig.desiredSize
declared as an ignored value, if I modify
desiredSize
I’m seeing Pulumi recognize the diff and want to update it
Do I need to also include
scalingConfig.maxSize
for some reason?
g

gentle-diamond-70147

03/23/2021, 9:53 PM
Can you share the
pulumi pre --diff
output? You should not need to ignore any other properties, unless you see Pulumi trying to update them too and you don't want that to occur.
I've just tested this with my own stack and
ignoreChanges
is working for all the properties I set.
q

quick-garden-9190

03/23/2021, 9:56 PM
Copy code
~ launchTemplate: {
              ~ version: "1" => "$Latest"
            }
          ~ scalingConfig : {
              ~ desiredSize: 7 => 9
            }
Copy code
{
            provider: awsProvider,
            ignoreChanges: ['scalingConfig.desiredSize']
}
g

gentle-diamond-70147

03/23/2021, 10:00 PM
Strange. Would you be able to hop a call tomorrow to troubleshoot this with you? We could talk through some of the other scenarios too.
I work for Pulumi in case that wasn't obvious. 😁
q

quick-garden-9190

03/23/2021, 10:03 PM
That would be great, I’m available at any point tomorrow. I’m in mountain time
g

gentle-diamond-70147

03/23/2021, 10:05 PM
Mind DM'ing me your email?
q

quick-garden-9190

03/23/2021, 10:13 PM
done, thanks
2 Views