https://pulumi.com logo
#general
Title
# general
b

busy-umbrella-36067

01/05/2019, 4:45 PM
Does https://github.com/pulumi/pulumi-eks do rolling upgrades of node AMI’s?
w

white-balloon-205

01/05/2019, 10:36 PM
Yes. It uses an AutoScalingGroup to manage the Node pool, and configures that ASG to do rolling updates that replace a single instance at a time and fail the update if the new instances don’t boot cleanly.
c

chilly-photographer-60932

01/06/2019, 4:06 PM
How do we achieve draining policy similar to
KOPS
?
b

busy-umbrella-36067

01/07/2019, 3:45 PM
I assume it doesn’t drain the nodes, should be fine for stateless apps but anything persisting data might cause an issue
c

chilly-photographer-60932

01/07/2019, 3:57 PM
So if the master is upgraded by aws to a newer version and if I change the AMI on the pulumi code and do an
apply
it will do a rolling upgrade using ASG?
Also will it taint nodes with something like this
kubectl taint nodes node_name key=value:NoSchedule
@white-balloon-205 Will it also drain?
<https://kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/>
Or do we have to do it manually?
b

busy-umbrella-36067

01/07/2019, 4:18 PM
@chilly-photographer-60932 The masters are entirely manged by AWS. You never need to worry about upgrades. That happens automatically. For the nodes, you will need to do a pulumi update.
c

chilly-photographer-60932

01/07/2019, 4:19 PM
I knew the masters are managed by aws
I meant for the nodes
if i wasn’t clear
b

busy-umbrella-36067

01/07/2019, 5:49 PM
@chilly-photographer-60932 Taking a look at it, it doesn’t looks like it uses the
drain
api call
I feel like it should trigger a drain if the node gets told to shutdown, but I might be wrong
w

white-balloon-205

01/07/2019, 5:56 PM
Sorry for the delay getting back on this. You are right that
drain
is not currently triggered during ASG turnover. This is not supported directly in EKS, but it is something that could be done on top - I'll open an issue to see if we can do something automatic in the
@pulumi/eks
package to support this.
cc also @microscopic-florist-22719
b

busy-umbrella-36067

01/07/2019, 5:57 PM
@white-balloon-205 for ECS we use a lamda function with lifecycle hooks that triggers the drain before kicking the node. Not sure how overkill that would be
c

chilly-photographer-60932

01/08/2019, 1:14 AM
@white-balloon-205 Is there a github issue for this?
w

white-balloon-205

01/08/2019, 1:17 AM
@busy-umbrella-36067 Yes - that's what would be needed here - either built-in to
@pulumi/eks
or built on top. Something along the lines of https://aws.amazon.com/blogs/compute/how-to-automate-container-instance-draining-in-amazon-ecs/, but for EKS.
@chilly-photographer-60932 Opened https://github.com/pulumi/pulumi-eks/issues/40 about this. Note that it should techincally be possible to build something like this yourself on top of existing
@pulumi/eks
, though we may need to expose the ASG as a property on the
Cluster
(easy PR if you want to go this direction).
c

chilly-photographer-60932

01/08/2019, 3:25 AM
Thanks @white-balloon-205.