https://pulumi.com logo
#general
Title
# general
d

damp-book-35965

03/18/2019, 7:39 PM
How can I add taints to a given node in k8s through pulumi ? Did not find any examples @gentle-diamond-70147
g

gorgeous-egg-16927

03/18/2019, 7:58 PM
I don’t think it’s currently possible to edit resources created outside of Pulumi, but this work is in progress. What’s your use case?
d

damp-book-35965

03/18/2019, 8:01 PM
I want to create a dedicated node pool which requires me to taint nodes and put tolerations on the pod among other things so that they are scheduled on the given node
👍 2
g

gorgeous-egg-16927

03/18/2019, 8:14 PM
i

important-leather-28796

03/18/2019, 8:18 PM
@damp-book-35965 not sure if useful, but here is a gist of a separately configured NodePool allowing change of node pools without deleting the cluster https://gist.github.com/rosskevin/9fe1b446d42bf5550bc44c246b13a909
e

early-musician-41645

03/18/2019, 8:19 PM
@gorgeous-egg-16927 what do you mean by
resources created outside of Pulumi
?
This use case is entirely created within Pulumi, e.g. the k8s cluster, the k8s resources & content, etc. Just looking for how to add taints to the node(s)/node-groups
d

damp-book-35965

03/18/2019, 8:21 PM
@important-leather-28796 Nice..Using this already but the limitation is it allows me to add only labels.. ideally it should also allow me to add taints as an option with the labels..
g

gorgeous-egg-16927

03/18/2019, 8:22 PM
@early-musician-41645 In this case, I meant the compute instance for the node. With cloud provider k8s, Pulumi is managing the monolithic cluster, not the individual controller/node instances
e

early-musician-41645

03/18/2019, 8:23 PM
So Pulumi can't handle the ASG node groupings in order to bulk-taint a set of nodes in k8s?
d

damp-book-35965

03/18/2019, 8:24 PM
@gorgeous-egg-16927 labels are supported, it's kind of the same thing right..
e

early-musician-41645

03/18/2019, 8:25 PM
e.g. https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
kubectl taint nodes node1 key=value:NoSchedule
g

gorgeous-egg-16927

03/18/2019, 8:26 PM
It depends on what is actually creating the ASG or node pool. If Pulumi is provisioning a “cluster” from a cloud provider, then Pulumi can’t yet edit the resources the cloud provider provisions on your behalf. However, if you’re creating those things directly with Pulumi, you should be able to specify labels, taints, etc
e

early-musician-41645

03/18/2019, 8:27 PM
I'm not sure what this has to do with provisioning of the cluster. We can already apply labels to nodes, why can't we do the same with taints? @white-balloon-205 FYI?
@microscopic-florist-22719 I can't remember if this is something we once discussed
g

gorgeous-egg-16927

03/18/2019, 8:29 PM
Sure, if you can apply labels, you should be able to apply taints. In the gist that @important-leather-28796 linked, he’s explicitly creating a NodePool, and you can set whatever you want on that
Again, it depends on the cloud provider, and what they are provisioning on your behalf
e

early-musician-41645

03/18/2019, 8:30 PM
So: 1) Create a node group 2) Apply labels to mark the nodes in the group 3) Apply taints to those same nodes ?
👍 1
d

damp-book-35965

03/18/2019, 8:31 PM
It's not supported @gorgeous-egg-16927 only labels are supported actually: https://github.com/pulumi/pulumi-eks/blob/master/nodejs/eks/nodegroup.ts#L27
g

gorgeous-egg-16927

03/18/2019, 8:32 PM
@damp-book-35965 Ah, sorry. I didn’t realize this was specific to the EKS package
d

damp-book-35965

03/18/2019, 8:51 PM
@white-balloon-205 @microscopic-florist-22719 Can we get some traction on this ?
w

white-balloon-205

03/18/2019, 9:09 PM
@damp-book-35965 and @early-musician-41645 - looking into this right now to see what the best option will be. Will get back to you in a little bit with a recommendation. /cc @breezy-hamburger-69619
b

breezy-hamburger-69619

03/18/2019, 9:14 PM
@damp-book-35965 For context, can you walk us through your dedicated nodepool use-case? e.g. - is this nodePool constantly changing, or is it fixed? - is having the
taints
prop the only thing missing to enable your use?
d

damp-book-35965

03/18/2019, 9:18 PM
- NodePool can increase or decrease in size to take advantage of autoscaling. This is yet to be determined as it will depend on the load on a given cluster and if it is drastically affecting performance as well - taints is what I have uncovered and I mostly think it will be the only one as the rest of the stuff is being handled through a custom helm chart
The idea is that I do not want any pods without tolerations to be scheduled on this node
@breezy-hamburger-69619 does this help ?
b

breezy-hamburger-69619

03/18/2019, 9:34 PM
Yes it does, thank you @damp-book-35965. We’re noodling on this as we speak, and trying to identify a path forward. Will update once we have something more concrete
d

damp-book-35965

03/18/2019, 9:40 PM
Check this as well: If you'll want to go this route, it's pretty straightforward as well https://github.com/awslabs/amazon-eks-ami/issues/29
b

breezy-hamburger-69619

03/18/2019, 9:41 PM
Thanks for the link - we were just discussing this flag and how to leverage it
👍 1
d

damp-book-35965

03/18/2019, 10:51 PM
Guys would love an ETA / estimate timeline on this ..Thanks
b

breezy-hamburger-69619

03/18/2019, 11:33 PM
There’s a PR in flight to handle this: https://github.com/pulumi/pulumi-eks/pull/63
🙂 1
d

damp-book-35965

03/18/2019, 11:40 PM
Super
b

breezy-hamburger-69619

03/20/2019, 1:49 AM
Update: the PR [1] for adding taints to NodeGroups has been merged. It is now available in
@pulumi/eks
v0.17.2
[2]. The updated example [3] shows how to make use of the
taints
prop. Let us know how this works for you all. 1 - https://github.com/pulumi/pulumi-eks/pull/63 2 - https://www.npmjs.com/package/@pulumi/eks/v/0.17.2 3 - https://github.com/pulumi/pulumi-eks/blob/master/nodejs/eks/examples/nodegroup/index.ts#L24-L28
👍 1
d

damp-book-35965

03/20/2019, 7:31 PM
Let me give it a shot