https://pulumi.com logo
Join Slack
Powered by
How can I add taints to a given node in k8s throug...
# general
d
How can I add taints to a given node in k8s through pulumi ? Did not find any examples @gentle-diamond-70147
g
I don’t think it’s currently possible to edit resources created outside of Pulumi, but this work is in progress. What’s your use case?
d
I want to create a dedicated node pool which requires me to taint nodes and put tolerations on the pod among other things so that they are scheduled on the given node
👍 2
g
i
@damp-book-35965 not sure if useful, but here is a gist of a separately configured NodePool allowing change of node pools without deleting the cluster https://gist.github.com/rosskevin/9fe1b446d42bf5550bc44c246b13a909
e
@gorgeous-egg-16927 what do you mean by
resources created outside of Pulumi
?
This use case is entirely created within Pulumi, e.g. the k8s cluster, the k8s resources & content, etc. Just looking for how to add taints to the node(s)/node-groups
d
@important-leather-28796 Nice..Using this already but the limitation is it allows me to add only labels.. ideally it should also allow me to add taints as an option with the labels..
g
@early-musician-41645 In this case, I meant the compute instance for the node. With cloud provider k8s, Pulumi is managing the monolithic cluster, not the individual controller/node instances
e
So Pulumi can't handle the ASG node groupings in order to bulk-taint a set of nodes in k8s?
d
@gorgeous-egg-16927 labels are supported, it's kind of the same thing right..
e
e.g. https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ 
kubectl taint nodes node1 key=value:NoSchedule
g
It depends on what is actually creating the ASG or node pool. If Pulumi is provisioning a “cluster” from a cloud provider, then Pulumi can’t yet edit the resources the cloud provider provisions on your behalf. However, if you’re creating those things directly with Pulumi, you should be able to specify labels, taints, etc
e
I'm not sure what this has to do with provisioning of the cluster. We can already apply labels to nodes, why can't we do the same with taints? @white-balloon-205 FYI?
@microscopic-florist-22719 I can't remember if this is something we once discussed
g
Sure, if you can apply labels, you should be able to apply taints. In the gist that @important-leather-28796 linked, he’s explicitly creating a NodePool, and you can set whatever you want on that
Again, it depends on the cloud provider, and what they are provisioning on your behalf
e
So: 1) Create a node group 2) Apply labels to mark the nodes in the group 3) Apply taints to those same nodes ?
👍 1
d
It's not supported @gorgeous-egg-16927 only labels are supported actually: https://github.com/pulumi/pulumi-eks/blob/master/nodejs/eks/nodegroup.ts#L27
g
@damp-book-35965 Ah, sorry. I didn’t realize this was specific to the EKS package
d
@white-balloon-205 @microscopic-florist-22719 Can we get some traction on this ?
w
@damp-book-35965 and @early-musician-41645 - looking into this right now to see what the best option will be. Will get back to you in a little bit with a recommendation. /cc @breezy-hamburger-69619
b
@damp-book-35965 For context, can you walk us through your dedicated nodepool use-case? e.g. - is this nodePool constantly changing, or is it fixed? - is having the 
taints
prop the only thing missing to enable your use?
d
- NodePool can increase or decrease in size to take advantage of autoscaling. This is yet to be determined as it will depend on the load on a given cluster and if it is drastically affecting performance as well - taints is what I have uncovered and I mostly think it will be the only one as the rest of the stuff is being handled through a custom helm chart
The idea is that I do not want any pods without tolerations to be scheduled on this node
@breezy-hamburger-69619 does this help ?
b
Yes it does, thank you @damp-book-35965. We’re noodling on this as we speak, and trying to identify a path forward. Will update once we have something more concrete
d
Check this as well: If you'll want to go this route, it's pretty straightforward as well https://github.com/awslabs/amazon-eks-ami/issues/29
b
Thanks for the link - we were just discussing this flag and how to leverage it
👍 1
d
Guys would love an ETA / estimate timeline on this ..Thanks
b
There’s a PR in flight to handle this: https://github.com/pulumi/pulumi-eks/pull/63
🙂 1
d
Super
b
Update: the PR [1] for adding taints to NodeGroups has been merged. It is now available in 
@pulumi/eks
v0.17.2
[2]. The updated example [3] shows how to make use of the 
taints
prop. Let us know how this works for you all. 1 - https://github.com/pulumi/pulumi-eks/pull/63 2 - https://www.npmjs.com/package/@pulumi/eks/v/0.17.2 3 - https://github.com/pulumi/pulumi-eks/blob/master/nodejs/eks/examples/nodegroup/index.ts#L24-L28
👍 1
d
Let me give it a shot
Open in Slack
PreviousNext
Powered by