Hi everyone, I just started to play with Pulumi an...
# general
Hi everyone, I just started to play with Pulumi and from the documentation/examples not clear to me how can I change a resource property after it was created? The resources are immutables, they have only read only properties, the "get" methods return read only results as well. I created an eks cluster based on the "aws-ts-eks" example, it worked fine, then I added a kubernetes load balancer, which is not working, because unable to find the public subnets. To make it work, I need to add a "kubernetes.io/cluster/${cluster_name}" tag to the public subnets. My problem is, that the subnets are created before the eks cluster, in fact the "eks.Cluster" constructor expects the subnet ids. I can use the "aws.ec2.getSubnet" method to get the subnet informations, but unable to modify it. I can create a new subnet with "aws.ec2.Subnet" based on the previous one, but I not sure how would pulumi handle that(overwrite the old one, create a new one), especially because the "aws.ec2.Subnet" constructor expects a name, which is not available on the "GetSubnetResult" interface. What is the correct way to do this?
You're entirely correct -- Pulumi resource objects in your program are immutable, meaning you need the necessary information to be available up front, before creating the resource that needs it. In your case, it sounds like we have a chicken-egg problem, because the subnet tags need the Kubernetes load balancer name ... which needs the subnets ... which needs ... From time to time, we discuss whether to support mutating existing resources. It's a little difficult to fit into our model -- and we worry it could let you easily create dependency cycles -- but it's possible we will support this someday. Unfortunately, absent that, I don't have a great answer for your specific example 😕 I'm sure @incalculable-sundown-82514, @white-balloon-205, or @stocky-spoon-28903 will have some thoughts.
Sometimes I've gotten around dependency cycles by creating the name as a hard string, I.e. You can probably pass in a name parameter to the LB, and also pass that same name in where it's expecting the LB name so that breaks the cycle by not needing a generated name.
not the LB name is the problem, but the cluster name, in terraform I can use a hard coded name, because it deletes the resources first, then create them, pulumi handles this the other way, I assume that's why it adds an auto generated string to the end of the resource name, but because of this, I'm unable to use hard coded strings, I have to know the assigned cluster name after it was created
if there's really no solution to this problem, that means the eks cluster is simply unusable an a deal breaker to us to may change from terraform to pulumi, I would be really sad, because pulumi is much more usable and easier to understand
I can imagine 1 workaround right now, use the aws api modifySubnetAttribute(https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/EC2.html#modifySubnetAttribute-property) function and add the tags after the cluster was created, and use "pulumi refresh" to save the changes to the state file(will the next "pulumi up" remove this?), I assume it possible with apply or with some other method, but it really hackish
We will pow wow tomorrow and see if we can come up with a solution. I'm sure we can make this work!
@cold-coat-35200 Am I right that https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html#vpc-private-subnet-tagging is the tag you want to make sure is set? If so, this tag doesn’t appear to need the name of the cluster, so you could in principle add it to the subnets at creation time. (This wouldn’t be possible with
new awsinfra.Network
currently, but you could create those resources manually or add a subnets tagging feature to that library). I believe the tag on the public subnets gets automatically added by EKS, are you not seeing that happen? If neither of those is an option, the workaround you describe should work - or even just a step that always applies the tag using aws-sdk in an
off the subnets to force the tag to be there as part of any deployment. That said - it would be nice to have more first class support for this. Lifecycle hooks would be one way, as would more general imperative actions.
@cold-coat-35200 @big-piano-35669 @white-balloon-205 wait… I’m confused. You can hard code a name, you just need to set the
attribute of the resource. Am I missing something here?
That should work in the immediate term.
In the longer term this is an instance of a problem we have already kind of had to deal with. It was, for example, one of the reasons we added
to the Kubernetes resources. To be blunt, I think the answer that you have to hard-code a name is not a great solution — a lot of the benefit of the Pulumi model is that you get autonaming.
I’m not sure how lifecycle hooks would fix this problem though. Seems like if we go down that route we should make 100% sure that EKS does not add the tag to the subnet itself, which is what I thought happened.
@white-balloon-205 not exacty, I need the tag from the "Subnet Tagging Requirement", which includes the cluster name, but you are right, it automatically added by EKS, I bumped into this issue, because I first created a vpc with private and public subnets and EKS added the tag only to the private subnets with nat gateway, then I removed the private subnets(set "usePrivateSubnets" to false), pulumi deleted them fine, so I left with the untagged public subnets and started to think how should I solve this without recreating the cluster btw, based on this comment, the other solution would be the "kubernetes.io/role/elb" tag for an external load balances, but it does not support sharing subnets between clusters https://github.com/kubernetes/kubernetes/issues/29298#issuecomment-356826381 out of curiosity, what happens, if EKS adds the tag, after that I do a "pulumi refresh", I assume the tag will be added to the state, then do a "pulumi up", will it remove the tag? I assume it will do, because it not exists in the code, just in the state file, right? @creamy-potato-29402 how can we add a hard coded name? in the cluster code eks is created like this: https://github.com/pulumi/eks/blob/master/nodejs/eks/cluster.ts#L188, so the name should be e.g.: "myCluster-eksCluster", but in reality it will be "myCluster-eksCluster-ca639p67", I don't know where the extra hash added
@cold-coat-35200 That seems like it should be a bug then? Normal Kubernetes resources do not get auto-named if you specify
, and my understanding is that AWS resources don’t get auto-named if you specify
? If that’s not an option that’s available to the EKS library, then I am guessing that was an oversight. @white-balloon-205, thoughts?
If I’m reading this right there are two separate discussions here: one about auto naming, and one about tagging resources that were not created by Pulumi in a manner which makes them easy to change (e.g. by a different module that doesn’t expose them, or by the provider automatically). For the tagging question, assuming you have the subnet ID, I think it’s possible to write an
dynamic resource using these APIs: https://docs.aws.amazon.com/cli/latest/reference/ec2/create-tags.html (et al for delete, describe), which would let you do what you need here. I’ll take a shot at it in a bit and reply on this thread.
If it works, I think the best way to go about making this easier is to upstream an
resource to Terraform and then pull it in via the bridge as a ‘first class’ thing that will work across all languages.
I’ve validated the approach, but hitting some issues with
and rejected promises that someone else can likely debug faster - I’ll post more when I have it figured out
Ok I got this figured out. I’ll post a gist shortly!
nice 🙂
To clarify, @cold-coat-35200: you have subnets created by something else that you want to tag as part of a pulumi program?
no, it was created by pulumi too, but I can't tag on creation time, because it depends on an other resource(needs the name of the eks cluster), but to create the eks cluster, I need the subnet ids, so as Joe said. it's a chicken-egg problem
OK - this will do the trick to solve this particular instance of the problem:
we already find other ways to solve this, but honestly, I'm curious about the code snippet you wrote, it may answer an other question I don't asked yet 🙂
There’s no reason in principle Terraform couldn’t have an
resource which then got pulled into Pulumi, implemented very similarly. For production code it likely wants a
@cold-coat-35200 just curious - how did you need up solving this? Would be interesting to understand in case it points to a more general solution or improvement we could make in libraries.
@stocky-spoon-28903 nice, thanks, just a few question, with this dynamic provider, the state of the resource will be saved to the state file, right? when I first add it, pulumi will call the create method, later if I do a destroy, it will call the delete, what if I change the "key" property in the "new AwsTagResource..." part? I assume I need to implement the update method too or if that missing pulumi will do a delete, then a create? And how this will look like during preview? I assume that will call the diff method and if that's missing won't show anything or just "computed"?
@white-balloon-205 nothing new, just as we talked earlier with a custom code like the one Jen created or use the "kubernetes.io/role/elb" tag in case no need to share the subnet between multiple cluster. I think currently you can't do too much in the libraries, because the two resource depends on each other. Generally mutating the properties would be a solution, but I can imagine that implementing an algorithm to check for cyclic dependencies(like Go does) would be hard