https://pulumi.com logo
#aws
Title
# aws
s

straight-fall-27130

03/13/2024, 12:10 PM
Hello all! Requesting help with EKS cluster. Details in thread
I am creating a cluster and a nodegroup using the eks.Cluster and eks.ManagedNodeGroup, I also have created vpc and all resources along with a security group, I attach the created security group to the cluster with cluster_security_group keyword, I require the default security group that gets created to be attached with a inbound rule so that pods can connect to internet, I can't seem to find a proper implementation or help for it, can you guys help me out? Thanks in advance!
p

purple-activity-54224

04/01/2024, 4:36 PM
Hey @straight-fall-27130 were you able to figure this out? I’m looking into creating a new EKS cluster, and I’m confused as well. From the docs it seems that the vpc_config args has an option to pass a cluster_security_group_id. The description states that this a SG created by Amazon for EKS, but from the cluster outputs or helper functions I don’t see a way to modify this SG. It seems perhaps I need to create the SG and pass it? But the description says that this is created on our behalf. I come from AWS CDK. There’s a method that allow this: cluster.cluster_security_group.add_ingress_rule(), for example, but I don’t see anything equivalent in Pulumi.
s

straight-fall-27130

04/03/2024, 11:16 AM
yes, @purple-activity-54224, I was experimenting with the pulumi_command package and ran the command command_string = `aws eks describe-cluster --name {clustername} --query cluster.resourcesVpcConfig.clusterSecurityGroupId --output text --region {region}`through this package. Then got the output from the command and used
var.stdout
to use this default security group id in the
aws.ec2.SecurityGroupRule
to add the ingress rule. Just add
opts=pulumi.ResourceOptions(depends_on=self.eks_cluster)
to the command package module so that it waits for the cluster to be created before it runs the command.
p

purple-activity-54224

04/03/2024, 12:18 PM
I did something similar using the get_cluster method, using the depends on resoruce option, but it did not work. Pulumi command seems like an option, but seems too hacky for me. I’m currently testing the eks package, where Im trying to create the cluster security group before hand and passing it as an argument to the resource creation. Hoping this will work.
s

straight-fall-27130

04/03/2024, 12:20 PM
it won’t work because the clust, on creation will create a security group that you have no control over, the workeer nodes will use that SG
get_cluster only works when the cluster is up, before the preview phase will try to get the cluster on preview and fail, it might work if you have the creation phase in a separate stack and the get_cluster in another stack
p

purple-activity-54224

04/03/2024, 12:27 PM
it kind of worked though. It’s true that it creates its own SG you don’t have any control over, but it allowed me to add a second SG wit the additional rules I need. Which is what I was looking for
so I guess that works for me. The default auto created SG will allow communications between the control plane and the worker nodes, and the second SG will allow the additional sockets I need
s

straight-fall-27130

04/03/2024, 12:28 PM
it does, but the nodes will get assigned the default sg, which was a problem for me as my pods didn’t have internet access
p

purple-activity-54224

04/03/2024, 12:28 PM
but I was only able to do it with the eks package, not the AWS Classic provider aws.eks.Cluster resource
s

straight-fall-27130

04/03/2024, 12:28 PM
maybe your use case is different
p

purple-activity-54224

04/03/2024, 12:29 PM
that’s a good point, let me check the SG attached to the worker nodes
b

billowy-garage-40145

04/16/2024, 4:08 PM
@purple-activity-54224 hey, can i trouble you for an example for this?
p

purple-activity-54224

04/16/2024, 6:37 PM
Hey @billowy-garage-40145 what exactly you need an example of?
b

billowy-garage-40145

04/16/2024, 6:44 PM
I want to create a EKS cluster with additional ingress rules to access node ports
p

purple-activity-54224

04/16/2024, 6:44 PM
the get_cluster did eventually proved to not work as @straight-fall-27130 predicted. Using Pulumi command did not work for me either, because I’m using private endpoints in my cluster, so the cluster won’t be reachable executing local commands. I suspect I will have the same type of problems using the Kubernetes provider to install add-ons, etc, because my cluster needs to have private endpoints. I was able to create the cluster and managed node groups, but I abandoned the effort there as the rest of the things needed (install add-ons, CNIs, etc..) would be to cumbersome to do with Pulumi. I did it using the AWS CDK, which has a concept of custom resources, already builtin to the EKS construct to deploy helm charts and other resources using a Lambda attached to the same VPC, and thus able to talk to the private cluster endpoint.
b

billowy-garage-40145

04/16/2024, 6:44 PM
the group gets created but not added to the cluster, i saw that you solved it.
I see, so basically this isn't possible with pulumi as far as you have tested as well?
p

purple-activity-54224

04/16/2024, 6:45 PM
I can give you the code I wrote to the create the barebones cluster and managed node group, using the AWS Classic provider. I had a hard time using the new EKS provider, and the documentation is awful.
b

billowy-garage-40145

04/16/2024, 6:46 PM
I appreciate it, thanks.
really unfortunate that that's the case with the eks provider tbh
p

purple-activity-54224

04/16/2024, 6:47 PM
it is possible. If you have public endpoints for your cluster you can reach it and deploy stuff. Otherwise, you could deploy the code to build the infra inside the same VPC . For me the headache and workarounds feel not worth the effort
b

billowy-garage-40145

04/16/2024, 6:49 PM
what i am trying to achieve is to set up a couple of ec2 instances that would reach the cluster in non web ports
do you suggest running them in the same vpc I use for eks?
p

purple-activity-54224

04/16/2024, 6:51 PM
you can. Give me a few minutes, I’ll make public what I had done so far in Github and share the linkl
b

billowy-garage-40145

04/16/2024, 6:51 PM
thank you so much, I really appreciate it, wasted a lot of time trying to get it working.
p

purple-activity-54224

04/16/2024, 6:52 PM
me too…which is to say I stopped working on it. I did it in CDK instead
b

billowy-garage-40145

04/16/2024, 6:52 PM
wish i could too, but i need my code to be cloud agnostic 😞
p

purple-activity-54224

04/16/2024, 6:54 PM
I mean if you are building AWS resources it will never be cloud agnostic. If you mean the actual IaC, then you are right, Pulumi can be used to create resources in multiple cloud
CDK has CDKTF…but that also tends to get a but tricky at times
you can see although I can't modify the default SG created for the cluster by AWS, you can create a SG and added to the cluster. Therefore, the cluster will the additional groups you add. If you need to allow EC2s to reach the cluster on whatever ports you need this is how you can enable them
b

billowy-garage-40145

04/17/2024, 4:08 AM
thank you so much!
p

purple-activity-54224

04/17/2024, 5:12 PM
Hope it helps