https://pulumi.com logo
Join Slack
Powered by
I used Pulumi to create an EKS cluster in another ...
# kubernetes
s
I used Pulumi to create an EKS cluster in another AWS account using an assumed role. But 
pulumi up
failed when trying to create kubernetes objects afterward, as part of the same stack. How can I access the EKS cluster I just created?
b
@steep-toddler-94095 did you use the 
ProfileArn
flag?
s
I used 
roleArn
in my Pulumi stack config file
b
Copy code
ProviderCredentialOpts: eks.KubeconfigOptionsArgs{
			ProfileName: pulumi.Sprintf("ssa-%s", awsCfg.Require("region")),
		},
I have this in my cluster creation call
Basically if your default AWS profile is not allowed to access this other cluster (which it is unlikely to be), then you will need to tell it which AWS profile to use
s
My default AWS profile has admin access to the other AWS account through an assumed role
I read that the iam user that created the EKS cluster has admin access to it, but in this case I am not sure how it works because my user does not exist within the aws account I created the cluster in.
b
I’m not sure that’s sufficient, it would need to be in the aws-auth ConfigMap for the new cluster, either your assumed role ARN (and explicitly that role ARN) or your user ID)
I asked about this here
It’s a huge shortcoming of both EKS and also how the AWS CLI works with it
Giant pain
🤕 1
s
just to make sure i understand correctly: it's impossible to access a kubernetes cluster with a user from another AWS account immediately after creation, even if that user is able to assume the role that created the EKS cluster
b
I believe that is correct.
s
Thanks Itay!
b
I am not 100% sure
But I will say you’re likely somewhere down this painful path
Actually sorry
Let me back up
I think that you could technically access it. The issue comes from how the kubeconfig works. If you look at the kubeconfig, it will have something like this call in it:
Copy code
aws eks get-token
and that invocation is very limited, it can basically either use a profile or a role ARN
In your case, I think a role ARN is what you need
So you can specify 
RoleArn
when you create the cluster
and then the provider will have that in there.
(note this is different than what you specify in your pulumi stack config - this is different than what Pulumi will use for your normal AWS calls)
s
thanks, so i was able to get connected to the cluster locally, though pulumi up still fails when modifying cluster resources. strange.
b
Define locally
Locally your 
kubeconfig
probably looks different than the one Pulumi is generating
s
oops, i mean the one in my 
~/.kube/config
. I manually edited the 
aws-auth
configmap to have my IAM ARN (from a different account) and it appears to be working now. I guess I don't totally understand what you mean by
So you can specify 
RoleArn
 when you create the cluster
where would this be specified? in the pulumi stack config or the eks cluster parameters?
b
EKS cluster parameters
Which is crazy, I know, because the EKS cluster doesn’t care - this is more to tell Pulumi how to create the local kubeconfig for the generated provider for that cluster.
s
can you show me you example code please? I can't find the 
roleArn
input parameter in https://www.pulumi.com/docs/reference/pkg/eks/cluster/
b
It’s inside providercredentialopts
s
ahh thanks. i will try this. thanks for your time today!!
b
See the snippet i pasted originally - I used profile name but rolearn is the other option
Let me know if it works - I could totally believe I’m wrong
And also no worries - you’ve answered a bunch of my questions 😀
😄 1
s
yeah i will let you know! might be tomorrow instead
it worked! partypus
b
Huzzah
🎉 1
p 1
😄 1
b
@bored-table-20691 thanks for helping out here, this comes up so often we probably need to create a post on it, as you said earlier, it's confusion from the kubeconfig creation 😞
b
Yeah. I am hopeful the EKS team is on a path to fix it, as they’ve said they will create an EKS level API to control what users and roles can access the cluster, so it should be configurable at that level once it’s there. Timeline is 🤷‍♀️ of course.
4 Views
Open in Slack
PreviousNext
Powered by