https://pulumi.com logo
#aws
Title
# aws
q

quiet-wolf-18467

04/21/2020, 12:06 PM
This comment actually encourages what I was trying to use
w

white-balloon-205

04/21/2020, 12:16 PM
Which subnets tags in particular are you not seeing? https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html Note that these tags get added by EKS itself, not by the
@pukumi/eks
provider. If they are not getting added, I would expect there is something misconfigured in the networking setup being used. Can you share any more details on your setup?
q

quiet-wolf-18467

04/21/2020, 12:30 PM
When I remove
publicSubnetIds
and
privateSubnetIds
and use
subnetids
, I get the tag added
the
<http://kubernetes.io/cluster|kubernetes.io/cluster>
tag
Changing these values pretty much destroys the stack though
So I'm cleaning up again now to retry
w

white-balloon-205

04/21/2020, 12:52 PM
Do you have a repro or any more details on your code? I certainly expect this to work in general - it’s a fairly heavily used component and this is the “normal” way to configure it. The component itself doesn’t do much related to these subnets - that’s mostly handled by EKS itself. Would love to understand what configuration is leading to this for you.
q

quiet-wolf-18467

04/21/2020, 12:55 PM
I've just started spinning up again with
subnetIds
, then I'll branch off and update to use
privateSubnetIds
and
publicSubnetIds
If this reproduces, I'll open up the code
👍 1
b

breezy-hamburger-69619

04/21/2020, 4:08 PM
@quiet-wolf-18467 Please let me know if you continue to experience issues here, or with EKS in general. Happy to sit down and help get you moving forward.
q

quiet-wolf-18467

04/21/2020, 4:27 PM
Thanks @breezy-hamburger-69619. Currently fighting with the service account to iam role stuff, there's not a lot of documentation around this
b

breezy-hamburger-69619

04/21/2020, 4:29 PM
Thanks for the feedback. We have the following resources available: an example, and a

YT video

to help with per Pod IAM. Where are you getting stuck that I could be of help with?
q

quiet-wolf-18467

04/21/2020, 4:31 PM
That's not the service account integration though, right? I'm using aws.iam.OpenIdConnectProvider
b

breezy-hamburger-69619

04/21/2020, 4:32 PM
Yes they’re the same, but it appears you’re trying to manually set this up. We’ve taken the liberty and handled all of the set up by using a cluster option `createOidcProvider`: https://www.pulumi.com/docs/reference/pkg/nodejs/pulumi/eks/#ClusterOptions-createOidcProvider
The example linked above should serve you well for this usecase
q

quiet-wolf-18467

04/21/2020, 4:33 PM
The only example I found was
b

breezy-hamburger-69619

04/21/2020, 4:33 PM
q

quiet-wolf-18467

04/21/2020, 4:33 PM
What would this become, using the
createOidcProvider
helper?
Copy code
const exampleAssumeRolePolicy = pulumi.all([exampleOpenIdConnectProvider.url, exampleOpenIdConnectProvider.arn]).apply(([url, arn]) => aws.iam.getPolicyDocument({
    statements: [{
        actions: ["sts:AssumeRoleWithWebIdentity"],
        conditions: [{
            test: "StringEquals",
            values: ["system:serviceaccount:kube-system:aws-node"],
            variable: `${url.replace("https://", "")}:sub`,
        }],
        effect: "Allow",
        principals: [{
            identifiers: [arn],
            type: "Federated",
        }],
    }],
}));
Ahh
That example is perfect
🙂 1
I've lost nearly 2 days on this 😂
Thank you so much
b

breezy-hamburger-69619

04/21/2020, 4:34 PM
I’m sorry you’ve wasted time on this 😞 We aim to take that on where possible
Setting up the OIDC provider is pretty complex, so I don’t recommend you try to go the manual route as you’ve found out
q

quiet-wolf-18467

04/21/2020, 4:36 PM
The problem I had was I checked
pulumi/examples
and then when I couldn't find what I needed there, I found the example I linked above
and then I started searching for
OpenIdConnectProvider
in Pulumi repos, which led me to where I am today
b

breezy-hamburger-69619

04/21/2020, 4:37 PM
Makes total sense. We should update
pulumi/examples
to call out that EKS has other, extensive examples at the link posted above.
q

quiet-wolf-18467

04/21/2020, 4:37 PM
When I get this working, I am happy to submit an example of running ExternalDNS on EKS with the iam role mappings
b

breezy-hamburger-69619

04/21/2020, 4:38 PM
The reason for the split is that
pulumi/examples
historically covers many different scenarios across languages, providers, stacks etc. and the examples in
pulumi/eks
are centered around EKS paritcular features, options, and usage scenarios that we actively test in our CI. The examples and tests subdir in it can shine a light on what type of coverage we have.
An example from you would be great! Please cc me on it
q

quiet-wolf-18467

04/21/2020, 4:43 PM
@breezy-hamburger-69619 and just like that, the magic happens and external-dns is working
Thank you
b

breezy-hamburger-69619

04/21/2020, 4:43 PM
🎉
q

quiet-wolf-18467

04/21/2020, 8:58 PM
What should be adding the
<http://kubernetes.io/role/elb|kubernetes.io/role/elb>
tag to the subnets, is that Pulumi or AWS?
Right now, my subnets have
<http://kubernetes.io/cluster|kubernetes.io/cluster>
, but not
role/elb
@white-balloon-205 I came across this: https://github.com/pulumi/pulumi-eks/issues/196
Are you saying I should add this tag manually?
Is
pulumi-eks
not doing this?
w

white-balloon-205

04/21/2020, 9:08 PM
What should be adding the 
<http://kubernetes.io/role/elb|kubernetes.io/role/elb>
 tag to the subnets, is that Pulumi or AWS?
Neither 🙂. You will need to add these. Pulumi doesn't necessarily manage the desired state of these subnets, and AWS doesn't take care of adding them, so you will want whatever code does manage the desired state of the subnets to add these (if that's Pulumi, then you can add it to the tags at the definition site of the Subnets).
This is actually exactly why I was asking which subnet tags you were referring to - as the answer is different for different tags (as described in https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html).
q

quiet-wolf-18467

04/21/2020, 9:11 PM
Ah, OK. Gotcha! Thanks Luke
So it's probably best not to use
awsx
for the VPC creation and do it the longer way, or can I add subnet tags with
awsx
?
b

breezy-hamburger-69619

04/21/2020, 9:13 PM
If you’re creating the vpc with
awsx
and passing it’s subnets into the cluster, they will get auto-tagged by the EKS service in AWS. e.g.: https://github.com/pulumi/pulumi-eks/blob/98f4a7b1ac71222af268f4357a9dbc9990262d88/nodejs/eks/examples/tests/migrate-nodegroups/index.ts#L14-L39
q

quiet-wolf-18467

04/21/2020, 9:14 PM
I'd need to get the
publicSubnetIds
and then tag them with
<http://kubernetes.io/role/elb|kubernetes.io/role/elb>
(I think)
I don't think I can/should tag private subnets with that
b

breezy-hamburger-69619

04/21/2020, 9:15 PM
Right
AWS only auto-tags the vpc and subnets with ``kubernetes.io/cluster/_<cluster-name>_`
q

quiet-wolf-18467

04/21/2020, 9:16 PM
It feels like pulumi-eks should do this bit
Copy code
const cluster: eks.Cluster = new eks.Cluster(
    infrastructure.generateResourceName("eks"),
    {
      vpcId: config.vpc.id,
      publicSubnetIds: config.vpc.publicSubnetIds,
      privateSubnetIds: config.vpc.privateSubnetIds,
    });
When I specify private and public, couldn't there be a flag to add the elb tag?
Rather than something like
Copy code
config.vpc.getSubnets().then(subnet => {
    if subnet.isPublic() {
      
    }
  })
b

breezy-hamburger-69619

04/21/2020, 9:26 PM
p/eks has no way of knowing if the subnets provided are being created in the same stack, another stack, or if the VPC exists outside of the context of the pulumi program if bringing an existing one. As Luke pointed to, you can set the tags in the subnet definition if you’re creating the vpc using pulumi, but if the vpc and subnets exist outside of pulumi, we cannot modify the tags on a resource pulumi does not manage.
q

quiet-wolf-18467

04/21/2020, 9:28 PM
OK. That makes sense
Thank you
👍 2