future-refrigerator-88869
05/09/2022, 4:43 PMaws-auth
configmap (or any configmap for that matter) after the cluster has been created ?
In my case, I have the eks cluster created with some roleMappings
but some workloads that get deployed might want to create a specific least-privileged user. I'd need to edit the existing ConfigMap
for aws-auth
after cluster creation but I can't figure a way to do it. Any help is appreciated 🙌roleMappings
. However, the problem is that the IAM policy for assumeRole
is relying on the arn and url of the oidc provider created for the cluster which don't exist yet since the cluster is not created.
The policy is about allowing a pod to run with a specific service account:
aws.iam.getPolicyDocument(
{
version: "2012-10-17",
statements: [
{
effect: "Allow",
principals: [
{
type: "Federated",
identifiers: [arn],
},
],
actions: ["sts:AssumeRoleWithWebIdentity"],
conditions: [
{
test: "StringEquals",
variable: `${url}:sub`,
values: [`${serviceAccountFullName}`],
},
],
},
],
},
{ parent: this }
);
Which is added to a role, which in turn is supposed to be added to roleMappings
How are we supposed to handle these cases? Create the OIDC provider ourselves and do the linking manually? Raw edit the string
for aws-auth
configmap and hope for the best ? Any advice ? 🙂polite-napkin-90098
05/10/2022, 6:29 PM// Create a security group for the ENIs which allows communication between the nodes and the control plane
const sg = new aws.ec2.SecurityGroup("EKS", {
description: "Group for the EKS cluster.",
vpcId: vpcid,
ingress: [
{
description: "ssh in from the ssh host",
fromPort: 22,
toPort: 22,
protocol: "tcp",
securityGroups: [ sshg.ids[0] ],
},
{
description: "https in from the ssh host",
fromPort: 443,
toPort: 443,
protocol: "tcp",
securityGroups: [ sshg.ids[0] ],
},
],
egress: [
{
description: "allow https out to anywhere",
fromPort: 443,
toPort: 443,
protocol: "tcp",
cidrBlocks: [ "0.0.0.0/0" ],
ipv6CidrBlocks: ["::/0"],
},
],
tags: {
name: `${nam}-EKS`,
},
});
// Create an EKS cluster
const cluster = new eks.Cluster(nam, {
vpcId: vpcid,
privateSubnetIds: [ privsub[0], privsub[1], privsub[2] ],
publicSubnetIds: [ pubsub[0], pubsub[1], pubsub[2] ],
clusterSecurityGroup: sg,
endpointPrivateAccess: true,
endpointPublicAccess: false,
nodeAssociatePublicIpAddress: false,
instanceType: "t3a.medium",
maxSize: 10,
minSize: 2,
serviceRole: serviceRole,
createOidcProvider: true,
roleMappings: [
{
groups: [ "system:masters" ],
roleArn: adminVMrole.arn,
username: "admin",
},
],
});
// and make a clusterrole to give the AdminVM role permissons to admin it.
const adminVM = new k8s.rbac.v1.ClusterRole("AdminVM", {
metadata: {
name: "AdminVM",
},
rules: [
{
verbs: ["*"],
resources: ["*"],
apiGroups: ["*"],
},
],
}, { provider: cluster.provider });
// and add a ClusterRoleBinding to tie it in to the cluster
const adminVMRB = new k8s.rbac.v1.ClusterRoleBinding("AdminVMRB", {
metadata: {
name: "cluster-admin-binding",
},
roleRef: {
apiGroup: "<http://rbac.authorization.k8s.io|rbac.authorization.k8s.io>",
kind: "ClusterRole",
name: "AdminVM",
},
subjects: [{
kind: "User",
name: "admin",
}],
}, { provider: cluster.provider });
Then to add the managed role with the AssumeRoleWebIdentity, I did this:
const oidcUrl = cluster.core.oidcProvider?.url;
const oidcArn = cluster.core.oidcProvider?.arn;
const saAssumeRolePolicy = pulumi
.all([oidcUrl, oidcArn])
.apply(([url, arn]) =>
aws.iam.getPolicyDocument({
statements: [
{
actions: ['sts:AssumeRoleWithWebIdentity'],
conditions: [
{
test: 'StringEquals',
values: [`system:serviceaccount:default:efs-csi-controller-sa`],
variable: `${url.replace('https://', '')}:sub`,
},
{
test: 'StringEquals',
values: [`<http://sts.amazonaws.com|sts.amazonaws.com>`],
variable: `${url.replace('https://', '')}:aud`,
},
],
effect: 'Allow',
principals: [{identifiers: [arn], type: 'Federated'}],
},
],
})
);
// Creare a manaagedPolicy to give access to sts:AssumeRoleWithWebIdentity
const webIdentityPolicy = new aws.iam.Policy(`${nam}-WebIdentityPolicy`, {policy: JSON.stringify({
Version: "2012-10-17",
Statement: [{
Action: ["sts:AssumeRoleWithWebIdentity"],
//should probably limit this to Test roles
Resource: "*",
Effect: "Allow",
}],
})});
const efsRole = new aws.iam.Role(`${nam}-efsRole`, {
assumeRolePolicy: saAssumeRolePolicy.json,
managedPolicyArns: [
efsPolicy.arn,
webIdentityPolicy.arn,
],
}, { dependsOn: [cluster]});
Which I cribbed from here:
https://github.com/jaxxstorm/pulumi-examples/blob/6207179e2c4a6edbf60628edcc8a886c360f72ab/typescript/aws/eks-platform/external-dns/index.ts#L17-L36
Thanks to @billowy-army-68599 for that 😄future-refrigerator-88869
05/10/2022, 8:40 PMefsRole
some read access to some resources in k8s but in order for that to happen I find myself in need to add it to the aws-auth configmap which i cannot seem to find a way to edit after cluster creation. If you have an idea, i'm all ears, or if there's a better way, please let me know 😄polite-napkin-90098
05/11/2022, 1:51 PMfuture-refrigerator-88869
05/11/2022, 1:53 PMpolite-napkin-90098
05/11/2022, 2:01 PMfuture-refrigerator-88869
05/11/2022, 2:05 PMConfigMap.get('default/aws-auth')
and edit the yaml string. I actually didn't think if pulumi would just do the edit automatically. I`ll give it a try and report back a bit later