bumpy-motorcycle-53357
07/16/2020, 2:12 PMvar configMap = new ConfigMap("aws-auth", new ConfigMapArgs()
{
Metadata = new ObjectMetaArgs()
{
Namespace = "kube-system",
Name = "aws-auth"
},
Data = new InputMap<string>()
{
["mapRoles"] = workerNodeRoleArn.Apply(arn =>
new[] {
//recreate default aws node role map
new
{
groups = new[]
{
"system:bootstrappers",
"system:nodes"
},
rolearn = arn,
username = "system:node:{{EC2PrivateDNSName}}"
}
}.ToYaml()
)
}
});
The issue is that as is, Pulumi complains that the resource already exists. I don't want to import it (EKS Crosswalk doesn't appear to import it either) as I want this to work without manual intervention on brand new EKS clusters. How does Crosswalk do it, and how can I get Pulumi to take control of this ConfigMap without importing it?
At this point, it would be fine if I could just delete that ConfigMap and re-create it, but don't think Pulumi supports that either.billowy-army-68599
07/16/2020, 2:19 PM$KUBECONFIG
?bumpy-motorcycle-53357
07/16/2020, 2:21 PMbillowy-army-68599
07/16/2020, 2:23 PMI want this to work without manual intervention on brand new EKS clustersit should work without issue, I guess I'm trying to determine what created the configmap in this case. if you're happy to delete it, can you just delete it manually and go from there?
bumpy-motorcycle-53357
07/16/2020, 2:24 PMbillowy-army-68599
07/16/2020, 2:25 PMbumpy-motorcycle-53357
07/16/2020, 2:25 PMbillowy-army-68599
07/16/2020, 2:31 PMbumpy-motorcycle-53357
07/16/2020, 2:33 PMbillowy-army-68599
07/16/2020, 2:38 PMworried-city-86458
07/16/2020, 10:09 PMbumpy-motorcycle-53357
07/17/2020, 6:15 PMprivate ConfigMap AwsAuthConfigMap(Output<string> workerNodeRoleArn)
{
static object CreateUserMap(GetUserResult user, params string[] groups)
{
return new
{
userarn = user.Arn,
username = user.UserName,
groups
};
}
var adminUsernames = _config.GetObject<List<string>>("eks-admins") ?? Enumerable.Empty<string>();
var devUsernames = _config.GetObject<List<string>>("eks-devs") ?? Enumerable.Empty<string>(); ;
var admins = adminUsernames.Select(x => GetUser.InvokeAsync(new GetUserArgs() { UserName = x }).Result).ToArray();
var devs = devUsernames.Select(x => GetUser.InvokeAsync(new GetUserArgs() { UserName = x }).Result).ToArray();
var mapUsers = admins.Select(x => CreateUserMap(x, "system:masters"))
.Concat(devs.Select(x => CreateUserMap(x, "system:basic-user")))
.ToArray()
.ToYaml();
var configMap = new ConfigMap("aws-auth", new ConfigMapArgs()
{
Metadata = new ObjectMetaArgs()
{
Namespace = "kube-system",
Name = "aws-auth"
},
Data = new InputMap<string>()
{
["mapRoles"] = workerNodeRoleArn.Apply(arn =>
new[] {
//recreate default aws node role map
new
{
groups = new[]
{
"system:bootstrappers",
"system:nodes"
},
rolearn = arn,
username = "system:node:{{EC2PrivateDNSName}}"
}
}.ToYaml()
),
["mapUsers"] = mapUsers
}
});
return configMap;
}
billowy-army-68599
07/17/2020, 6:19 PMworried-city-86458
07/17/2020, 10:06 PMDiagnostics:
kubernetes:core:ConfigMap (alpha-eks-auth):
error: configured Kubernetes cluster is unreachable: unable to load schema information from the API server: Get "https://<endpoint>.<http://gr7.us-west-2.eks.amazonaws.com/openapi/v2?timeout=32s|gr7.us-west-2.eks.amazonaws.com/openapi/v2?timeout=32s>": dial tcp <ip>:443: i/o timeout
I've tried configuring a create timeout to no avail:
var k8sProvider = new Provider($"{name}-k8s", new ProviderArgs { KubeConfig = KubeConfig });
// aws-auth config map
var authConfigMap = new ConfigMap($"{name}-auth", new ConfigMapArgs
{
Metadata = new ObjectMetaArgs
{
Namespace = "kube-system",
Name = "aws-auth"
},
Data =
{
["mapRoles"] = IamHelpers.GetRoleMappings(nodeRole, awsAccountId),
["mapUsers"] = IamHelpers.GetUserMappings()
}
},
new CustomResourceOptions { CustomTimeouts = new CustomTimeouts { Create = TimeSpan.FromMinutes(1) }, Provider = k8sProvider });
timeout=32s
query parameter above?bumpy-motorcycle-53357
07/21/2020, 1:57 PMgorgeous-egg-16927
07/21/2020, 3:56 PMworried-city-86458
07/22/2020, 2:36 AMdate && pulumi up --skip-preview --suppress-outputs
date && mkdir -p /home/user/.kube && pulumi stack output --show-secrets KubeConfig > /home/user/.kube/alpha
while true; do date && kubectl --kubeconfig=/home/user/.kube/alpha version; done
With the following output:user@e6d07659f7d8:/workspaces/gemini-pulumi/eks-infra$ date && pulumi up --skip-preview --suppress-outputs
Wed Jul 22 02:16:41 UTC 2020
Updating (pharos/alpha):
Type Name Status
pulumi:pulumi:Stack eks-infra-alpha
+ ββ aws:iam:Role alpha-eks-cluster-role created
+ β ββ aws:iam:RolePolicyAttachment alpha-eks-cluster-rp-cluster created
+ ββ aws:iam:Role alpha-eks-node-role created
+ β ββ aws:iam:RolePolicyAttachment alpha-eks-node-rp-cni created
+ β ββ aws:iam:RolePolicyAttachment alpha-eks-node-rp-node created
+ β ββ aws:iam:RolePolicy alpha-eks-node-rp-alb created
+ β ββ aws:iam:RolePolicyAttachment alpha-eks-node-rp-ecr created
+ ββ aws:eks:Cluster alpha-eks-cluster created
Resources:
+ 8 created
1 unchanged
Duration: 10m20s
Permalink: <https://app.pulumi.com/pharos/eks-infra/alpha/updates/79>
user@e6d07659f7d8:/workspaces/gemini-pulumi/eks-infra$ date && mkdir -p /home/user/.kube && pulumi stack output --show-secrets KubeConfig > /home/user/.kube/alpha
Wed Jul 22 02:27:05 UTC 2020
user@e6d07659f7d8:/workspaces/gemini-pulumi/eks-infra$ while true; do date && kubectl --kubeconfig=/home/user/.kube/alpha version; done
Wed Jul 22 02:27:07 UTC 2020
Client Version: <http://version.Info|version.Info>{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-17T11:41:22Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Unable to connect to the server: dial tcp 35.155.125.57:443: i/o timeout
Wed Jul 22 02:27:37 UTC 2020
Client Version: <http://version.Info|version.Info>{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-17T11:41:22Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Unable to connect to the server: dial tcp 35.155.125.57:443: i/o timeout
Wed Jul 22 02:28:07 UTC 2020
Client Version: <http://version.Info|version.Info>{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-17T11:41:22Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Unable to connect to the server: dial tcp 44.227.187.110:443: i/o timeout
Wed Jul 22 02:28:37 UTC 2020
Client Version: <http://version.Info|version.Info>{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-17T11:41:22Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Unable to connect to the server: dial tcp 44.227.187.110:443: i/o timeout
Wed Jul 22 02:29:07 UTC 2020
Client Version: <http://version.Info|version.Info>{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-17T11:41:22Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Unable to connect to the server: dial tcp 35.155.125.57:443: i/o timeout
Wed Jul 22 02:29:37 UTC 2020
Client Version: <http://version.Info|version.Info>{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-17T11:41:22Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: <http://version.Info|version.Info>{Major:"1", Minor:"17+", GitVersion:"v1.17.6-eks-4e7f64", GitCommit:"4e7f642f9f4cbb3c39a4fc6ee84fe341a8ade94c", GitTreeState:"clean", BuildDate:"2020-06-11T13:55:35Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Wed Jul 22 02:29:38 UTC 2020
Client Version: <http://version.Info|version.Info>{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-17T11:41:22Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: <http://version.Info|version.Info>{Major:"1", Minor:"17+", GitVersion:"v1.17.6-eks-4e7f64", GitCommit:"4e7f642f9f4cbb3c39a4fc6ee84fe341a8ade94c", GitTreeState:"clean", BuildDate:"2020-06-11T13:55:35Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
...
breezy-hamburger-69619
07/22/2020, 2:50 AMREADY
In most cases delays are eventual consistency, but 2+ min is more indicative of EKS just having an odd weekworried-city-86458
07/22/2020, 2:55 AMbreezy-hamburger-69619
07/22/2020, 3:06 AM