bored-table-20691
07/09/2021, 8:47 PM+-kubernetes:core/v1:Secret: (replace)
[id=itay9/odas-secrets-o46s05yb]
[urn=urn:pulumi:tenant-itay9::okera-trial-tenants::kubernetes:core/v1:Secret::odas-secrets]
[provider=urn:pulumi:tenant-itay9::okera-trial-tenants::pulumi:providers:kubernetes::k8s-ssa-provider::b438617b-1a9e-4bd4-94f4-9ab2a01529a2]
~ stringData: {
~ SYSTEM_TOKEN: "[secret]" => "[secret]"
}
In the code, the secret is created like this:
...
StringData: pulumi.StringMap{
...,
"SYSTEM_TOKEN": systemToken,
and systemToken
is defined like this:
systemToken := pulumi.All(jwtKey.PrivateKeyPem).ApplyT(
func(args []interface{}) (string, error) {
...
},
).(pulumi.StringOutput)
This would all make sense if there was an update in systemToken
, but there isn’t - that’s not one of the resources that Pulumi is saying requires an update.
Any idea why this might be the case? How do I debug something like this?important-sandwich-62391
07/12/2021, 6:15 PMENABLE_POD_ENI
environment variable? Any ideas on how to best do this with Pulumi?
The AWS documentation says to run ”
kubectl set env daemonset aws-node -n kube-system ENABLE_POD_ENI=true
worried-city-86458
07/13/2021, 3:51 AMworried-city-86458
07/13/2021, 5:11 PMDiagnostics:
pulumi:pulumi:Stack (k8s-dev):
could not get token: RequestError: send request failed
caused by: Post <https://sts.amazonaws.com/>: dial tcp: lookup <http://sts.amazonaws.com|sts.amazonaws.com> on 192.168.65.5:53: no such host
could not get token: RequestError: send request failed
caused by: Post <https://sts.amazonaws.com/>: dial tcp: lookup <http://sts.amazonaws.com|sts.amazonaws.com> on 192.168.65.5:53: no such host
error: update failed
kubernetes:<http://apiextensions.k8s.io/v1:CustomResourceDefinition|apiextensions.k8s.io/v1:CustomResourceDefinition> (appmesh-system/appmesh-controller-selfsigned-issuer):
error: Delete "https://[REDACTED].<http://gr7.us-west-2.eks.amazonaws.com/apis/cert-manager.io/v1alpha2/namespaces/appmesh-system/issuers/appmesh-controller-selfsigned-issuer|gr7.us-west-2.eks.amazonaws.com/apis/cert-manager.io/v1alpha2/namespaces/appmesh-system/issuers/appmesh-controller-selfsigned-issuer>": getting credentials: exec: executable
aws-iam-authenticator failed with exit code 1
error: post-step event returned an error: failed to save snapshot: performing HTTP request: Patch "<https://api.pulumi.com/api/stacks/pharos/k8s/dev/update/27f96f11-ba00-4a38-b812-1a87d668ec8b/checkpoint>": dial tcp: lookup
<http://api.pulumi.com|api.pulumi.com> on 192.168.65.5:53: no such host
Resources:
- 199 deleted
Duration: 2h44m7s
During the hang I checked resources and the aws appmesh controller and cert manager controllers had already been deleted.
I didn't think it would matter since I didn't see any finalizers or owner refs associated with the <http://cert-manager.io/v1/Issuer|cert-manager.io/v1/Issuer>
resource being deleted, but maybe I'm missing something?
Looks like this is due to https://github.com/pulumi/pulumi-kubernetes/issues/861better-shampoo-48884
07/14/2021, 10:48 AMerror: Duplicate resource URN 'urn:pulumi:prod.k8s.devsecops::baseline-k8s::app:k8s:foundation$kubernetes:<http://helm.sh/v3:Chart$kubernetes:apiextensions.k8s.io/v1beta1:CustomResourceDefinition::{uniquevar}-kongconsumers.configuration.konghq.com';|helm.sh/v3:Chart$kubernetes:apiextensions.k8s.io/v1beta1:CustomResourceDefinition::{uniquevar}-kongconsumers.configuration.konghq.com';> try giving it a unique name
where {uniquevar} is an alphanumeric stringbetter-shampoo-48884
07/14/2021, 10:49 AMlemon-monkey-228
07/14/2021, 2:29 PMpulumi refresh
as I hadn’t used this repo/stack in a while and it seems to have untracked all of my k8s resourceslemon-monkey-228
07/14/2021, 2:29 PMpulumi preview
/ pulumi up
, it’s trying to create them from freshlemon-monkey-228
07/14/2021, 2:29 PMbland-cat-29878
07/16/2021, 1:07 PMproud-pizza-80589
07/18/2021, 1:55 PMripe-shampoo-80285
07/19/2021, 3:05 PMnumerous-thailand-80976
07/20/2021, 8:45 PMripe-shampoo-80285
07/21/2021, 2:50 PMripe-shampoo-80285
07/21/2021, 2:51 PMripe-shampoo-80285
07/21/2021, 2:51 PMripe-shampoo-80285
07/21/2021, 2:53 PMripe-shampoo-80285
07/21/2021, 3:04 PMfreezing-quill-32178
07/22/2021, 10:53 AMripe-shampoo-80285
07/22/2021, 2:38 PMambitious-afternoon-55254
07/22/2021, 5:19 PM(venv) a devops git:(master) ✗ pulumi up
Previewing update (dev):
Type Name Plan Info
pulumi:pulumi:Stack devops-dev 4 errors
+ └─ eks:index:Cluster test-cluster create
+ ├─ eks:index:ServiceRole test-cluster-eksRole create
+ │ ├─ aws:iam:Role test-cluster-eksRole-role create
+ │ ├─ aws:iam:RolePolicyAttachment test-cluster-eksRole-90eb1c99 create
+ │ └─ aws:iam:RolePolicyAttachment test-cluster-eksRole-4b490823 create
+ ├─ eks:index:ServiceRole test-cluster-instanceRole create
+ │ ├─ aws:iam:Role test-cluster-instanceRole-role create
+ │ ├─ aws:iam:RolePolicyAttachment test-cluster-instanceRole-03516f97 create
+ │ ├─ aws:iam:RolePolicyAttachment test-cluster-instanceRole-e1b295bd create
+ │ └─ aws:iam:RolePolicyAttachment test-cluster-instanceRole-3eb088f2 create
+ ├─ eks:index:RandomSuffix test-cluster-cfnStackName create
+ └─ aws:iam:InstanceProfile test-cluster-instanceProfile create
Diagnostics:
pulumi:pulumi:Stack (devops-dev):
error: Error: invocation of aws:ec2/getVpc:getVpc returned an error: invoking aws:ec2/getVpc:getVpc: 1 error occurred:
* no matching VPC found
# ...... Long stacktace continues
steep-toddler-94095
07/23/2021, 12:29 AMpulumi up
failed when trying to create kubernetes objects afterward, as part of the same stack. How can I access the EKS cluster I just created?purple-plumber-90981
07/24/2021, 10:38 PMpurple-plumber-90981
07/26/2021, 4:18 AMhelm template --debug
to generate the template final state yaml before deploying ?faint-dog-16036
07/26/2021, 6:26 PMtls.PrivateKey
pem
into openssh
format and throwing it into a k8s secret using stringData
The issue is that pulumi thinks the secret changes every single update (despite it remaining stable), and so it recreates the secret every time. Anyone run into this?busy-journalist-6936
07/29/2021, 8:22 PMk8s.helm.v3.Chart
inherit the credentials for the cluster it built? How do I go from building the cluster to also deploying stuff to it? if I manually pull the kubeconfig (future will be oidc) and re-run pulumi up
everything deploys fine.
index.ts LINKdazzling-sundown-39670
08/02/2021, 11:59 AMgetting credentials: exec plugin is configured to use API version <http://client.authentication.k8s.io/v1alpha1|client.authentication.k8s.io/v1alpha1>, plugin returned version <http://client.authentication.k8s.io/v1beta1|client.authentication.k8s.io/v1beta1>
. Can it be pulumi related?quiet-state-42882
08/03/2021, 3:33 AMpulumi up
: 404 could not find version. Any thoughts on what might be causing this? Would it be pulumi or digital ocean?delightful-gigabyte-39334
08/03/2021, 5:22 PMDiagnostics:
pulumi:pulumi:Stack (eks-test-dev):
error: Program failed with an unhandled exception:
error: Traceback (most recent call last):
File "/Users/username/Documents/Projects/infrastracture/eks-test/venv/lib/python3.9/site-packages/pulumi/runtime/resource.py", line 519, in do_rpc_call
return monitor.RegisterResource(req)
File "/Users/username/Documents/Projects/infrastracture/eks-test/venv/lib/python3.9/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/Users/username/Documents/Projects/infrastracture/eks-test/venv/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNKNOWN
details = "'dependsOn' was passed a value that was not a Resource."
debug_error_string = "{"created":"@1628011197.637119000","description":"Error received from peer ipv4:127.0.0.1:62720","file":"src/core/lib/surface/call.cc","file_line":1070,"grpc_message":"'dependsOn' was passed a value that was not a Resource.","grpc_status":2}"
quiet-architect-91246
08/03/2021, 7:20 PM"Container runtime not ready: cni not initialized".
This can be solved by running export kubever=$(kubectl version | base64 | tr -d '\n')
and kubectl apply -f "<https://cloud.weave.works/k8s/net?k8s-version=$kubever>"
. This obviously requires manual input which doesnt work for a automatic cicd pipeline, so Id love to do it in the declaration of the cluster with pulumi. I could hand in a recreation of the used .yaml file as ClusterRole
and ClusterRoleBinding
, but that would lead to very large, hardly readable codeblock and would require manual changes when the kubernetes version changes. Which leads me to my question: Is there a better way to handle the network initialization?quiet-architect-91246
08/03/2021, 7:20 PM"Container runtime not ready: cni not initialized".
This can be solved by running export kubever=$(kubectl version | base64 | tr -d '\n')
and kubectl apply -f "<https://cloud.weave.works/k8s/net?k8s-version=$kubever>"
. This obviously requires manual input which doesnt work for a automatic cicd pipeline, so Id love to do it in the declaration of the cluster with pulumi. I could hand in a recreation of the used .yaml file as ClusterRole
and ClusterRoleBinding
, but that would lead to very large, hardly readable codeblock and would require manual changes when the kubernetes version changes. Which leads me to my question: Is there a better way to handle the network initialization?billowy-army-68599
08/03/2021, 7:22 PMquiet-architect-91246
08/03/2021, 7:42 PM