bitter-father-26598
07/18/2023, 12:05 PMhelm repo add cockroachdb <https://charts.cockroachdb.com/>
helm repo add zitadel <https://charts.zitadel.com>
# Install CockroachDB. Contains helm hooks
helm install crdb cockroachdb/cockroachdb \
--set fullnameOverride=crdb \
--set single-node=true \
--set statefulset.replicas=1
# Install ZITADEL
helm install sifauth zitadel/zitadel \
--set zitadel.masterkey="1FhB0hgbG9sU5Yzw6NRknfstbQbuADNP" \
--set zitadel.configmapConfig.ExternalSecure=false \
--set zitadel.configmapConfig.TLS.Enabled=false \
--set zitadel.secretConfig.Database.cockroach.User.Password="admin" \
--set replicaCount=1
However when I try to use Pulumi Helm Releases with the same values, the release consistently fails. What am I missing?
const provider = new k8s.Provider('base-kube-provider', {
kubeconfig: cluster.kubeConfigs[0].rawConfig
}, { dependsOn: [cluster] })
new k8s.helm.v3.Release('crdb', {
name: 'crdb',
chart: 'cockroachdb',
waitForJobs: true,
repositoryOpts: {
repo: '<https://charts.cockroachdb.com>',
},
values: {
fullnameOverride: 'crdb',
'single-node': true,
statefulset: {
replicas: 1
}
},
}, {dependsOn: [provider], provider})
new k8s.helm.v3.Release('sifauth', {
name: 'sifauth',
chart: 'zitadel',
waitForJobs: true,
repositoryOpts: {
repo: '<https://charts.zitadel.com>',
},
values: {
replicaCount: 1,
zitadel: {
masterkey: authConfig.getSecret('masterKey'),
configmapConfig: {
ExternalSecure: false,
TLS: {
Enabled: false
},
secretConfig: {
Database: {
cockroach: {
User: {
Password: authConfig.getSecret('dbPassword')
}
}
}
}
}
},
},
}, {dependsOn: [crdb], provider})
Error:
kubernetes:<http://helm.sh/v3:Release|helm.sh/v3:Release> (crdb):
warning: Helm release "crdb" was created but has a failed status. Use the `helm` command to investigate the error, correct it, then retry. Reason: timed out waiting for the condition
error: 1 error occurred:
* Helm release "default/crdb" was created, but failed to initialize completely. Use Helm CLI to investigate.: failed to become available within allocated timeout. Error: Helm Release default/crdb: timed out waiting for the condition
# kubectl get pods
NAME READY STATUS RESTARTS AGE
crdb-0 0/1 Running 0 9m10s
full-eve-52536
07/18/2023, 1:32 PMRelease
resource in Pulumi, especially on our ephemeral environments where we are constantly building fresh clusters. Our issues are primarily around not being able to find the CRDs and resources in time, thoughbitter-father-26598
07/18/2023, 1:33 PMfull-eve-52536
07/18/2023, 1:36 PMatomic: true
and dependencyupdate: True
directives and that has help us a little bit, not sure that will help your specific use case though.bitter-father-26598
07/18/2023, 1:36 PMupdate
run.curved-kitchen-24115
07/18/2023, 9:26 PMdependsOn: []
can be a little tricky - it just sets up the dependency graph, not a timing aspect. I think you want to use crdb.status
but I’m not 100% sure on that.full-eve-52536
07/18/2023, 9:30 PMpulumi up
curved-kitchen-24115
07/18/2023, 9:31 PMcrdb.status
in the dependsOn… I read that somewhere — but I cannot remember where or whether it worked.
In our case the CRDs we create occur within the <timeout> window that helm.release waits (5m maybe?) so it reconciles itself.crdb
exists when it is defined, so that passes the dependsOn check immediately. So you want to depend on something that takes time. I think the rationale is that status
isn’t ready until the install has occurred.dependsOn
.full-eve-52536
07/18/2023, 9:33 PMRelease
resources that we use end up installing the CRDs on their owncrbd.status
is and where it's coming fromcurved-kitchen-24115
07/18/2023, 9:34 PMhelm.Release
; additionally it’s Resource itself, so you can dependsOn
itfull-eve-52536
07/18/2023, 9:36 PMcrbd
was it's own baked in resource in PulumiRelease
as part of our stack and almost everytime we run pulumi up
We get the server could not find the requested resource
curved-kitchen-24115
07/18/2023, 9:38 PMhelm.Release.status
is an Output; and you can dependsOn
those if I remember correctlymy-app-package-v1
depends on CRDs defined in my-crd-package-v1
, how can you upgrade to my-crd-package-v2
without breaking my-app-package-v1
?
So, typically, you end up with MyCRD-V1
and MyCRD-v2
, etc, and then the app package can decide which one it wants to implement.bitter-father-26598
07/18/2023, 9:46 PMfull-eve-52536
07/18/2023, 9:47 PMcurved-kitchen-24115
07/18/2023, 9:47 PMbitter-father-26598
07/18/2023, 9:51 PMcreated
.