https://pulumi.com logo
Join Slack
Powered by
This message was deleted.
# general
s
This message was deleted.
c
@important-leather-28796 fascinating, how are you getting the YAML?
just 
helm template
?
i
via raw url from the recommended install docs for each (avoiding helm altogether) e.g.
Copy code
// Install the CustomResourceDefinitions and cert-manager itself
    const certManager = new k8s.yaml.ConfigFile(
      '<https://raw.githubusercontent.com/jetstack/cert-manager/release-0.7/deploy/manifests/cert-manager.yaml>',
      undefined,
      { parent: this },
    )
It feels like I never should have tried helm when a component’s release publishes the raw manifest. Pulumi allows the simple customization of anything I need so no need to obfuscate through another helm developer’s interpretation of the original manifest (or that’s the way I feel)
doubling the toolchain just caused me to chase unnecessary things - a secret on the k8 dashboard and namespace/webhook issue on the cert-manager
raw manifests pretty much worked out of the box.
So - not much of a revelation - just that helm won’t be my first look anymore
c
helm also has a bunch of small gotchas, like ~most charts re-generate secrets every invocation of the chart.
i
it just seems like unnecessary complexity in hindsight. I wish I would have known that but not sure how you could introduce that thought to other users diplomatically.
o
also having some helm issues
c
cc @breezy-hamburger-69619 🙂
m
I have actually had really good luck using helm charts with pulumi.
o
for example deploying a newrelic chart. in the function, create a namespace, pass namespace as dependsOn to subsquent helm install of the kube-state-metrics and newrelic charts. Even if changing the namespace on which the helm object depends, it does not redeploy the app
(this is in a case where we initially deployed the apps to the ‘wrong’ namespace)
c
@orange-tailor-85423 
dependsOn
does not set the namespace
it just causes the namespace to be provisioned first.
o
correct
I hard-coded that change as well in the chart
m
With one small exception also related to namespaces. It seems setting the 
namespace
input field does not actually cause the resources to end up there. Instead I do this:
Copy code
transformations: [(y) => {
        y.metadata.namespace = 'kube-system'; // eslint-disable-line
      }],
c
yeah, we will have a fix for that very soon.
it should definitely re-provision the chart if you’re changing the namespace of the resources in it.
m
I have found that to be true, just requiring the 
transformations
hack to actually change them.
o
wow
maybe this our issue then
m
Also should still set 
namespace: kube-system
on the 
k8s.helm.v2.Chart
, since the chart may interpolate on 
.Release.namespace
For now, need both, sounds like fix incoming to only need 
namespace: kube-system
, and no 
transformations
type hack
o
Tim, larger example of the transformations hack?
m
That's literally all I add if I want to move everything to a namesapce
do you mean a full resource?
Copy code
const chartIgnored = new k8s.helm.v2.Chart('ingress', {
      repo: 'incubator',
      chart: 'aws-alb-ingress-controller',
      version: '0.1.4',
      namespace: 'kube-system',
      values: {
        clusterName: name,
        autoDiscoverAwsRegion: true,
        autoDiscoverAwsVpcID: true,
      },
      transformations: [(y) => {
        y.metadata.namespace = 'kube-system'; // eslint-disable-line
      }],
    }, defaultOpts);
c
so we will eventually fix the 
namespace: kube-system
thing, but this week is very likely to fix the transforms stuff.
the 
namespace: kube-system
thing is so, so annoying, because it’s something helm shoudl be doing itself, and we’ll have to work around.
m
Huh, I don't understand what the transforms fix is.
What is the issue there?
One other interesting helm chart hack I have had to do that may be of interest to people:
Copy code
transformations: [(y) => {
        // Fix stupid name changing
        if (y.kind === 'Pod' && y.metadata.name.indexOf('jenkins-ui-test') !== -1) {
          y.metadata.name = 'jenkins-ui-test'; // eslint-disable-line
        }
      }],
The jenkins helm chart has a pod that gets random characters appended to its name, causing a constant diff. This fixes it.
o
code:
Copy code
export function Install(
    name: string,
    kubestateMetricsVersion: string,
    version: string,
    licenseKey: string,
    env: string,
    cluster: k8s.Provider
): k8s.helm.v2.Chart {
    const namespace = new k8s.core.v1.Namespace(
        `newrelic3`,
        {
            metadata: { name: `newrelic3` }
        },
        { provider: cluster }
    );

    new k8s.helm.v2.Chart(
        name,
        {
            repo: 'stable',
            chart: 'kube-state-metrics',
            version: kubestateMetricsVersion,
            namespace: 'newrelic3'
        },
        {
            dependsOn: [namespace],
            providers: { kubernetes: cluster }
        }
    );
    return new k8s.helm.v2.Chart(
        `${name}-infra`,
        {
            repo: 'stable',
            chart: 'newrelic-infrastructure',
            namespace: 'newrelic3',
            version,
            values: { licenseKey, cluster: env }
        },
        {
            dependsOn: [namespace],
            providers: { kubernetes: cluster }
        }
    );
}
this is going from ‘newrelic’ to ‘newrelic2’ to ‘newrelic3’
Copy code
Type                                                           Name                                    Plan       Info
     pulumi:pulumi:Stack                                            drogon-casey-robertson
 >   ├─ pulumi:pulumi:StackReference                                networkStorageCluster                   read
 +   ├─ kubernetes:core:Namespace                                   newrelic3                               create
     ├─ kubernetes:helm.sh:Chart                                    newrelic-infra
 ~   │  └─ kubernetes:<http://rbac.authorization.k8s.io:ClusterRoleBinding|rbac.authorization.k8s.io:ClusterRoleBinding>  newrelic-infra-newrelic-infrastructure  update     [diff: ~subjects]
     ├─ kubernetes:helm.sh:Chart                                    newrelic
 ~   │  └─ kubernetes:<http://rbac.authorization.k8s.io:ClusterRoleBinding|rbac.authorization.k8s.io:ClusterRoleBinding>  newrelic-kube-state-metrics             update     [diff: ~subjects]
 -   └─ kubernetes:core:Namespace                                   newrelic2                               delete

Resources:
    + 1 to create
    ~ 2 to update
    - 1 to delete
creates new namespace but nothing happens to the deployed app
m
Add the 
transformations
hack:
Copy code
return new k8s.helm.v2.Chart(
        `${name}-infra`,
        {
            repo: 'stable',
            chart: 'newrelic-infrastructure',
            namespace: 'newrelic3',
            version,
            values: { licenseKey, cluster: env },
            transformations: [(y) => {
                y.metadata.namespace = 'newrelic3'; // eslint-disable-line
            }],
        },
        {
            dependsOn: [namespace],
            providers: { kubernetes: cluster }
        }
    );
@orange-tailor-85423 ^
c
@millions-judge-24978 I mean that transforms don't allow you to put output<t> in them right now. will fix that
but also you should just be able to set the namespace in chart opts and have it work
m
Ah I see, when does that become an issue? I haven't noticed any problem.
I guess just some case in which you need something more dynamic than what I'm doing in there
c
the release namespace issue you mentioned
I'm on phone
I think you mentioned it. it's the option that helm template populates
o
working now Tim - thanks so much
💯 1
I need to read up on that - I’m not a helm expert
m
Oh my only issue is the namespace option in chart inputs does not move resources
o
is there an open issue to track this?
4 Views
Open in Slack
PreviousNext
Powered by