Following the install of a kubernetes cluster I'd like to automate deployment of a few helm charts. The charts are hosted in a private helm repo hosted in S3 that I normally access with

helm repo add my-repo <s3://my-helm-repo/charts>

. Given the example service deployment (copied below) how do I specify custom repo endpoints using the S3 plugin?

// Sample copied from <https://github.com/pulumi/examples/blob/master/kubernetes-ts-helm-wordpress/index.ts>

// Deploy the latest version of the stable/wordpress chart.
const wordpress = new k8s.helm.v2.Chart("wpdev", {
    repo: "stable",
    version: "2.1.3",
    chart: "wordpress"
});

@early-musician-41645 this is a thing I did not know you could do 🙂

There's a Helm S3 plugin that allows accessing an s3-based chart repo

@early-musician-41645 I’ll file an issue and fix this release.

There's a chance that this just works if you configure the repo before running Pulumi

oh oh

ah, good point... so Pulumi is using the local Helm, right?

correct

If that's the case then I can just do the

helm repo add

before calling

pulumi up

. I'll try it out

yeah.

Question is - how do I reference the repo in the code? Normally I do the command I have above, but then to reference the repo/package I do

helm install my-repo/my-chart

@early-musician-41645

{repo: "my-repo", chart: "my-chart"}

?

do you have the link to the opts code?

hmmm... specifically I want to pass in a file for values, e.g.

--values $filepath

@early-musician-41645 you mean from shell?

Typically I pass in values by giving a filepath. Here's a more complete example of things I need to do:

cmd="helm upgrade -i backgrounder $HELM_REPO/backgrounder \
      --tiller-namespace $namespace \
      --namespace $namespace \
      --values $podsettings_file \
      --set global.podSettings.podSettingsVersion=$configuration_version \
      --set images.tag=$service_version
      --set-string global.source.commit=$CI_COMMIT_SHA \
      --set-string global.source.repo=$CI_COMMIT_REF_SLUG \
      --set-string global.deploymentId=$deployment_id"

@early-musician-41645 we pick up the default values file, but not a custom file

You use helm without Tiller?

Yeah.

Does that work?

we are basically equivalent to

helm template

The scenario I have is that we have some services with LOTS of environment-specific configuration, and we have lots of environments, and lots of services. When deploying a service I download a config, specific to an environment, and versioned, and then feed that into the helm command via

--values

that makes sense, and that should be doable with Pulumi too

We model environments using namespaces. And to track services running in a single namespace we deploy tiller into that one namespace

mmmmmm

If there's a better practice then I'm open to it

@early-musician-41645 for using Helm, I’m not sure there is

Okay, I may be able to work around this by forming the

values

from a file input and injecting them at runtime

So the “Pulumi way” is that you should parameterize your app on namespaces. This is somewhat different from the Helm way.

Yeah, unfortunately that's not an option. The "environment" is the top-level resource in our deployments, not the app. It's because we are deploying a distributed-monolith where apps must be deployed together...

hmm, not sure I understand. you’re trying to deploy the application to a specific namespace based on where tiller is located?

I'm trying to deploy the app to a namespace based on need. E.g. we need Foo app, upgraded to v123, in the Bar environment. We've added another layer of isolation by running a tiller instance in each environment to track that environment's deployments

I’ve got to go, but I’ll be back in a bit

In terms of scenarios for locking down access to the kubernetes cluster, what's the model for Pulumi? Here's a for-instance: https://medium.com/@amimahloof/how-to-setup-helm-and-tiller-with-rbac-and-namespaces-34bf27f7d3c3 In that article the namespaces are controlled with service accounts and tiller is installed under that service account to enable RBAC control, e.g. devs are only allowed to deploy via the "dev" service account which grants access to tiller in a set of namespaces, but admins can also deploy to kube-system, etc. Via Pulumi, how can I set up that type of RBAC control for deployments?

@early-musician-41645 Pulumi uses the official kubernetes client under the covers … so the story is basically identical to however you would do this with

kubectl

Thanks for explaining it. That makes sense. I'll discuss with my team about it to see what we can work with. Is there a doc or link detailing best practices or examples of typical setups like this with Pulumi?

@early-musician-41645 there’s a bug here: https://github.com/pulumi/examples/issues/158

Okay, that works