TLDR; DigitalOcean rotates the kubeconfig, and everytime it does that, pulumi is unable to connect to the cluster due to server asking for credentials. I tried doing a refresh with

--target

into the

digitalocean.KubernetesCluster

, which brought a new kubeconfig content, but for some reason it doesn't seem to be used by the provider at all. Still getting

configured Kubernetes cluster is unreachable: unable to load schema information from the API server: the server has asked for the client to provide credentials

Does Pulimi understand states outside of its s3 bucket? Like, say docker images? Hmm, if the only thing I change in my Pulimi is the replica count for a k8 deployment would Pulimi do a full redeployment by making a new image in AWS ECR even though the codebase has not changed?

I think it’ll depend on how your Pulumi code is written.

I don’t think Pulumi implicitly knows if the codebase has changed or not compared to what is on ECR.

We manage the ECR deployment separate from the image tags deployed in our cluster. So we have a CICD process that will trigger a new container being tagged, created, and deployed to ECR based on our rules. Then in our Pulumi config we have an image version that is consumed by the Pulumi code and we manually manage that to control what version of the container is deployed where.

@loud-battery-37784 Honestly I’d chalk it up to a bug. If a

pulumi up

runs and the only thing that has changed is the replica count in

index.ts

(written using Pulumi APIs) then it shouldn’t really build the docker image again. But what bothers me is how unclear it is right now.

If i'm trying to pull an image from a private self hosted gitlab registry (under custom dns), to deploy on a kubs cluster

should i first deploy my private token using this

and then reference the image re yaml for k8s

ts
export const image = repo.buildAndPushImage("./app");

Running this twice where

./app

hasn’t changed since last time, should be no-op the second time and re-used the image already pushed since last time. I’d consider it a bug, not sure if I should post an issue, though.

@loud-battery-37784 I wonder if there is an approach where we can first try to “fetch” an image based on a tag/version and if it doesn’t exist, use

buildAndPushImage

This way I should technically get what I want - where the only “new” thing is the replica count, while the image stays the same. Depending on how Pulimi does its

kubectl

command where the only “new” thing should be the replica count.

I would think so. One of the reasons we moved to Pulumi from Terraform is being able to write our IaC in a turing complete language like TypeScript. Much easier to customize functionality like what you’re saying. Create an

if/else

block and do work.

Personally, I like the way it’s implemented currently. If the code executes it happens. It’s up to me to decide if the code should execute not the tool.

I’m leaving this open ended in case I’ve misunderstood something but… That seems counter intuitive, my assumption is that Pulumi manages state, if that contract is broken, then the tool becomes unreliable (I am not saying that is the case!) I am just saying that if the only thing I change between “pulumi up” is the

replica

count then that’s the only thing I want to change. Docker today already has the concept of layer hash. If no layer has changed, then that means no new image is necessary.

Pulumi knows when the image changes locally

If nothing changes in the image then it will just redeploy the same image

Pulumi doesn’t run Kubectl / it talks to the APIs directly

@broad-dog-22463 Oh so how does it work? In

pulumi up

a second time where

replica

number has changed, how does it make sure it’s not using a new image when using

buildAndPushImage

?

So the state has the name of the image associated with the deployment and will scale out because that’s the parameter that has changed

It’s an idempotent operation

Or it should be and if it reports differently then it’s a bug and something needs fixed

If a change is made manually then you need to run “Pulumi refresh” to update the Pulumi state with the latest changes from this manual update

@loud-battery-37784 In our case, we’re using

branch_names:latest

for everything except

master

&

develop

With separate repositories for each branch.

develop

and

master

branches share the same repository.

develop

will use automated

commit sha

as a tag.

master

will use hand selected (manual typed)

commit sha

as image tag from the same ECR/Docker repository as

develop

.

@broad-dog-22463 My question is Do I use

const image = buildAndPushImage(name: "app:23", pathOrBuild: "./app"): RepositoryImage

or

const image = new RepositoryImage(repository: repo, image: "app:23")

? if I want to get a specific

image

or build it, if it’s missing.

Build and push image doesn’t talk to kubernetes - it’s talking to ECR AFAIcR

Yeh, and that should have sufficient information to pick apart details for knowing if it’s fetching an existing image, or building a new one because something changed, correct?

BuildAndPushImage will build a local image and push the changes to ecr if there are any

It’s a wrapper that sits over Pulumi-docker