@lemon-agent-27707 Biggest challenges for now • gcloud install is super slow. I need the following pre start script to make it work (the base code is from the pulumi version from 2018 so might be outdated, but a gke k8s cluster needs these packages + the creds magic to connect)

sh -c "$(curl --location <https://taskfile.dev/install.sh>)" -- -d -b .bin
curl -sLS <https://get.arkade.dev> | sh
arkade get yq
mv /root/.arkade/bin/yq /usr/local/bin/ 
./.bin/task codegen:charts
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] <https://packages.cloud.google.com/apt> cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
curl <https://packages.cloud.google.com/apt/doc/apt-key.gpg> | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - 
apt-get update
apt-get install google-cloud-sdk google-cloud-sdk-gke-gcloud-auth-plugin
echo "$GOOGLE_CREDENTIALS" > creds.json
gcloud auth activate-service-account --key-file=creds.json

• i prefer to put my secrets in the pulumi config, but for AWS just setting the accessKey and secret in there does not work, so i have to add the env vars in the UI • i have not pulumified the deployments setup, but i would love to set up deployments and the webhooks up at a project level, as my projects are 10-20 stacks of the same things (e.g. clusters in different cloud providers in different regions)

Thanks for the feedback @proud-pizza-80589 and sorry for the delayed reply!

gcloud install is super slow

You could consider using a custom docker image that has these tools already installed: https://www.pulumi.com/docs/pulumi-cloud/deployments/reference/#customizing-the-deployment-environment The tradeoff here being that it will take time to pull the image as we cache the pulumi base image on the deployment runner. Depending on how long your tools take to install, and how large the docker image is, it could end up being faster though.

i prefer to put my secrets in the pulumi config, but for AWS just setting the accessKey and secret in there does not work

This sounds like a bug. Would you mind opening an issue via github.com/pulumi/pulumi-aws/issues/new In the meantime, have you checked out the Deployments OIDC integration for AWS? It takes about 10 minutes to set up. We utilize it for all of our stacks internally. It works quite nicely with the added benefit that credentials are temporary and scoped for an improved security profile. We do plan on eventually supporting the concept of "environments" which allow you to set up cloud credentials once, and then reuse them by reference across multiple stacks. (ie this stack deploys to the prod environment)

but i would love to set up deployments and the webhooks up at a project level

There is an issue open for this: https://github.com/pulumi/pulumi-cloud-requests/issues/239 would love your feedback there. This is definitely something we plan on doing, but don't have an ETA yet. The pulumiservice provider does work quite nicely for managing deployment settings programmatically. You can even define the deployment settings for a stack in the same program, which kind of gives you project level settings but with some caveats. https://www.pulumi.com/docs/pulumi-cloud/deployments/reference/#defined-as-code-with-the-pulumi-service-provider

i tried our own deployment image we use in our platform but it is a huge one, that job was just "stuck" until i found the cancel job button a few hours later. 🙂 gcloud cli is just huge, i think the .deb is even 350MB

Where did the deployment get stuck? Did you get any setup logs that indicated the image pull was in progress?

Yes, it hung on pulling the image

Can you share the dockerfile? Based on that error, it sounds like the image was pulled successfully but failed to start. I wonder if you're trying to bind to a port that is blocked?

FROM node:18.16.0-bullseye-slim
LABEL org.opencontainers.image.source="<https://github.com/settlemint/bpaas>"

RUN export DEBIAN_FRONTEND=noninteractive && \
  apt-get update && \
  apt-get install -y --no-install-recommends dumb-init curl jq git python3 ca-certificates unzip gnupg && \
  echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] <https://packages.cloud.google.com/apt> cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \
  curl <https://packages.cloud.google.com/apt/doc/apt-key.gpg> | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - && \
  apt-get update && \
  apt-get install -y --no-install-recommends google-cloud-sdk google-cloud-sdk-gke-gcloud-auth-plugin && \
  curl "<https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip>" -o "awscliv2.zip" && \
  unzip awscliv2.zip && \
  mkdir -p /aws && \
  ./aws/install && \
  rm -Rf aws awscliv2.zip && \
  apt-get clean && \
  rm -rf /var/lib/apt/lists/* && \
  mkdir -p /pulumi/projects/.pulumi/workspaces && \
  mkdir -p /pulumi/projects/.pulumi/stacks

WORKDIR /pulumi/projects
ENV PULUMI_HOME /pulumi/projects/.pulumi
ENV PATH "/pulumi/bin:/gcloud/google-cloud-sdk/bin:${PATH}"

COPY --from=pulumi/pulumi-nodejs:3.70.0 --chmod=0777 /pulumi /pulumi
COPY --from=lachlanevenson/k8s-kubectl:v1.25.4 --chmod=0777 /usr/local/bin/kubectl /usr/local/bin/kubectl
COPY --chmod=0777 entrypoint.sh /entrypoint.sh

entrypoint.sh

#!/bin/bash -e

export GOOGLE_APPLICATION_CREDENTIALS="$(mktemp).json"
# Check if GOOGLE_CREDENTIALS is base64 encoded
if [[ $GOOGLE_CREDENTIALS =~ ^([A-Za-z0-9+/]{4})*([A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)?$ ]]; then
  echo "$GOOGLE_CREDENTIALS"|base64 -d > $GOOGLE_APPLICATION_CREDENTIALS
  # unset for other gcloud commands using this variable.
  unset GOOGLE_CREDENTIALS
else
  echo "$GOOGLE_CREDENTIALS" > $GOOGLE_APPLICATION_CREDENTIALS
fi
gcloud config set 'auth/service_account_use_self_signed_jwt' false
gcloud auth activate-service-account --key-file=$GOOGLE_APPLICATION_CREDENTIALS

exec "$@"

exit 0