https://pulumi.com logo
Title
p

proud-pizza-80589

03/07/2021, 8:16 PM
Did something change in the way to create an EKS cluster? Just tonight suddenly
const cluster = new eks.Cluster(name, {
  instanceType: "t2.medium",
  desiredCapacity: 1,
  minSize: 1,
  maxSize: 2,
  storageClasses: 'gp2',
  deployDashboard: false,
});
fails with
Diagnostics:
  eks:index:VpcCni (chartmuseum-vpc-cni):
    error: Command failed: kubectl apply -f /var/folders/4z/2zvwp19d343djqc78xywc6p00000gn/T/tmp-215018e7GR5EVRaFT.tmp
    error: You must be logged in to the server (the server has asked for the client to provide credentials)
 
  kubernetes:<http://storage.k8s.io/v1:StorageClass|storage.k8s.io/v1:StorageClass> (chartmuseum-gp2):
    error: configured Kubernetes cluster is unreachable: unable to load schema information from the API server: the server has asked for the client to provide credentials
 
  kubernetes:core/v1:ConfigMap (chartmuseum-nodeAccess):
    error: configured Kubernetes cluster is unreachable: unable to load schema information from the API server: the server has asked for the client to provide credentials
 
  pulumi:pulumi:Stack (chartmuseum-tutorial-chartmuseum):
    error: You must be logged in to the server (the server has asked for the client to provide credentials)
 
    error: update failed
The cluster is created so not sure why this suddenly no longer works
b

billowy-army-68599

03/07/2021, 8:28 PM
how are you creating the chart museum part?
p

proud-pizza-80589

03/07/2021, 8:31 PM
this is even before it gets there, the eks:index:VpcCni is part of the new Cluster
b

broad-dog-22463

03/07/2021, 8:32 PM
No Ned releases of Pulumi-eks have happened
Not since Jan 28th
p

proud-pizza-80589

03/07/2021, 8:33 PM
It is a blog post i’m wirting i’ve left for 2 weeks and am picking up tonight. https://github.com/roderik/chartmuseum-tutorial/blob/main/index.ts
this code worked then 🙂
b

broad-dog-22463

03/07/2021, 8:33 PM
Have you just started picking up 0.22 or was it always on 0.22?
p

proud-pizza-80589

03/07/2021, 8:34 PM
eks was always 22
this is how the pretty logs looks like
the parts failing are in the new eks.Cluster()
b

billowy-army-68599

03/07/2021, 8:36 PM
what's in your
kubeconfig
?
p

proud-pizza-80589

03/07/2021, 8:36 PM
destroyed nd recreated everything, including the full stack as well
b

billowy-army-68599

03/07/2021, 8:36 PM
do you have an old
KUBECONFIG
in
~/.kube/config
?
p

proud-pizza-80589

03/07/2021, 8:36 PM
i have 20 something clusters in there
b

billowy-army-68599

03/07/2021, 8:36 PM
ah wait, no. This is the aws credentials I think
how do you auth to aws?
p

proud-pizza-80589

03/07/2021, 8:38 PM
aws configure witht he same creds as the aws:accessKey and secretKey in the pulumi config
i also reinstalled my computer, it is a m1 arm but pulumi is installed as amd64
b

billowy-army-68599

03/07/2021, 8:39 PM
did you install the aws cli
p

proud-pizza-80589

03/07/2021, 8:39 PM
➜ chartmuseum-tutorial (main) ✗ aws --version                        (arn:aws:eks:eu-west-3:711839938093:cluster/chartmuseum-eksCluster-8e59b65/chartmuseum) aws-cli/2.1.29 Python/3.9.2 Darwin/20.4.0 source/arm64 prompt/off
it is installed as arm64 though
b

billowy-army-68599

03/07/2021, 8:40 PM
okay, well I'm not totally sure, but the issue is definitely coming from the created cluster. I would output the kubeconfig and try that
p

proud-pizza-80589

03/07/2021, 8:40 PM
➜ chartmuseum-tutorial (main) ✗ aws sts get-caller-identity                 (arn:aws:eks:eu-west-3:711839938093:cluster/chartmuseum-eksCluster-8e59b65/chartmuseum) Could not connect to the endpoint URL: “https://sts.eu-west3.amazonaws.com/
that is weird,,,
ah crap i know
i forgot a - between west and 3
b

broad-dog-22463

03/07/2021, 8:41 PM
That’d be it
And we turned off region validation by default in Pulumi-aws
Ok that’s kinda the last straw @billowy-army-68599 - I’m going to revert those defaults I changed as it’s proving to be painful
b

billowy-army-68599

03/07/2021, 8:42 PM
👍
p

proud-pizza-80589

03/07/2021, 8:42 PM
weird though that i have aws:region: eu-west-3 in my yaml file
does the local env take precedence?
b

broad-dog-22463

03/07/2021, 8:42 PM
It should not
b

billowy-army-68599

03/07/2021, 8:43 PM
it does for the
kubectl
part
b

broad-dog-22463

03/07/2021, 8:43 PM
Wel there we go ☝️
TIL
b

billowy-army-68599

03/07/2021, 8:43 PM
the kubectl call out doesn't use the provider I think, because if you look in the generated kubeconfig unless you set the aws profile and region, it uses the defaults
p

proud-pizza-80589

03/07/2021, 8:44 PM
i have had problems with that before, you can make GKE clusters with the config in the yml file, but not talk to the cluster because the kubernetes provider does not use those vaiables
b

billowy-army-68599

03/07/2021, 8:44 PM
it runs
aws eks get-token
which will use your default profile if it's set
p

proud-pizza-80589

03/07/2021, 8:45 PM
i have bypassed this by duplicating some stuff i found in a pulumi repo somehwere
- name: Setup credentials
        run: |
          export GOOGLE_APPLICATION_CREDENTIALS="$(mktemp).json"
          # Check if GOOGLE_CREDENTIALS is base64 encoded
          if [[ $GOOGLE_CREDENTIALS =~ ^([A-Za-z0-9+/]{4})*([A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)?$ ]]; then
            echo "$GOOGLE_CREDENTIALS"|base64 -d > $GOOGLE_APPLICATION_CREDENTIALS
            # unset for other gcloud commands using this variable.
            unset GOOGLE_CREDENTIALS
          else
            echo "$GOOGLE_CREDENTIALS" > $GOOGLE_APPLICATION_CREDENTIALS
          fi
          gcloud auth activate-service-account --key-file=$GOOGLE_APPLICATION_CREDENTIALS
          gcloud --quiet auth configure-docker $GOOGLE_DOCKER_HOSTNAME_LIST
as i do not use the pulumi action but use the cli manually
b

broad-dog-22463

03/07/2021, 8:47 PM
Yeah all of that has disappeared now :)
p

proud-pizza-80589

03/07/2021, 8:47 PM
Not sure what the intended behaviour is (as all these providers are standalone), but as a user, if i set my creds in the pulumi yaml, i kind of expect them to be used everywhere.
FYI, deploy works perfectly now, 👍