Hi! I have a statefulset: ```const labels = { app...
# getting-started
b
Hi! I have a statefulset:
Copy code
const labels = { appClass: 'app' };

export const appSvc = new k8s.core.v1.Service('app-svc', {
  metadata: {
    labels,
    name: 'app-svc',
  },
  spec: {
    type: 'ClusterIP',
    clusterIP: 'None',
    selector: labels,
  },
}, {
  provider,
});

export const interactionsApp = new k8s.apps.v1.StatefulSet('app', {
  metadata: {
    labels,
  },
  spec: {
    serviceName: 'app-svc',
    replicas: 3,
    selector: {
      matchLabels: labels,
    },
    template: {
      metadata: {
        labels,
      },
      spec: {
        imagePullSecrets: [
          { name: registryCredentials.metadata.name },
        ],
        containers: [
          { /* ... */ },
        ],
      },
    },
  },
}, {
  provider,
  dependsOn: [
    registryCredentials,
    databaseUser,
    redisInstance,
  ],
});
I hardcode the service's
metadata.name
to
'app-svc'
and the statefulset's
spec.serviceName
to
'app-svc'
as well. While this works, it feels wrong because now the resource name is not managed by pulumi. If I don't specify
metadata.name
in the service definition, and pass
appSvc.metadata.name
to the statefulset
spec.serviceName
, the statefulset is never created.. it waits for the
appSvc
to become ready, but the
appSvc
will never become ready because the statefulset hasn't been made yet:
Copy code
kubernetes:core/v1:Service (app-svc):
  error: 2 errors occurred:
      * resource default/app-svc-4kc42v1c was successfully created, but the Kubernetes API server reported that it failed to fully initialize or become live: 'app-svc-4kc42v1c' timed out waiting to be Ready
      * Service does not target any Pods. Selected Pods may not be ready, or field '.spec.selector' may not match labels on any Pods
how do I solve this issue? Is hardcoding the service name a good idea?
l
Hardcoding the name is fine, so long as you understand the implications around delete/replace and the lifecycle of resources. I don't understand why interactionsApp isn't created if appSvc.metadata.name isn't set. Is that property automatically set when you don't provide a value? If it is, then as I understand it, Pulumi should be able to figure out dependencies and it should work... maybe the other error message is important, and the Service won't finish creating unless you define a pod?
b
The typings indicate that the property is required
I'm currently testing a more isolated environment, the other error was related to a service depending on the application
Specifically, I'm testing the example on the documentation here https://www.pulumi.com/registry/packages/kubernetes/api-docs/apps/v1/statefulset/#create-a-statefulset-with-auto-naming. I'm waiting for it to time out to be sure, but it looks like the exact same thing is going to happen: the statefulset isn't being created because it's waiting on the service to be ready, but the service isn't ready because it can't find pods matching it's selector, and it can't find pods matching it's selector because the statefulset isn't being created. 🔃
Yeah, it timed out with the example from the documentation
Copy code
Type                           Name                    Status                  Info
     pulumi:pulumi:Stack            _redacted_              **failed**              1 error
 +   └─ kubernetes:core/v1:Service  nginx-service           **creating failed**     1 error

Diagnostics:
  kubernetes:core/v1:Service (nginx-service):
    error: 2 errors occurred:
        * resource default/nginx-service-nm3byu6a was successfully created, but the Kubernetes API server reported that it failed to fully initialize or become live: 'nginx-service-nm3byu6a' timed out waiting to be Ready
        * Service does not target any Pods. Selected Pods may not be ready, or field '.spec.selector' may not match labels on any Pods
 
  pulumi:pulumi:Stack (_redacted_):
    error: update failed
Is this the documentation being wrong or Pulumi being wrong?
l
I'm afraid I don't know. Could it have to do with
labels
? You're specifying it as an object, but the examples all have it as an array of strings....
b
I destroyed my stack and ran the example code from the documentation above, so my
labels
isn't a thing anymore
Considering it's just example code I'm pretty certain either the documentation example is wrong, or something regressed in Pulumi (I assume the example worked at some point)
l
Do any pods get created after the 10 minute timeout?
b
Nope. The statefulset itself just doesn't get created (
kubectl get statefulset
returns an empty list). I also don't see a
creating...
in the pulumi cli output for the statefulset; it starts by trying to create the service first, it'll say the service is waiting for
n
pods matching the selector, and this times out because those pods never get spawned (because the statefulset isn't being made)
If I hardcode the service name, everything works perfectly
l
Then I'm at a loss. Maybe cross-post to #kubernetes?
b
Will do, thanks
👍 1
b
@bright-horse-50102 it's quite rare to use the servicename like this, is there a reason you're doing it?
b
Use it like what? Hardcoding it? My end goal here is to simply make a StatefulSet following best practices, I have no other specific requirements
b
I would use the example here: https://www.pulumi.com/registry/packages/kubernetes/api-docs/apps/v1/statefulset/ note this section:
Copy code
spec: {
        selector: {
            matchLabels: {
                app: "nginx",
            },
        },
The reason you're having a hard time using
serviceName: 'app-svc',
is because pulumi has logic in it to ensure the service has healthy pods behind it. by setting the name this way, you're creating a circular dependency. remove the
serviceName
property and use the matchlabels property instead
b
I've used that exact example, message with details here: https://pulumi-community.slack.com/archives/C01PF3E1B8V/p1650600176789779?thread_ts=1650599095.566589&cid=C01PF3E1B8V The line below the snippet you've sent includes
serviceName
anyway (right below the selector). It's included in the example. Am I missing something? 😅
b
🤦‍♂️ forgive me, you're right
what image are you running, I'll take a look in a minute
b
I'm running that exact example, so the
<http://k8s.gcr.io/nginx-slim:0.8|k8s.gcr.io/nginx-slim:0.8>
image. Copied the entire codeblock and didn't change anything else
b
okay, give me 10 mins
👍 1
b
I did change one thing actually, I renamed the service from
nginxService
to
nginx-service
on line 4 because I was getting an error about service names having to be valid DNS names (i.e. lowercase). No changes other than that
here's what I ran specifically
Copy code
import * as kubernetes from '@pulumi/kubernetes';

const nginxService = new kubernetes.core.v1.Service('nginx-service', {
  metadata: {
    labels: {
      app: 'nginx',
    },
  },
  spec: {
    ports: [ {
      port: 80,
      name: 'web',
    } ],
    clusterIP: 'None',
    selector: {
      app: 'nginx',
    },
  },
});

const wwwStatefulSet = new kubernetes.apps.v1.StatefulSet('wwwStatefulSet', {
  spec: {
    selector: {
      matchLabels: {
        app: 'nginx',
      },
    },
    serviceName: nginxService.metadata.name,
    replicas: 3,
    template: {
      metadata: {
        labels: {
          app: 'nginx',
        },
      },
      spec: {
        terminationGracePeriodSeconds: 10,
        containers: [ {
          name: 'nginx',
          image: '<http://k8s.gcr.io/nginx-slim:0.8|k8s.gcr.io/nginx-slim:0.8>',
          ports: [ {
            containerPort: 80,
            name: 'web',
          } ],
          volumeMounts: [ {
            name: 'www',
            mountPath: '/usr/share/nginx/html',
          } ],
        } ],
      },
    },
    volumeClaimTemplates: [ {
      metadata: {
        name: 'www',
      },
      spec: {
        accessModes: [ 'ReadWriteOnce' ],
        storageClassName: 'my-storage-class',
        resources: {
          requests: {
            storage: '1Gi',
          },
        },
      },
    } ],
  },
});
b
@bright-horse-50102 I am shocked this has never come up before, but it's actually a bug: https://github.com/pulumi/pulumi-kubernetes/issues/1974 I added a workaround at the bottom, you need to add an annotation to skip the await logic
b
I see 😅 Glad to have helped then! The workaround makes sense, thanks.
b
fyi: if you don't have a volume controller that'll work for the storageclass, it'll also faul
b
Gotcha, luckily I'm not dealing with volumes for my scenario
b
? why use a statefulset then, out of interest?
b
Our application is a service that consumes data from a third party platform (Discord - a chat application comparable to Slack). Discord applications have the concept of "shards", these shards are assigned a static range of Discord groups that they receive events for. Each of these shards have to identify with Discord's websocket with a numeric "shard ID", which the stable unique ordinal of a statefulset pod is perfect for
b
oh sweet, makes sense!
😄 1
b
Also, I suppose the name of the service being camelCase is a separate issue?
b
yep I'm fixing that now
👍 1