Hi Ben! I think I can help you, let me know when you're free.
Here's some background: there's a bug in the Pulumi Kubernetes provider that manifested in CLI 3.92.0 and was fixed recently but not released quite yet (reference). The Pulumi CLI has a workaround since 3.94.0 (reference). Which version of the CLI is the CI job using?