https://pulumi.com logo
Powered by
Title
c

calm-vr-6039

04/07/2023, 2:26 AM
Does anyone have experience with splitting up your infra into multiple stacks and referencing resources from one stack in another? Specifically I have a core stack that creates an EKS cluster, and I'm looking to use that cluster in my other stacks. I'm trying to do this with: pulumi.StackReference I can get the reference to the cluster with: cluster = coreInfrastructureReference.requireOutput("cluster"); But how can I then use that cluster variable. I need to access the provider property from it, but Property 'provider' does not exist on type 'Output<any>'. How do I convert this Output<any> to a cluster object?
s

steep-toddler-94095

04/07/2023, 2:43 AM
I haven't verified this works but what do you think about trying: • Just export the provider as its own variable rather than the entire Cluster • Use a Type Assertion on the Provider 
import { ProviderArgs } from '@pulumi/kubernetes'
import { Unwrap } from '@pulumi/pulumi'

const eksProvider = coreInfrastructureReference.requireOutput("eksProvider") as Unwrap<ProviderArgs>
c

calm-vr-6039

04/07/2023, 2:44 AM
I was just attempting to do that, running into issues attempting to export the provider though.
s

steep-toddler-94095

04/07/2023, 2:45 AM
what sort of issues?
c

calm-vr-6039

04/07/2023, 2:48 AM
Got that side working but this threw an error: Type 'UnwrappedObject<ProviderArgs>' is missing the following properties from type 'ProviderResource': id, urn, getProvider
s

steep-toddler-94095

04/07/2023, 2:53 AM
can you paste your code?
c

calm-vr-6039

04/07/2023, 2:54 AM
On the importation side?
s

steep-toddler-94095

04/07/2023, 2:54 AM
yeah
c

calm-vr-6039

04/07/2023, 2:55 AM
This is a condensed version: 
import * as pulumi from "@pulumi/pulumi";
import { ProviderArgs } from '@pulumi/kubernetes'
import * as k8s from '@pulumi/kubernetes';

const config = new pulumi.Config();

// Obtain the declared configs for the dev stack in api-gateway
const referrerOrganizationName = config.require("core-org");
const referrerProjectName = config.require("core-project-name");
const referrerStackName = config.require("core-stack");

// Create a stack reference to obtain resources from the referrer stack
const coreInfrastructureReference = new pulumi.StackReference(`${referrerOrganizationName}/${referrerProjectName}/${referrerStackName}`);

// Export any required outputs from the core-infrastructure
export const clusterProvider = coreInfrastructureReference.requireOutput("clusterProvider") as pulumi.Unwrap<ProviderArgs>

// Create secrets
export const secrets = new k8s.core.v1.Secret(
    'hasura-secrets',
    {
      metadata: {
        namespace: namespaceName,
      },
      data: {
        hasuraGraphqlJWTSecret: HASURA_GRAPHQL_JWT_SECRET.apply(toBase64),
        backendJWTPrivateKey: AuthJwtPrivateKey.privateKeyPem.apply(toBase64),
        backendJWTPublicKey: AuthJwtPrivateKey.publicKeyPem.apply(toBase64),
        dbConnectionUrl: connectionUrl.apply(toBase64),
      },
    },
    { provider: clusterProvider }
  );
s

steep-toddler-94095

04/07/2023, 2:56 AM
instead of 
{ provider: clusterProvider }
can you try 
{ provider: new Provider('whateveryouwanttonamethis', clusterProvider) }
where Provider is imported from '@pulumi/kubernetes'
basically we want to create a 
new Provider
object with the same params as the outputted one. Since the output is a json serialized Provider, it's not an actual Provider object
c

calm-vr-6039

04/07/2023, 3:00 AM
That threw no errors, so this is looking good but it's failing for other reasons (The provider isn't available as an output yet, when I ran a pulumi up after adding it as an export it doesn't seem to have added to the outputs). Hoping I don't have to do a down and up.
s

steep-toddler-94095

04/07/2023, 3:02 AM
did you add the export to index.ts? you should absolutely not have to do a down and up
c

calm-vr-6039

04/07/2023, 3:04 AM
You're right, it's just because I have so many different resources and commenting things out to make it quicker to debug something got left out which it had a requireOutput for on the other end. I'm rerunning it now.
Didn't work: <ref *1> TypeError: Cannot read properties of undefined (reading 'driver') and TypeError: Cannot read properties of undefined (reading 'burst')
My cluster is an EKS cluster, is it possible it's a different provider?
s

steep-toddler-94095

04/07/2023, 3:09 AM
that is a very odd error message. This is a compile error? Can you post your export and import code?
c

calm-vr-6039

04/07/2023, 3:11 AM
import * as eks from '@pulumi/eks';
import { Output } from '@pulumi/pulumi';

import * as config from '@project/infrastructure-config';
import { vpc } from './vpc';

const cluster = new eks.Cluster(config.CLUSTER_NAME, {
    name: config.CLUSTER_NAME,
    vpcId: vpc.id,
    publicSubnetIds: vpc.publicSubnetIds,
    privateSubnetIds: vpc.privateSubnetIds,
    instanceType: config.CLUSTER_NODE_INSTANCE_TYPE,
    createOidcProvider: true,
    desiredCapacity: 2,
    minSize: 1,
    maxSize: 3,
});

const kubeconfig: Output<unknown> = cluster.kubeconfig;

const urn: Output<string> = cluster.urn;

const clusterProvider = cluster.provider

export { urn, kubeconfig, clusterProvider };
That's the export.
import { Provider } from '@pulumi/kubernetes';
import * as pulumi from "@pulumi/pulumi";
import { ProviderArgs } from '@pulumi/kubernetes'
import * as k8s from '@pulumi/kubernetes';

const config = new pulumi.Config();

// Obtain the declared configs for the dev stack in api-gateway
const referrerOrganizationName = config.require("core-org");
const referrerProjectName = config.require("core-project-name");
const referrerStackName = config.require("core-stack");

// Create a stack reference to obtain resources from the referrer stack
const coreInfrastructureReference = new pulumi.StackReference(`${referrerOrganizationName}/${referrerProjectName}/${referrerStackName}`);

// Export any required outputs from the core-infrastructure
export const clusterProvider = coreInfrastructureReference.requireOutput("clusterProvider") as pulumi.Unwrap<ProviderArgs>

// Create secrets
export const secrets = new k8s.core.v1.Secret(
  'hasura-secrets',
  {
    metadata: {
      namespace: "default",
    },
    data: {
      secret: "test",
    },
  },
  { provider: new Provider('EksCluster', clusterProvider) }
);
s

steep-toddler-94095

04/07/2023, 3:18 AM
ah i noticed that the type assertion should have been 
Output<Unwrap<ProviderArgs>>
since requireOutput returns an Output. my mistake. but then it means youd have to create your secret within an apply 😕
c

calm-vr-6039

04/07/2023, 3:21 AM
I think the issue is using 2 different provider types, I'm saving an eks.provider, but then trying to create a kubernetes provider, and they have completely different args.
But what's strange is, I can't seem to find the eks.Provider type, despite it being in the docs: https://www.pulumi.com/registry/packages/eks/api-docs/provider/#eks-provider
s

steep-toddler-94095

04/07/2023, 3:22 AM
No, the args should be the same. The EKS cluster provider is of type 
k8s.Provider
c

calm-vr-6039

04/07/2023, 3:22 AM
Strange, the provider args in the docs for eks.provider show only an ID param?
s

steep-toddler-94095

04/07/2023, 3:23 AM
as Unwrap<ProviderArgs>
is wrong though. it needs to be an 
Output<Unwrap<ProviderArgs>>
or a 
Promise<Unwrap<ProviderArgs>>
if you use 
requireOutputValue
if you look at the exported value you'll see that it's a JSON object with the same params that k8s Provider takes in its args
but i wonder if 
requireOutput
is reading it in as a stringified json instead? Pulumi making it a Output<any> sort of masks the actual type. if you 
coreInfrastructureReference.requireOutput("clusterProvider").apply(console.log)
you should have a better idea what you are importing
c

calm-vr-6039

04/07/2023, 3:28 AM
{
      id: '8079f57d-31b6-456a-8ad5-625f489a0db6',
      kubeconfig: '....',
      pkg: 'kubernetes',
      urn: 'urn:pulumi:prod::core::eks:index:Cluster$pulumi:providers:kubernetes::core-k8-cluster-provider'
    }
s

steep-toddler-94095

04/07/2023, 3:30 AM
not directly related to the current issue, but i don't personally configure my providers from StackReferences. EKS clusters are not really ephemeral resources, so I just hardcode the cluster ARN in a new provider I create
c

calm-vr-6039

04/07/2023, 3:31 AM
The plan is to have multiple clusters, per env, so that's the reason for not hardcoding them.
s

steep-toddler-94095

04/07/2023, 3:31 AM
hmm that's interesting, that value is completely different from what I see in my Stack Output when I export my EKS cluster provider. I'm using the same package as you to create the cluster version 1.0.1
c

calm-vr-6039

04/07/2023, 3:32 AM
That's why I'm extremely confused.
s

steep-toddler-94095

04/07/2023, 3:34 AM
ill see if i can reproduce the import issue a bit later
c

calm-vr-6039

04/07/2023, 3:37 AM
Thanks, I appreciate all the help. I'll keep plugging away and seeing why I'm getting this provider as my export.
s

steep-toddler-94095

04/07/2023, 3:49 AM
actually, turns out i get the same as you. the initial output of 
pulumi up
when you are first creating the provider output is misleading.
It looks like the exported Provider isn't what you want. Maybe you just need to export the properties individually and construct a new Provider with the multiple exported properties. It's not convenient, but it would work... you'd still need to create all your resources within a 
.apply
or 
.then
though
Or export a custom object with all the properties you'd want in the provider. e.g. 
export const eksProviderArgs = {
  kubeconfig: cluster.kubeconfig,
  context: ...,
  ...
}
c

calm-vr-6039

04/07/2023, 3:52 AM
I'll give that a try then and see if I can get that working. Thanks for all your help.
Got it working. 
// Create secrets
export const secrets = new k8s.core.v1.Secret(
  'hasura-secrets',
  {
    metadata: {
      namespace: "default",
    },
    data: {
      dbConnectionUrl: "test",
    },
  },
  {
    provider: new Provider('EksCluster', 
      {
        ...clusterProvider,
        helmReleaseSettings: {
          driver: "secret"
        },
        kubeClientSettings: {
          burst: 10
        }
      }
    )
  }
);
s

steep-toddler-94095

04/07/2023, 4:00 AM
oh nice!! i wouldn't have guessed those props you added would have been required. And I'm also confused why you didn't need to do this inside an 
apply
given that the 
clusterProvider
is an 
Output<T>
But glad you got it working!!
c

calm-vr-6039

04/07/2023, 4:01 AM
Ya, I won't question the pulumi gods there lol strange that both those values should have defaults but they're required in there. And as you said why don't I need to do an apply!? But it works!
w

worried-rain-74420

04/10/2023, 3:04 PM
Very glad to hear you got this working! Just wanted to drop this blog post. We recently shipped a feature allowing you to access StackReferences without Output. https://www.pulumi.com/blog/stack-reference-output-details/
c

calm-vr-6039

04/10/2023, 4:17 PM
That is awesome, thank you! I'll look to implement this immediately!
#typescriptJoin Slack
Powered by