high-lamp-12603
04/02/2020, 4:50 PM// VPC Stack
import * as awsx from '@pulumi/awsx';
const vpc = new awsx.ec2.Vpc('application-vpc', {
cidrBlock: '10.128.0.0/19',
enableDnsSupport: true,
enableDnsHostnames: true,
subnets: [...Array(3).keys()].map((i) => ({ name: `private-${i}`, type: 'private' })),
numberOfNatGateways: 1,
});
export const { id, privateSubnetIds } = vpc;
// EKS Stack - Stack References Only
import * as awsx from '@pulumi/awsx';
import * as eks from '@pulumi/eks';
import * as pulumi from '@pulumi/pulumi';
import { last } from 'lodash';
const env = last(pulumi.getStack().split('.'));
const vpcStack = new pulumi.StackReference(`infra.vpc.${env}`);
const vpcId = vpcStack.getOutput('vpcId');
const privateSubnetIds = vpcStack.getOutput('privateSubnetIds');
export const eksCluster = new eks.Cluster('applicatin-eks-cluster', {
vpcId,
privateSubnetIds,
});
// EKS Stack - Only pass minimal references (ideal)
import * as awsx from '@pulumi/awsx';
import * as eks from '@pulumi/eks';
import * as pulumi from '@pulumi/pulumi';
import { last } from 'lodash';
const env = last(pulumi.getStack().split('.'));
const vpcStack = new pulumi.StackReference(`infra.vpc.${env}`);
const vpcId = vpcStack.getOutput('vpcId');
const vpc = awsx.ec2.Vpc.fromExistingIds('application-vpc', {
vpcId,
});
export const eksCluster = new eks.Cluster('applicatin-eks-cluster', {
vpcId: vpc.id,
privateSubnetIds: vpc.privateSubnetIds,
});
Option 1 works but outputting every will get messy quickly with increased complexity. Option 2 doesn’t actually work because fromExistingIds
doesn’t actually hydrate the subnets. Any suggestions or better examples?white-balloon-205
Hey new to Pulumi and looking for some best practices around cross stack references that go beyond the simple examples.Welcome!
The docs seem to heavily suggest using many small stacksThis doesn't sound intentional - curious which doc pointed you that direction? In general, I suggest starting off with a single stack, and breaking it up if/when you know there are layers that are truly independent and are owned/versioned by different teams. It is definitely more constraining to introduce cross-stack boundaries in an application - so I wouldn't suggest this unless you know you need it.
Option 1 works but outputting every will get messy quickly with increased complexity.Yes - you will in general want to draw boundaries between layers such that there are not a huge number of cross-stack dependencies (that is, such that the two layers are not overly coupled to one another). In many cases, just a couple of pieces of information are enough to re-hydrate what you need in the other stack, and those should be the outputs and the higher-level stack.
Option 2 doesn’t actually work becauseI'll have to look into this one. What do you see in this case for you example?doesn’t actually hydrate the subnets.fromExistingIds
high-lamp-12603
04/02/2020, 5:02 PMwhite-balloon-205
high-lamp-12603
04/02/2020, 5:04 PMwhite-balloon-205
migrate an existing lerna monorepo with some shared infra (vpc & eks) and a bunch of micro services that “own” their own infra and get deployed into eksOkay - so keeping that same breakdown totally makes sense in Pulumi. One additional thing you can do in Pulumi is introduce a shared JS library that populates information in the higher-level stacks from the outputs available from the lower-level stack. That library can then be used by all the higher-level stacks as a single simple way to bootstrap themselves by referencing the vpc+eks stack they want to build on. You would still need to export all of the true surface area of the VPC+EKS stack - but ideally that's still just a handful of exports. Did your existing setup avoid needing to have as much on the boundary between these components?
high-lamp-12603
04/02/2020, 5:16 PMDid your existing setup avoid needing to have as much on the boundary between these components?Its loosely coupled now which has its own problems 🙂 and is why we’re looking at Pulumi
aws-cdk
works closer to the later optionwhite-balloon-205
Ideally I think we’d want to pass the minimum identifiers and then hydrate read-only objectsYes - this is what you would want to do. I think it's roughly: vpcID, subnetIDs, kubeconfig. Everything else should be able to be re-constructed from those.
high-lamp-12603
04/02/2020, 5:30 PMconst vpcId = vpcStack.getOutput('id');
const privateSubnetIds = vpcStack.getOutput('privateSubnetIds');
const vpc = awsx.ec2.Vpc.fromExistingIds('application-vpc', {
vpcId,
privateSubnetIds,
});
white-balloon-205
high-lamp-12603
04/02/2020, 5:31 PMType 'Output<any>' is not assignable to type 'Input<string>[] | undefined'.
Type 'OutputInstance<any>' is missing the following properties from type 'Input<string>[]': length, pop, push, concat, and 26 more
white-balloon-205
type Input<T> = Output<T> | T
high-lamp-12603
04/02/2020, 5:32 PMpulumi.Output<string[]>
white-balloon-205
awsx
VPC in the higher-level stacks? Or can you just use the raw VPC and Subnets? How are you using those IDs from the higher-level stacks?high-lamp-12603
04/02/2020, 5:47 PMwhite-balloon-205
StackRefrences
.high-lamp-12603
04/02/2020, 5:57 PMwhite-balloon-205
fromExistingIds
with StackReference
- the following works, though how useful it is depends on what you are planning to use the vpc
for ultiamtely.
const vpcStack = new pulumi.StackReference("vpc");
const vpcId = vpcStack.getOutput("vpcId") as pulumi.Output<string>;
const publicSubnets = vpcStack.getOutput("publicSubnetIds") as pulumi.Output<string[]>;
const privateSubnets = vpcStack.getOutput("privateSubnetIds") as pulumi.Output<string[]>;
const vpc = pulumi.all([publicSubnets, privateSubnets]).apply(([publicSubnetIds, privateSubnetIds]) => {
return awsx.ec2.Vpc.fromExistingIds("name", {
vpcId,
publicSubnetIds,
privateSubnetIds,
})
});
high-lamp-12603
04/02/2020, 6:07 PMpublicSubnets
need the apply()
logic but vpcId
does not?white-balloon-205
fromExistingIds
currently accepts vpcId: Input<string>
but publicSubnetIds: Array<Input<string>>
. That means you can pass an Output<string>
to the former, but not an Output<Array<string>>
to the latter (since you need an Array<Output<string>>
.
A key question is "is the length known yet"? For an Output<Array<string>>
the length might not be known yet, and can only be queried inside an .apply
. For an Array<Output<string>>
the length is always known.
This is important when you need to promptly loop over the input ( to create one resource per item in the loop).high-lamp-12603
04/02/2020, 6:18 PM