https://pulumi.com logo
Join Slack
Powered by
Hi, guys. I have a quick question. how do you buil...
# getting-started
b
Hi, guys. I have a quick question. how do you build a project that have mutiple 
Pulumi.<cluster>.yaml
And some yaml file with resource A, some yaml with resource B. Seems each resource will trying to build with all stack files. Here is a example project structure I am going with:
Copy code
├── infrastructure
│   ├── iac
│   │   ├── aws
│   │   │   ├── containers
│   │   │   │   ├── index.ts
│   │   │   │   ├── package.json
│   │   │   │   ├── Pulumi.yaml
│   │   │   │   ├── Pulumi.foo.yaml
│   │   │   │   ├── Pulumi.bar.yaml
│   │   │   │   ├── ecr
│   │   │   │   │   ├── index.ts
│   │   │   │   ├── fargate
│   │   │   │   │   ├── index.ts
Like a example here 
Pulumi.foo.yaml
will only have 
ecr
resource. 
Pulumi.bar.yaml
will only have 
fargate
resource. How do I make each resource 
index.ts
smartly know if there is no config about ecr in the stack yaml file. I will do nothing.
s
I would like to know that too
n
Is something like this what you're looking for?
Copy code
import { Config } from "pulumi";

import { createSomeEcrResource } from "./ecr";
import { createSomeFargatResource } from "./fargate";

const ecrConfig = Config("ecr");
if (ecrConfig.getBoolean("enabled")) {
  createSomeEcrResource(ecrConfig);
}

const fargateConfig = Config("fargate");
if (fargateConfig.getBoolean("enabled")) {
  createSomeFargateResource(fargateConfig);
}
d
Hmm do the 
Pulumi.*.yaml
correspond to separate Pulumi stacks of the same project?
b
@nice-pilot-2521 Yeah. using a if else on all the index code is something I am thinking of, just feel there maybe something smarter to do so?
d
Hmm, I’m not sure I fully understand here… but my impression of stacks is to set up different “environments” of the same exact infrastructure. There are some cases where if statements might make sense (like in a non-production web environment, you omit some piece of infra), but in general, I think it’s probably a smell if the shape of your infra is different from Pulumi.stack-1.yaml to Pulumi.stack-2.yaml of the same project. Rather, it would make sense to separate different “shaped” infra into separate stacks and use stack outputs and stack references to link them together if necessary
E.g. we have an 
application
stack that has RDS, Amazon MQ, DNS, datadog, S3 buckets, SSM params, etc. and sets up the cluster… we have Pulumi.prod.yaml for the production site, Pulumi.staging.yaml for pre-prod, and Pulumi.dev.yaml for development…
Then we have a separate project for our CI/builds that sets up the ECR resource where buildkite pushes our images
That separate project has a single stack, 
Pulumi.ci.yaml
and does 
pulumi.export("ecr-url", repository.url)
(python)
b
Yeah, what happen if your 
Pulumi.staging.yaml
have S3 bucket. and 
Pulumi.dev.yaml
dont have any S3 bucket. The S3 bucket index.ts file will try to load both yaml and needs to decide this file doesnt have S3 bucket config, and I will skip doing nothing.
d
The app project stacks have:
Copy code
build_stack = pulumi.StackReference("our-org/builds/ci")
BUILDS_ECR_URL = build_stack.get_output("ecr-url")
Yeah so for the cases where different stacks have different resources requirements (e.g. different sized RDS instances) we use a combination of config variables and if statements
image.png
b
Yeah. I guess I am looking for a good coding style how your index.ts file can decided cause on the configs in the yaml file. I think @Willem showed me a good example. Can you share some example too. thanks!
d
Yeah I think 
if
statements are totally fine, and there are different approaches to the code design
I was just checking to make sure that the different stacks within a single project weren’t to set up different types of resources
b
I have this style of stack
Copy code
├── infrastructure
│   ├── iac
│   │   ├── aws
│   │   │   ├── containers
│   │   │   │   ├── index.ts
│   │   │   │   ├── package.json
│   │   │   │   ├── Pulumi.yaml
│   │   │   │   ├── ecr
│   │   │   │   │   ├── index.ts
│   │   │   │   ├── fargate
│   │   │   │   │   ├── index.ts
│   │   │   ├── storage
│   │   │   │   ├── index.ts
│   │   │   │   ├── package.json
│   │   │   │   ├── Pulumi.yaml
│   │   │   │   ├── s3
│   │   │   │   │   ├── index.ts
Do you think I should do 1 project per resource or like what shown 1 project per category
Copy code
I was just checking to make sure that the different stacks within a single project weren't to set up different types of resources
If I have 1 project including many resource, I will have to do that right? with the structure I have above.
d
yeah so from what I’m seeing above, you have 2 “projects” => 
containers
and 
storage
. To me, it doesn’t make sense for 
ecr
and 
fargate
to be separate “stacks” within the 
containers
project
a “stack” is an instance of a project, or at least that’s how I’ve been thinking of it
ecr
and 
fargate
are groups of resources that should live in the 
containers
project and when you want to launch those resources you should create a “stack” of the 
containers
project
you could just put all the stuff from 
containers/ecr/index.ts
and 
containers/fargate/index.ts
into the 
containers/index.ts
. For code organization, it’s nice to break it apart, and you can just use JS/TS imports
b
I am sorry, maybe I am miss understanding the term here: When I am creating a stack here means a yaml file like 
Pulumi.dev.yaml
, this mean I want to have a dev cluster. and this cluster will have some resources such as ecr, fargate, s3. And with my setup I will have 2 
Pulumi.dev.yaml
file under 
storage
& 
containers
. My questions will be should I keep ecr&fargate under same project of contaienrs or should I do this:
Copy code
├── infrastructure
│   ├── iac
│   │   ├── aws
│   │   │   ├── containers
│   │   │   │   ├── ecr
│   │   │   │   │   ├── index.ts
│   │   │   │   │   ├── package.json
│   │   │   │   │   ├── Pulumi.yaml
│   │   │   │   │   ├── index.ts
│   │   │   │   ├── fargate
│   │   │   │   │   ├── index.ts
│   │   │   │   │   ├── package.json
│   │   │   │   │   ├── Pulumi.yaml
│   │   │   │   │   ├── index.ts
│   │   │   ├── storage
│   │   │   │   ├── s3
│   │   │   │   │   ├── index.ts
│   │   │   │   │   ├── package.json
│   │   │   │   │   ├── Pulumi.yaml
│   │   │   │   │   ├── index.ts
d
Up to you—we don’t keep s3 and fargate in separate projects. We organize our projects by “workload” or actual use case. Like are these resources for a web app?
If so, I’d have one project called “web app” and put all the resources needed to stand up that web app’s infrastructure in there. The stacks would be for the production/staging/qa environments of that web app
Does that make sense?
Copy code
For now, Zephyr has decided to go with a monorepo approach—a single repository that contains both their application code and the Pulumi code to manage the infrastructure resources. All of the resources are defined in a single Pulumi project, with multiple stacks that correspond to development and production environments. Over the course of this series, we'll see how Zephyr's use of Pulumi changes as Zephyr grows and their application evolves.
b
I am worry about organize our projects by “workload”. what if you have many workload need for example S3? frontend, backend. do you create 2 S3 resource? or you doing the cross workload resource provision?(which I worry it will be messy?)
d
Yeah I’d say you definitely create a separate s3 resource for each workload
Each workload’s infra should be isolated
Otherwise you end up with entanglement and resources you cannot alter or refactor as needs change over time
b
I see, thanks. The copy here event mean copying the index.ts resource code right?
There can be 2 set of S3 generation code of index.tx file that live in different workload.
d
yeah you could create a library of repeated patterns that you use that you just import into each project
you could use https://www.pulumi.com/docs/intro/concepts/resources/components/ to group sets of resources too
but if you have:
Copy code
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const bucket = new aws.s3.Bucket("bucket", {
    acl: "private",
    tags: {
        Environment: "Dev",
        Name: "My bucket",
    },
});
… I think it’s ok to repeat declarative code like this sometimes
Like what is in your 
s3/index.ts
?
b
Got it. thanks! is there any github repo of this setup I can look at?
d
hmm our code is in a private repo… I’m seeing if there’s anything I can find out there
Have you looked through https://github.com/pulumi/examples
b
Thanks
Open in Slack
PreviousNext
Powered by