Hi all, am new to Pulumi and just getting started with evaluating a migration from Terraform to Pulumi. I know there is a #C01PF3E1B8V channel so will post in there for any any noobie gotchas but wondered if there were any posts, either on here or the blog, that share peoples experiences and architecture tips? Would be great to read a “if only I knew then what I know now” post or “what I would do differently” post. Thanks.

I too have started my journey into pulumi and having been a long term user of TF and using it in extremely large infras, I dont find Pulumi that great. The whole outputs being a promise really triggers a lot of difficulty when trying to do things which rely on outputs from one thing to get manipulated(or not) and then trigger creations of new things. There are ways to force it to work through use of the apply function, but its not recommended by pulumi. Honestly I cant recommend pulumi from TF unless the inability to even read TF is something you're looking to solve. I just joined a team of 4 devs w/ myself being sole devops. None of the devs know TF, but they can at least read Python, so I decided to go with Pulumi so they can at least be semi capable of following what is happening. We're also not migrating from TF to pulumi. Knowing what I now now, wouldnt recommend doing that. For part of your PoC, I'd suggest creating a transit gateway and associating two different VPCs from different accounts. This is what I'm going through and I've found it to not be quite as straight forward as I feel it would have been in TF.

Thanks for the feedback Ryan. Very interesting and seems like a couple of big gotcha’s. I believe other tools also suffer a little bit from the promise “issue”. I’ll look into the example you suggested and see what happens. I wonder if any of the Pulumi team could comment on the above as I’m sure you cannot be the first person to raise this. Does this mean that if you have 2 resources and the 2nd resource requires the arn/id of the first resource it cannot work as we don’t have access to the arn?

not exactly. once again i'm new to pulumi so perhaps i've missed something. But take the following example:

a = create_a_thing()
b = create_b_thing(a.id)

would seem to work. But if you were to want to create a list of things, this would be a bit more cumbersome. For example if you were retrieving the list from a source (say an AWS secret), then because that value would be of the Output type, and not a string, you cant simply do a for loop and iterate across the list

l = get_secret('key').value
for item in l:
  do_more(item)

wouldnt work Once again i'd encourage doing more than just a simple poc. My first day of pulumi was great as I was creating singular objects and passing outputs directly to inputs. It becomes more troublesome when you start looking at doing more advanced things. I think the transit gw is a good example for seeing some of the caveats as you have to do things like get the list of route tables in order to associate a new route into them pointing to the tgw attachment across two different accounts.

Super. Thanks Ryan. We are using AWS infra so will give the TGW a try. I would say though that even with TF it ain’t that easy setting it up across accounts and VPCs.

oh agreed, but i'm finding the req of dealing with Outputs and them not being the same as a str a headache. At least w/ TF a str is a str and yo dont need to worry about it as TF dependency map will figure out things and make sure its available to be used downstream when needed. GL and if you figure things out in a better fashion with pulumi than I seem to have so far, LMK

Yes, the output of string and string stuff is a big headache me. But if you can do infra components in a async way, you can split them into different stacks, and call stack references in the code.

there are some helpers that can help smooth out working with Outputs: https://www.pulumi.com/docs/reference/pkg/python/pulumi/#pulumi.Output for instance, Output.format is equivalent to str.format, but is Output aware. there are similar helpers for concatenation and json. I find these are usually enough to not need to reach for apply as often

format() still returns an output, not a str

right, but it knows how to deal with Output arguments

• If you have an option of doing something with a stackref vs not, opt for not using the stackref. Once added it's hard to refactor it away, and the lookups are tricky. • Not all resource providers are equal. Those built with the lightweight TF bridge can be much more kludgy, for example showing constant diffs that are actually just ordering issues of order-irrelevant lists. • don't put stack names/etc in resource_name values because it'll make renaming things later harder • stack export && vim && stack import are sadly a thing you'll need from time to time.

Thank you all so much for the comments so far. Provides a great insight into what people have learned to date and those gotchas you only get after everything is live!

I am a long-time user of TF and often build complex modules/abstractions for users to simplify their use-case. I like Pulumi more, because it is easier to build a good abstraction with reusable classes and functions + with programming languages ecosystem (e.g. pydantic for python), you can express input validation as complex as you want. I believe multiple TF deployments from a single module kinda suck. there is so much files repetition (e.g. variables.tf everywhere), so you reach out to other tools like terragrunt to make it more sane and they have their own tradeoffs. it is much integrated process with pulumi. I also think cross-lang Pulumi components (e.g. python or TS) and then users consume it in yaml/no-code is very powerful and recent idp announcement contributes to this. I am currently building a VPC abstraction in Pulumi that implements every AWS VPC feature and I am excited to get it done and released. then users can then leverage the power of what I build with a simple yaml input. I do agree that working with pulumi outputs is gotcha. To be fair, it just can't be a string - the value of an output is not known before the resource is created. it works with TF, because that it was designed with that use-case in mind, but porting to a general programming language that behavior is not possible. Though I am curious if it is theoretically possible to rework pulumi outputs to support awaitable promises and retrieve values this way, e.g.

await vpc.id

Great feedback @famous-ambulance-44173 . I too want to make as many consumable components as possible so it’s easy for users to consume and the yaml for user input as the abstraction layer is a good one. Any thoughts on my stack question? Do you approach everything in one stack or many? If many, how do you decide what should be in each one?

We use multiple stacks. The infra-oriented stacks are supposed to be stable, like defining dns, vpc, ecs, ecr, etc. The disposable stacks are mostly micro services, which update very frequently.

I don't think there is one size fits all with stacks as well as cross-stack references. it is the same-ish problem with terraform / cfn / etc personally I don't like micro stacks and like to think about them in terms of lifecycle and team ownership. for example, I have a single stack for networking vpc component which includes vpc, rt, natgw, gateways, attachments, dns settings, flow logs and endpoints. it has its own lifecycle and can be potentially managed by networking team then another stack for application infra, e.g. ec2+sg+load balancer + cloudfront (if relevant). if it is eks it would probably be separate for the cluster and the application itself. I know that some other folks like to make it more granular. cross-stack reference, e.g. how to pass subnet ids from networking stack to app stack - I don't have a definite answer here. it can be done based on tags, ssm or pulumi stack references. stack references is the "easiest" but only works in very IaC mature orgs where no infra is managed outside of IaC and the contracts are enforced. probably I wouldn't start with that and first focus on some metadata so you can lookup ids inside the module I'd suggest to just start and build something resembling real world infra, then you can iterate upon it later.

Thanks @famous-ambulance-44173 and @adamant-lawyer-19698 👍