Hi. (I've tried googling, but I felt I need to look for help here, so perhaps docs need to be clearer on this.) I have a (Python) program with a bunch of imported (AWS) resources, but my plan is to make this a template, so I can create and destroy these at will. The problem I have is that if I start with a clean slate, (no stack state file) then • I can't use

pulumi stack import

, because I don't have a stack state file like that, • I can't use

pulumi up

, because that wants to create things that already exist, • I can't use

pulumi refresh

, because it (apparently) doesn't care what I've declared, only what's in the stack state file. • I refuse to believe that the idea is to manually import each resource again, like a peasant. I should also point out that it would be nice if nomenclature and docs could be clearer on what's a resource in a program, in a stack state and with the actual cloud provider, at least two of these are very different, right?

a (Python) program with a bunch of imported resources.

I’m not sure I understand. If you have a program with a bunch of imported resources, these resources should then be in your state file and

pulumi up

shouldn’t create these a second time. Either your description of “imported resources” is not what we understand of being truly imported resources into the state file, or I am missing something about the current state of your setup.

Yeah, but I want to vary the the resource names, to create (at least) dozens of these.

That is indeed the hard part of IaC: the resources which need to be imported might make sense to you because you know your program. But given the flat namespace AWS is, you could deploy 10s, even hundreds of differents stacks from the same program. If we don’t keep track of the resources set up by each stack in a corresponding state file, how would we know which resources we need to update or additionally create?

Easy. They match the spec.

Tags are AWS specific. Pulumi’s engine needed a model which works for any technology. Whether it is Pulumi or another existing IaC tool, they all keep track of state whether it is visible to you or not.

Fair, but there's no way to even try

That is why our Pulumi SaaS is a hosted and fault tolerant backend to manage state, with ACL, audit logging, etc.

Right, so the fix is to put all the eggs in a really, really safe basket?

Whether it is our SaaS, or a self-managed backend (e.g S3 with bucket object versioning), you can always get back to the last working version.

Yeah, but that's not going to recognize the state of the remote provider, but that's not a thing that it does.

that is what

pulumi refresh

is for.

There is an issue we have tracking this idea https://github.com/pulumi/pulumi/issues/6667 But as suggested in the ticket there isn't really a "good" way to do this.

Yes, it's a large-ish json file. It's editable, without something throwing a hissy-fit, which is nice

That won't work because

refresh

works via IDs

With my previous employer, we wrote a service that deploys services, and it had some of these concepts and design issues as well. One of the things it did do very well was to discover and replace resources that had been fat-fingered by the proletariat*. (* "dress like teenagers, paid like doctors, expect to eat for free")

discover and replace resources

Yeh that's only doable if it's safe to assume that Pulumi is fully managing the entire cloud account. Which is so rarely the case that we haven't put effort into supporting it.

It's safe enough if all the resources you manage are tagged

Not every cloud supports tags so we can't do that

Well, they need to have something, otherwise it would just be too chaotic

We are at the mercy of what aws/microsoft/google/oracle/everyone else decides to implement here 🤷

Yeah. It just strikes me as crazy that the easiest way to work with this is to load up the SDK, munge the cloud account, and forge a state file. (I still like the rest of it, so this is actually a real possibility)

Well don't forge a state file, just use the import machinery, that's what

pulumi import

is for, to build a statefile for you from existing resources.

Hm. Sure. The annoying part is finding all the resources, anyway

Not much we can do to help with looking these resources up (we've got ideas of extending import to look up by attributes rather than ID) but at least to save some amount of shell-ing back and forth

pulumi import

can take a JSON file with a list of all the resource:id pairs to import.

Yeah, it's something. Anyway, thanks for the support!