https://pulumi.com logo
Join Slack
Powered by
This message was deleted.
# general
s
This message was deleted.
l
a (Python) program with a bunch of imported resources.
I’m not sure I understand. If you have a program with a bunch of imported resources, these resources should then be in your state file and 
pulumi up
shouldn’t create these a second time. Either your description of “imported resources” is not what we understand of being truly imported resources into the state file, or I am missing something about the current state of your setup.
s
Yeah, but I want to vary the the resource names, to create (at least) dozens of these.
What I want is to discover the remote resources matching the descriptions I've given. After that, maybe some need to be created and some updated, but that's a separate thing.
program <-> state <-> cloud, right? Well, if the state file is necessary, it's not really fair to say you just declare your resources in your IaC program, because you also need for them to be in the state file.
If I want to import dozens of stacks that look just like this, I apparently need to forge state files, or do something equally strange.
l
That is indeed the hard part of IaC: the resources which need to be imported might make sense to you because you know your program. But given the flat namespace AWS is, you could deploy 10s, even hundreds of differents stacks from the same program. If we don’t keep track of the resources set up by each stack in a corresponding state file, how would we know which resources we need to update or additionally create?
s
Easy. They match the spec.
If that's not enough, you need to tighten the spec. Tags are pretty good for that.
l
Tags are AWS specific. Pulumi’s engine needed a model which works for any technology. Whether it is Pulumi or another existing IaC tool, they all keep track of state whether it is visible to you or not.
s
Fair, but there's no way to even try
Aside from this use case, it makes the state file a really dangerous single point of failure.
Having N stacks just make N anonymous resources seems really crazy to me. They'd at least have to be connected to something, right? Otherwise, 🤔 maybe you could just pick one at random?
l
That is why our Pulumi SaaS is a hosted and fault tolerant backend to manage state, with ACL, audit logging, etc.
s
Right, so the fix is to put all the eggs in a really, really safe basket?
Anyway, I'm not really worried about losing the state file, I'm worried about getting the state file corrupted.
I guess versioning could help there, but even trivial operations on 100+ version histories? Yeah, that's going to be pretty nasty.
l
Whether it is our SaaS, or a self-managed backend (e.g S3 with bucket object versioning), you can always get back to the last working version.
FYI, we keep a version of the complete state, not a version per resource.
s
Yeah, but that's not going to recognize the state of the remote provider, but that's not a thing that it does.
l
that is what 
pulumi refresh
is for.
e
There is an issue we have tracking this idea https://github.com/pulumi/pulumi/issues/6667 But as suggested in the ticket there isn't really a "good" way to do this.
s
Yes, it's a large-ish json file. It's editable, without something throwing a hissy-fit, which is nice
Ah, ok, how's this for a plan: 1. Write up, (or import, whatever,) the program, 2. Generate a blank-slate state file, where everything is represented, but id:s are trashed 3. 
pulumi refresh
e
That won't work because 
refresh
works via IDs
The point of refresh is to ask "I had resource X and Y that had attributes A and B, can you check those resources still exist and look like that"
s
With my previous employer, we wrote a service that deploys services, and it had some of these concepts and design issues as well. One of the things it did do very well was to discover and replace resources that had been fat-fingered by the proletariat*. (* "dress like teenagers, paid like doctors, expect to eat for free")
Yeah, that's what I'd like, to do, too. I just don't have the id:s for anything. I have a very good idea of what I expect things to look like, though.
e
discover and replace resources
Yeh that's only doable if it's safe to assume that Pulumi is fully managing the entire cloud account. Which is so rarely the case that we haven't put effort into supporting it.
s
It's safe enough if all the resources you manage are tagged
(Or, ofc, you can be quite sure some of way)
e
Not every cloud supports tags so we can't do that
s
Well, they need to have something, otherwise it would just be too chaotic
Or maybe it just is.. 😱
e
We are at the mercy of what aws/microsoft/google/oracle/everyone else decides to implement here 🤷
I'd suggest leaving your thoughts on https://github.com/pulumi/pulumi/issues/6667, it's a very interesting idea of we could work out a general way to support this, but for the time being us (and everyone other IaC tool in the market) need to use state files.
s
Yeah. It just strikes me as crazy that the easiest way to work with this is to load up the SDK, munge the cloud account, and forge a state file. (I still like the rest of it, so this is actually a real possibility)
e
Well don't forge a state file, just use the import machinery, that's what 
pulumi import
is for, to build a statefile for you from existing resources.
s
Hm. Sure. The annoying part is finding all the resources, anyway
Looking at my shell history, most of these are imported by name, so there's that, I guess.
o 0 O (Damn it, shell scum is what I'm trying to avoid, here)
e
Not much we can do to help with looking these resources up (we've got ideas of extending import to look up by attributes rather than ID) but at least to save some amount of shell-ing back and forth 
pulumi import
can take a JSON file with a list of all the resource:id pairs to import.
s
Yeah, it's something. Anyway, thanks for the support!
If there's anything I can give you it's probably the observation that nomenclature could be clarified a bit. In our tool/service, we used 'plan' and 'fact' to refer to objects describing desired resources and actual resources referred to.
👍 1
14 Views
Open in Slack
PreviousNext
Powered by