This message was deleted.
# general
s
This message was deleted.
b
There is definitely a way to go about this, although it isn't as straightforward as we would like. For example, if you create a checkpoint with all the right URNs and IDs, then
pulumi refresh
will read everything back. We are discussing an
import
command that automates this: https://github.com/pulumi/pulumi/issues/1635. In the absence, @white-balloon-205 or @microscopic-florist-22719 may have tips.
d
I don’t yet have a mental model of the state internals, so attempting this makes manually me a bit nervous at this stage
If I had to imagine how I’d want this to work, I’d want some kind of interactive refresh mode
where it’s like “hey, this resource doesn’t exist - do you know it’s id/arn/etc?”
ie i’d want to iteratively “preview” and “refresh” until i was confident “update” wouldn’t screw me
maybe that’s what your import thing is saying?
also, how do i see my checkpoint file? is that just
pulumi stack export
?
b
Yes,
pulumi stack export
will dump it to stdout, and
pulumi stack import
will re-read it from stdin. (Although we track history, we don't have great support for going back in time to get historical checkpoints, so I'd recommend backing things up before manual edits.)
Agree, interactive mode would be pretty killer.
d
i’m usually not a fan of interactive things, as they make automation harder, but i think it is appropriate here, since this is inherently a one-off task, not something to be automated
b
Just so I understand, you're saying you'd want to write your program, then instead of
pulumi up
, you'd say
pulumi import
and it'd compute the resources and then let you pick resources from your current AWS account to form the checkpoint state? Basically, adopting them one at a time?
d
yeaaaaah - i think so … this is just a random idea
b
I like it 🙂
d
so i basically made this pulumi script by copying the existing manual deployment
so in theory, there should only be minor, inconsequential differences
b
The nice thing about this is that it could warn you and/or help you reconcile if your program couldn't possibly have produced the state you're adopting.
d
the reason my mind went to interactive is b/c you have a lot of info to drive the process - like you know resource types and the like, so you can provide good choices to match things up
and “screw it, just make a new one” is a valid choice as well
b
Sort of like an interactive merge (take theirs, take mine, etc)
d
yeah - i think you need to support “merge markers” of some kind in your snapshot files too
b/c you want to capture the fact that you made a merge choice, so that when the next update happens, the right thing occurs
not sure what that looks like in practice tho
with respect to the manual approach for now - what do i need to know? i see this big scary json file - what values do i actually need to capture?
b
I like this approach better than what we'd previously discussed, which was more of a "start from nothing and start importing". This is difficult, but with the interactive, program-first standpoint, at least -- as you say -- you know what you're trying to find (types, etc). So it actually seems feasible. /cc @microscopic-florist-22719 and @lemon-spoon-91807 as I think be keen on this
In theory if you just have the URNs and IDs, then you can do a
pulumi refresh
and it'll use those values to look up the resources and populate all of the state for them.
d
just testing my mental model: urn == arn on aws, yes?
and id = what exactly?
b
URN is assigned by Pulumi, actually.
d
oh, hm
b
ID is the AWS identifier, often the ARN, but it depends on the resource type.
For example:
d
got it - how would i figure out URNs then for a manual reconciliation?
b
This is an S3 Bucket:
Copy code
{
                "urn": "urn:pulumi:joe-thumbnailer::video-thumbnailer::cloud:bucket:Bucket$aws:s3/bucket:Bucket::bucket",
                "custom": true,
                "id": "bucket-e2ed56f",
                "type": "aws:s3/bucket:Bucket",
                "inputs": {
                    "acl": "private",
                    "bucket": "bucket-e2ed56f",
                    "forceDestroy": true,
                    "serverSideEncryptionConfiguration": {
                        "rule": {
                            "applyServerSideEncryptionByDefault": {
                                "sseAlgorithm": "AES256"
                            }
                        }
                    }
                },
                "outputs": {
                    "accelerationStatus": "",
                    "acl": "private",
                    "arn": "arn:aws:s3:::bucket-e2ed56f",
                    "bucket": "bucket-e2ed56f",
                    "bucketDomainName": "<http://bucket-e2ed56f.s3.amazonaws.com|bucket-e2ed56f.s3.amazonaws.com>",
                    "bucketRegionalDomainName": "<http://bucket-e2ed56f.s3.us-west-2.amazonaws.com|bucket-e2ed56f.s3.us-west-2.amazonaws.com>",
                    "corsRules": [],
                    "forceDestroy": true,
                    "hostedZoneId": "Z3BJ6K6RIION7M",
                    "id": "bucket-e2ed56f",
                    "loggings": [],
                    "region": "us-west-2",
                    "requestPayer": "BucketOwner",
                    "serverSideEncryptionConfiguration": {
                        "rule": {
                            "applyServerSideEncryptionByDefault": {
                                "kmsMasterKeyId": "",
                                "sseAlgorithm": "AES256"
                            }
                        }
                    },
                    "tags": {},
                    "versioning": {
                        "enabled": false,
                        "mfaDelete": false
                    }
                },
                "parent": "urn:pulumi:joe-thumbnailer::video-thumbnailer::cloud:bucket:Bucket::bucket",
                "dependencies": [],
                "initErrors": null,
                "provider": "urn:pulumi:joe-thumbnailer::video-thumbnailer::pulumi:providers:aws::default::5532a60a-758b-4589-bddb-98f4670e6fb9"
            },
If I just trimmed that to
Copy code
{
                "urn": "urn:pulumi:joe-thumbnailer::video-thumbnailer::cloud:bucket:Bucket$aws:s3/bucket:Bucket::bucket",
                "custom": true,
                "id": "bucket-e2ed56f",
                "type": "aws:s3/bucket:Bucket",
                "inputs": {
                },
                "outputs": {
                },
                "parent": "urn:pulumi:joe-thumbnailer::video-thumbnailer::cloud:bucket:Bucket::bucket",
                "dependencies": [],
                "initErrors": null,
                "provider": "urn:pulumi:joe-thumbnailer::video-thumbnailer::pulumi:providers:aws::default::5532a60a-758b-4589-bddb-98f4670e6fb9"
            },
And ran
pulumi refresh
, the properties would get read back from my actual AWS account.
So, TL;DR, this is way harder than it should be 😞
Maybe @white-balloon-205 or @microscopic-florist-22719 will have a better idea.
d
hm yeah - pretty cumbersome
gotta run, but happy to discuss this more soon. going to try to make the switchover of prod to pulumi sometime this week probably
mainly b/c i now have my first interesting lambda function that i don’t feel like managing myself 🙂
b
Nice! 👍
Let us chat and hopefully we can come up with something.
d
Thanks