https://pulumi.com logo
Powered by
Title
h

hallowed-australia-10473

11/30/2022, 9:47 PM
I have a stack on DigitalOcean with three components: a domain, a bucket, and a droplet. I would like to destroy and rebuild the droplet, but leave the domain and bucket alone. What’s the best practices way of doing this? I’m hoping for something more elegant than “just manually delete the droplet then run pulumi up” 🙂
l

little-cartoon-10569

11/30/2022, 10:02 PM
If this is a one-time thing, it can be as simple as commenting out the code for the droplet and running up, then commenting it back in. You can also change its Pulumi name, that will force Pulumi to destroy and recreate. If this is likely to happen repeatedly, the best approach would be to split your project in two, based on the deployment cycle. The domain and bucket would be in the base project, and export whatever properties the droplet needs. And the droplet would be in its own project, and get the required properties via StackReferences. Then you can destroy/up the droplet project any time you need.
h

hallowed-australia-10473

11/30/2022, 10:07 PM
It turns out the address is specific to the droplet unless I do some other shenanigans — but if it were an elastic IP or something, it’d be as simple as making two directories, splitting the Pulumi.yaml across them appropriately to get two projects, and making the droplet project depend on the base project. I will definitely look into that approach.
l

little-cartoon-10569

11/30/2022, 10:20 PM
So the domain is dependent on the droplet and needs to be updated when the droplet is? Then keep it in the same project as the droplet.
h

hallowed-australia-10473

11/30/2022, 10:22 PM
For now, but if I use reserved IPs (I think that’s their name for elastic IPs) I can keep the same IP and just switch between droplets, which makes replacing the droplet much easier as you described above.
l

little-cartoon-10569

11/30/2022, 11:00 PM
And DigitalOcean domains can't refer to droplets, they need to refer to their IP addresses? That's not awesome.
h

hallowed-australia-10473

11/30/2022, 11:01 PM
It is more probable to assume I am not taking full advantage of their provider than to assume defects in the provider, heh.
https://www.pulumi.com/registry/packages/digitalocean/api-docs/reservedipassignment/ shows what looks like a reasonable approach: create the reserved IP, create the droplet, assign the reserved IP to the droplet
The domain and reserved IP could go in the base project with the bucket. The droplet project would perform the assignment as necessary.
l

little-cartoon-10569

11/30/2022, 11:03 PM
You can't modify a resource after it's created. If you create the domain in one project, then you can't update it in the other.
h

hallowed-australia-10473

11/30/2022, 11:04 PM
Right — so the domain <-> reserved IP connection would be made in the base project
foo.example.com responding to 172.16.0.1
droplet-foo would be created, and the reserved IP 172.16.0.1 would be assigned to it
Rebuild the droplet project and create droplet-bar, the foo droplet is removed and destroyed, the bar droplet gets assigned
The assignment is a resource in the droplet project
Unless I am completely misunderstanding this
l

little-cartoon-10569

11/30/2022, 11:10 PM
It does look weird. Is there no way to swap between which droplet is exposed via the domain? Maybe a load balancer, or a droplet version?
h

hallowed-australia-10473

11/30/2022, 11:10 PM
Most of what I do is in Kubernetes where the ingress handles a lot of this
l

little-cartoon-10569

11/30/2022, 11:10 PM
Ah there is a load balancer. Use that to make your life might easier.
https://www.pulumi.com/registry/packages/digitalocean/api-docs/loadbalancer/
When you replace your droplet, the LB will look after it. The domain should point at the LB.
h

hallowed-australia-10473

11/30/2022, 11:11 PM
Okay, so the LB acts like the reserved IP. That works. Wonder which one costs more per month? Heh.
l

little-cartoon-10569

11/30/2022, 11:12 PM
Yea, cost is the perpetual scourge to the dream architecture...
h

hallowed-australia-10473

11/30/2022, 11:13 PM
Oooh, reserved IPs are free when attached to a droplet, $5/mo if not (because IPv4)
Load balancers are $12/mo
And since I’m not near the volume or capacity for a LB, I’m gonna go with the cheap-to-free option
l

little-cartoon-10569

11/30/2022, 11:16 PM
Right. Well for now then, you're doing to have to destroy the domain any time you destroy the droplet.
So put the bucket in a different project. Best you can do.
h

hallowed-australia-10473

11/30/2022, 11:20 PM
Wait, you don’t think I can put the domain and the reserved IP and the bucket in the base project, and then the droplet and assignment in the droplet project?
w

worried-rain-74420

12/01/2022, 12:44 AM
I like to protect the domain and bucket, then run with 
--exclude-protected
h

hallowed-australia-10473

12/01/2022, 12:45 AM
Yeah, right now I’m seeing idempotency failures — running pulumi up twice on the same code wants to destroy and recreate the bucket
Protecting things sounds wise
l

little-cartoon-10569

12/01/2022, 3:15 AM
running pulumi up twice on the same code wants to destroy and recreate the bucket
What does Pulumi say is the reason for this? This happens only because the name of the resource is changing, or some property that triggers replacement is changing.
What type of resource is the bucket? Is it a DigitalOcean volume? I see that description is a triggering property.. that's kinda weird.
h

hallowed-australia-10473

12/02/2022, 1:37 AM
It was a spaces bucket. Yesterday I deleted the entire stack and then manually cleaned everything up so I could try again.
Pulumi didn’t say why it wanted to destroy and recreate the bucket. I protected the bucket in my code and reran pulumi up. Here’s what I saw: 
View Live: <https://app.pulumi.com/mathuin/pocket-lightning/prod/previews/bc5e1ae0-849f-495b-930c-cf8d63f46bcc>

     Type                                Name                   Plan     Info
     pulumi:pulumi:Stack                 pocket-lightning-prod           
     └─ digitalocean:index:SpacesBucket  pl-bucket                       1 error


Diagnostics:
  digitalocean:index:SpacesBucket (pl-bucket):
    error: unable to replace resource "urn:pulumi:prod::pocket-lightning::digitalocean:index/spacesBucket:SpacesBucket::pl-bucket"
    as it is currently marked for protection. To unprotect the resource, remove the `protect` flag from the resource in your Pulumi program and run `pulumi up`
This is the code that creates the bucket:
bucket = do.SpacesBucket("pl-bucket", region=region, opts=pulumi.ResourceOptions(protect=True))
region = config.get("region") or "sfo2"
is where region is defined
w

worried-rain-74420

12/03/2022, 11:14 PM
@hallowed-australia-10473 I believe you are missing the 
--exclude-protected
CLI flag. If I'm right, try: 
pulumi destroy --exclude-protected
h

hallowed-australia-10473

12/03/2022, 11:14 PM
I’m not destroying, I’m rerunning up. There’s no exclude protected for up
w

worried-rain-74420

12/03/2022, 11:15 PM
Oh, I gotcha, the issue you're having is that your protected resource now needs to be replaced, I see.
h

hallowed-australia-10473

12/03/2022, 11:15 PM
Well, Pulumi thinks so, and I can’t figure out why
w

worried-rain-74420

12/03/2022, 11:21 PM
Hmm... something that's odd to me is that the bucket name isn't replace-on-change in the DO schema. You're not getting a rich diff during preview that shows the difference between the old and new resource?
Could you screencap your terminal output by chance?
h

hallowed-australia-10473

12/03/2022, 11:22 PM
I am not getting a rich diff during preview, even when I ask for one
I posted a screencap about 29 minutes ago
Here’s a fresh one: 
~/g/pocket-lightning ❯❯❯ pulumi up                          reserved-ips ✚ ✱ ◼
Previewing update (prod)

View Live: <https://app.pulumi.com/mathuin/pocket-lightning/prod/previews/c21bd27c-45f7-40a1-b2d4-191db031c3e3>

     Type                                Name                   Plan     Info
     pulumi:pulumi:Stack                 pocket-lightning-prod
     └─ digitalocean:index:SpacesBucket  pl-bucket                       1 erro


Diagnostics:
  digitalocean:index:SpacesBucket (pl-bucket):
    error: unable to replace resource "urn:pulumi:prod::pocket-lightning::digitalocean:index/spacesBucket:SpacesBucket::pl-bucket"
    as it is currently marked for protection. To unprotect the resource, remove the `protect` flag from the resource in your Pulumi program and run `pulumi up`
w

worried-rain-74420

12/03/2022, 11:25 PM
Sure, I wasnt sure if the text you had posted was the full terminal output or if it was a sample.
Thanks for reposting.
h

hallowed-australia-10473

12/03/2022, 11:25 PM
~/g/pocket-lightning ❯❯❯ pulumi up --diff               ✘ 1 reserved-ips ✚ ✱ ◼
Previewing update (prod)

View Live: <https://app.pulumi.com/mathuin/pocket-lightning/prod/previews/2091b01e-1306-4d95-9f66-cb9b74f5bb86>

  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:prod::pocket-lightning::pulumi:pulumi:Stack::pocket-lightning-prod]
error: unable to replace resource "urn:pulumi:prod::pocket-lightning::digitalocean:index/spacesBucket:SpacesBucket::pl-bucket"
as it is currently marked for protection. To unprotect the resource, remove the `protect` flag from the resource in your Pulumi program and run `pulumi up`
Resources:
    2 unchanged
That’s with the diff argument
w

worried-rain-74420

12/03/2022, 11:25 PM
Oh, what's the output of 
pulumi preview
?
h

hallowed-australia-10473

12/03/2022, 11:26 PM
~/g/pocket-lightning ❯❯❯ pulumi preview               ✘ 255 reserved-ips ✚ ✱ ◼
Previewing update (prod)

View Live: <https://app.pulumi.com/mathuin/pocket-lightning/prod/previews/8e643fe8-b1fb-4a25-9457-5a389f7df783>

     Type                                Name                   Plan     Info
     pulumi:pulumi:Stack                 pocket-lightning-prod
     └─ digitalocean:index:SpacesBucket  pl-bucket                       1 erro


Diagnostics:
  digitalocean:index:SpacesBucket (pl-bucket):
    error: unable to replace resource "urn:pulumi:prod::pocket-lightning::digitalocean:index/spacesBucket:SpacesBucket::pl-bucket"
    as it is currently marked for protection. To unprotect the resource, remove the `protect` flag from the resource in your Pulumi program and run `pulumi up
`
w

worried-rain-74420

12/03/2022, 11:26 PM
Man, that message really is unhelped! I should open an issue if there isn't one already.
Well, here's my next pitch to try to debug this:
Try changing your code to: 
bucket = do.SpacesBucket("pl-bucket", region=region, opts=pulumi.ResourceOptions(protect=False))
...and running 
pulumi preview
, just so we can see what Pulumi thinks is supposed to change.
h

hallowed-australia-10473

12/03/2022, 11:29 PM
~/g/pocket-lightning ❯❯❯ pulumi preview               ✘ 255 reserved-ips ✚ ✱ ◼
Previewing update (prod)

View Live: <https://app.pulumi.com/mathuin/pocket-lightning/prod/previews/b8c217ab-59de-41fb-bd79-97fb66a61b0a>

     Type                                        Name                   Plan
     pulumi:pulumi:Stack                         pocket-lightning-prod
 +-  ├─ digitalocean:index:SpacesBucket          pl-bucket              replace
 +-  ├─ digitalocean:index:ReservedIpAssignment  pl-ripa                replace
 +-  └─ digitalocean:index:Droplet               pl-droplet             replace


Outputs:
  ~ bucket_name: "pl-bucket-66ee4ad" => "pl-bucket-4077b39"
  ~ droplet_ip : "138.197.208.46" => output<string>
  - reserved_IP: "138.68.36.145"
  + reserved_ip: "138.68.36.145"

Resources:
    +-3 to replace
    3 unchanged
So this would trash the bucket and the droplet with no change in the config files at all? (IP -> ip was the “minor” non-impacting change I added)
Oooh actually the user data for the droplet has the bucket name
w

worried-rain-74420

12/03/2022, 11:29 PM
Aha! It looks like... the bucket name, which is autogenerated, does indeed trigger replace-on-change.
h

hallowed-australia-10473

12/03/2022, 11:30 PM
replace-on-change when the change is the hash seems a little unexpected
w

worried-rain-74420

12/03/2022, 11:31 PM
replace-on-change when the change is the hash seems a little unexpected
Yeah, that's a known ergonomic issue with the way we do auto-naming. There's work underway to revamp auto-naming but I'm not sure the current status.
h

hallowed-australia-10473

12/03/2022, 11:32 PM
I’m now wondering why the domain and reserved IP are not affected
# create the reserved IP
rip = do.ReservedIp("pl-rip", region=region, opts=pulumi.ResourceOptions(protect=True))
pulumi.export("reserved_ip", rip.ip_address)

# create the domain
domain = do.Domain(
    "pl-domain",
    name=hostname,
    ip_address=rip.ip_address,
    opts=pulumi.ResourceOptions(protect=True),
)
pulumi.export("domain_name", domain.name)
w

worried-rain-74420

12/03/2022, 11:36 PM
As a general rule, to work around this difficulty with "replace-on-change" and autonaming, if there's a 
name
field for your particular resource, then you should try to specify it. If you're making a one-off resource, then you can hardcode it. If your resource is part of a component, you should try to incorporate the component's name (as provided by the component constructor). You can template the name into a string to do this. So if this bucket was in a component named 
MyDOComponent
you might want to name the resource to be 
MyDOComponent-bucket
vis a vis 
"{0}-{1}".format(component_name, "bucket")
Sorry about the ergonomics there. I'd say this idiosyncrasy is more than a "papercut" due to Pulumi's autonaming...
h

hallowed-australia-10473

12/03/2022, 11:38 PM
Is there a way to turn off replace-on-change for the bucket?
w

worried-rain-74420

12/03/2022, 11:38 PM
If we've correctly diagnosed the issue, then there's a bug in the DO schema documentation for 
SpacesBucket
where 
name
should be 
replace-on-change
.
No, that's a rule determined by the provider. ReplaceOnChange means "hey this thing is not longer fundamentally the same as it was if you change this property." In the case of a bucket, it's URI is determined by its name. So if you create a bucket 
jack-twilley
, and try to change it's name, you're fundamentally creating a new bucket according to the cloud provider (because you're changing what is expected to be a permanent URI for the bucket).
h

hallowed-australia-10473

12/03/2022, 11:41 PM
There is a plus to the random tag on the end of the name: it can take a while for a bucket destroy to finish, and I can pulumi up immediately after pulumi destroy because it’ll make a new name for the bucket.
Not sure if that’s worth this downside
w

worried-rain-74420

12/03/2022, 11:42 PM
Put another way, if you had, say, a 
CreditCard
resource, and you wanted to change the 
CreditCard.cc_number
field, your cloud provider would be within their rights to say "hey man this resource isn't editable! You've got to delete the credit card and make a new one!" It's the same idea with buckets and their names. Or, like "database cluster and engine type".
There is a plus to the random tag on the end of the name: it can take a while for a bucket destroy to finish, and I can pulumi up immediately after pulumi destroy because it’ll make a new name for the bucket.
Hmm... yeah I'm not sure I have any insight to offer; it sounds like a personal choice. I personally don't want my infrastructure to have nondeterminism -- I want to minimize the randomness as much as possible to make it as reproducible as possible. But if that's something you like for the purpose of performance an convenience, I won't yuck your yum 😸
h

hallowed-australia-10473

12/03/2022, 11:46 PM
I’m past the part of my dev cycle where I destroy/up very often so I’m okay moving past that and into the warm comforting arms of determinism
I guess I figured it made a name and would keep that name if nothing else changes, but there’s nothing else except a name for a bucket and how would it necessarily know that the old name was good
And “replace” isn’t quite what it sounds like for a bucket, or a database — it’s not like the new bucket gets created and the files all get copied over from the old one, right?
w

worried-rain-74420

12/03/2022, 11:49 PM
And “replace” isn’t quite what it sounds like for a bucket, or a database — it’s not like the new bucket gets created and the files all get copied over from the old one, right?
Correct! It's not a "move", it destroys the old and creates a fresh one. Which is exactly the opposite of what happens when you change a non-
replace-on-change
property. Conceptually, that's more like a "mv", in that it updates the properties of a resource without needing to destroy it and create a new one.
h

hallowed-australia-10473

12/03/2022, 11:50 PM
I think I’m going to set the bucket name and embed the stack name in it — I already embed the project name sorta so it’s not that big a stretch
w

worried-rain-74420

12/03/2022, 11:50 PM
For example, you can change the maintenance windows for a database cluster without having to replace the cluster with a new instance.
h

hallowed-australia-10473

12/03/2022, 11:51 PM
Still don’t know if I’d call it a bug in DO or in my understanding
Now to destroy that stuff I just made hard to destroy, haha
… and now I do think I have a bug, or at least an ugly wart. 
pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:prod::pocket-lightning::pulumi:pulumi:Stack::pocket-lightning-prod]
    +-digitalocean:index/spacesBucket:SpacesBucket: (replace) 🔓
        [id=pl-bucket-prod]
        [urn=urn:pulumi:prod::pocket-lightning::digitalocean:index/spacesBucket:SpacesBucket::pl-bucket]
        [provider=urn:pulumi:prod::pocket-lightning::pulumi:providers:digitalocean::default_4_16_0::1533d91c-1447-40ca-ae1c-142b5698f63b]
      ~ region: "sfo2-zg02" => "sfo2"
What the heck is that on the region?
Another day’s pain
#generalJoin Slack
Powered by