Hi! I'm new here (especially to slack) but have been planning to use Pulumi for the launch of my project for over a year and have gotten into it very seriously recently after refreshing on concepts with the continuous delivery and infrastructure as code books. I actually started with a homebrew solution to get stuff operational and SSH in and set stuff up with the digitalocean API directly, then tried terraform, then discovered and went for the pulumi automation API and have been playing around with it and pulumi for weeks now, and I gotta say I absolutely love it and am at the point where I've achieved an ease and separation of concerns typical of good application code (attaching earlier screenshot for reference of the high point I've experienced recently with this). This leads me to my problem... and I wouldn't ask here unless I was desperate. I'm using pulumi and doing infrastructure as code because I want to be able to deploy manage servers in a largely immutable fashion, e.g. if I have some files on my computer I want hosted on (maybe a contrived example given that S3 buckets can be used) but that I want hosted on NGINX somewhere, the ideal workflow would be to create a server, upload them, assign a floating IP to the server and use that to connect, then when I change the files, I'd also want this immutable workflow where I create another server, upload all of the files over SSH (separately from the creation of the droplet in the lifecycle) and only when all of them are there switch to the new server. Although this particular example is contrived, I was really looking towards writing components like this and to incorporate some sort of smoke tests. Things I tried include: 1) Trying to get a ComponentResource which includes my droplet replaced, but when I did this replace_on_changes didn't work for my input property and it did not get replaced, just a change detected iirc. 2) I also tried tainting the component resource manually and getting it replaced with ``pulumi state taint``but it changed none of the children. 3) I tried adding a local command to my component, having it depend on the droplet and having it fail, and I know that failing dependent resources won't change anything, I did this in isolation, see screenshot 3. This also leads me to an adjacent but equally relevant issue 1) What am I supposed to do If I want to make a component where I take the name of a docker image to install and port to publish, and then have the encapsulated droplet get redeployed/replaced with a new one? If the name if the input to the component changes then all of the internal components stay the same especially if I'm setting them up with a command, I've considered making a hash of the inputs and changing the name of the droplets for this but haven't thought far enough ahead about the implications. Any help would be appreciated, even if it's that Pulumi doesn't support this use case in any capacity, or if it requires something more complex, I'm not above getting my hands dirty if need be.

If I understand correctly, what you're looking for is a setup where you replace one server instance with a new one, by 1) bringing up the new instance, 2) waiting for the new instance to be healthy, and 3) removing the old instance. Typically, this requires a setup with a load balancer (so that traffic is routed to a live server instance at all times and clients seamlessly switch over) and some sort of health check on the server instance that allows Pulumi to determine whether a new server instance is ready. Usually, the latter is a feature of the resource provider. E.g., when you deploy a container somewhere, the Pulumi resource will only be considered "ready" when the container is ready to accept traffic (as determined by some configurable healthcheck, which might or might not be the same healthcheck that your load balancer is using). If this sounds approximately like what you're trying to achieve, I'd say that you're probably trying to do too much on the Pulumi side. Any container deployment that I know or can imagine will replace the resource when you change the image name and tag (e.g., docker.Container, kubernetes.core.v1.Pod, or gcp.cloudrunv2.Service), and they all consider a deployment "completed" only if the container is considered ready. This is also true if this container resource is part of a component resource.

That is approximately what I'm trying to do and it would be ideal to do it on the pulumi side. And I'd rather not bring in or pay for a load balancer in front when I can use a floating IP address and just switch them. For example there's this article demonstrating using Ansible to setup wordpress and an ec2 instance with an elastic IP address (floating IP basically). But the thing is if the pulumi lifecycle, if the droplet or resource that the floating IP depends on is destroyed, there's going to be downtime before it's ready. I was hoping that it was possible to make a resource that was composed of many and could be replaced atomically (once the entire thing is healthy, then it's changed out as a whole). This isn't just about deploying containers either, I'd like to be able to use Pulumi for serious infrastructural tasks I have that aren't container related or as trivial. I'm not an enterprise and I'm not made of money but I'm experiencing a situation on a project adjacent to mine where using an S3 bucket on a certain service isn't economical due to bad actors in the community hitting it with the intent to run up the bills, (in an experiment I did I was able to do $13 of damage on my own bucket with a single 1mb file in 3 hours, paying only pennies to do so) cloudflare not being an option due to it being a majority of image content, and there being no controls to rate limit what goes out/stop what comes in, which leads to my serious, use case: It's more economical at our scale to host our content off of server instances than S3, and it's is still more convenient and desirable to manage our content off of S3 to do it on our own servers, and maybe this is a bad idea that won't play out well in the end, but I think it's besides the point, I'd like to make a server instance that has nginx with some rate limiting rules, something to replicate everything from s3 to our servers, and to only replace a droplet or EC2 instance and it's public access via floating IP when it's done. About configurable health checks (not sure if you meant for those specific ones or in general): I looked at the hooks available too and saw the health check example, but adding a healtcheck to a server resource still means that if it fails or even if it doesn't, the old one that my floating IP or something gets assigned to later on won't exist anymore, and there will be down time. Since posting this: I've basically given up on the idea (at least outside of the automation API) as I don't believe it is possible to have pulumi handle anything like this within a single stack/up operation. So I won't try to do this within a stack/resource, or to do it atomically. I've come to the conclusion that If I want this functionality I'll have to forego the idea of having components change based on inputs, and just separate/tear down/bring up additional stack instances as needed and manage them externally to Pulumi. What I have in mind is: keeping a primary/secondary stack instance, bringing up the primary for the first time, for upgrades, bringing up the secondary, the smoke tests will be baked in, and if the up operation fails, failing it, tearing it down, upgrade fails, if it succeeds, the stack for the domain names/floating IPs gets up'd again, and the secondary stack becomes the primary, then the original primary gets brought down, ideally in a logged and mostly idempotent fashion. If something goes wrong, which I hope that it won't, I'll tear down the whole environment and start over. TLDR: I'll break stuff down into more pulumi programs/stacks and use the automation API and some custom state/locking to manage two primary/secondary stacks and smoke test around that where an UP will fail if the tests within do. I'm not seeking assistance with any implementation, maybe feedback if I'm doing something backwards or frowned upon or going in the right direction (began work on it last night, haven't ran it yet). <https://www.pulumi.com/blog/deploy-wordpress-aws-pulumi-ansible/> <https://www.pulumi.com/docs/iac/automation-api/concepts-terminology/> <https://www.pulumi.com/docs/iac/concepts/options/hooks/>

Just picking two points: As the blog post you linked describes, Ansible and Pulumi are tools from two distinct categories, and it sounds like that the way you think about the problem could be better suited for Ansible. Trying to create something like an Ansible playbook in Pulumi will generally not work or require a lot of hacks (as I believe you already discover e.g. in your upgrade_server function), in the same way that trying to replicate IaC infrastructure management through Ansible won't work. If you already have a VM you deploy your servers/containers to, a load balancer does not incur additional costs. You can run e.g. Traefik on the same VM.

I'm aware that they're two different categories of tools, however, a server isn't useful until configured, and when employing the immutable server pattern you don't want one replaced with the other prior to confirming that it works, which I think is a fair expectation for Pulumi to support in some way which I suppose isn't possible. My code for upgrading it has nothing to do with running or creating a playbook, it has to do with creating a second stack like the first one, and bringing up another stack for the floating IP to change to it. Quoting the infrastructure as code book under the immutable server pattern "Rather than applying a change to a running server instance, you create a new server instance. You have the opportunity to test the new instance, **then swap out the previous instance**". I don't particularly want to maintain a VM or have stuff up ahead of time, I'd rather provision and replace server instances as needed and have the added plus of upgrading specs/scaling up. The upgrade server function doesn't run a playbook to go into the server and do anything, it's meant to bring up a new stack with the server instance, if it doesn't error, bring up the floating IP stack with the details, and then tear down the original. The upgrade is in the immutable server sense.

Pulumi can do that, and usually does it, but not quite in the way that you set it up. When you have a

Server(image="firstImage")

resource and deploy a stack, and then you change your program to

Server(image="secondImage")

, and you then run

pulumi up

(or the equivalent in the Automation API) again, your server provider will determine that

image

has changed. If a change to

image

requires a resource replacement (rather than an in-place update), it will then 1) bring up a new server instance with the new image, 2) wait for the new server instance to be ready, 3) shut down the old server instance. If the server never becomes ready, the old one will just stay there (unless you set

destroyBeforeReplace

to true) and Pulumi will fail with an error. Your

Server

resource should have an

ipAddress

output that you can pass as an input to other resources in your program. If the IP address changes, these other resources will then be updated accordingly. The key idea here is that you update the same stack to a new version of your program. The mechanics of provisioning a server are handled by the provider and/or the APIs the provider calls into. It's their job to determine whether the server is ready.

Not every situation though is a server image or provided from a provider like that. I want to employ this pattern for any type of server or just a ComponentResource like a droplet or an EC2 instance. I wanted custom infrastructure to be replaceable in it, and to be able to be changed out atomically in the same fashion. Using/only being able to do this with an existing provider is not what I want, and I feel like, though I'm not completely certain that terraform for example is more capable in this regard because when it's local provisioner fails the resource will become tainted and probably result in the old one not getting replaced. But, I understand now that Pulumi made design decisions that don't allow for this, and that the automation API may be better suited towards implementing the broader pattern I'm after so I'll just work on using that as hinted at on one of the pages for it. I understand this all loud and clear now. Thanks, anyway.

Hmm. I don't think Terraform and Pulumi are any different here. What would what you're trying to achieve look like in Terraform? I use both in my day-to-day work, so I'd be quite curious to hear about that. You said that "when it's local provisioner fails the resource will become tainted and probably result in the old one not getting replaced", which is the process that I tried to describe above and is exactly how Pulumi handles it as well. In fact, you can use any Terraform provider with Pulumi, and many Pulumi providers are derived from Terraform providers. As a side note, the Automation API is "just" a wrapper around the Pulumi CLI, allowing you to call it from within a program, so generally there's no difference when it comes to what you can and cannot do with either.

I think that they are significantly different in capabilities here. I took the time to write a terraform file as of a few hours ago to produce the behavior I wanted. You've emphasized that "The mechanics of provisioning a server are handled by the provider and/or the APIs the provider calls into. It's their job to determine whether the server is ready" but this isn't the case in the lifecycle of terraform, Pulumi considers the resource done, and fine enough as replaced when the underlying API has a new one, meaning there is always down time in deployments for a resource like a droplet or EC2 instance. Terraform (provided you set "lifecycle { create_before_destroy = true }" on your resource won't get rid of the old one until your command is complete and won't replace the old one unless it succeeds. If the provisioner fails, the resource will be tainted, and on the next apply (If I paid attention correctly), it will be deleted and assuming the same configuration options are passed on the next apply, The lifecycle extends beyond the shallow definition of "create" on a resource going through, onto provisioners, allowing for almost zero downtime deployments, which is proving impossible in the same fashion in pulumi, and it means when the deployment fails for some reason, the old server won't be replaced. Fundamentally, it can also be any resource, not just a droplet. If you use hooks in pulumi and there's a health check, your old one is already gone. I'm not actively or seriously using Terraform at this time, so I've just thrown everything into a main.tf for this. This file takes weather or not it should intentionally fail on provisioning the server, and an option for which webserver to install (apache or nginx). It sets the server up over SSH and on the first successful deployment a server will be operational with a floating IP address, when it's upgraded, since it has ``create_before_destroy`` set when it comes to upgrading it (which I've made it trigger by modifying the user data with the software choice) it will create a replacement resource, SSH into it, and only get rid of the old one once the command has succeeded and none of the provisioners fail. If they fail, the old one still exists, because it didn't get deleted the second the new one was technically on the provider. I can't do this basic pattern in Pulumi. Also I'm aware that the automation API is just a wrapper over top of pulumi, but I'm also aware that what I want to do in it is impossible within a single stack, so scripting what I want and keeping everything automatic in code leads that to be my go to since I've been using it/intending to use it anyway later for writing simple deployment scripts that make spinning up production/test environments easier.

terraform {
  required_providers {
    digitalocean = {
      source  = "digitalocean/digitalocean"
      version = "~> 2.0"
    }
    tls = {
      source  = "hashicorp/tls"
      version = "4.0.6"
    }
  }
}
variable "do_token" {}
variable "should_fail" { description = "true or false" }
variable "software_choice" { description = "apache or nginx" }
provider "digitalocean" {
  token = var.do_token
}
resource "tls_private_key" "rsa_ssh_key" {
  algorithm = "RSA"
  rsa_bits = 2048
}
resource "digitalocean_ssh_key" "do_ssh_key" {
  name       = "my_throw_away_ssh_key_name"
  public_key = tls_private_key.rsa_ssh_key.public_key_openssh
}
resource "digitalocean_droplet" "software-droplet" {
  name  = "nginx-droplet"
  region = "lon1"
  image = "rockylinux-8-x64"
  size  = "s-1vcpu-2gb-intel"
  user_data = "# ${var.software_choice}"
  ssh_keys = [
    digitalocean_ssh_key.do_ssh_key.id
  ]
  provisioner "local-exec" {
    command = var.should_fail == "true" ? "exit 1" : "exit 0"
  }
  provisioner "remote-exec" {
    connection {
      host = self.ipv4_address
      private_key = tls_private_key.rsa_ssh_key.private_key_pem
    }
    inline = [
      var.software_choice == "nginx" ? "dnf update -y\ndnf install -y nginx\nsystemctl enable nginx --now" : "dnf update -y\ndnf install -y httpd\nsystemctl enable httpd --now"
    ]
  }

  lifecycle { create_before_destroy = true }
}

resource "digitalocean_reserved_ip" "reserved_ip" {
  droplet_id = digitalocean_droplet.software-droplet.id
  region = digitalocean_droplet.software-droplet.region
}

output "floating_ip_address" {
  value = digitalocean_reserved_ip.reserved_ip.ip_address
}
output "droplet_ip_address" {
  value = digitalocean_droplet.software-droplet.ipv4_address
}
output "running_software_choice" {
  value = var.software_choice
}

I don't have time to check this in detail right now, but Pulumi has the https://www.pulumi.com/docs/iac/concepts/options/deletebeforereplace/ resource option, which is the inverse of

create_before_destroy

but should otherwise be equivalent. Also, https://github.com/pulumi/pulumi-digitalocean is directly built on https://github.com/digitalocean/terraform-provider-digitalocean, so I'd be surprised to learn that Pulumi does something fundamentally different here when it comes to creating resources or determining their state. I'll look at your TF example later.

The changes in pulumi are meaningless and not tied to anything I do to connect and set it up. Even if they are meaningful, like changing the user data, that replacement only changes it at a very shallow level. Replace does not mean the resource is ready. These types of upgrades without downtime are not possible with these options in pulumi in the same way they are in Terraform. It doesn't matter if it creates it before or after, it's definition of done is not good enough to be meaningful for the use case. In terraform they're tied to the provisioner, it's not done until provisioned when setup and it is especially impactful this way when create_before_destroy is set.

Hmm. I'm pretty sure that the "definition of done" depends on the specific provider. I don't work with Digital Ocean, but your desired behavior is quite common and I don't have the impression that e.g. GCP deployments behave differently between TF and Pulumi when it comes to considering a resource created or updated. I have never had the need to use TF provisioners, so I'll have to take some time to look into their precise behavior. My guess is that you can achieve the same behavior with hooks or by writing some chained apply() like this:

const server = new Server();
const ip_address = server.ip_address.apply(async (ip_address) => {
// do whatever you have to do on the server
return ip_address;
});
const proxy = new Proxy(backend_ip=ip_address);

The

ip_adress

output will only resolve after the

apply

returns.