https://pulumi.com logo
Join Slack
Powered by
Can someone talk some sense into me for multi-regi...
# general
c
Can someone talk some sense into me for multi-region setups? I think I am on-board with the idea of there being one stack per region (e.g. 
staging-us-east-1
, 
prod-us-west-2
, etc), but then I started thinking about the global resources (like route53, global accelerator, MRAPs) and then I thought that we’d need a 
prod-globals
… but some regional resources need the global resource to exist FIRST and sometimes the global resource needs the REGIONAL resource to exist first. So that sorta suggests the need for having stack variants for 
pre-globals
and 
post-globals
. Am I nuts? I.e. in order to support 2 “environments” like 
staging
and `prod`in 2 regions, we’d need 8 pulumi stacks (!!!). That’s a lot but it would at least avoid any circular headaches with deployments/automation. alternatively, there could be a single multi-region stack for each “environment” — that way there’s no cross-stack dependencies and the relationship between the variants of global- vs. regional-resources is explicit. But doing something like canary deployments where you release one region at a time would become a real bear to pull off…
w
So there is a tradeoff here and either way can be good depending on your use case. One way is as you said (although not so complex, you shouldnt need pre and post globals if you are careful with your stack refs). You set up a stack per region and then your stacks are small and the resources are specific to that region. This makes for faster "up" but more complex reference logic between stacks. another option is to use one stack like "prod" and put two aws providers in there with different region configs. Then all the resources for each region are deployed with that stack (based on how you do the config of the code and deployment). This can be much simpler and if you dont anticipate the stacks becoming massive it may be a better way to model things for you.
b
Yes, using multiple explicit providers within a stack can alleviate a lot of headache here:
Copy code
var useast1 = new Aws.Provider("useast1", new Aws.ProviderArgs { Region = "us-east-1" });  // also use for global
var uswest2 = new Aws.Provider("uswest1", new Aws.ProviderArgs { Region = "us-west-2" });
var testRole = new Aws.Iam.Role("global_role", new()
{
    Name = "global_role",
    AssumeRolePolicy = ...
}, { provider = useast1 });
var eastCert = new Aws.Acm.Certificate("cert", new Aws.Acm.CertificateArgs
{
    DomainName = "<http://foo.com|foo.com>",
    ValidationMethod = "EMAIL",
}, new CustomResourseOptions { Provider = useast1 });

var westCert = new Aws.Acm.Certificate("cert", new Aws.Acm.CertificateArgs
{
    DomainName = "<http://foo.com|foo.com>",
    ValidationMethod = "EMAIL",
}, new CustomResourseOptions { Provider = useast2 });
But yes, if you are trying to do canary deployments per region then you would probably separate the stacks, and use stack-references or similar (
pulumi-stacks
provider in ESC). Orchestration in this fashion could be augmented using Pulumi Automation API in you pipelines
c
I think the canary deployments are a huge benefit… from my experience a majority of the Big Messes™ caused by a bad deploy could have mitigated or prevented entirely by doing canary deployments… so that alone might push me towards having the multiple stacks per “environment” and just stitch together the deployments of component Pulumi stacks that make up a “Stack” stack. (Ugh, I need some new words here) I used the explicit regional providers to good effect in the initial project and I love the simplicity that Pulumi offers with this, but that project did not require nuanced deployments…
b
I call what you are referring to a "stack hierarchy", and its a very common pattern. e.g.: global->env-regional->env-tenant
in pulumi, stack is not environment, its just "config applied to program", and an encapsulation boundry
c
do you go for the one pulumi stack per region? With possible variants for global resources? Or do you go for the single multi-region stack?
and if you go for a single multi-region stack, how do you do canary deploys?
b
stack is a one-shot deployment, so you cannot do canary, unless you are talking about the canary just being the same exact program with different inputs (since stack is just applied inputs) orchestrated externally
c
Right… if I were rolling my own deployment service, my initial thought was that the deployment could pin specific versions for each service in each region — a manifest or lock file e.g.
Copy code
my-service-image-us-east-1: hash123
my-service-image-us-west-2: hash123
my-service-env-us-east-1: <some vars>
# ...etc...
and then I could manually apply the resources in one region at a time. But 
pulumi up
is certainly a lot simpler…
b
Here is an example of what I mean by external automation (using Automation API library) https://www.pulumi.com/ai/conversations/55f2dc01-dbee-4a46-a8fd-767c988d8f64
c
Hmmm… sorry… that example doesn’t make sense. I get the steps involved in doing a canary deployment. Drain traffic, “dark region deploy”, etc… The example there doesn’t show what to do when you have an existing stack — you wouldn’t be defining stack infrastructure in a deployment script.
I think the overall automation still only really works if you have stacks defined per region… 🤔
b
its a silly example, meant to show control flow using the automation API (orchestration of stacks)
createOrSelectStack will use an existing one
and yes, i am saying, if you want to do canary you would have to do something along those lines
👍 1
(separate stacks)
c
ok, that makes sense. Where would an automation script like that run? In a lambda? Or?
b
anywhere, its just a standard program (a program that executes pulumi programs). Typically you run it in CI/CD
👍 1
Here are some docs: https://www.pulumi.com/docs/iac/automation-api/getting-started-automation-api/ be sure to check out the examples repo at the end
👍 1
Open in Slack
PreviousNext
Powered by