https://pulumi.com logo
Title
l

little-cartoon-10569

08/15/2020, 7:59 AM
Are there any gotchas with try-catch blocks in Pulumi? I have a snippet like this:
try {
    const stack = new pulumi.StackReference('stackName');
    const someId = peerStack.requireOutput("resourceId") as pulumi.Output<string>;
  } catch (e) {
    if (e instanceof Error) {
      pulumi.log.error(e.message);
      return pulumi.output(`${e.message}`);
    } else {
      pulumi.log.error("Unknown object thrown");
      return pulumi.output("Unknown error");
    }
  }
I'm expecting this code to report the error and continue working. But I'm getting
error: Running program '/pulumi/projects/main/pulumi/projects/example' failed with an unhandled exception:
Error: Required output 'someId' does not exist on stack 'stackName'.
at /pulumi/projects/node_modules/@pulumi/pulumi/stackReference.js:72:23
...
And preview fails. Even though I have handled the exception... How can I conditionally skip a section of my code if a required output isn't present in another stack?
Also tried with
requireOutputValue() as pulumi.Input<string>
. No change.
Haven't figured this out, would love to know how to achieve this. Currently I'm working around this by having a config value that means "the other stack exists and has the correct values ready to go". The exception thrown when I look up config and the value is missing is fine, it works with my exception handling. That way, I avoid throwing an exception when using StackReference, and all is well.
a

ancient-megabyte-79588

08/17/2020, 2:07 PM
I haven't had a chance to test this out yet, but from what I understand looking at this code, your try catch block won't fail because it is returning a object that promises to have that stackReference eventually. (deferred execution) The same with the someId. So that try catch block won't fail. And then, you've scoped stack and someId within the try/catch block which means it isn't availabe outside of that block? If you use
const var1 = stackRef.getOutput("someConfigValue") || "default";
you can detect that you didn't get a stack value or adapt in the case it wasn't available with a default value.
l

little-cartoon-10569

08/17/2020, 7:37 PM
The docs for requireObject and requireObjectValue imply that an error should be thrown if the stack doesn't have the object:
Fetches the value of the named stack output, or throws an error if the output was not found.
Fetches the value promptly of the named stack output. Throws an error if the stack output is not found.
The stack doesn't have the object I'm requesting (there is no
export
for it, it will never be there), so I believe the exception should be thrown. If it's not, then there's a bug in the docs. (The scope is not an issue, this is just example code.)
And the exception does happen: it's just not caught. The Pulumi app fails with an exception, my resources don't get created, an error message is displayed, but my error code isn't called.
My best guess is that the exception is being thrown asynchronously and not from within the same try block.
a

ancient-megabyte-79588

08/17/2020, 9:23 PM
That is what I was trying to convey. Most things in pulumi are async and since you are not using an await on the call (I don't know if it works in pulumi), you've actually left your try/catch block context and you've moved onto the next statement.
l

little-cartoon-10569

08/17/2020, 9:31 PM
Yes. I think that's the bug. It may be intended to work that way, but the docs imply something else: a synchronous exception. And there should be no reason not to have a synchronous exception in my use case: if the stack reference doesn't contain the output, then no amount of waiting it is going to change that, so don't wait, just throw.
a

ancient-megabyte-79588

08/18/2020, 3:03 PM
I think we need to understand if the acquisition of the StackReference is synchronous and may throw exceptions or the registration of the intention to get the StackReference and all of its config options is synchronous. If we look here at the StackReference code in github, we can see that all of the properties on StackReference are
Output<>
, which will not necessarily be fulfilled at the time of creation since Output instantiations are usually deferred. I 100% agree that when you ask the StackReference for an
getRequired("name");
, if it doesn't exist, throw an exception but that is isn't the same as creating the initial StackReference object that you are using to get deferred objects.
I haven't looked at the docs and maybe that is the problem and not the actual SDK itself. (I'm using the nodejs/TypeScript sdk as a reference point btw).
l

little-cartoon-10569

08/18/2020, 9:55 PM
Me too. I think we need synchronous ways to find out if the other stack exists and if it contains an output. I totally accept that getting that reference must be asynchronous. My particular use case is VPC peering: I want to know whether do not to start my peering attempt, but my app crashes when I try to get the VPC ID from the other stack, because that stack hasn't been upped yet. And the same problem happens when I want to accept the connection: the 2nd stack needs to get the connection ID from the 1st stack, but the app will crash if the connection hasn't been requested. All of this should be doable synchronously and "catchably". Currently I'm side-stepping the problem by coping my stack outputs to config. When I call config.require('PeerVpcId') and the PeerVpcId hasn't been set, I get a synchronous exception thrown. Nice and "catchable". But not as idiomatic as calling stackRef.require... (imo).
a

ancient-megabyte-79588

08/18/2020, 10:38 PM
All of this should be doable synchronously and "catchably".
I think this is the hard part. Provisioning cloud resources can't be done synchronously, unless you are willing to wait, even in the event that the provisioning failed. And a Stack is a cloud resource when it is being shared via StackReference.
l

little-cartoon-10569

08/18/2020, 10:41 PM
But you can't deploy to one stack while waiting for another stack to deploy, can you? StackReferences are different in that when you create a reference, you know that the stack behind has already finished deploying.
a

ancient-megabyte-79588

08/18/2020, 10:42 PM
Pulumi has the
dependsOn
property that you can use to wait until the resource is provisioned, but in your case, your first stack/pulumi app may not have even run, so your still going to have a undefined string, and the app requesting the StackReference has to wait until another process has run.
I'm wondering, are you using stacks as a part of a workflow, like, run stack 1, run stack 2 (but only if stack 1 run), run stack 3 (but only if stack 2 have run), etc?
l

little-cartoon-10569

08/18/2020, 10:44 PM
And this applies even to
requireOutputValue
, which has this doc?
Fetches the value promptly of the named stack output. Throws an error if the stack output is not found.
Yes, something like that.
a

ancient-megabyte-79588

08/18/2020, 10:45 PM
I don't use stacks as a part of a workflow like that. I do have stacks that dependOn each other, but if the second stack blows up, that is ok because I have to run a CI/CD or manually build the core platform.
So for example, I have: • a pulumi app that provisions our Azure Network resources (vNet + subnets) and we have stacks for different environments. (dev, test, prod) • a pulumi app that provisions our Azure Kubernetes Service instances (6 in total) into those vNets (dev, test, stage, dev-shared, prod, prod-shared), which will fail if the vnets aren't there • a pulumi app that provisions k8s core applications into a k8s cluster (which will fail if the cluster isn't there) • a pulumi app per deployed product with stacks for each environment (which will fail if the cluster isn't there)
l

little-cartoon-10569

08/18/2020, 10:48 PM
We use projects in a monorepo for that. In my case, I have admin, dev and prod stacks, and I need peering between admin and the other two. Which means that some resources in admin depend on resources in all three stacks.
To map my use case to your solution, I think I'd need a whole new peering app, which seems OTT?
In fact, that's what we had, and we decided to rework to merge the VPC-creation and peering apps... and now we have this problem 😞
a

ancient-megabyte-79588

08/18/2020, 10:51 PM
I theory, I could have crafted that as a single massive pulumi app with all the configuration values required for all of the apps, but I prefer smaller apps and smaller configuration-per-stack to minimize how much configuration is managed.
l

little-cartoon-10569

08/18/2020, 10:52 PM
👍 That's how I've done it too.
a

ancient-megabyte-79588

08/18/2020, 10:52 PM
In a single app, you can manage dependencies with the
dependsOn
attribute.
l

little-cartoon-10569

08/18/2020, 10:53 PM
Can I dependsOn a stackRef value?
a

ancient-megabyte-79588

08/18/2020, 10:54 PM
I don't know. Generally, pulumi is very good about detecting
Output<T>
and automatically building in the dependency for you, but that still doesn't solve the problem that the dependency value doesn't currently exist.
I think that if you were creating all VPCs in one app, you could
dependsOn
all of them and then when they are all complete, create the Peering.
l

little-cartoon-10569

08/18/2020, 10:57 PM
Hmm.. sounds nice, but I don't know how to do that. We run the stacks separately. Is it even possible to run
pulumi stack select
during a
pulumi up
? Is it possible to depend on two things in two stacks at the same time? 🤔
a

ancient-megabyte-79588

08/18/2020, 10:57 PM
Two console windows, yes
I can type
pulumi up
in as many separate console windows as my fast little fingers can type. 😄
That doesn't ensure the order in which they will finish, or that they will finish at all.
l

little-cartoon-10569

08/18/2020, 10:59 PM
Yeah nah, not for me thanks. That's not pipeline-able, the potential for race conditions is extreme, and there's no way it would get past a peer review.
a

ancient-megabyte-79588

08/18/2020, 10:59 PM
And I should clarify, I haven't tried running
pulumi up
on the same app on multiple stacks concurrently. Pulumi cli MIGHT stop that from happening. But in theory, you could run two stacks of the same app at the same time and they would not conflict.
l

little-cartoon-10569

08/18/2020, 11:01 PM
But if one of those stacks has a dependsOn on a resource in the other stack, then there will be long waits. If they both have dependsOn the other stack, then there will be race conditions. I feel like the best thing here for Pulumi would be to prohibit dependsOn between stacks, and to fall back to the synchronous exceptions.
a

ancient-megabyte-79588

08/18/2020, 11:02 PM
I think if you have important dependencies like you've described, I would try to do that in one app with the
dependsOn
functionality.
l

little-cartoon-10569

08/18/2020, 11:02 PM
Needs to be one stack, too, I think.
Which is absolutely possible with assume-role providers. I'm just not going down that road right now. The current workaround isn't elegant but it isn't complicated either. I even have a nice exception saying "You've forgotten to run stack X, I'm aborting now, go run it and update the config in this stack with its output."
👍 1
Thanks for your input in this. The idea that a StackReference is a cloud resource hadn't occurred to me, and explains why the current implementation is why it is.
a

ancient-megabyte-79588

08/18/2020, 11:05 PM
It was great to talk about all of this! I hope it was therapeutic if not helpful. 😄
l

little-cartoon-10569

08/18/2020, 11:06 PM
🙂 It would just help a lot to have a SynchronouslyFetchedResource or somesuch, for when we know the resource / StackReference exists before the app is run...
a

ancient-megabyte-79588

08/18/2020, 11:07 PM
I don't think that is an unreasonable ask for a synchronously execute inspection. Add it as a feature request. I'll add the caveat that there may already be a way to do it that I don't know about. 😛
l

little-cartoon-10569

08/18/2020, 11:43 PM