Is there a good example of using the resolved promise of a data source function to modify the control flow of the rest of a program? For example, if I wanted to use the

names.length

result of

aws.getAvailabilityZones()

to pass into something that wants a

number

, and then use the output of that to map to

aws.ec2.Subnet

objects?

This is definitely not obvious, so great question. The

getAvailabilityZones

function returns a

Promise<number>

, so you'll need to either

await

its value or use a callback to access it. I prefer

await

, especially if you're using TypeScript, but it does come with one unfortunate "catch": in JS, you cannot

await

at the top level (yet, there is a proposal to enable this). So, this example code will fetch the AZs and simply print them out. From there you can create a subnet-per-AZ, etc.

import * as aws from "@pulumi/aws";
(async () => {
    const zones = (await aws.getAvailabilityZones()).names;
    console.log(`${zones.length} AZs found:`);
    for (const zone of zones) {
        console.log(`\t${zone}`);
    }
})();

If you're creating these in a function, you won't need the cumbersome

(async () => { ... })()

stuff. Instead, it'd just be something like

async function createSubnets() { ... }

. We do have an example of this in the AWS repo, but it's fairly buried and largely only there to serve as a test case: https://github.com/pulumi/pulumi-aws/tree/master/examples/webserver/variants/zones.

Does this lead to (for now) wrapping basically the entire program in an async lambda?

I wouldn't say there's one answer that works for all cases, however for complex programs we do see this a fair bit. Another pattern is to do something like:

async function main() {
    // all the things happen here
}
main();

We don't await the promise returned from main here, but that's okay, since Node.js keeps the program alive until the event loop quiesces. Another approach is to try to scope the places where you need to do this. For instance, in the subnets case, you might have something like

async function createSubnets() {
    const zones = (await aws.getAvailabilityZones()).names;
    const subnets = [];
    ... create them ...
    return subnets;
}

This can just "push the problem around", because of course the returned subnets array will be wrapped in a promise, but for some cases this is more appropriate. Because our

Input<T>

type accepts promises, doing it like this can sometimes let you achieve a more dataflow style rather than needing to await all over the place.

Right, that’s where I’m currently hitting the problem: exporting the IDs of a bunch of created subnets.

A pattern we use in some program is:

async function createResources() {
    // do everything here, including awaiting
    return {
        a: ...,
        b: ...,
    };
}

let outputs = createResources();
export let a = outputs.then(out => out.a);
export let b = outputs.then(out => out.b);

I'm personally not happy with all this ceremony, but as you note, most of this comes from standard JS/TS promises stuff. @lemon-spoon-91807 and @white-balloon-205 had been exploring how to do data sources without needing promises, but I've lost track of where we landed (Node.js makes it hard, by design, to block the message loop).

Aha, that pattern works reasonably well: https://gist.github.com/jen20/2d8451034c0a15858af8561f69a79ac4

Great! cc @lemon-spoon-91807 @white-balloon-205 @microscopic-florist-22719 as this pattern feels like something we probably want to make more obvious and/or give idiomatic support to

If wonder if there couldn’t be something which allows you to export an object from the module directly, and then have the framework take care of separating out each of the values?

Make from the root will build and run tests, and install a local dev copy at /opt/pulumi, which you will want on your path, as the language runtime is loaded dynamically.

Thanks. I’m playing with a few ideas for how to improve the situation above now, which means digging further into Typescript than I’ve had to thus far…

I tried finding the corresponding GitHub issue in the Typescript repo -- since they usually lead in front on any features likely to make it in -- but came up empty. Since several teammates were involved with Typescript and frequently hang out with those folks, would be good to find out (@white-balloon-205 @lemon-spoon-91807 @bitter-oil-46081).

Yes. interesting question

Given the issues and overall velocity around module loading generally, I wouldn't be surprised if this one takes a while to settle. Even Babel seems to be reluctant to pursue it https://github.com/babel/proposals/issues/44. I wish there was a more pleasant way to write code like this. We also had to do this same trickery in our PPC service (our private hosted cloud project).

Humorously relevant from 4 years ago when we were standardizing

await

in JavaScript. https://github.com/tc39/ecmascript-asyncawait/issues/9#issuecomment-39000615

@big-piano-35669 I wonder whether an acceptable answer here would be to allow exporting an

async function(): Promise<any>

from the (user-facing) entry-point module of a Pulumi stack, and if that is what you get back call it from the runtime and use the returned object as the outputs? That way the majority of the ceremony in the pattern above could be removed and replaced with:

interface StackOutputs {
    subnetIds: pulumi.Output<string>[]
}

export default async function(): Promise<StackOutputs> {
    // Do everything here including awaiting
    return {
        subnetIds: someSubnets.map(subnet => subnet.id)
    };
}

Ooh that's quite clever! 👍 Do you mind filing a GitHub issue as a suggestion and we can discuss over there?

Yup, will do.