https://pulumi.com logo
#python
Title
# python
h

hundreds-gpu-71155

03/13/2024, 4:54 PM
ok, i really gotta work out how pulumi async works, because i just wrote this and I'd rather not do that again.
Copy code
nics = vm.network_profile.network_interfaces.apply(
        lambda nics: [
            network.get_network_interface_output(
                resource_group_name=resource_group.name,
                network_interface_name=get_name(nic),
            )
            for nic in nics
        ]
    )
Copy code
nics.apply(
            lambda nics: pulumi.Output.all(
                *[nic.ip_configurations for nic in nics]
            ).apply(
                lambda ipcss: [
                    ipc["private_ip_address"] for ipcs in ipcss for ipc in ipcs
                ]
            )
        ),
m

miniature-musician-31262

03/13/2024, 5:02 PM
Yep, this is definitely one of the toughest Pulumi concepts to get comfortable with. Takes a bit, but once it clicks, it can be super useful (if occasionally cumbersome -- although we do have some helpers that can make it a bit less awkward). We actually just refreshed a bunch of the docs around this and would love to know whether they help and what could still be better explained: https://www.pulumi.com/docs/concepts/inputs-outputs/
h

hundreds-gpu-71155

03/13/2024, 5:04 PM
ok, but in this case, i have
list[Output] -> list[list[Output]] -> list[list[Output]]
(getting all the IP addresses from every Network Interface on a VM)
and i've written some hideous comprehensions in my time
but since Outputs and comprehensions is hobbled, I don't think there's a better way to write... that
like, the normal python way to write this would be
Copy code
[
    ipc["private_ip_address"]
    for nic in vm.network_profile.network_interfaces
    for ipc in nic.ip_configurations
]
and, like, if i could sprinkle some
await
through there and make it work, that would be great. excellent, even. but doing so requires extra work to get into the async context.
ok, the answer is that pulumi/python does not currently support python async, although it's using asyncio under the hood
correction:
Output.apply()
accepts an async function
and you could await on
Output._future
, although I bet that a bunch of metadata is lost
i think the ideal version would be something like
Copy code
[
    ipc["private_ip_address"]
    for nic in await vm.network_profile.network_interfaces
    for ipc in await (await get_network_interface_output(nic)).ip_configurations
]
i'm just rambling at this point
m

miniature-musician-31262

03/13/2024, 8:16 PM
No this is super helpful, thanks. Python isn't my strongest language, but I think I get it.
Thanks for sharing that AI Answer also -- looks like that's one's just wrong?
h

hundreds-gpu-71155

03/13/2024, 8:22 PM
i did figure out the hack to allow async,
Output.apply()
accepts async functions (or you combine
asyncio.ensure_future()
with
Output()
), and that'll let you get into an async context. You can then use
await some_output._future
inside to access data. The big problem with this hack is that it'll lose a bunch of metadata (resource tracking, secret status). Pulumi could probably commit some sins to preserve some of that tracking (eg, add code to
__await__
to mark when a value's been used in a coroutine, add encapsulation types), but there might be caveats (eg, it might only be able to track the metadata of the overall coroutine, not track the flow of data inside the coroutine).
and yeah, the AI answer made up
pulumi.runtime.run()
, which i didn't figure out until i had looked through the docs and source to find it. the proper answer is "pulumi does not support python async. use async with caution."
m

miniature-musician-31262

03/13/2024, 8:25 PM
I just took that answer down, yeah. Thanks again for calling that out
h

hundreds-gpu-71155

03/13/2024, 8:26 PM
(there are ways to use async if you need to, but there's a strong chance of oddities if you're not careful)
(eg, if you have a connection pool in the async domain, you need to make sure it gets cleaned up, or the OS will just close connections when python exits)
b

busy-journalist-6936

03/13/2024, 8:30 PM
These complex apply cases can be especially useful for learning efforts. @hundreds-gpu-71155 if you happen to have code that’s public would you mind sharing? (PS no pressure, no pressure at all, thank you for wrestling through it regardless ❤️)
h

hundreds-gpu-71155

03/13/2024, 8:33 PM
nope, everything i'm working on currently is proprietary. maybe someday, i'll be able to release some of the utility magic i've written. this case starts pretty simple, though: Given a pile of
pulumi_azure_native.compute.VirtualMachine
, how do I aggregate their IP addresses?
the answer is by re-requesting the half-filled Network Interfaces off the VM and then traversing two layers of Outputs
b

busy-journalist-6936

03/13/2024, 8:36 PM
that’s enough info to work with and build an example from. thank you!
h

hundreds-gpu-71155

03/13/2024, 9:12 PM
you're welcome