https://pulumi.com logo
Join Slack
Powered by
what is the right(generic) way to make outputs(kno...
# python
k
what is the right(generic) way to make outputs(known after provisioning; in this case a data source) an string that I can use as an input for other resources? (tried to use the output directly which was not successful)
m
You cannot convert an output to a string within the Pulumi program. The result of 
apply()
will always be another output. But you can use a 
pulumi.Output
as a 
pulumi.Input
, its value will become available while Pulumi deploys your infrastructure. (You can use 
apply()
to convert between different output types, though, e.g., converting an 
pulumi.Output[int]
to 
pulumi.Output[str]
) If you post the code (snippet) that doesn't work, I'm sure we can figure out the problem.
k
at the moment I just try to set a resource name out of an pulumi output from a data source:`existing_rcg = network.FirewallPolicyRuleCollectionGroup(` selected_rcg.name.apply(lambda name: '' + str(name)), resource_group_name=resource_group_name, firewall_policy_name=firewall_policy.name, opts=ResourceOptions( provider=virtual_wan_provider, import_=resource_id, ) )
pyright complains about that I can't use an output[str] as parameter resource_name of type str
background: selected_rcg is reading all available "groups" and selects one and gives back a rcg from
Copy code
get_firewall_policy_rule_collection_group
sadly I couldn't update the rcg directly with my firewall rules so I decided to import the rcg if needed and do the update on "existing+rcg" later on. (will rename to imported_rcg)
Copy code
def get_or_import_existing_rule_collection_group(
    selected_rcg: network.FirewallPolicyRuleCollectionGroup,
    firewall_policy: network.FirewallPolicy,
    resource_group_name: str,
    subscription_id: Input[str],
    virtual_wan_provider: pulumi.ProviderResource
) -> network.FirewallPolicyRuleCollectionGroup:
    """
    Imports an existing FirewallPolicyRuleCollectionGroup if it exists, otherwise returns it
    from the Pulumi state.
    """
    # Construct the resource ID
    resource_id = pulumi.Output.all(resource_group_name, firewall_policy.name, selected_rcg.name).apply(
        lambda args: f"/subscriptions/{subscription_id}/resourceGroups/{args[0]}/providers/Microsoft.Network/firewallPolicies/{args[1]}/ruleCollectionGroups/{args[2]}"
    )

    # Attempt to import the resource. It will create the resource if the import doesn't exist.
    imported_rcg = network.FirewallPolicyRuleCollectionGroup(
        selected_rcg.name.apply(lambda name: '' + str(name)),
        resource_group_name=resource_group_name,
        firewall_policy_name=firewall_policy.name,
        opts=ResourceOptions(
            provider=virtual_wan_provider,
            import_=resource_id,
        )
    )

    return imported_rcg
m
The resource name cannot depend on a Pulumi 
Output
, it has to be available at the point in time the Python code is executed. You can do something like this:
Copy code
groups: list[str] = ["group-a", "group-b"]
for group in groups:
 the_group = MemberResource(f"member-of-{group}", ...)
Here, the value of the resource name is known when the program runs. But an `Output`'s value only becomes known later.
k
I guess I will have to use more static names then
m
I find it helpful to think of it like this: Your Python code describes a system of resources and connections between them (e.g., an output of one resource becomes the input of another), which resembles a graph that you hand over to the Pulumi engine to create this system. By the time you hand it over to the Pulumi engine, every node in that graph has to have a unique name, and every connection between nodes has to be defined.
k
I think my bigger problem now is: resource_id = pulumi.Output.all(resource_group_name, firewall_policy.name, selected_rcg.name).apply( lambda args: f"/subscriptions/{subscription_id}/resourceGroups/{args[0]}/providers/Microsoft.Network/firewallPolicies/{args[1]}/ruleCollectionGroups/{args[2]}" )
I wanted to construct resource_id for the import of the pulumi resource
I will try selected_rcg.id but I'm afraid it wants a string too
m
That's generally speaking not a good idea. I think you can get around this somewhat by creating resources inside an "apply()" but that's discouraged and usually not needed in my experience.
> I will try selected_rcg.id but I'm afraid it wants a string too I think you should focus on the core issue: You want static resource names that never change between runs of your program. As soon as you change the resource name, Pulumi believes it's a new resource, which is why you want them to be static/deterministic.
k
first I just try to get the rcg as data source and update its rulecollections directly but that failed also
selected_rcg is returns always the same name(it just contains some small logic for the first time usage)
maybe I run in some dead end. I though I could read, import and update an rule collection group in one pulumi app
m
If it's always the same anyway, why can't you just hard-code the name? An import should be a one-time thing, not something you do every time the program runs.
k
it's selects the rule collection groups from 90 groups(which are provisioned outside this app). it contains a logic to balance all the rule collections across of the groups
m
> maybe I run in some dead end. I though I could read, import and update an rule collection group in one pulumi app You can do that, but to me this starts to sound like you're trying to write an imperative program (like a shell script of Azure CLI calls) rather than a declarative IaC program.
k
90 groups and n landing zones that add rule collections to the groups. each landing zone picks one group and stays with it
m
Pulumi is excellent for the latter, but if you want a program that runs through your resources and patches them, an Azure SDK script is likely the better choice.
Are you managing the landing zones and groups with Pulumi?
k
I move away from terraform because I hoped to actually have the flexibility to add additional logic before keeping them in a declarative state
m
You can do this, but not as part of the actual Pulumi program. At the level we're talking about here, TF and Pulumi take the same approach: You bring resources into the state (either by creating them or by importing them) and then you evolve them by changing your infrastructure code. Neither tool is made for ad-hoc updates.
k
maybe I described it wrong. the goal and the whole app is about importing and updating the resources from then on. the app just have balancing logic to choose the resource to import
m
Then I'd say think about how you want to name your resources. The names should be determined within your program, not from the outside. Otherwise, your infrastructure state would depend on the infrastructure, leading to a circular dependency. For example, imagine you delete a resource you imported. Now, you can no longer figure out its name.
k
worst case I really have to run the balancing logic beforehand and pass it to the pulumi app
m
Yes, I would look at it this way: In your Pulumi program, you describe a certain infrastructure setup. Now, while importing resources, your task is to match what's already there to this setup.
k
the resource name is something I can make generic enough, just getting the resource id to import is an issue. the dependency on the infrastructure is on purpose and part of the chain.
m
But that's a one-time activity. Once you've imported the resources, they are part of your state.
k
seems like I just can't build the resource id based on pulumi outputs for the import even all is sequential in my case
thanks for the replies @modern-zebra-45309
h
Late to this party, but this whole 'Outputs' system is definitely the hardest thing to learn in Pulumi at least for me. I tried these same things that you are mentioning, and there is no magic invocation/syntax that will do what you want. Here is the key takeaway in my mind: # Outputs can only be evaluated as properties of Pulumi resources. This is because the actual order of execution of creating resources isn't the order of your code, like a normal, procedural language. Pulumi is basically "executes" your code to create a DAG that it will then execute/apply in the same way that Terraform does. As such, the order of operations is controlled by the DAG. Lines of code that aren't part of a Resource are executed during DAG-creation and the problem that you are seeing is that the values of Outputs is not known at that time. So this works: 
x = aws.s3.bucket("backup",  bucket=primary_bucket.name.apply(lambda primary_bucket: f"{primary_bucket}-backup"))
This won't work:
Copy code
bucket_name = primary_bucket.name.apply(lambda primary_bucket: f"{primary_bucket}-backup")
x = aws.s3.bucket("backup",  bucket=bucket_name)
Another tricky bit is about 
Output.json_dump
which is that f-strings are evaluated before the function is, so you can't use it if you will be using an f-string in the JSON, which is really common when you need an ARN that ends in 
/*
.
m
Your second example works as well, they are identical, no? (In both cases, it has to be 
primary_bucket.bucket.apply(lambda primary_bucket: f"{primary_bucket]-backup")
to get the bucket's name, though.)
h
Ah you are right about the .name. I was thinking after I wrote this that actually that second one may work, but, there is something here that doesn't work. You definitely can't print the value of 
primary_bucket
. Overall, I've just tried to avoid evaluating any Outputs anywhere except within properties of resources and that seems to have helped me avoid chasing my tail over it.
m
And can you say more about your f-String/json_dump scenario? I don't fully understand what situation you're talking about
h
This doesn't work:
Copy code
policy = pulumi.Output.json_dumps({Resources:[my_s3_bucket.arn, f"{my_s3_bucket.arn}/*"]})
m
I was thinking after I wrote this that actually that second one may work, but, there is something here that doesn't work.
I'm very certain that this works, I use similar patterns a lot to keep resource definitions legible.
You definitely can't print the value of 
primary_bucket
.
Yes, but that's unrelated to whether you assign the 
Output
to an intermediate variable or not.
This doesn't work:
Yes, this needs an additional "apply" for the second entry.
h
I'm not sure if maybe in my first example, pulumi sometimes would try to create the second bucket before the first and then you might get the Output issue
m
There must be something else going on there. The Outputs you pass determine the DAG.
h
But it is super irritating of an error with many times I end up getting the error message from AWS because the text string telling me not to do this has become part of a setting in AWS
m
Yeah, I know that feeling when you open an IAM policy in the console and half of it is a Pulumi warning string 😉
But I find that once you get the hang of it, and discover utilities like Outputs.json_dumps, it's a great and powerful system. I recently had to merge two YAMLs in TF and it was neither fun nor is the resulting code maintainable.
k
Actually merging yamls is one of the reasons why I move away from TF too. Actually anything more advanced than a static resource makes TF just unreadable.
Thanks both for the input
just to share my progress. after I learned that the docs don't give a working example for a firewall policy rule collection group for the azure native provider, I just tried the azure classic one. it worked normally to create rcgs. my solution for now. I do an pulumi stack export --file state.json before running the preview/up in the pipeline. that way I check if the urn for the rcg is already present in my app. if it is not present I import the rcg, if present I just update it. works
Also I used the azure sdk to get all the rule collection groups to avoid issues with outputs.
7 Views
Open in Slack
PreviousNext
Powered by