pulumi-community #python

future-daybreak-16512

12/07/2021, 7:35 AM

Hi All, Getting the following error while calling get functions from Pulumi Policy class using Python :

Copy code

error: Exception calling application: There is no current event loop in thread 'ThreadPoolExecutor-0_0'.

The sample code :

Copy code

from pprint import pprint
from pulumi_policy import (
  ReportViolation,
  ResourceValidationArgs,
)
import pulumi
import pulumi_aws as aws

config = pulumi.Config()

def ec2_validator(args: ResourceValidationArgs, report_violation: ReportViolation):

  if args.resource_type == "aws:ec2/instance:Instance":
    if args.props.get('vpcSecurityGroupIds'): 
      sg_id = aws.ec2.get_security_groups(tags={
        "key": "val",
        "key": "val",
      })

      pprint(vars(sg_id))

Any idea how to fix this error?

polite-mechanic-60124

12/07/2021, 9:15 PM

I'm running into a python pulumi oddity. I'm rendering a jinja template with variables that are pulumi secrets ie of type pulumi.Output[str] and base64ing it to be used by a user_data field.

Copy code

user_data=pulumi.Output.all(
    {
        "cluster_size": cluster_size,
        "secret_id": secret_id,
    }
).apply(
    lambda args: render_user_data_output(
        templating_env(),
        "userdata-server.sh.jinja",
        *args,
    )
),

Now for the weird part. If render_user_data_output returns a string, I get a base64 encoded pulumi object address in memory. If it returns a pulumi Output, then it renders the base64 encoded secret. Both versions defined below

Copy code

def render_user_data_output(
        env: Environment, template_name: str, kwargs: Mapping[str, Any]
) -> str:
    rendered_template = render_template_file(env, template_name, kwargs)
    return base64.b64encode(rendered_template.encode("utf-8")).decode("utf-8")

Copy code

def render_user_data_output(
        env: Environment, template_name: str, kwargs: Mapping[str, Any]
) -> pulumi.Output:
    rendered_template = render_template_file(env, template_name, kwargs)
    return pulumi.Output.from_input(base64.b64encode(rendered_template.encode("utf-8")).decode("utf-8"))

Does anyone know what's going on here?

polite-mechanic-60124

12/07/2021, 9:18 PM

Even more mind-blowing, if I just change the type hint for the return to a pulumi.Output it works (but fails mypy since its a string)

Copy code

def render_user_data_output(
        env: Environment, template_name: str, kwargs: Mapping[str, Any]
) -> pulumi.Output:
    rendered_template = render_template_file(env, template_name, kwargs)
    return base64.b64encode(rendered_template.encode("utf-8")).decode("utf-8")

It's kind of like python loses track of the coroutine and doesn't know to await

nutritious-shampoo-16116

12/13/2021, 5:39 PM

hey Pulumi team, we are being bitten again by silence swallowing of exceptions happening in futures. Is there a way to deal with this issue? I don't think swallowing exceptions, which effectively results in a given resource being not seen by Pulumi (and even worse, if the resource was already created , this leads to a destroy) is an option. How this can be reproduced: • Make some target groups • Export or pass them to ECSService in the

load_balancers_info

• Apply on the target groups and build the payload expected by AWS with a loop or something • Put an exception in the loop • Exception is not reraised • ECSService is not created or it is dropped

nutritious-shampoo-16116

12/13/2021, 5:41 PM

but really any exception raised in a callback silently passes

glamorous-kitchen-14786

12/15/2021, 3:43 PM

Can anyone point me in the direction of a good example project in Python where the resources have been broken up into different python modules? I've engineered a fairly complex project at this point where I've kept all of my resources discoverable to the pulumi cli by doing imports in the module level

__init__.py

files, but I'm running into some difficulty with circular dependencies that I'm not really sure how I would resolve at this point. Essentially I have a file creating secrets in AWS that depends on some infrastructure in another module, and that module would also like to be able to reference the secret values. I tried doing a

pulumi.export

on the secret value to get around this, but it seems I cannot actually reference my stack outputs in flight (either they aren't created until the end, or the resource creation ordering is "wrong" -- not sure which). Open to any advice on structuring a project that isn't all in one big

__main__.py

or any specific advice about this problem if I've explained it well enough. Sadly I cannot share the actual code, but if it would he helpful I could try to whip up a sample dummy project that replicates the issue without getting me in trouble.

rich-easter-89163

12/16/2021, 4:51 AM

Hi people! I'm having an strange issue with aws provider, i reduced my code to this:

Copy code

from pulumi_aws import iam

iam.Policy('test', policy='{}')

It doesn't matter that this will produce an error due to missing fields on policy because I'm not getting an error because my stack just hangs (and sometimes when I interrupt it with ctrl+c it show an

transport is closing

error, but not always). The funny thing is that I wrote my code applied on one stack (lets say staging) then when trying to apply it to other stack (lets say production) is when it hangs, I'm using different aws profiles for each stack, but I verified that is properly configured (other stacks with other code but same profile works). Any clues on how to debug this issue? I think the issue is while triying to instanciate the default aws provider but even using very verbose logs I cant see anything that let me fix this.

glamorous-kitchen-14786

12/16/2021, 2:33 PM

Have you tried stepping through the execution with something like pdb?

glamorous-kitchen-14786

12/16/2021, 2:34 PM

This way you could see where the code is hanging, which might give you some clues.

rich-easter-89163

12/16/2021, 4:56 PM

I did something more rudimentary: commenting resources until stoped to hang, so I know that is everytime y try to create any aws resource

rich-easter-89163

12/16/2021, 5:31 PM

More news: all my stacks of the same AWS account are broken

quiet-plastic-34312

12/21/2021, 2:01 PM

Hi Folks there any way to run a python pulumi code as a python code to use debug line by line ... when i try run it i get a "pulumi.errors.RunError: Program run without the Pulumi engine available; re-run using the

pulumi

CLI"

dry-answer-66872

12/22/2021, 6:04 AM

Hi Team, https://www.pulumi.com/registry/packages/aws/api-docs/s3control/bucketlifecycleconfiguration/ when I try with this example, I am getting _*NameError: name 'aws_s3control_bucket' is not defined*_ Could anyone help in this regard? I am assuming here ''example' as - Lifecycle Name

Copy code

bucket=aws_s3control_bucket["example"]["arn"],   --> Bucket ARN

great-sunset-355

12/22/2021, 9:37 AM

Is the only option to create VPC endpoints within

apply()

when using this code?

Copy code

# vpc endpoints
        vpc_endpoints = ["s3", "ecr.dkr", "ecr.api", "logs"]
        region_output = self._config.aws_provider.region
        for endpoint in vpc_endpoints:
        
            service_name = region_output.apply(
                        lambda region: f"com.amazonaws.{region}.{endpoint}"  # endpoint here does not update from the loop variable
                    )
            pulumi_aws.ec2.VpcEndpoint(
                f"{self._config.name}-{endpoint}-vpc-endpoint",
                args=pulumi_aws.ec2.VpcEndpointArgs(
                    service_name=service_name,
                    vpc_id=self.vpc.id,
                    tags=self._config.tags,
                ),
                opts=self._opts,
            )

helpful-van-82564

12/22/2021, 1:18 PM

Hi, I'm deploying linkerd on a pulumi-provisioned cluster (gke) but need to set an annotation to the kube-system namespace to ensure linkerd never runs there. Since I dont own or maintain the namespace or any resources inside it, I dont really want to import it into pulumi; I just want to ensure an annotation exists. is there any straightforward way of doing this?

victorious-exabyte-70545

12/22/2021, 3:43 PM

Hi all, I am trying to write a unit test and am running into an issue with mocking azuread.get_group. In MyMocks I hav created the function "call" which is catching the token "azureadindex/getGroupgetGroup" and written to return a value (in this case object_id). The function runs (and should return something) but resource azure.authorization.Assignment receives a None value. This is strange because the mock works for these:

victorious-exabyte-70545

12/22/2021, 3:43 PM

Copy code

azure:core/getSubscription:getSubscription
azure-native:network:getVirtualNetwork
azure-native:network:getSubnet
azure:keyvault/getKeyVault:getKeyVault
azure:containerservice/getRegistry:getRegistry
azure-native:compute:getSshPublicKey

victorious-exabyte-70545

12/22/2021, 3:43 PM

but not for

victorious-exabyte-70545

12/22/2021, 3:43 PM

Copy code

azuread:index/getGroup:getGroup

victorious-exabyte-70545

12/22/2021, 3:47 PM

Any ideas?

victorious-exabyte-70545

12/22/2021, 3:47 PM

My other question is how do I mock stack configurations? Thanks in advance for any tips.

great-sunset-355

12/24/2021, 12:32 PM

I really do not enjoy debugging pulumi 😞 this is error says absolutely nothing

Copy code

transformed: Input[U] = func(value)
    TypeError: sequence item 6: expected str instance, NoneType found
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

👍 1

victorious-exabyte-70545

12/24/2021, 6:15 PM

Still struggling with unit test stuff

victorious-exabyte-70545

12/24/2021, 6:16 PM

so id is mocked but display_name and tenant_id values are seen as None in the project.

victorious-exabyte-70545

12/24/2021, 6:20 PM

Copy code

if args.token == "azure:core/getSubscription:getSubscription":
            return {
                "id": "00000000-0000-0000-0000-000000000000",
                "display_name": "subscription",
                "tenant_id": "11111111-1111-1111-1111-111111111111"
            }

victorious-exabyte-70545

12/24/2021, 6:22 PM

print statements from the project:

victorious-exabyte-70545

12/24/2021, 6:22 PM

Copy code

azure:core/getSubscription:getSubscription
here is id of my sub
00000000-0000-0000-0000-000000000000
here is tenant id of my sub
None
here is displayname of my sub
None

victorious-exabyte-70545

12/24/2021, 6:23 PM

tenant_id and display_name are ignored for some reason.

victorious-exabyte-70545

12/24/2021, 6:23 PM

@billowy-army-68599 any ideas on this?

victorious-exabyte-70545

12/24/2021, 6:25 PM

Copy code

subscription = azure.core.get_subscription()
print("here is id of my sub")
print(subscription.id)
print("here is tenant id of my sub")
print(subscription.tenant_id)
print("here is displayname of my sub")
print(subscription.display_name)