pulumi-community #python

handsome-state-59775

05/10/2021, 8:17 PM

For creating a k8s StorageClass, I'm trying to use the second constructor from here with

StorageClassArgs

, but it errors out with:

Copy code

File ".../venv/lib/python3.9/site-packages/pulumi_kubernetes/storage/v1/StorageClass.py", line 237, in __init__
        __self__._internal_init(resource_name, *args, **kwargs)
    TypeError: _internal_init() got an unexpected keyword argument 'args'

Call site looks something like this:

Copy code

pulumi_kubernetes.storage.v1.StorageClass(
            resource_name='string',
            args=pulumi_kubernetes.storage.v1.StorageClassArgs(...),
            opts=pulumi.ResourceOptions(),
)

Any insights please?

curved-cartoon-90054

05/11/2021, 5:59 PM

HI team i am getting below error

Copy code

pulumi new -y
Please choose a template: aws-python                   A minimal AWS Python Pulumi program
Created project 'pulumi-bridge'

error: stack 'dev' already exists

magnificent-motherboard-76110

05/12/2021, 10:29 PM

Hi', in my

Pulumi.env.yaml

I have declared:

Copy code

config:
  tools:data:
    vpn:
      vm:
        name: servername
        size: Standard_D4s_v2
    registry:
      config:
        name: registryname

then from my

___main___.py

Copy code

cfg = pulumi.Config().require_object('data')
vm = cfg.get('vpn').get('vm')
registry = cfg.get('registry').get('config')

later on when I use such var

Copy code

vm = azure_native.compute.VirtualMachine(
  "vpn-VirtualMachine",
  hardware_profile=azure_native.compute.HardwareProfileArgs(
   vm_size=config.vm.get('size'),
  ),
# ...

everything works, but when I use

Copy code

registry = azure_native.containerregistry.Registry(
  "container-Registry",
  admin_user_enabled=True,
  location=config.region,
  registry_name=config.registry.get('name'),

even if it sees to be the same logic, I'm getting this error:

Copy code

File "/<MY_PATH>/venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 181, in massage
        if not key.startswith("_"):
    TypeError: startswith first arg must be bytes or a tuple of bytes, not str
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

any idea of what I'm doing wrong?

clever-lawyer-94920

05/16/2021, 7:06 PM

Is there a way to use https://registry.terraform.io/modules/terraform-aws-modules/eks/aws with Pulumi?

echoing-dawn-86996

05/19/2021, 1:25 PM

Is there a way to get the provider configuration? For example I want to know the

gcp:project

that is configured in my stack. Is there a way to retrieve this?

icy-policeman-83782

05/19/2021, 4:38 PM

Hello everyone I tried the pulumi for the first time and had issue with pip3 - grpcio installation dependencies on mac M1 I solved it with

CFLAGS="-I /opt/homebrew/opt/openssl/include" LDFLAGS="-L /opt/homebrew/opt/openssl/lib" GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 pip install -r requirements.txt

👍 1

happy-ability-61825

05/20/2021, 10:36 AM

Hi all, I am writing unit tests for our Pulumi stack and I am trying to understand how to use

Mocks.call()

to mock the return value from a get request (in this case

azure_native.network.get_subnet()

)

Copy code

class MyMocks(pulumi.runtime.Mocks):
    def call(self, args: pulumi.runtime.MockCallArgs):
        return {}

What should I be returning from in this function to simulate the response from

get_subnet()

shy-author-33795

05/20/2021, 7:23 PM

I'm attempting to create an S3 bucket with the following code:

Copy code

"""An AWS Python Pulumi program"""
from pulumi_aws import iam, s3

bucket = s3.Bucket(resource_name = 'elenas-cool-new-bucket'

and I'm getting a weird error:

Copy code

Type                     Name            Plan     Info
     pulumi:pulumi:Stack      s3_buckets-dev           8 messages
     └─ pulumi:providers:aws  default_4_5_0            1 error
 
Diagnostics:
  pulumi:pulumi:Stack (s3_buckets-dev):
    panic: strconv.ParseUint: parsing "": invalid syntax
    goroutine 1 [running]:
    <http://github.com/pulumi/pulumi-terraform-bridge/v3/pkg/tfbridge.GetModuleMajorVersion(0x0|github.com/pulumi/pulumi-terraform-bridge/v3/pkg/tfbridge.GetModuleMajorVersion(0x0>, 0x0, 0xc00153f5b0, 0x1)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi-terraform-bridge/v3@v3.0.0/pkg/tfbridge/info.go:730 +0x1dc
    <http://github.com/pulumi/pulumi-aws/provider/v4.Provider(0x0|github.com/pulumi/pulumi-aws/provider/v4.Provider(0x0>, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /home/runner/work/pulumi-aws/pulumi-aws/provider/resources.go:4173 +0x52e1b
    main.main()
        /home/runner/work/pulumi-aws/pulumi-aws/provider/cmd/pulumi-resource-aws/main.go:26 +0x37
 
  pulumi:providers:aws (default_4_5_0):
    error: could not read plugin [/Users/elena.washington/.pulumi/plugins/resource-aws-v4.5.0/pulumi-resource-aws] stdout: EOF

any idea how to fix this?

thousands-art-55778

05/24/2021, 10:49 AM

Hey folks, how are you doing? I'm facing a little problem here when using Pulumi, Docker and AWS Fargate/TaskDefinition. I guess I'm missing some of the concepts of outputs. I'm sending the Docker Image to ECR repository, but I can't access its name in execution time (probably because of a json.dumps method). (context in the thread)

miniature-architect-54920

05/24/2021, 3:31 PM

Hi, I would like to see if someone can point me into the right direction. I would like to use a folder structure like the example below where I can add and remove py files and the instance(or any other asset) would be created/removed. I am not sure how to get the main.py file to import/execute all the py files in the subfolders.

Copy code

aws/instances/app1
aws/instances/app2
aws/instances/app3

sticky-bear-14421

05/25/2021, 11:45 AM

Hi folks, I've got a question about chaining providers in Python. My code has three providers (AWS) in total, most of the code runs with the default provider. I then create an iam role which the second provider assumes. This one I call intermediate, because it is just used to assume into the target account to create the last ressources over there. Or should I say I hoped to do so, because my code failes.

Copy code

current_region = aws.get_region()
intermediate_provider = aws.Provider("IntermediateProvider", region=current_region.id,
                        assume_role={'role_arn': created_role.arn, 'session_name': 'abc'})
target_provider = aws.Provider("TargetProvider", region=current_region.id,
                        assume_role={'role_arn': external_role.arn, 'session_name': 'def'},
                        opts=ResourceOptions(provider=intermediate_provider))

some_ressource = aws.ec2...(opts=ResourceOptions(provider=target_provider))

This code fails with an ominous error message:

Copy code

TypeError: Explicit providers may not be used with provider resources

hallowed-animal-47023

05/30/2021, 1:39 PM

Hey everyone, I just started playing with Pulumi a few days ago. I’m manually converting some terraform over to Pulumi for the learning experience. I’m getting a duplicate URN error and I don’t see why

hallowed-animal-47023

05/30/2021, 1:40 PM

Copy code

test (aws:ec2:Vpc)
+  aws:ec2/vpc:Vpc (create)
    [urn=urn:pulumi:test::pulum::aws:ec2/vpc:Vpc::test]
    __defaults                  : [
        [0]: "assignGeneratedIpv6CidrBlock"
        [1]: "instanceTenancy"
    ]
    assignGeneratedIpv6CidrBlock: false
    cidrBlock                   : "10.0.0.0/16"
    enableDnsHostnames          : true
    enableDnsSupport            : true
    instanceTenancy             : "default"
    tags                        : {
        Name      : "test"
        __defaults: []
    }
 
NAT-Gateway-IP-0 (aws:ec2:Eip)
+  aws:ec2/eip:Eip (create)
    [urn=urn:pulumi:test::pulum::aws:ec2/eip:Eip::NAT-Gateway-IP-0]
    __defaults: []
    tags      : {
        Name      : "NAT-Gateway-IP-0"
        __defaults: []
    }
    vpc       : true
Diagnostics:
error: Duplicate resource URN 'urn:pulumi:test::pulum::aws:ec2/eip:Eip::NAT-Gateway-IP-0'; try giving it a unique name
 
NAT-Gateway-IP-1 (aws:ec2:Eip)
+  aws:ec2/eip:Eip (create)
    [urn=urn:pulumi:test::pulum::aws:ec2/eip:Eip::NAT-Gateway-IP-1]
    __defaults: []
    tags      : {
        Name      : "NAT-Gateway-IP-1"
        __defaults: []
    }
    vpc       : true
 
Stack Outputs:
    vpc_id: Output<T>

hallowed-animal-47023

05/30/2021, 1:40 PM

The URNs appear to be different, so I’m lost as to why I’m getting this error.

hallowed-animal-47023

05/30/2021, 1:45 PM

I did try completely deleting my stack in the event something was left over but no dice.

hallowed-animal-47023

05/30/2021, 1:48 PM

Copy code

import pulumi
from pulumi_aws import ec2, get_availability_zones

# Create AWS VPC with specified name, CIDR block, and enable DNS


class VPC(pulumi.ComponentResource):
    """
    This creates a base VPC with subnets in every available availability zone
    """
    def __init__(
            self,
            name,
            opts=None
    ):
        vpc = ec2.Vpc(name,
                          cidr_block="10.0.0.0/16",
                          enable_dns_support=True,
                          enable_dns_hostnames=True,
                          tags={
                              'Name': name
                      })

        availability_zones = get_availability_zones(state="available")
        public_subnets = []
        private_subnets = []
        for x in range(len(availability_zones.names)):
            public_subnets.append(
                ec2.Subnet(f"Prod-Public-{availability_zones.names[x]}",
                           availability_zone=availability_zones.names[x],
                           cidr_block=f"10.0.{x + 1}.0/24",
                           vpc_id=vpc.id,
                           tags={
                               'Name': f"Prod-Public-{availability_zones.names[x]}"
                           })
            )
            private_subnets.append(
                ec2.Subnet(f"Prod-Private-{availability_zones.names[x]}",
                           availability_zone=availability_zones.names[x],
                           cidr_block=f"10.0.{x + len(availability_zones.names) + 1}.0/24",
                           vpc_id=vpc.id,
                           tags={
                               'Name': f"Prod-Private-{availability_zones.names[x]}"
                       })
            )

            nat_gateways = []
            nat_eips = []
            for x in range(2):
                nat_eips.append(
                    ec2.Eip(
                        f"NAT-Gateway-IP-{x}",
                        vpc=True,
                        tags={
                            "Name": f"NAT-Gateway-IP-{x}"
                        }
                    )
                )
                # nat_gateways.append(
                #     ec2.NatGateway(
                #         f"{name}-NAT-Gateway",
                #         allocation_id=nat_eips[x],
                #         subnet_id=public_subnets[x].id,
                #     tags = {
                #         "Name": f"NAT-Gateway-{x}"
                #     })
                # )



        # Export VPC Id
        pulumi.export('vpc_id', vpc.id)
        url = vpc.id.apply(lambda vpc_id: "https//"+vpc_id)

hallowed-animal-47023

05/30/2021, 1:48 PM

^ Code for reference

few-pillow-1133

05/31/2021, 9:32 AM

Looks like elasticache provisioning error-ed while trying to get tags post deployment workaround mentioned here not applicable, as the pulumi resource doesn't expose such field

Copy code

aws:elasticache:ReplicationGroup (mompltprdafelasticache-gitlab):
    error: Preview failed: refreshing urn:pulumi:nonprod-plt-gitlab::elasticache::aws:elasticache/replicationGroup:ReplicationGroup::mompltprdafelasticache-gitlab: 1 error occurred:
        * error listing tags for resource (arn:aws:elasticache:af-south-1::cluster:mompltprdafelasticache-gitlab-001): InvalidARN: account field cannot be empty.
        status code: 400, request id: 2f3cce92-6d51-4006-8102-6fb635d11e0d

sticky-bear-14421

05/31/2021, 9:58 AM

Hi, I've got a question reagarding the apply() function in Pulumi-Python, as I assume I have a problem with concurrent execution of my stack. My code does similar things as the example of @hallowed-animal-47023 from Sunday. I create a VPC and the the whole network setup. Including the attachment of the the newly created vpc and it's subnet to our TransitGateway .. For the aws.ec2transitgateway.VpcAttachment() I need the private subnets in my vpc .. as there is the ec2.get_subnet_ids() function I call it to filter my subnets:

Copy code

private_subnet_ids = aws.ec2.get_subnet_ids(
    vpc_id=vpc.id, # created as the first resource in the stack
    tags={"visibility": "private"} # filter by tags for the correct subnets
)

But, this consistently fails with the error message of:

Copy code

Exception: invoke of aws:ec2/getSubnetIds:getSubnetIds failed: Missing required argument: The argument "vpc_id" is required, but no definition was found. ()

This worked before, while developing the code, as the VPC was created some days ago, but then we deleted the stack, and now everything should be created in a single run. I guess the call to aws.ec2.Vpc() hasn't returned yet with an vpc and its id when the call to get_subnet_ids() is being issued.

sticky-bear-14421

06/04/2021, 1:50 PM

Hello everyone, After solving my Routing and other funny issues I have created an AWS Redshift instance and now I want to create users and grants in this cluster.

Copy code

redshift_cluster = aws.redshift.cluster(..)

pg_provider = postgresql.Provider('redshift_cluster',
    database=redshift_cluster.database_name,
    host=redshift_cluster.dns_name,
    port=redshift_cluster.port,
    username=redshift_cluster.master_username,
    password=redshift_cluster.master_password)

my_user = postgresql.Role('my_user',
    name="my_user",
    password="123456",
    login=True,
    create_database=False,
    create_role=False,
    superuser=False,
    opts=pulumi.ResourceOptions(provider=pg_provider)

But, when I try to apply this code it will fail with an SQL error:

Copy code

error: error creating role my_user: pq: syntax error at or near "ROLE"

Any ideas what is wrong with my Role() definition?

sparse-tomato-5980

06/08/2021, 8:51 PM

Hey folks - is there a way to quiet things that are logged with

pulumi.log.warn()

gorgeous-minister-41131

06/11/2021, 10:36 PM

Not sure if this is a Python-specific thing, but is there not a way to modify the values of an Input of an object after-the-fact it has been instantiated? for example:

Copy code

d = self.create_deployment(
            *args,
            **kwargs,
            extra_annotations={"<http://app.kubernetes.io/group|app.kubernetes.io/group>": "worker"},
            extra_labels={"<http://app.kubernetes.io/group|app.kubernetes.io/group>": "worker"},
        )
        d.spec.replicas = 2

I find myself having to write custom classes that use Mixins to modify object properties before they’re passed to the constructor, to allow the object to be modified (just-before) it’s actually created…

gorgeous-minister-41131

06/11/2021, 10:37 PM

TL;DR: d is a Deployment object in K8s.. create_deployment creates the object… but after the object’s created, I assume there’s no way to ever change replicas.. without creating a whole new deployment object of the same URN to replace the old…?

gorgeous-minister-41131

06/11/2021, 11:17 PM

I’m going to gather it’s not possible since the object has already been created and the URN exists, and the method of using helper classes to allow Input changes prior to the object creation is the only way…

gorgeous-lifeguard-69736

06/18/2021, 11:34 AM

Hi, how can I assign a role to a gcp service account? I tried the below but it fails with

Copy code

gcp:serviceAccount:IAMMember (log-writer-iam):
  error: 1 error occurred:
        * Error applying IAM policy for service account 'projects/secret-stash-stadium/serviceAccounts/web-sa@secret-stash-stadium.iam.gserviceaccount.com': Error setting IAM policy for service account 'projects/secret-stash-stadium/serviceAccounts/web-sa@secret-stash-stadium.iam.gserviceaccount.com': googleapi: Error 400: Invalid service account (<pulumi.output.output object at 0x10c8f3310>)., badRequest

What am I doing wrong?

Copy code

sa = serviceaccount.Account(
        resource_name="sa",
        account_id="web-sa",
    )

    log_writer_iam = serviceaccount.IAMMember(
        resource_name="log-writer-iam",
        member=f"serviceAccount:{sa.email}",
        role="roles/logging.logWriter",
        service_account_id=sa.name,
    )

full-island-88669

06/18/2021, 2:37 PM

Hi! I'm having an issue using pulumi Python package:

Copy code

Could not find a version that satisfies the requirement grpcio==1.38.0

More info: https://stackoverflow.com/questions/68036977/cannot-use-my-app-on-bitbucket-pipelines-could-not-find-a-version-that-satisfie

great-sunset-355

06/19/2021, 8:37 AM

Hi was anyone able to run pulumi with python debugger? when I run

python __main__.py

I get an error

Program run without the Pulumi engine available

. I found this issue https://github.com/pulumi/pulumi/issues/1372#issuecomment-583086422 I tried what @white-balloon-205 suggested but I could not find the process. Here is the code I tried from: https://www.pulumi.com/docs/intro/concepts/config/#structured-configuration

Copy code

import pulumi

config = pulumi.Config()
data = config.require_object("data")
print("Active:", data.get("active"))

For some reason I do not even get the output of

print

when I run

pulumi up

Any clues?

purple-appointment-84502

06/22/2021, 8:52 AM

Hi all, struggling with the pulumi automation API in python, and some import issues, wondering if you can help. I'm trying to import a common component resource from a sibling directory as a module. In my main__.py files, i'm trying to import:

from ..common.commonComponentResource

And I'm getting relative import errors. From my understanding, the

work_dir

specified as part of the automation api, needs to be where the stack .yaml files are, and this is then used as the python working directory too. Is there any way to have a different work directory to get around the relative import errors? So the project structure I have is: -- Common ---- CommonComponentResource.py ---- __init.py -- Stack1 ---- _main.py ---- yaml -- Stack2 Thanks for your help!

full-artist-27215

06/22/2021, 2:45 PM

I may be having similar issues as @purple-appointment-84502 above, if I'm understanding correctly (I'm having a bit of trouble following the directory layout structure as well). I'm trying to manage multiple Pulumi projects in a single repository that share common resource code (e.g., I have defined some component resources that each project uses). I happen to also be using Pants (https://pantsbuild.org) to manage our other Python code, and I can use the Automation API (based on the "Local Program" pattern in the documentation) to run things under Pants, and therefore take advantage of its Python dependency management to resolve the common code. However, I don't want to forego the use of the

pulumi

CLI to interact with these projects and their stacks (e.g., if something goes wrong, I'd like to be easily do something like

pulumi export

-> edit file ->

pulumi import

, without having to also encode that via the Automation API). The best I've managed so far (though I'm still investigating) is to manipulate the

sys.path

in my various

__main__.py

in order to get the shared code importing correctly, but that seems like a hack I'd rather avoid. Is there any guidance for how to structure things (with directory structures, configuration parameters, etc.) to be able to juggle multiple related Pulumi Python projects that share code? Thanks in advance.

enough-leather-70274

06/22/2021, 11:18 PM

@red-match-15116 - would really value your input on @full-artist-27215 & @purple-appointment-84502's feedback above. Looks like this has been raised before as far back as 2018 even by Joe himself (see https://github.com/pulumi/pulumi/issues/3635, https://github.com/pulumi/pulumi/issues/1641), but has yet to be resolved. Is there any mechanism allowing pulumi to be invoked from (or for automation's

work_dir

param to be set to) a higher level directory than the project dir, e.g. a directory further up the hierarchy from the one containing the yaml and main.py files? That way common code can be shared between multiple pulumi projects as sibling or parent modules/ packages and imported in _main.py and python would resolve them (as they'd naturally be included in sys.path). Or is there a different more obvious solution or workaround that I'm missing?