pulumi-community #python

hallowed-ice-8403

06/23/2021, 8:48 AM

Hello.. I am trying to create s3 and dynamo table in the same stack and trying to run it in parallel as they are not dependent on each other. I have used the

auto.create_or_select_stack

function, am running it as 2 separate tasks. I am noticing this, after the dynamo is created it deletes the s3. I am guessing its because its replacing the state file.

Copy code

aws:dynamodb:Table msd-test-dynamo creating
INFO - +  aws:dynamodb:Table msd-test-dynamo created
INFO - -  aws:s3:Bucket msd-test-s3 deleting
INFO - -  aws:s3:Bucket msd-test-s3 deleted

How do i add resources to existing stack without deleting existing resources ? Also it would be helpful if there a best practises doc for state, stack and projects management.

happy-alarm-59675

06/23/2021, 10:07 AM

Hi, I try to output a list for my custom provider, like in this example here: https://github.com/pulumi/examples/blob/master/aws-py-ec2-provisioners/provisioners.py#L159-L160 But when I do this,

pulumi up

either hangs, or I get this error:

Copy code

File ".venv/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 79, in _get_list_element_type
        raise AssertionError(f"Unexpected type. Expected 'list' got '{typ}'")
    AssertionError: Unexpected type. Expected 'list' got '<class 'list'>'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

The code, which is just some playing around with dynamic providers: https://gist.github.com/ederst/406438f594dd82b3c614df43658b3bf8 My guess is i am running into https://github.com/pulumi/pulumi/pull/7049 and I have to wait until this is resolved, and use some different method to expose a list as output (Output[str] and ','.join()) in the meantime?

great-sunset-355

06/23/2021, 9:13 PM

Hi does anyone have an example of moderately complex project with packages and functions?

great-sunset-355

06/25/2021, 11:55 AM

Hi was anyone able to get the code completion working in PyCharm? There seem to be something with _utilities.lazy_import so that PyCharm does not provide any completion at all. Then in VScode, you get completion working but it's not great either when I use Show Bover on the resource it does not show almost anything. For this example:

Copy code

"""A Python Pulumi program"""

import pulumi

from pulumi_aws import iam

iam.Role()

Show Hover action only shows

(class) Role

- while I expect the whole docstring to show up. But it looks like the cause of this is that how docstrings are written and possibly usage of overload decorator as well

Copy code

class MyClass:
    @overload
    def __init__(self, a:int)->None:
        """init INT"""
        ...
    def __init__(self, *args, **kwargs) -> None:
        pass # real implementation

If Classes had a class docstring like this it would at least shown that:

Copy code

class MyClass:
    """A class doc."""
    @overload
    def __init__(self, a:int)->None:
        """init INT"""
        ...
    def __init__(self, *args, **kwargs) -> None:
        pass # real implementation

I wonder if pulumi adheres to any of the styling guides or "just generates docstrings"

alert-mechanic-59024

06/25/2021, 4:03 PM

Hey guys. Currently using the latest versions of pulumi (python sdk) and pulumi-kubernetes, and seeing this: File “/pulumi/projects/venv/lib/python3.9/site-packages/pulumi_kubernetes/apiextensions/v1/CustomResourceDefinition.py”, line 121, in init self._internal_init(resource_name, *args, **kwargs) TypeError: _internal_init() got an unexpected keyword argument ‘status’ I’ve considered modifying the upstream yaml files to remove the offending code, but I’d prefer to either wait for the bug to be resolved, or roll back to an earlier version of pulumi - I believe v2 did not have this issue. Could anyone tell me if they have seen this issue and, if so, how they went about resolving? Alternatively, could someone offer up their requirements.txt file with a list of compatible python library versions? Thanks!

mammoth-refrigerator-77806

06/28/2021, 7:34 PM

in a

aws.ecs.TaskDefinition()

resource I’d like to use other resource attributes in the

container_definitions

parameter which requires a valid JSON document (currently declaring within

json.dumps

). However, when I try this I get the error

TypeError: Object of type Output is not JSON serializable

. Is there a way to do this?

great-sunset-355

06/29/2021, 6:02 PM

Generated objects in python have unreal names - this is not pythonic it looks like Java or C# at best 😞

Copy code

ServiceSourceConfigurationAuthenticationConfigurationArgs

Is there any chance to remedy this? without dropping into dictionaries? PEP8 python line length suggest 79 characters the name above is 57!

incalculable-action-69391

06/30/2021, 3:42 AM

Is there anyone please who know how to set environment vars in pulumi with python, this information is NOT covered even one time in the documentation. Is it a dictionary {key:value} .

incalculable-action-69391

06/30/2021, 3:42 AM

ambitious-father-68746

06/30/2021, 2:38 PM

Hi, I currently need to create a set of resources conditionally based on some Outputs. Using

if

doesn't work because an Output is not a boolean. I've found a solution where I can shove the creation of the resource inside an

apply()

, but then I can't refer to that resource from outside the

apply().

Any ideas?

great-sunset-355

06/30/2021, 4:37 PM

How can I do this

apply

correctly, it keeps biting me?

Copy code

cfg = pulumi.Config()
    ssm.Parameter(
        f"{prefix}-app-dn-credentials",
        name=f"/{prefix}",
        value=json.dumps(
            {
                "user": "user",
                "password": cfg.require_secret('secret').apply(lambda x: x)
            }
        )
    )

few-pillow-1133

07/01/2021, 12:59 PM

Any idea why cli deployment could be failing We're trying to deploy from gitlab to azure, and it seems not to work

raise invoke_error

Exception: invoke of azure-native:resources:getResourceGroup failed: invocation of azure-native:resources:getResourceGroup returned an error: building auth config: obtain subscription(f15d5330-8e98-43e0-ac1e-3a08e5702508) from Azure CLI: Error parsing json result from the Azure CLI: Error waiting for the Azure CLI: exit status 1

error: an unhandled error occurred: Program exited with non-zero exit code: 1

Config looks like below

Copy code

config:
  azure-native:clientId: xxx
  azure-native:clientSecret:
    secure: xxx
  azure-native:environment: public
  azure-native:location: xxx
  azure-native:subscriptionId: xxx
  azure-native:tenantId: xxx
  azure:clientId: xxx
  azure:clientSecret:
    secure: xxx
  azure:environment: public
  azure:location: xxx
  azure:subscriptionId: xxx
  azure:tenantId: xxx
  service-bus:data:
     ....

numerous-pencil-44890

07/02/2021, 4:15 PM

Hi, does anyone know how to set a provider’s config programmatically? I have a stack that creates a GKE cluster which uses the

gcp:project

and other configs, but in my application (k8s namespace) stack I just want to inherit the GCP project, region, zone from a stack reference so that it can’t be screwed up when deploying between multiple clusters. The docs mention that you can create the provider with those values: “passed to the constructor of

new gcp.Provider

to construct a specific instance of the GCP provider”. But I don’t see any examples doing that. [https://www.pulumi.com/docs/intro/cloud-providers/gcp/#configuration]

hallowed-ice-8403

07/05/2021, 6:39 AM

Hello.. I am trying to provision base infrastructure components in aws using pulumi. Is there a sample git project using python automation api, that provision micro stacks for each module (group of aws components) within same project ?

ambitious-article-39970

07/05/2021, 11:58 AM

hey guys qq trying to deploy ECS fargate and pass it a secret in container definitions

Copy code

```task_definition = aws.ecs.TaskDefinition('pulumi-app-task',
    family='fargate-task-definition',
    cpu='256',
    memory='512',
    network_mode='awsvpc',
   tags = global_tags,
    requires_compatibilities=['FARGATE'],
    execution_role_arn=role.arn,
    container_definitions=json.dumps([{
      'secrets' : json.dumps([{'db_password': f"{db_password.id}"}]),
      'name': 'pulumi-test-app',
      'image': 'nginx',
      'portMappings': [{
         'containerPort': 80,
         'hostPort': 80,
         'protocol': 'tcp'
      }]
   }])
)
having issues with this line  'secrets' : json.dumps([{'db_password': f"{db_password.id}"}]),

I assume this is because its returning an aws ARN im getting an unmarshelled string error converting to json. is this the case?

rror: aws:ecs/taskDefinition:TaskDefinition resource 'pulumi-app-task' has a problem: ECS Task Definition container_definitions is invalid: Error decoding JSON: json: cannot unmarshal string into Go struct field ContainerDefinition.Secrets of type []*ecs.Secret. Examine values at 'TaskDefinition.ContainerDefinitions'.

im new to pulumi and havent quite figured out how to use a debugger with it yet so havning to guess alot at what values which come back look like. (I know the second json dumps isnt needed but there for debugging)

ambitious-article-39970

07/05/2021, 11:58 AM

Copy code

db_password = aws.ssm.Parameter("pulumi-db-secret",
   type =  "SecureString",
   value = db_password_ssm)

ambitious-article-39970

07/05/2021, 1:37 PM

so I figure out abit further and the issue seems to resolve around the db_password object as the values it spits out work when hardcoded

many-yak-61188

07/06/2021, 3:32 PM

hello all 👋 complete noob here. Running into an issue utilizing

pulumi

in a github

workflow

where all dependencies are managed via

poetry

. I'm think I do not understand the combination of

poetry

in github

workflows

which is causing the error in pulumi and not directly an issue with pulumi itself. Describing the issue in the thread with more details

many-yak-61188

07/07/2021, 3:27 AM

Following up here from the thread above. After a lot of painful experimentation I settled on this work around - bottom line

pulumi/actions@v3

and

poetry

don't play nice together in a github workflow environment. So instead of simply being able to do

Copy code

# ----------------------------------------------
            #          install & configure poetry
            # ----------------------------------------------
            - name: Install Poetry
              uses: snok/install-poetry@v1.1.6
              with:
                  virtualenvs-create: true

            - name: Install dependencies
              run: |
                  poetry install --no-interaction

            # ----------------------------------------------
            #          Run pulumi in preview mode
            # ----------------------------------------------

            - name: Run pulumi preview
              uses: pulumi/actions@v3
              with:
                  command: preview
                  stack-name: dev
                  cloud-url: <s3://accrue-pulumi>

(poetry installs

pulumi

packages) I had to install

pulumi

packages via

pip

... again

Copy code

# ----------------------------------------------
            #          install & configure poetry
            # ----------------------------------------------
            - name: Install Poetry
              uses: snok/install-poetry@v1.1.6
              with:
                  virtualenvs-create: true

            - name: Install dependencies
              run: |
                  poetry install --no-interaction

            # ----------------------------------------------
            #          Run pulumi in preview mode
            # ----------------------------------------------

            - name: Install pulumi via pip
              run: |
                  pip install pulumi
                  pip install pulumi-aws

            - name: Run pulumi preview
              uses: pulumi/actions@v3
              with:
                  command: preview
                  stack-name: dev
                  cloud-url: <s3://accrue-pulumi>

If I come up with a better way, I'll post it in the thread

some-twilight-56575

07/07/2021, 4:44 PM

I could use some help with this use case I need to create a k8s secret (tls) from a SelfSigned Cert generated by pulumi

Copy code

key = pulumi_tls.PrivateKey(
        "cluster-issuer-key", algorithm="RSA", rsa_bits=4096
    )
    # private_key_pem public_key_pem
    ca = pulumi_tls.SelfSignedCert(
        "cluster-issuer-cert",
        is_ca_certificate=True,
        private_key_pem=key.private_key_pem,
        validity_period_hours=87600,
        key_algorithm="RSA",
        subjects=[
            pulumi_tls.SelfSignedCertSubjectArgs(
                common_name=f"{stack_name} Communication CA"
            )
        ],
        allowed_uses=[
            "cert_signing",
            "key_encipherment",
            "digital_signature",
            "server_auth",
        ],
    )

    issuer_secret = pulumi_kubernetes.core.v1.Secret(
        "cluster-issuer-secret",
        metadata={"name": "ca-issuer", "namespace": "cert-manager"},
        type="<http://kubernetes.io/tls|kubernetes.io/tls>",
        data={"tls.key": key.private_key_pem, "tls.crt": key.cert_pem},
    )

it appears data must be pre base64 encoded

high-cartoon-83388

07/09/2021, 10:44 AM

Hello guys, when using the pulumi python sdk do you know if there is there any way to tell if the CLI was invoked with

pulumi up

pulumi destroy

pulumi preview

? Something like

pulumi.runtime.command

perhaps ?

great-sunset-355

07/12/2021, 8:02 AM

I'm learning about ComponentResources my code: https://gist.github.com/1oglop1/c563f25b317ecd6149e5465bd9eea597 And I'm trying to understand how

register_outputs

should work with secrets

opts=ResourceOptions(additional_secret_outputs=['password'])

The result of the code above is this for both the

First

and the

Second

component. I'm probably doing something wrong that the

password

is not marked as a secret value in resource outputs. here is the result of

pulumi stack export

Copy code

{
    "version": 3,
        "resources": [
            {
                "urn": "urn:pulumi:dev::tests::pulumi:pulumi:Stack::tests-dev",
                "custom": false,
                "type": "pulumi:pulumi:Stack"
            },
            {
                "urn": "urn:pulumi:dev::tests::custom:first::first",
                "custom": false,
                "type": "custom:first",
                "outputs": {
                    "password": "222222",
                    "w2": "222222"
                },
                "parent": "urn:pulumi:dev::tests::pulumi:pulumi:Stack::tests-dev",
                "additionalSecretOutputs": [
                    "password"
                ]
            },
            {
                "urn": "urn:pulumi:dev::tests::custom:second::second",
                "custom": false,
                "type": "custom:second",
                "outputs": {
                    "password": "Secret"
                },
                "parent": "urn:pulumi:dev::tests::pulumi:pulumi:Stack::tests-dev",
                "additionalSecretOutputs": [
                    "password"
                ]
            }
        ]
    }
}

astonishing-dinner-89046

07/12/2021, 3:47 PM

I’m trying to incorporate a certificate validation workflow. I’m trying to use the example below but

dvo.domainName

as well as (name, record) parameters in the

for range in

loop doesn’t seem to be working. Any ideas?

witty-dentist-91954

07/13/2021, 4:40 PM

I'm setting up an gke nodeport service and want to add the new node_port to a firewall rule, so the default health check works. I'm trying to get the port like this:

my_nodeport = my_svc.spec.apply(lambda p: p.ports[0]['node_port'])

But this failes with an

KeyError: 'node_port'

in outputs.py. It works, if I first create the NodePort and create the firewall in a second run. I would like to avoid setting a fixed node_port in the service. Is there a way to do that?

broad-hairdresser-1495

07/15/2021, 2:54 PM

Hi,

Copy code

Do you want to perform this update?  [Use arrows to move, enter to select, type to filter]
  yes
> no
  details

Is there a Pulumi function that can retrieve user input when running

pulumi up

average-school-38756

07/15/2021, 11:33 PM

i'd like to create an

aws.secretsmanager.Secret

and a corresponding

SecretVersion

, but not have the value stored anywhere in state, not even encrypted - i just want Secrets Manager to keep track of the value. Is there a way to write this way?

shy-author-33795

07/21/2021, 3:59 PM

I'm following this example of post-provisioning with a dynamic provider here. I have copied the

provisioners.py

file without revision and I'm importing it in my

__main__.py

with

import provisioners

. I am using a dynamic provider to remote exec into a server with network access to an aurora cluster to run a MySQL command,

show databases

. However, my program errors:

Copy code

└─ pulumi-python:dynamic:Resource  show-dbs                                      1 error

Diagnostics:
  pulumi-python:dynamic:Resource (show-dbs):
    error: Exception calling application: No module named 'provisioners'

I'm not sure what i'm doing wrong to make this module unavailable.

aloof-jelly-80665

07/21/2021, 8:36 PM

Working with Google Artifact Registry, I am having trouble finding an example of initializing a google_beta provider in Python. All the examples I find reference a

google_beta

variable, but I can’t find an instantiation of it.

great-alligator-26951

07/28/2021, 12:48 AM

I am really having some trouble I was hoping someone would be able to help. I am trying to feed a Pulumi Python Output to a configuration that expects a string. For the life of me I cant seem to figure out how to get the output object to a string. The string I need to feed is to another python module for configuring a SaaS application, but I can imagine I am going to have the same problem if I need to use a pulumi program inline with AWS boto3. Can someone share a snippet? This is the only way I have found that somewhat works. Write to a file and then read it when I need to assign the variable which is horrible

Copy code

# define a function to write an arn to a file
def write_to_file(arn):
    f = open("arn.txt", "a")
    f.write(arn)
    f.close()

json = lb.arn.apply(lambda a: write_to_file(arn=a))

great-sunset-355

07/29/2021, 7:28 AM

What's the best way to get a proper type hinting for Outputs? Is there any cast method or anything available? acm certificate has the output of type

CertificateDomainValidationOptionArgs

https://www.pulumi.com/docs/reference/pkg/aws/acm/certificate/#outputs I'd like to be able to use it as

cert_validation_option.domain_name

, etc...