pulumi-community #python

python

incalculable-dream-27508

08/12/2020, 6:22 PM

I'm trying to figure out how to use https://cloudinit.readthedocs.io/en/latest/topics/modules.html#set-hostname (with OpenStack). As I understand it, I need to pass

user_data

as follows:

#cloud-config
fqdn: host.name.goes.here

which is simple enough... But I want to create multiple machines, and I have considerably more in

user_data

. So far I'm defining

PROVISION

which holds the string with a bunch of data, with the intention of using it in mine

INSTANCES = {
  i: openstack.compute.Instance(f'instance-{i + 1}.test.instance',
                                flavor_name="1C-1G-20G",
                                image_name="Debian-Buster-amd64",
                                key_pair="my_key",
                                networks=[{
                                  'name': 'test-net',
                                  'accessNetwork': 'true'
                                }],
                                user_data=PROVISION,
                                security_groups=SECGROUPS_TEST)
  for i in range(INSTANCE_COUNT)
  }

Any suggestions how I can set / inject / template in

PROVISION

passed here

fqdn: instance-{i + 1}.test.instance

incalculable-dream-27508

08/12/2020, 6:36 PM

Oh, seems like https://docs.python.org/3/library/string.html#template-strings might be what I'm looking for

nutritious-shampoo-16116

08/13/2020, 10:49 AM

hi people, was anybody able to mock StackReference when running tests? We are seeing a weird problems where the object is mocked correctly but for some reasons Pulumi is till trying to use StackReference to the stack and the coroutine fails because of that

witty-ice-69000

08/13/2020, 2:53 PM

Any ideas what this error is trying to tell me? Google doesn't help and neither does Pulumi docs.

error: Program failed with an unhandled exception:
    error: Traceback (most recent call last):
      File "/usr/local/bin/pulumi-language-python-exec", line 85, in <module>
        loop.run_until_complete(coro)
      File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
        return future.result()
      File ".venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 83, in run_in_stack
        await run_pulumi_func(lambda: Stack(func))
      File ".venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 51, in run_pulumi_func
        await RPC_MANAGER.rpcs.pop()
      File ".venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 35, in run_pulumi_func
        func()
      File ".venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 83, in <lambda>
        await run_pulumi_func(lambda: Stack(func))
      File ".venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 106, in __init__
        func()
      File "/usr/local/bin/pulumi-language-python-exec", line 84, in <lambda>
        coro = pulumi.runtime.run_in_stack(lambda: runpy.run_path(args.PROGRAM, run_name='__main__'))
      File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 282, in run_path
        return _run_code(code, mod_globals, init_globals,
      File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "./__main__.py", line 41, in <module>
        main()
      File "./__main__.py", line 37, in main
        create_stack_deployment_iam(this_provider, "production", permission_boundaries)
      File "/asi/aws/infrastructure.py", line 62, in create_stack_deployment_iam
        instance_assume_role_policy = iam.get_policy_document(
      File ".venv/lib/python3.8/site-packages/pulumi_aws/iam/get_policy_document.py", line 317, in get_policy_document
        __ret__ = pulumi.runtime.invoke('aws:iam/getPolicyDocument:getPolicyDocument', __args__, opts=opts).value
      File ".venv/lib/python3.8/site-packages/pulumi/runtime/invoke.py", line 127, in invoke
        return InvokeResult(_sync_await(asyncio.ensure_future(do_rpc())))
      File ".venv/lib/python3.8/site-packages/pulumi/runtime/sync_await.py", line 95, in _sync_await
        return fut.result()
      File ".venv/lib/python3.8/site-packages/pulumi/runtime/invoke.py", line 124, in do_rpc
        raise exn
      File ".venv/lib/python3.8/site-packages/pulumi/runtime/rpc_manager.py", line 67, in rpc_wrapper
        result = await rpc
      File ".venv/lib/python3.8/site-packages/pulumi/runtime/invoke.py", line 108, in do_invoke
        resp = await asyncio.get_event_loop().run_in_executor(None, do_invoke)
      File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/concurrent/futures/thread.py", line 57, in run
        result = self.fn(*self.args, **self.kwargs)
      File ".venv/lib/python3.8/site-packages/pulumi/runtime/invoke.py", line 106, in do_invoke
        raise Exception(details)
    Exception: invocation of aws:iam/getPolicyDocument:getPolicyDocument returned an error: grpc: error while marshaling: proto: repeated field Values has nil element
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

The snippet of my code that it's referencing is this:

witty-ice-69000

08/13/2020, 2:53 PM

instance_assume_role_policy = iam.get_policy_document(
        opts=pulumi.ResourceOptions(depends_on=[user], provider=provider),
        statements=[
            {
                "actions": ["sts:AssumeRole"],
                "effect": "Allow",
                "principals": [
                    {"identifiers": [user.arn.apply(lambda arn: arn)], "type": "AWS"}
                ],
            },
        ],
    )

helpful-processor-86468

08/14/2020, 11:48 AM

any clue why im getting:

gcp:organizations/getIAMPolicy:getIAMPolicy returned an error: grpc: error while marshaling: proto: repeated field Values has nil element

with:

scheduler_policy_func_invoker = organizations.get_iam_policy(bindings=[
        {
            "members": [job_account_member],
            "role": "roles/cloudfunctions.invoker",
        },
    ])

fierce-market-67222

08/14/2020, 12:02 PM

hi i have some problems with pulumi_gcp ….i use it to create dns records on google and it works from on-prem clusters but i am getting forbidden when trying to do the same thing from a gke cluster …same user same code… but no clue why..thank you

stocky-spoon-28903

08/15/2020, 10:52 PM

Is there a way to catch exceptions from a data source in a python Pulumi program?

astonishing-branch-30086

08/17/2020, 12:45 PM

Is there a hook for the event that Pulumi creates a resource? I would like to print a message if and only if a resource of a particular type was created by Pulumi in the current

pulumi up

call.

early-match-56268

08/18/2020, 12:40 PM

Hi 👋 , does anyone know how pulumi figures out which plugin version to install? I started with a fresh installation of pulumi (cleaned the

~/.pulumi

dir and reinstalled) and then tried to run my project but the wrong plugin versions are being installed when I bring up my project:

❯ poetry run pulumi up
Previewing update (prod):
[resource plugin kubernetes-2.4.0] installing
Downloading plugin: 23.95 MiB / 23.95 MiB [=========================] 100.00% 1s
Moving plugin... done.

error: could not load plugin for aws provider 'urn:pulumi:prod::new-data::pulumi:providers:aws::aws-east': no resource plugin 'aws' found in the workspace or on your $PATH

the aws plugin is not installed and the kubernetes version in my project is 2.4.3 but 2.4.0 plugin is being installed. I can workaround by manually installing the plugins but it is a bit inconvenient

echoing-breakfast-73834

08/18/2020, 4:04 PM

Hey all, does anybody know how to properly return

CheckResult

object with failures? I've tried

return CheckResult(None, check_failures)

return CheckResult([], check_failures)

and

return CheckResult(_news, check_failures)

but each of them raise a different error.

echoing-dawn-86996

08/19/2020, 4:09 PM

Hey guys! I'm trying to upgrade from '2.2.1' to '2.8.2' and I'm getting import errors:

from pulumi.resource import ComponentResource, ResourceOptions, export
from pulumi_gcp import service_account, projects

This is the error:

File "./components/service_accounts_binding.py", line 2, in <module>
    from pulumi_gcp import service_account, projects
ImportError: cannot import name 'service_account' from 'pulumi_gcp' (/usr/local/lib/python3.7/site-packages/pulumi_gcp/__init__.py)

My requirements.txt:

pulumi>=2.0.0,<3.0.0
pulumi-gcp>=3.0.0,<4.0.0

echoing-dawn-86996

08/19/2020, 4:10 PM

Not sure what is going on in that package, but all seem correct to me.. What am I missing?

echoing-dawn-86996

08/19/2020, 4:20 PM

Looks like changing service_account to serviceaccount fixed the issue. This is similar issue to this: https://github.com/pulumi/pulumi-gcp/issues/410 I'll comment there too

bumpy-beach-79487

08/19/2020, 4:41 PM

Hi there!

bumpy-beach-79487

08/19/2020, 4:42 PM

i'm having the same issue

bumpy-beach-79487

08/19/2020, 4:42 PM

I have already changed the name to

serviceaccount

bumpy-beach-79487

08/19/2020, 4:43 PM

from pulumi import Config
from pulumi import export as output
from pulumi import ResourceOptions
from pulumi_gcp import cloudrun, serviceaccount

bumpy-beach-79487

08/19/2020, 4:44 PM

This is the error I received:

File "./__main__.py", line 4, in <module>
        from pulumi_gcp import cloudrun, serviceaccount
    ImportError: cannot import name 'serviceaccount' from 'pulumi_gcp' (/usr/local/lib/python3.7/site-packages/pulumi_gcp/__init__.py)
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

bumpy-beach-79487

08/19/2020, 5:02 PM

I updated to python3.8, that fixed my error

broad-dog-22463

08/19/2020, 7:57 PM

Hi all We must apologise for the breaking change that occured in the minor version bump of the GCP provider. v3.19.0 introduced a codegen change that ensured consistent package names. This changed the following package names: Go

serviceAccount -> serviceaccount

NodeJS

serviceAccount -> serviceaccount

Python

service_account -> serviceaccount

This was an unintended consequence of the codegen change being released and is something that we should have ensured happened in a controlled manner rather than forcing the breakage on the customer We are going to have a followup post-mortem event to ensure we learn from this and that we add better quality bars to our releases so that we don't release these breaking changes without any warning or notice. We will publish the results of this post-mortem to make sure our customers understand what we have learned from this I am sorry for this issue. If this issue will cause you any unnecessary replacement of any infrastructure, then please do let us know and we can work with you on making sure this isn't the case Paul

busy-iron-87492

08/20/2020, 8:49 AM

Hi there! I'm doing some investigation to do some integration tests like described here: https://github.com/pulumi/examples/tree/31056c3480cc445e5d4d3a8a0a86977adce2bc5e/testing-integration only thing is that I would do it in python, does anyone have any example git project where this is illustrated?

rhythmic-napkin-82334

08/25/2020, 10:56 AM

message has been deleted

limited-knife-15571

08/26/2020, 10:04 PM

I'm trying to reproduced the script from here (in typescript) in python to create a digitalocean container registry credential

ContainerRegistryDockerCredentials

in python. https://github.com/pulumi/pulumi-kubernetesx/issues/49#issuecomment-651498775 My code look like that

container_registry = ContainerRegistry.get("source", "hyperwave-research")
        registry_creds = pulumi.Output.secret( ContainerRegistryDockerCredentials(
            "container-registry-creds", registry_name=container_registry.name).docker_credentials)

        def encode_base64_dc(dc):
            message_bytes = dc.encode('ascii')
            base64_bytes = base64.b64encode(message_bytes)
            print(f"===> Docker  : {dc}")
            print(f"===> Docker Credential : {base64_bytes.decode('utf-8')}")
            return base64_bytes.decode("utf-8")

        dc = registry_creds.apply(encode_base64_dc)

        self.secret = Secret(
            "registry-creds-kube-secret",
            type="<http://kubernetes.io/dockerconfigjson|kubernetes.io/dockerconfigjson>",
            metadata={"namespace": "default", "name": f"registry-{do_registry_name}"},
            string_data={".dockerconfigjson": dc},
            opts=opts,
        )

However whatever I tried, the docker.credential is everytime None. I tried other property of the

ContainerRegistryDockerCredentials

and I can confirm I have value like registry_name

```
      File "/home/dzucker/git/Hyperwave.Infrastructure/venv/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 192, in serialize_property
        obj[transformed_key] = await serialize_property(v, deps, input_transformer)
      File "/home/dzucker/git/Hyperwave.Infrastructure/venv/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 173, in serialize_property
        value = await serialize_property(output.future(), deps, input_transformer)
      File "/home/dzucker/git/Hyperwave.Infrastructure/venv/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 159, in serialize_property
        future_return = await asyncio.ensure_future(awaitable)
      File "/home/dzucker/git/Hyperwave.Infrastructure/venv/lib/python3.8/site-packages/pulumi/output.py", line 112, in get_value
        val = await self._future
      File "/home/dzucker/git/Hyperwave.Infrastructure/venv/lib/python3.8/site-packages/pulumi/output.py", line 174, in run
        transformed: Input[U] = func(value)
      File "./digitalocean/docker_registry.py", line 42, in print_dc
        message_bytes = dc.encode('ascii')
    AttributeError: 'NoneType' object has no attribute 'encode'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

I tried other idea that I found from the slack https://pulumi-community.slack.com/archives/C84L4E3N1/p1596457104392200?thread_ts=1596452646.389800&cid=C84L4E3N1 however same result. Am I doing something ? or could it be they are a mapping error on the python property naming.

worried-plastic-45846

08/27/2020, 4:52 PM

Hello, I had a codebuild.Project() configured to source from github, with

source_version="master"

. I switched this CodeBuild project to an S3 source, and removed the

source_version

parameter entirely (it's optional for S3, and defaults to the latest version of the source artifact). After a

pulumi up

, CodeBuild kept the value at "master", I had to explicitly add

source_version=""

to my codebuild.Project(). Is this the expected behavior? FYI @broad-dog-22463 @bright-orange-69401

gentle-soccer-81698

08/27/2020, 6:28 PM

Hi all, I'm trying to get vsphere host thumbprint using the new resource and I'm getting an error when trying to retrieve it.

raise ValueError(f"unexpected input of type {type(value).__name__}")
    ValueError: unexpected input of type GetHostThumbprintResult

gentle-soccer-81698

08/27/2020, 6:28 PM

hosts = ['<http://vlab-esx-80.vballin.com|vlab-esx-80.vballin.com>','<http://vlab-esx-90.vballin.com|vlab-esx-90.vballin.com>','<http://vlab-esx-100.vballin.com|vlab-esx-100.vballin.com>','<http://vlab-esx-110.vballin.com|vlab-esx-110.vballin.com>','<http://vlab-esx-120.vballin.com|vlab-esx-120.vballin.com>','<http://vlab-esx-130.vballin.com|vlab-esx-130.vballin.com>','<http://vlab-esx-140.vballin.com|vlab-esx-140.vballin.com>','<http://vlab-esx-150.vballin.com|vlab-esx-150.vballin.com>']

host_thumb_list = []
def get_host_thumbprint():
    for h in hosts:
        thumbprint = vsphere.get_host_thumbprint(address=h, insecure=True)
        host_thumb_list.append(thumbprint)
    return
get_host_thumbprint()

gentle-soccer-81698

08/27/2020, 6:28 PM

https://www.pulumi.com/docs/reference/pkg/vsphere/gethostthumbprint/

gentle-soccer-81698

08/27/2020, 6:33 PM

pulumi version
v2.9.0
pulumi plugin ls
NAME     KIND      VERSION  SIZE   INSTALLED   LAST USED
vsphere  resource  2.8.0    61 MB  2 days ago  2 days ago

many-psychiatrist-74327

08/28/2020, 3:31 AM

Hi all. I’m having troubles using

tags

on AWS resource, and the error seems to be related the RPC python implementation…

pulumi up

fails with:

File "/<path-redacted>/python3.8/site-packages/pulumi/runtime/rpc.py", line 451, in translate_output_properties
        raise AssertionError(f"Unexpected type. Expected 'list' got '{typ}'")
    AssertionError: Unexpected type. Expected 'list' got '<class 'str'>'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

With this,

pulumi up

works:

for zone in zones.names:
    vpc_subnet = ec2.Subnet(
        f'vpc-subnet-{zone}',
        assign_ipv6_address_on_creation=False,
        vpc_id=vpc.id,
        map_public_ip_on_launch=True,
        cidr_block=f'10.100.{len(subnet_ids)}.0/24',
        availability_zone= zone,
        tags={
            'Name' : f'pulumi-sn-{zone}'
        }
    )

With this, it doesn’t:

for zone in zones.names:
    vpc_subnet = ec2.Subnet(
        f'vpc-subnet-{zone}',
        assign_ipv6_address_on_creation=False,
        vpc_id=vpc.id,
        map_public_ip_on_launch=True,
        cidr_block=f'10.100.{len(subnet_ids)}.0/24',
        availability_zone= zone,
        tags={
            'Name' : f'pulumi-sn-{zone}',
            'new-tag': 'new-value'  # <---- this is the only change
        }
    )

Any ideas/suggestions on what might be going on here?

Title

many-psychiatrist-74327

08/28/2020, 3:31 AM

Hi all. I’m having troubles using

tags

on AWS resource, and the error seems to be related the RPC python implementation…

pulumi up

fails with:

File "/<path-redacted>/python3.8/site-packages/pulumi/runtime/rpc.py", line 451, in translate_output_properties
        raise AssertionError(f"Unexpected type. Expected 'list' got '{typ}'")
    AssertionError: Unexpected type. Expected 'list' got '<class 'str'>'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

With this,

pulumi up

works:

for zone in zones.names:
    vpc_subnet = ec2.Subnet(
        f'vpc-subnet-{zone}',
        assign_ipv6_address_on_creation=False,
        vpc_id=vpc.id,
        map_public_ip_on_launch=True,
        cidr_block=f'10.100.{len(subnet_ids)}.0/24',
        availability_zone= zone,
        tags={
            'Name' : f'pulumi-sn-{zone}'
        }
    )

With this, it doesn’t:

for zone in zones.names:
    vpc_subnet = ec2.Subnet(
        f'vpc-subnet-{zone}',
        assign_ipv6_address_on_creation=False,
        vpc_id=vpc.id,
        map_public_ip_on_launch=True,
        cidr_block=f'10.100.{len(subnet_ids)}.0/24',
        availability_zone= zone,
        tags={
            'Name' : f'pulumi-sn-{zone}',
            'new-tag': 'new-value'  # <---- this is the only change
        }
    )

Any ideas/suggestions on what might be going on here?

Actually.. even if I just change the tag value (from

f'pulumi-sn-{zone}'

test

pulumi up

fails with that error…

seems to happen on any changes to a Vpc, actually

i destroyed my stack, created a new one. deleted ALL my resources except the Vpc, and it’s still failing. if I comment out the Vpc, everything works just fine

microscopic-pilot-97530

08/28/2020, 9:40 PM

Hi @many-psychiatrist-74327, I’ve reproduced the problem. It looks like it’s due to some extra type checks we added in the

pulumi

Python SDK in v2.9.0, which fails when certain values come back from the engine that aren’t expected. I’m going to open an issue and work on a fix. In the meantime, you can workaround by downgrading you Python package dependencies to

pulumi

v2.8.2 and

pulumi_aws

v3.1.0.

many-psychiatrist-74327

08/28/2020, 9:41 PM

Hi @microscopic-pilot-97530! Thanks for taking a look!

I also cross-posted this to #general, and found another person who ran into the same issue

Threads: https://pulumi-community.slack.com/archives/C84L4E3N1/p1598590542226600?thread_ts=1598590542.226600&cid=C84L4E3N1 https://pulumi-community.slack.com/archives/C84L4E3N1/p1598631037246000?thread_ts=1598631037.246000&cid=C84L4E3N1 Thanks again!

microscopic-pilot-97530

08/28/2020, 9:47 PM

Thanks, also posting responses there.

https://github.com/pulumi/pulumi/issues/5250

The fix has been released in version

2.9.2

of the

pulumi

package, which will work with

pulumi-aws

version

3.2.1

View count: 6