https://pulumi.com logo
Docs
Join the conversationJoin Slack
Channels
announcements
automation-api
aws
azure
blog-posts
built-with-pulumi
cloudengineering
cloudengineering-support
content-share
contribex
contribute
docs
dotnet
finops
general
getting-started
gitlab
golang
google-cloud
hackathon-03-19-2020
hacktoberfest
install
java
jobs
kubernetes
learn-pulumi-events
linen
localstack
multi-language-hackathon
office-hours
oracle-cloud-infrastructure
plugin-framework
pulumi-cdk
pulumi-crosscode
pulumi-deployments
pulumi-kubernetes-operator
pulumi-service
pulumiverse
python
registry
status
testingtesting123
testingtesting321
typescript
welcome
workshops
yaml
Powered by Linen
python
  • w

    worried-needle-99800

    08/30/2020, 8:32 AM
    I believe it’ll start from
    __main__.py
  • w

    worried-needle-99800

    08/30/2020, 8:34 AM
    Not 100% sure but I don’t think there’s any particular naming convention for multiple files
  • w

    worried-needle-99800

    08/30/2020, 8:36 AM
    Most programs in the examples repo (https://github.com/pulumi/examples/) seem pretty basic, just putting Python files under the same directory as the
    __main__.py
    file
  • h

    hallowed-beach-15050

    08/31/2020, 3:40 AM
    yeah,
    __main__.py
    is the way to go
  • a

    astonishing-quill-88807

    08/31/2020, 7:14 PM
    I'm trying to use a dynamic provider and it's throwing an exception, but it doesn't show the full traceback. Is there a way to see the full Python traceback to understand where and why it's actually erroring?
  • h

    hallowed-beach-15050

    08/31/2020, 9:10 PM
    @astonishing-quill-88807 what version of pulumi? and can you paste the error you are getting along with the code that you think may be generating the error?
  • h

    hallowed-beach-15050

    08/31/2020, 9:11 PM
    inline or a gist would be awesome
  • a

    astonishing-quill-88807

    08/31/2020, 9:11 PM
    I figured out the problem. I'm using v2.9.2 and my issue was trying to address the input properties as an attribute, rather than via dictionary access. Here's the code for posterity: https://github.com/mitodl/ol-infrastructure/blob/dagster_instance/src/ol_infrastructure/providers/salt/minion.py
  • a

    astonishing-quill-88807

    08/31/2020, 9:12 PM
    It was throwing attribute errors about
    salt_api_url
  • h

    hallowed-beach-15050

    08/31/2020, 9:47 PM
    makes sense
  • d

    damp-elephant-82829

    09/02/2020, 6:36 AM
    Guys when you write custom components in Python, does the framework offers standard logging facilities or do you use plain python logging?
    g
    • 2
    • 2
  • d

    damp-elephant-82829

    09/02/2020, 7:24 AM
    Awesome. One additional question related to ComponentResources: is it an anti pattern to give external access to child resources instead of registering their output?
  • s

    strong-musician-98782

    09/02/2020, 8:39 AM
    Hey guys, i'm trying to create a class to create ec2 instance and then call it from my main but pulumi keeps thinking my userdata is changing even when its not, here's part of my class(tried to edit out as much as i can so some args might be missing):
    import pulumi
    from pulumi_aws import ec2
    
    class KSDBServerArgs:
        """
        The arguments necessary to construct an `KSDBServer` resource.
        """
        def __init__(self,
                     ks_id: str):
            """
            Constructs an KSArgs.
            :param ks_id:
            :param user_data:
            """
    
            self.ks_id = ks_id
            config = pulumi.Config()
    
            self.user_data = '''
            #!/bin/bash
            /usr/sbin/useradd -d /home/user -m -s /bin/bash user
            echo "user ALL=(ALL:ALL) NOPASSWD: ALL" >> /etc/sudoers
            mkdir -p /home/user/.ssh
            chown -R user:user /home/user/.ssh
            chmod 700 /home/user/.ssh
            echo 'ssh-rsa {1}' >> /root/.ssh/authorized_keys
            '''.format(config.require('build_token'),config.require('certificate'))
    
    
    
    class KSDBServer(pulumi.ComponentResource):
        def __init__(self,
                     name: str,
                     args: KSDBServerArgs,
                     opts:pulumi.ResourceOptions = None):
            super().__init__('kenshoo:KS:KSDBServer', name, None, opts)
            self.name = name
    
            # Creating ec2 instance
            self.db_instance = ec2.Instance("aws-"+ args.ks_id + "-db",
                                                    instance_type=args.instance_size,
                                                    vpc_security_group_ids=args.db_sec_groups,
                                                    subnet_id=args.subnet,
                                                    ami=args.ami,
                                                    user_data=args.user_data,
                                                    opts=pulumi.ResourceOptions(parent=self,delete_before_replace=True))
    g
    • 2
    • 2
  • s

    shy-football-10348

    09/02/2020, 5:48 PM
    Hey folks, I have narrowed down an issue with pulumi preview hanging to the AWS
    cloudformation.Stack
    module, I am running the following command to bring up pulumi preview:
    pulumi up --logtostderr -v=9 --debug 2> /tmp/out.txt
    the log file always hangs at the following entry
    I0902 10:44:30.806398   96259 eventsink.go:59] adding resolver weighted_routing_policies
    I0902 10:44:30.806446   96259 eventsink.go:62] eventSink::Debug(<{%reset%}>adding resolver weighted_routing_policies<{%reset%}>)
    I0902 10:44:30.806722   96259 eventsink.go:59] adding resolver zone_id
    I0902 10:44:30.806768   96259 eventsink.go:62] eventSink::Debug(<{%reset%}>adding resolver zone_id<{%reset%}>)
    I0902 10:44:30.807050   96259 eventsink.go:59] adding resolver fqdn
    I0902 10:44:30.807108   96259 eventsink.go:62] eventSink::Debug(<{%reset%}>adding resolver fqdn<{%reset%}>)
    here is a copy of my stack
    stack = aws.cloudformation.Stack(
        "jupyterhub-stack",
        capabilities=['CAPABILITY_NAMED_IAM'],
        template_url="redacted",
        parameters={
            "ServiceName": "jupyterhub",
            "TeamName": "redacted",
            "TeamEmail": "redacted",
            "AdditionalDomains": "redacted",
            "InternalFQDN": "redacted",
            "AlertingEnabled": "disabled",
            "AlertsEmail": "",
            "PagerDutyIntegrationURL": "",
            "AuthenticationType": "mtls_saml",
            "PingIntegrationName": "redacted",
            "SandstormRole": "redacted", #TODO Update
            "SandstormPrefix": "redacted", #TODO Update
            "VPC": jupyterhub_vpc,
            "PublicSubnets": ','.join(jupyterhub_pub_subnets),
            "PrivateSubnets": ','.join(jupyterhub_priv_subnets),
            "InstanceType": "t3.medium",
            "BackendScheme": "https",
            "Environment": "production",
            "PreserveHostname": "enabled",
            "XFrameOptions": "deny",
            "ASGMaxSize": "6",
            "ASGMinSize": "2"
        },
    )
  • s

    shy-football-10348

    09/02/2020, 5:49 PM
    I understand the values of my parameters may be the source of the issue, which I am happy to share if folks think that’s the case.
  • s

    shy-football-10348

    09/02/2020, 5:49 PM
    has anyone run into this before and may know where to look next? thanks everyone
  • s

    shy-football-10348

    09/02/2020, 5:52 PM
    i suspect it may have something to do with the subnets and how I am building that value with join, i’ll give that a shot
  • s

    shy-football-10348

    09/02/2020, 6:01 PM
    yea, pulumi implodes if you do this
    priv_subnets = ','.join(jupyterhub_priv_subnets)
  • g

    green-school-95910

    09/02/2020, 6:03 PM
    I suspect
    jupyterhub_priv_subnets
    is an
    Output
    from somewhere else
  • s

    shy-football-10348

    09/02/2020, 6:03 PM
    correct
  • g

    green-school-95910

    09/02/2020, 6:03 PM
    If that is the case you should do
    jupyterhub_priv_subnets.apply(','.join)
  • g

    green-school-95910

    09/02/2020, 6:05 PM
    Or less cryptic if you are not used to passing bound methods as arguments:
    jupyterhub_priv_subnets.apply(lambda subnets: ','.join(subnets))
  • s

    shy-football-10348

    09/02/2020, 6:05 PM
    wow, that works. thank you so much. i have a hard time understanding
    .apply
    for whatever reason
  • g

    green-school-95910

    09/02/2020, 6:13 PM
    It is a problem for multiple people. I tried to explain why is that a few days ago here
  • d

    damp-elephant-82829

    09/02/2020, 7:48 PM
    Guys I have created a composite component which creates a GCP project, an associated service account and key, a new provider associated with the key
    from pulumi import ComponentResource, ResourceOptions
    from pulumi import Output
    import pulumi_gcp
    from pulumi_gcp import (
        organizations,
        projects,
        pubsub,
        storage,
        serviceAccount,
    )
    import base64
    
    
    class ProjectArgs:
        def __init__(
            self, project_name: str, root_project_name: str, organization_name: str
        ):
            self.project_name = project_name
            self.root_project_name = root_project_name
            self.organization_name = organization_name
    
    
    class Project(ComponentResource):
    
        new_project_id: Output[str]
        ephemeral_project_provider: pulumi_gcp.Provider
        project_owner_service_account: pulumi_gcp.serviceAccount.Account
    
        def __init__(self, name: str, args: ProjectArgs, opts: ResourceOptions = None):
    
            super().__init__("my-org:modules:Project", name, {}, opts)
    
            root_project = organizations.Project.get(
                f"{name}-root-project", id=args.root_project_name
            )
            organization = organizations.get_organization(
                organization=args.organization_name
            )
    
            # Create an ephemeral project
            ephemeral_project = organizations.Project(
                f"{name}-new-project",
                name=args.project_name,
                project_id=args.project_name,
                billing_account=root_project.billing_account,
                org_id=organization.org_id,
            )
    
            self.project_owner_service_account = serviceAccount.Account(
                resource_name=f"{args.project_name}-project-owner-service-account",
                account_id="projectowner",
                project=ephemeral_project.project_id,
            )
    
            project_owner_service_account_key = serviceAccount.Key(
                resource_name=f"{args.project_name}-project-owner-service-account-key",
                service_account_id=self.project_owner_service_account.name,
            )
    
            project_owner_serviceaccount_iam_membership = projects.IAMMember(
                resource_name=f"{args.project_name}-project-owner-service-account-iam-member",
                project=ephemeral_project.project_id,
                role="roles/owner",
                member=self.project_owner_service_account.email.apply(
                    lambda service_account_email: f"serviceAccount:{service_account_email}"
                ),
            )
    
            resourceManagerService = projects.Service(
                f"{args.project_name}-enable-resorce-management",
                project=ephemeral_project.project_id,
                service="<http://cloudresourcemanager.googleapis.com|cloudresourcemanager.googleapis.com>",
            )
    
            self.ephemeral_project_provider = pulumi_gcp.Provider(
                resource_name="ephemeral_project_provider",
                credentials=project_owner_service_account_key.private_key.apply(
                    lambda k: base64.b64decode(k).decode("utf-8")
                ),
                opts=ResourceOptions(
                    depends_on=[
                        resourceManagerService,
                        project_owner_serviceaccount_iam_membership,
                    ],
                ),
            )
    
            self.register_outputs({"new_project_id": ephemeral_project.project_id})
    Now when I try to use it in my stack, I get an exception because the event loop is already closed
  • d

    damp-elephant-82829

    09/02/2020, 7:49 PM
    Step #9 - "Pulumi Create Stack":     error: Program failed with an unhandled exception:
    Step #9 - "Pulumi Create Stack":     error: Traceback (most recent call last):
    Step #9 - "Pulumi Create Stack":       File "/pulumi/pulumi-language-python-exec", line 85, in <module>
    Step #9 - "Pulumi Create Stack":         loop.run_until_complete(coro)
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
    Step #9 - "Pulumi Create Stack":         return future.result()
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi/runtime/stack.py", line 83, in run_in_stack
    Step #9 - "Pulumi Create Stack":         await run_pulumi_func(lambda: Stack(func))
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi/runtime/stack.py", line 51, in run_pulumi_func
    Step #9 - "Pulumi Create Stack":         await RPC_MANAGER.rpcs.pop()
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi/runtime/stack.py", line 35, in run_pulumi_func
    Step #9 - "Pulumi Create Stack":         func()
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi/runtime/stack.py", line 83, in <lambda>
    Step #9 - "Pulumi Create Stack":         await run_pulumi_func(lambda: Stack(func))
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi/runtime/stack.py", line 106, in __init__
    Step #9 - "Pulumi Create Stack":         func()
    Step #9 - "Pulumi Create Stack":       File "/pulumi/pulumi-language-python-exec", line 84, in <lambda>
    Step #9 - "Pulumi Create Stack":         coro = pulumi.runtime.run_in_stack(lambda: runpy.run_path(args.PROGRAM, run_name='__main__'))
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/runpy.py", line 280, in run_path
    Step #9 - "Pulumi Create Stack":         run_name, mod_spec, pkg_name).copy()
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    Step #9 - "Pulumi Create Stack":         exec(code, run_globals)
    Step #9 - "Pulumi Create Stack":       File "./__main__.py", line 56, in <module>
    Step #9 - "Pulumi Create Stack":         organization_name=organization,
    Step #9 - "Pulumi Create Stack":       File "./ephemeral_project.py", line 37, in __init__
    Step #9 - "Pulumi Create Stack":         organization=args.organization_name
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi_gcp/organizations/get_organization.py", line 101, in get_organization
    Step #9 - "Pulumi Create Stack":         __ret__ = pulumi.runtime.invoke('gcp:organizations/getOrganization:getOrganization', __args__, opts=opts).value
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi/runtime/invoke.py", line 133, in invoke
    Step #9 - "Pulumi Create Stack":         return InvokeResult(_sync_await(asyncio.ensure_future(do_rpc())))
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi/runtime/sync_await.py", line 95, in _sync_await
    Step #9 - "Pulumi Create Stack":         return fut.result()
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi/runtime/invoke.py", line 130, in do_rpc
    Step #9 - "Pulumi Create Stack":         raise exn
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi/runtime/rpc_manager.py", line 67, in rpc_wrapper
    Step #9 - "Pulumi Create Stack":         result = await rpc
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi/runtime/invoke.py", line 90, in do_invoke
    Step #9 - "Pulumi Create Stack":         inputs = await rpc.serialize_properties(props, {})
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi/runtime/rpc.py", line 71, in serialize_properties
    Step #9 - "Pulumi Create Stack":         result = await serialize_property(v, deps, input_transformer)
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi/runtime/rpc.py", line 163, in serialize_property
    Step #9 - "Pulumi Create Stack":         return await serialize_property(future_return, deps, input_transformer)
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/site-packages/pulumi/runtime/rpc.py", line 210, in serialize_property
    Step #9 - "Pulumi Create Stack":         raise ValueError(f"unexpected input of type {type(value).__name__}")
    Step #9 - "Pulumi Create Stack":     ValueError: unexpected input of type GetOrganizationResult
    Step #9 - "Pulumi Create Stack":     error: an unhandled error occurred: Program exited with non-zero exit code: 1
    Step #9 - "Pulumi Create Stack":  
    Step #9 - "Pulumi Create Stack":     exception calling callback for <Future at 0x7f58bd80f510 state=finished returned ReadResourceResponse>
    Step #9 - "Pulumi Create Stack":     Traceback (most recent call last):
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 324, in _invoke_callbacks
    Step #9 - "Pulumi Create Stack":         callback(self)
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/asyncio/futures.py", line 365, in _call_set_state
    Step #9 - "Pulumi Create Stack":         dest_loop.call_soon_threadsafe(_set_state, destination, source)
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/asyncio/base_events.py", line 736, in call_soon_threadsafe
    Step #9 - "Pulumi Create Stack":         self._check_closed()
    Step #9 - "Pulumi Create Stack":       File "/usr/local/lib/python3.7/asyncio/base_events.py", line 479, in _check_closed
    Step #9 - "Pulumi Create Stack":         raise RuntimeError('Event loop is closed')
    Step #9 - "Pulumi Create Stack":     RuntimeError: Event loop is closed
    m
    g
    • 3
    • 17
  • d

    damp-elephant-82829

    09/02/2020, 7:49 PM
    message has been deleted
  • d

    damp-elephant-82829

    09/02/2020, 7:49 PM
    What I find interesting is this line: ValueError: unexpected input of type GetOrganizationResult
  • a

    ancient-yacht-19759

    09/02/2020, 9:03 PM
    is there somewhere i can find the correct python syntax for creating and selecting AWS resource providers to create resources in multiple regions ?
    g
    • 2
    • 22
  • c

    clever-plumber-29709

    09/02/2020, 9:34 PM
    since what version of
    pulumi_aws
    should the Data classes (https://www.pulumi.com/blog/announcing-python-tooling-improvements/#data-classes) be working? i'm on 3.2.1, but the
    s3.BucketWebsiteArgs
    are not working
    g
    m
    • 3
    • 12
Powered by Linen
Title
c

clever-plumber-29709

09/02/2020, 9:34 PM
since what version of
pulumi_aws
should the Data classes (https://www.pulumi.com/blog/announcing-python-tooling-improvements/#data-classes) be working? i'm on 3.2.1, but the
s3.BucketWebsiteArgs
are not working
g

gentle-diamond-70147

09/02/2020, 9:36 PM
When you say they are not working, what do you mean?
I believe they're available as of 3.2.0 - https://github.com/pulumi/pulumi-aws/blob/master/CHANGELOG.md#320-2020-08-24.
m

microscopic-pilot-97530

09/02/2020, 9:43 PM
Best results with the latest
pulumi
2.9.2 and
pulumi_aws
3.2.1 packages.
c

clever-plumber-29709

09/02/2020, 9:45 PM
i think my pulumi cli was not updated, not sure if that affects, i'm testing again
do i need to configure something on vs code? i installed mypyls and the extension
m

microscopic-pilot-97530

09/02/2020, 9:59 PM
I use VS Code with Pylance installed. https://marketplace.visualstudio.com/items?itemName=ms-python.vscode-pylance I also have Pylance’s basic type checking enabled via the VS code editor setting (it’s not perfect, but I’ve found it to be helpful):
"python.analysis.typeCheckingMode": "basic"
When I open a project, after the Python extension is finished loading (shown in the status bar of VS Code), I click the Python interpreter that it is using and set it to the
venv
for my project.
c

clever-plumber-29709

09/02/2020, 10:14 PM
yes. i saw the post and installed pylance and set it as default. also the interpreter is correctly set to the venv. but when typing i dont see the box with the types indication (image 1) And i see an error that BucketWebsiteArgs is not member of s3 module (image 2)
m

microscopic-pilot-97530

09/02/2020, 10:19 PM
You’re sure you have
pulumi_aws
3.2.1 installed in the venv? I don’t get any red squiggles and this is what I see when hovering over
BucketWebsiteArgs
.
What does
venv/bin/python -m pip list
show?
This is for the project I’m currently working in:
Package    Version
---------- -------
Arpeggio   1.9.2
attrs      20.1.0
dill       0.3.2
grpcio     1.31.0
parver     0.3.0
pip        20.2.2
protobuf   3.13.0
pulumi     2.9.2
pulumi-aws 3.2.1
semver     2.10.2
setuptools 50.0.3
six        1.15.0
wheel      0.35.1
c

clever-plumber-29709

09/02/2020, 11:05 PM
versions are fine on my project. i switched to a newly created project just to check, and now I dont see the previous error. but i see a new one related to mypy (image 1) hovering BucketWebsiteArgs says unknown (image 2)
View count: 1