pulumi-community #general

Channels

general

many-psychiatrist-74327

08/28/2020, 1:59 AM

👋 hello! qq: is it possible to edit the tags (or really, any Input) of a given Resource after it’s been declared? The context is how to break circular dependencies. In my immediate situation, I’m creating a Cluster that needs the VPC id. But my VPC also needs the cluster name in its tags so it can be discovered by ingress controllers. I suppose a workaround is to specify the physical name of the Cluster when creating it.. but I’d rather avoid that if possible.

many-psychiatrist-74327

08/28/2020, 4:55 AM

Cross-posting from #python: I’m getting the following error when running `pulumi up`: Tried on pulumi 2.9.0 and 2.9.1. Problem seems to be related to the an AWS VPC resource.

$ pulumi up
Previewing update (test):
     Type                 Name             Plan       Info
 +   pulumi:pulumi:Stack  kubernetes-test  create     3 errors; 37 messages
 +   └─ aws:ec2:Vpc       eks-vpc-2        create

Diagnostics:
  pulumi:pulumi:Stack (kubernetes-test):
    error: Program failed with an unhandled exception:
    error: Traceback (most recent call last):
      File "/usr/local/bin/pulumi-language-python-exec", line 85, in <module>
        loop.run_until_complete(coro)
      File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
        return future.result()
      File "/Users/<redacted>/venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 83, in run_in_stack
        await run_pulumi_func(lambda: Stack(func))
      File "/Users/<redacted>/venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 51, in run_pulumi_func
        await RPC_MANAGER.rpcs.pop()
      File "/Users/<redacted>/venv/lib/python3.8/site-packages/pulumi/runtime/rpc_manager.py", line 67, in rpc_wrapper
        result = await rpc
      File "/Users/<redacted>/venv/lib/python3.8/site-packages/pulumi/runtime/resource.py", line 427, in do_register
        await rpc.resolve_outputs(res, resolver.serialized_props, resp.object, resolvers)
      File "/Users/<redacted>/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 487, in resolve_outputs
        translated_value = translate_output_properties(value, res.translate_output_property, types.get(key))
      File "/Users/<redacted>/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 434, in translate_output_properties
        return {
      File "/Users/<redacted>/venv/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 436, in <dictcomp>
        translate_output_properties(v, output_transformer, get_type(k))
      File "/Users/<redacted>/venv/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 451, in translate_output_properties
        raise AssertionError(f"Unexpected type. Expected 'list' got '{typ}'")
    AssertionError: Unexpected type. Expected 'list' got '<class 'str'>'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

To narrow down the problem, I created an entirely new stack, and commented out my resources until the error went away. If I define a VPC (like in the config below), the error shows up. If I remove the VPC resource, everything works fine.

from pulumi_aws import ec2

vpc = ec2.Vpc(
    'eks-vpc',
    cidr_block='10.100.0.0/16',
    instance_tenancy='default',
    enable_dns_hostnames=True,
    enable_dns_support=True,
    tags={
        'Name' : 'pulumi-eks-vpc'
    }
)

Has anyone encountered this before or has any pointers/suggestions?

sticky-receptionist-53934

08/28/2020, 7:31 AM

Hello, just a quickie, are there any good practices on how to deploy a pulumi stack in different gcp projects? for example, common to me is that I have one prod project and one dev project and I want to easily be able to deploy on one or the other.

quick-finland-93793

08/28/2020, 7:37 AM

Hello! I’m having issues creating AWS RDS aurora-mysql DB instances. Referring to the AWS documentation, the value for

AllocatedStorage

is not applicable, but still is required when instanciating an

Rds.Instance()

with Engine

aurora-mysql

. Furthermore, the CLI reports, that a value for

iops

has to be set, but setting an

iops

value for engine

aurora-mysql

seems to be invalid (

InvalidParameterCombination: Specifying IOPs is not allowed for this engine

). I can’t find an applicable example of how to create AWS RDS Aurora-Mysql instances on the web - may someone give me some hints of what I'm doing wrong?

✅ 1

cuddly-dusk-95227

08/28/2020, 8:42 AM

Hello! Just tried out my first project in Pulumi, love it! Can’t wait to see what’s coming next :) and learning to use pulumi more in anger :)

🎉 2

victorious-helmet-11068

08/28/2020, 12:45 PM

hello! I use quite often StackReference.

victorious-helmet-11068

08/28/2020, 12:45 PM

In the output and programming model view it is better to export an object or an attribute of an object? It is better to export “nework” or “network.id” ?

breezy-cricket-40277

08/28/2020, 1:55 PM

Hello! Is it possible to read config secrets from

files

stdin

? Such as

pulumi config set property --file myfile

astonishing-quill-88807

08/28/2020, 3:03 PM

I'm running into some trouble with trying to write a dynamic resource provider. When I try to preview the state it's throwing an error about the resource object not having an attribute that is defined as one of the outputs. The relevant code for the provider is here: https://github.com/mitodl/ol-infrastructure/blob/dagster_instance/src/ol_infrastructure/providers/salt/minion.py#L127-L149

astonishing-quill-88807

08/28/2020, 3:03 PM

The code where I'm trying to use the attribute is here: https://github.com/mitodl/ol-infrastructure/blob/dagster_instance/src/ol_infrastructure/infrastructure/applications/dagster/__main__.py#L162-L176

astonishing-quill-88807

08/28/2020, 3:08 PM

Here's the output that I'm seeing:

Previewing update (<http://applications.dagster.QA|applications.dagster.QA>):
     Type                                          Name                                                           Plan        Info
     pulumi:pulumi:Stack                           <http://ol-infrastructure-dagster-application-applications.dagster.QA|ol-infrastructure-dagster-application-applications.dagster.QA>              3 errors
 ~   ├─ aws:ec2:SecurityGroup                      dagster-db-access-qa                                           update      [diff: ~ingress]
 +   ├─ pulumi-python:dynamic:Resource             dagster-data-qa-0                                              create      
     └─ ol:infrastructure:aws:database:OLAmazonDB  ol-etl-db-qa                                                               
 +-     └─ aws:rds:Instance                        ol-etl-db-qa-postgres-instance                                 replace     [diff: ~name]
 
Diagnostics:
  pulumi:pulumi:Stack (<http://ol-infrastructure-dagster-application-applications.dagster.QA|ol-infrastructure-dagster-application-applications.dagster.QA>):
    error: Program failed with an unhandled exception:
    error: Traceback (most recent call last):
      File "/usr/bin/pulumi-language-python-exec", line 85, in <module>
        loop.run_until_complete(coro)
      File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
        return future.result()
      File "/home/tmacey/code/mit/ops/infra/ol-infrastructure/.venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 83, in run_in_stack
        await run_pulumi_func(lambda: Stack(func))
      File "/home/tmacey/code/mit/ops/infra/ol-infrastructure/.venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 35, in run_pulumi_func
        func()
      File "/home/tmacey/code/mit/ops/infra/ol-infrastructure/.venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 83, in <lambda>
        await run_pulumi_func(lambda: Stack(func))
      File "/home/tmacey/code/mit/ops/infra/ol-infrastructure/.venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 106, in __init__
        func()
      File "/usr/bin/pulumi-language-python-exec", line 84, in <lambda>
        coro = pulumi.runtime.run_in_stack(lambda: runpy.run_path(args.PROGRAM, run_name='__main__'))
      File "/usr/lib/python3.8/runpy.py", line 282, in run_path
        return _run_code(code, mod_globals, init_globals,
      File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "./__main__.py", line 173, in <module>
        {salt_minion.minion_public_key}
    AttributeError: 'OLSaltStack' object has no attribute 'minion_public_key'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

astonishing-quill-88807

08/28/2020, 3:16 PM

Nevermind, figured it out. I had forgotten to update the dict of attributes passed into the resource provider.

purple-dawn-53665

08/28/2020, 3:37 PM

I'm not sure if this would be suited for this channel or others, but is there any plan to build a provider for interfacing with Infoblox DNS/DHCP/IPAM to manage it?

echoing-breakfast-73834

08/28/2020, 4:10 PM

I0828 15:57:58.508597    3262 eventsink.go:59] incoming output value translated: 128.0 -> 128.0
I0828 15:57:58.508613    3262 eventsink.go:62] eventSink::Debug(<{%reset%}>incoming output value translated: 128.0 -> 128.0<{%reset%}>)
I0828 15:57:58.508993    3262 eventsink.go:59] incoming output property translated: handler -> handler
I0828 15:57:58.509008    3262 eventsink.go:62] eventSink::Debug(<{%reset%}>incoming output property translated: handler -> handler<{%reset%}>)
I0828 15:57:58.509383    3262 eventsink.go:59] incoming output value translated: lambda_transform_alert.lambda_handler -> lambda_transform_alert.lambda_handler
I0828 15:57:58.509398    3262 eventsink.go:62] eventSink::Debug(<{%reset%}>incoming output value translated: lambda_transform_alert.lambda_handler -> lambda_transform_alert.lambda_handler<{%reset%}>)
I0828 15:57:58.509782    3262 eventsink.go:59] incoming output property translated: kmsKeyArn -> kms_key_arn
I0828 15:57:58.509797    3262 eventsink.go:62] eventSink::Debug(<{%reset%}>incoming output property translated: kmsKeyArn -> kms_key_arn<{%reset%}>)
I0828 15:57:58.510183    3262 eventsink.go:59] incoming output value translated: arn:aws:kms:us-east-2:ACCOUNT_ID:key/KEY_ID -> arn:aws:kms:us-east-2:ACCOUNT_ID:key/KEY_ID
I0828 15:57:58.510199    3262 eventsink.go:62] eventSink::Debug(<{%reset%}>incoming output value translated: arn:aws:kms:us-east-2:ACCOUNT_ID:key/KEY_ID -> arn:aws:kms:us-east-2:ACCOUNT_ID:key/KEY_ID<{%reset%}>)
I0828 15:57:58.510781    3262 eventsink.go:59] RPC failed with exception:
I0828 15:57:58.510797    3262 eventsink.go:62] eventSink::Debug(<{%reset%}>RPC failed with exception:<{%reset%}>)
I0828 15:57:58.511812    3262 eventsink.go:59] Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc_manager.py", line 67, in rpc_wrapper
    result = await rpc
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/resource.py", line 426, in do_register
    await rpc.resolve_outputs(res, resolver.serialized_props, resp.object, resolvers)
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 492, in resolve_outputs
    translated_value = translate_output_properties(value, res.translate_output_property, types.get(key))
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 432, in translate_output_properties
    translated_values = {
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 433, in <dictcomp>
    k: translate_output_properties(v, output_transformer, get_type(k))
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 439, in translate_output_properties
    return {
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 441, in <dictcomp>
    translate_output_properties(v, output_transformer, get_type(k))
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 456, in translate_output_properties
    raise AssertionError(f"Unexpected type. Expected 'list' got '{typ}'")
AssertionError: Unexpected type. Expected 'list' got '<class 'str'>'
I0828 15:57:58.511831    3262 eventsink.go:62] eventSink::Debug(<{%reset%}>Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc_manager.py", line 67, in rpc_wrapper
    result = await rpc
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/resource.py", line 426, in do_register
    await rpc.resolve_outputs(res, resolver.serialized_props, resp.object, resolvers)
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 492, in resolve_outputs
    translated_value = translate_output_properties(value, res.translate_output_property, types.get(key))
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 432, in translate_output_properties
    translated_values = {
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 433, in <dictcomp>
    k: translate_output_properties(v, output_transformer, get_type(k))
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 439, in translate_output_properties
    return {
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 441, in <dictcomp>
    translate_output_properties(v, output_transformer, get_type(k))
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pulumi/runtime/rpc.py", line 456, in translate_output_properties
AssertionError: Unexpected type. Expected 'list' got '<class 'str'>'
    raise AssertionError(f"Unexpected type. Expected 'list' got '{typ}'")
<{%reset%}>)

echoing-breakfast-73834

08/28/2020, 4:10 PM

I've found what appears to be a nice bug involving the pulumi resource for AWS KMS keys. During preview of an existing key specifically on Ubuntu 18.04 (running in an Azure DevOps pipeline) I receive the below error. This doesn't happen on Win10 with identical settings, on 2.9.1 or 2.9.0. It looks like the terraform bridge is not able to correctly transform an output. Has anybody else experienced this issue?

quaint-egg-65511

08/28/2020, 5:06 PM

We are approaching pulumi from a terraform background, we looked everywhere, every guide, every, github example but I can't figure out how is supposed to work in a fully automated environment. How can I create a stack entirely from CI? Stack init edits pulumi.stack.yml and we are supposed to save it

damp-elephant-82829

08/28/2020, 6:11 PM

Hello, I have a general question about how to use Pulumi, even if in this sense it has very much to do with GCP. Imagine you use pulumi to create a bucket in microservice A, and in microservice B you want to consume data from the bucket. How do you publish the output of pulumi from the CI/CD pipeline of A so that it can be consumed by B?

chilly-hairdresser-56259

08/28/2020, 7:37 PM

Hello, for RDS Parameter Groups, when creating Parameter values. Is there a specific order that Pulumi is looking for? On initial apply everything worked fine, however now I have 34 updates everytime I run a pulumi up. I even tried copying the Resource Order from Pulumi UI which still results in a re-order in the diff. I have some 34 Parameters that need set. pulumi.config.yaml below

stack:instanceparameters:
- name: tmp_table_size
  value: 6.4e+07
- name: max_connections
  value: "10000"
- name: read_buffer_size
  value: "1000000"
- name: connect_timeout
  value: "1200"
- name: innodb_stats_method
  value: nulls_equal
- name: innodb_adaptive_hash_index
  value: "1"
- name: slow_query_log
  value: "1"
- name: net_read_timeout
  value: "4000"
- name: key_buffer_size
  value: "1000000000"
- name: innodb_stats_persistent
  value: "true"
- name: read_rnd_buffer_size
  value: "1000000"
- name: innodb_stats_auto_recalc
  value: "1"
- name: lock_wait_timeout
  value: "210"
- name: log_output
  value: FILE
- name: general_log
  value: "0"
- name: wait_timeout
  value: "240"
- name: event_scheduler
  value: "ON"
- name: long_query_time
  value: "5"
- name: innodb_stats_on_metadata
  value: "0"
- name: join_buffer_size
  value: "1000000"
- name: group_concat_max_len
  value: "1048576"
- name: innodb_sort_buffer_size
  value: "65536"
- name: log_queries_not_using_indexes
  value: "1"
- name: max_allowed_packet
  value: "128000000"
- name: autocommit
  value: "1"
- name: tx_isolation
  value: READ-COMMITTED
- name: innodb_buffer_pool_size
  value: '{DBInstanceClassMemory*1/4}'
- name: sort_buffer_size
  value: "524288"
- name: default_tmp_storage_engine
  value: MyISAM
- name: query_cache_size
  value: '{DBInstanceClassMemory/12}'
- name: innodb_stats_transient_sample_pages
  value: "256"
- name: bulk_insert_buffer_size
  value: 3.2e+07
- name: innodb_stats_persistent_sample_pages
  value: 256
- name: interactive_timeout
  value: 250
- name: performance-schema-instrument
  value: wait/lock/metadata/sql/mdl=ON

hallowed-beach-15050

08/28/2020, 8:10 PM

I want to update some of the docs in https://github.com/pulumi/docs/tree/master/content/docs/reference/pkg/python/pulumi_pagerduty but where does that content come from?

bright-restaurant-97414

08/28/2020, 8:37 PM

When you use Pulumi how does one get an attribute from a resource https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/data-sources/ingress#attributes-5

millions-furniture-75402

08/28/2020, 8:49 PM

I’m not understanding how to provide an existing LoadBalancer to an awsx.LoadBalancer. There this feature here: https://github.com/pulumi/pulumi-awsx/pull/541 My attempts to use it:

// loadBalancer property not available in ApplicationLoadBalancer

const alb = new awsx.lb.ApplicationLoadBalancer(`${appName}-lb`, {
  loadBalancer: aws.lb.LoadBalancer.get("sandbox", "arn:aws:elasticloadbalancing:us-east-1:xxxxxxxxxxx:loadbalancer/app/sandbox/xxxxxxx", {
    vpcId: vpc.vpc.id
  }),
});


// TS2511: Cannot create an instance of an abstract class.
 
const alb = new awsx.lb.LoadBalancer("application", `${appName}-lb`, {
  loadBalancer: aws.lb.LoadBalancer.get("sandboxs", "arn:aws:elasticloadbalancing:us-east-1:xxxxxxxxxxx:loadbalancer/app/sandbox/xxxxxxx", {
    vpcId: vpc.vpc.id
  }),
});

sticky-receptionist-53934

08/29/2020, 2:23 PM

trying to enable workload identity on a gke cluster but I get the following error:

error: 1 error occurred:
    	* updating urn:pulumi:test::taskforce-infra::taskforce:cluster$gcp:container/cluster:Cluster::cluster-provisioner: googleapi: Error 400: Currently, the only supported value for workload pool is [PROJECT_ID].svc.id.goog and [PROJECT_ID].hub.id.goog., badRequest

sticky-receptionist-53934

08/29/2020, 2:24 PM

there are is only one property that can be set to "default" and I find no examples on how to do this, anyone knows?

workloadIdentityConfig: {
          identityNamespace: "default",
 },

nice-portugal-52925

08/29/2020, 3:40 PM

Has anyone had any luck uploading a docker image to

GCP container registry

? I keep running into this issue:

error: Error: ' docker login' failed with exit code 1

        at /Documents/work/infra/node_modules/@pulumi/docker.ts:583:15
        at Generator.next (<anonymous>)
        at fulfilled (/Documents/work/infra/node_modules/@pulumi/docker/docker.js:18:58)
        at processTicksAndRejections (internal/process/task_queues.js:86:5)

    error: Error response from daemon: login attempt to <https://www.googleapis.com/v2/> failed with status: 404 Not Found

I am able to push to the registry when I use a

docker push

command but unable to with Pulumi

damp-elephant-82829

08/30/2020, 10:31 AM

I am unable to get more detailed logs from pulumi, I am running the following commands: export TF_LOG=TRACE pulumi refresh -v 9 --yes && pulumi up --yes -v 9

millions-microphone-64941

08/30/2020, 6:29 PM

Hi all. Thanks for this great service/product. I was able to create a production ready stack for our open sourced project using Pulumi's Node.js libraries and AWS that includes: • VPC and RDS instance in private subnet • ECS managing containers in private subnet • Continuous deployment with CodePipeline and CodeBuild from Github to ECR and then update ECS Task definition • ASM to securely provide secrets as environment variables into containers • Configure Coudflare to secure connection to ALB through HTTPS/SSL • Separate foundation stack (VPC and RDS) from application stack (ECS and etc) through Node script • Cloudwatch of metrics of services If you'd like to fork it or use it as an example: https://github.com/khatm-org/khatm-infrastructure/wiki/Architecture Thanks!

damp-table-85095

08/31/2020, 1:30 AM

Hey guys. I noticed https://www.pulumi.com/docs/reference/pkg/docker/image/ contains an example of how to push a custom image into Azure, but is it possible to push custom images to docker hub?

damp-table-85095

08/31/2020, 1:31 AM

Google Compute Engine allows one to run images off Docker Hub in the context of the Free Tier but if you push images to their private repository you will get billed.

damp-table-85095

08/31/2020, 6:05 AM

And on a related note, where is the pulumi equivalent of https://registry.terraform.io/modules/terraform-google-modules/container-vm/google/2.0.0 ? I want to deploy a docker container onto a Google Compute Engine instance without using Kubernetes.

icy-train-33339

08/31/2020, 6:54 AM