pulumi-community #general

general

alert-pillow-96521

03/03/2021, 10:59 AM

Hi, anybody has experience with deploying CosmosDB through Azure devops? I am trying to deploy CosmosDB Account with EnableServerless option, as Pipeline task Pulumi@1 version 1.0.8, downloads latest PulumiSDK 2.21.2 I have added to the project

Capabilities = { new AccountCapabilityArgs { Name = "EnableServerless" } }

, however the deployment fails on

expected capabilities.0.name to be one of [EnableAggregationPipeline EnableCassandra EnableGremlin EnableTable EnableMongo MongoDBv3.4 mongoEnableDocLevelTTL], got EnableServerless

I have found it could be related to old Terraform version. Is possible to upgrade Pulumi task somehow? or I need to wait for new version? in the Devops pipeline it is enabled via "Pulumi Azure Pipelines Task" I have seen this preview already cosmos serverless , but that was using Pulumi with TypeScript.

breezy-apartment-46543

03/03/2021, 2:55 PM

Can I set an input to pulumi up from the CLI? Or how do I assign an input to pulumi?

agreeable-ram-97887

03/03/2021, 3:01 PM

Is the only way to access resource properties via the CLI by adding them to the pulumi exports? Or (hopefully) is there a way to extract any arbitrary property when you know the resource's URN? I ask because I am considering using Pulumi to create many user accounts and to keep track of their metadata, but it would be unwieldy to simply dump all of this information in the exports.

tall-shoe-66375

03/03/2021, 4:09 PM

Istio: Hello, I’m working with DigitalOcean and K8s, do you have a tutorial for implement the istio ingress from scratch?

modern-dress-7144

03/03/2021, 7:38 PM

👋 I'm using Pulumi to manage ~100 Datadog monitors and it's awesome. But for some reason today, previews & updates started taking 5+ minutes instead of seconds. Are there any Pulumi system issues that would cause this? I don't even know who to ask about it - Datadog? I don't really know where to find any debugging information to give them

stocky-student-96739

03/03/2021, 10:53 PM

This is happening again right now: https://pulumi-community.slack.com/archives/C84L4E3N1/p1613513722156300

stocky-student-96739

03/03/2021, 10:53 PM

@gentle-diamond-70147 ^^

breezy-apartment-46543

03/04/2021, 12:07 AM

I have the following lambda function defined in index.js:

const fn = new aws.lambda.Function('InfraLambda', {
  code: new pulumi.asset.AssetArchive({
    '__index.js': new pulumi.asset.FileAsset('./handler.js'),
  }),
  handler: '__index.fnInfra',
  runtime: aws.lambda.NodeJS12dXRuntime,
  role: role.arn,
  tracingConfig: {
    mode: 'Active', // Enable X-ray
  },
});

The handler.js file have require(‘@pulumi/pulumi’) and require(‘@pulumi/aws’) at the top and is using some classes from those packages. When deploying the app and then calling the apigateway for the lambda, ‘internal server error’ occurs and cloudwatch logs ’Error: Cannot find module ‘@pulumi/pulumi’' etc. How can I solve this problem, it looks like pulumi only respects dependencies declared in index.js and not the ones in handler.js

brash-airline-37413

03/04/2021, 2:02 AM

I’ve built an arm64 binary of pulumi-aws and moved it onto my gopath with it’s default name pulumi-resource-aws but when I run pulumi up I’m getting:

pulumi:providers:aws (default_3_31_0):
    error: no resource plugin 'aws-v3.31.0' found in the workspace or on your $PATH, install the plugin using `pulumi plugin install resource aws v3.31.0`

I’ve confirmed my gopath with

go env

and I get the same thing when renaming the artifact to aws-v3.31.0 and verbose logs say nothing additional. Is there something special I need to do in order for pulumi to pick up a locally compiled version of pulumi-aws?

clever-cartoon-41433

03/04/2021, 5:10 AM

be me

install loft helm chart

after installation, every update brings errors where pulumi tries to create CRD resources that already exist

try removing loft from index.ts so that I can just manually install it via helm myself

the removal of the chart also breaks my stack, as now the namespace is stuck in a perpetually terminating state

clever-lawyer-94920

03/04/2021, 5:46 AM

Can I rename a pulumi stack?

elegant-island-39916

03/04/2021, 10:08 AM

Hi, how would I go about ignoring the changes to my provider? This does not seem to work

ResourceOptions(provider=eks_provider, ignore_changes=["provider"])

👀 1

miniature-leather-70472

03/04/2021, 11:15 AM

Is there anyway to run some code on a destroy action? I've got some code I run to create a resource through a REST API on up, and I want to delete it on destroy. I can't find something like isDryRun() for destroy. This is C# so no custom providers

brash-kite-78002

03/04/2021, 12:18 PM

Hello Everyone! As far as I can see, there are plugins for the CLI and modules for the code itself in the Pulumi. In a normal case scenario, I would want to lock modules to the specific version, but it turns out, that I need to lock plugins also. Can I avoid it?

tall-shoe-66375

03/04/2021, 5:52 PM

Attach a Firewall to my droplets on Digital Ocean On the link below mention how to link a FW to each Droplet created https://www.digitalocean.com/community/questions/can-i-assign-a-firewall-while-creating-a-droplet-via-api However the Pulumi Docs mention to attach one FW to One Droplet https://www.pulumi.com/docs/reference/pkg/digitalocean/firewall/ I’ve tried on a for-each loop with TS attaching the droplet.id but this trigger an error. Exists an example to solve this?

colossal-australia-65039

03/04/2021, 7:54 PM

I'm trying to remove

aws:region

from my Pulumi config file and instead have it as a provider in code so I have fewer dependencies on the config file. But this is causing every resource to be recreated even though it's ultimately being applied to the same cloud environment. Is there no way around this?

clever-cartoon-41433

03/05/2021, 6:15 AM

Hi; I need to .get() a specific secret created by another pulumi resource, and pulumi is erroring because that resource doesnt exist, even though I put a dependsOn in the secrets' config. Is there any way around this?

chilly-analyst-14900

03/05/2021, 8:02 AM

With the new azure-native provider, how do I get the PrimaryAccessKey from a storageaccount?

damp-school-17708

03/05/2021, 10:12 AM

Hello what do you use for local testing? Starting lambdas locally, etc. Sam Cli?

tall-shoe-66375

03/05/2021, 1:50 PM

Userdata update on Droplets trigger an Error on Firewall previously created

Diagnostics:
  digitalocean:index:Firewall (doroplets-webmago-firewall):
    error: transport is closing
 
  pulumi:pulumi:Stack (digitalocean-ts-droplets-dev):
    panic: interface conversion: interface {} is string, not int
    goroutine 13 [running]:
    <http://github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.SerializeValueForHash(0xc000561100|github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.SerializeValueForHash(0xc000561100>, 0x148c740, 0x1927410, 0xc0006cb900)
    	/home/runner/go/pkg/mod/github.com/pulumi/terraform-plugin-sdk/v2@v2.0.0-20201218231525-9cca98608a5e/helper/schema/serialize.go:24 +0x959
    <http://github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.HashSchema.func1(0x148c740|github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.HashSchema.func1(0x148c740>, 0x1927410, 0xc0008feec0)
    	/home/runner/go/pkg/mod/github.com/pulumi/terraform-plugin-sdk/v2@v2.0.0-20201218231525-9cca98608a5e/helper/schema/set.go:43 +0x68
    <http://github.com/pulumi/pulumi-terraform-bridge/v2/pkg/tfshim/sdk-v2.v2Schema.SetHash(0xc0006cba40|github.com/pulumi/pulumi-terraform-bridge/v2/pkg/tfshim/sdk-v2.v2Schema.SetHash(0xc0006cba40>, 0x148c740, 0x1927410, 0x0)
    	/home/runner/go/pkg/mod/github.com/pulumi/pulumi-terraform-bridge/v2@v2.19.0/pkg/tfshim/sdk-v2/schema.go:176 +0x62
    <http://github.com/pulumi/pulumi-terraform-bridge/v2/pkg/tfbridge.visitPropertyValue(0xc000a729a4|github.com/pulumi/pulumi-terraform-bridge/v2/pkg/tfbridge.visitPropertyValue(0xc000a729a4>, 0xb, 0xc000b693e0, 0xa, 0x146d820, 0xc00069e040, 0x198e3a0, 0xc0006cba40, 0x0, 0xc0006cba00, ...)
    	/home/runner/go/pkg/mod/github.com/pulumi/pulumi-terraform-bridge/v2@v2.19.0/pkg/tfbridge/diff.go:114 +0x486
    <http://github.com/pulumi/pulumi-terraform-bridge/v2/pkg/tfbridge.doIgnoreChanges(0x19800a0|github.com/pulumi/pulumi-terraform-bridge/v2/pkg/tfbridge.doIgnoreChanges(0x19800a0>, 0xc0004e9650, 0xc00057bec0, 0xc000397e90, 0xc0006f8cf0, 0x0, 0x0, 0x0, 0x19845a0, 0xc000792220)
    	/home/runner/go/pkg/mod/github.com/pulumi/pulumi-terraform-bridge/v2@v2.19.0/pkg/tfbridge/diff.go:259 +0x4e5
    <http://github.com/pulumi/pulumi-terraform-bridge/v2/pkg/tfbridge.(*Provider).Diff(0xc00000a1e0|github.com/pulumi/pulumi-terraform-bridge/v2/pkg/tfbridge.(*Provider).Diff(0xc00000a1e0>, 0x19765c0, 0xc000397d70, 0xc0002da000, 0xc00000a1e0, 0x1520001, 0xc000953e40)
    	/home/runner/go/pkg/mod/github.com/pulumi/pulumi-terraform-bridge/v2@v2.19.0/pkg/tfbridge/provider.go:689 +0x785
    <http://github.com/pulumi/pulumi/sdk/v2/proto/go._ResourceProvider_Diff_Handler.func1(0x19765c0|github.com/pulumi/pulumi/sdk/v2/proto/go._ResourceProvider_Diff_Handler.func1(0x19765c0>, 0xc000397d70, 0x1678e80, 0xc0002da000, 0x1687960, 0x22d3870, 0x19765c0, 0xc000397d70)
    	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v2@v2.20.1-0.20210212181059-f4b0fa86fedc/proto/go/provider.pb.go:2251 +0x89
    <http://github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1(0x19765c0|github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1(0x19765c0>, 0xc000396600, 0x1678e80, 0xc0002da000, 0xc000b57de0, 0xc000b57e00, 0x0, 0x0, 0x1950560, 0xc00036de90)
    	/home/runner/go/pkg/mod/github.com/grpc-ecosystem/grpc-opentracing@v0.0.0-20180507213350-8e809c8a8645/go/otgrpc/server.go:57 +0x2f6
    <http://github.com/pulumi/pulumi/sdk/v2/proto/go._ResourceProvider_Diff_Handler(0x16c0c20|github.com/pulumi/pulumi/sdk/v2/proto/go._ResourceProvider_Diff_Handler(0x16c0c20>, 0xc00000a1e0, 0x19765c0, 0xc000396600, 0xc000600e40, 0xc0007085c0, 0x19765c0, 0xc000396600, 0xc000950a80, 0x9d6)
    	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v2@v2.20.1-0.20210212181059-f4b0fa86fedc/proto/go/provider.pb.go:2253 +0x150
    <http://google.golang.org/grpc.(*Server).processUnaryRPC(0xc0003321c0|google.golang.org/grpc.(*Server).processUnaryRPC(0xc0003321c0>, 0x1984d80, 0xc00008a780, 0xc0009a8800, 0xc0006dab40, 0x2290410, 0x0, 0x0, 0x0)
    	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.32.0/server.go:1194 +0x522
    <http://google.golang.org/grpc.(*Server).handleStream(0xc0003321c0|google.golang.org/grpc.(*Server).handleStream(0xc0003321c0>, 0x1984d80, 0xc00008a780, 0xc0009a8800, 0x0)
    	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.32.0/server.go:1517 +0xd05
    <http://google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc0001262c0|google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc0001262c0>, 0xc0003321c0, 0x1984d80, 0xc00008a780, 0xc0009a8800)
    	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.32.0/server.go:859 +0xa5
    created by <http://google.golang.org/grpc.(*Server).serveStreams.func1|google.golang.org/grpc.(*Server).serveStreams.func1>
    	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.32.0/server.go:857 +0x1fd

helpful-van-82564

03/05/2021, 2:17 PM

I imported a GKE cluster and have the kubeconfig as output in the stack, however it's not being marked as a secret - although it does for a gke cluster I did deploy through pulumi. Is this expected?

colossal-australia-65039

03/05/2021, 5:10 PM

is there a way to mark some stack outputs as secret and others as non-secret, so only the secret ones require the

PULUMI_CONFIG_PASSPHRASE

(I'm using S3 to store state)

shy-oxygen-8874

03/05/2021, 5:26 PM

Does anybody know a way to power something like a tf plan apply workflow? What we're trying to do is run pulumi through gitops. We want to run pulumi preview at pull request time, generate previews automatically there, then apply just those previews when the pr is merged and the change flows through our CI. In terraform, we could just generate a plan and apply that in each step. Is there a similar thing for pulumi? If not, is there a blessed way to diff pulumi preview outputs (the thinking there is that we can save off pr-time previews for each stage, then our ci pipe can run a preview before deployment and compare/check for unexpected changes).

purple-jelly-68780

03/05/2021, 6:30 PM

Does anyone know if the

Pulumi.stackname.yaml

file is meant to be committed to git? This might be a matter of personal choice but I wanted to hear if there's any kind of best practice. Especially if it has values like:

config:
  aws:region: us-west-2
  health-directory:bastionPublicKey:
    secure: <encryptedString>
  health-directory:dbPassword:
    secure: <encryptedString>

clever-cartoon-41433

03/05/2021, 8:10 PM

Any way to shell out during a pulumi apply to get a value? At the moment, I can only consistently retrieve a certain value created by my resources by running a binary (which automatically waits until a k8s secret is created before printing it to stdout). Alternatively, is there a way to "await" for that k8s secret in pulumi, and then use its value elsewhere? I.e., does pulumi have some way to do this: https://www.terraform.io/docs/language/resources/provisioners/local-exec.html?

clever-cartoon-41433

03/06/2021, 8:42 PM

^ This is another way of saying "How do I add an explicit dependency to an output .apply()"?

purple-jelly-68780

03/06/2021, 11:59 PM

This might seem like a really minor issue but I'm curious if anyone knows of a way to get the nicely formatted Pulumi output (like when running from the CLI) when running

.up()

from Automation API? I'm currently hacking my way through to try to do it myself but it sure would be nice if there was some out of the box way of doing this 😅

gifted-dentist-89608

03/07/2021, 12:48 AM

Hey guys am I crazy or am missing something I have a config value setup I am just trying to get it and have it display in a http get request. I can get the config name so it exist. But I get this this

"Cannot find module '@pulumi/pulumi/metadata.js'"

in cloudwatch when I try to http request. But I have @pulumi/pulumi installed in my package.json.

purple-processor-6610

03/07/2021, 5:14 AM

Hi folks, I am trying to deploy docker containers using pulumi docker. I am building and tagging containers with a git tag which increments for every commit. I looked at the pulumi docker example and tried to use RemoteImage to find my tagged image in our private registry. RemoteImage is returning me incorrect value for which I get a error in Container function. Error below. For right now I am deploying just the Container with a tagged image, but that means my resources change with every pulumi up. Has anyone else used tags in images? Thx

invalid reference format

clever-cartoon-41433

03/07/2021, 8:35 PM

I'm specifying a kubeconfig to a kubernetes provider but it seems stubbornly refusing to handle it, and instead uses the Gitlab-generated user account (passed in by KUBECONFIG env var) to try to create stuff, which (thankfully) fails. Last time I fixed this issue I had to pass in the kubeconfig via filename and that somehow solved the problem, but how do I pass it in via value this time (as it's in the form of an Output<string>)? I'm setting the kubeconfig, cluster, and context here.

Title

clever-cartoon-41433

03/07/2021, 8:35 PM

billowy-army-68599

03/07/2021, 8:45 PM

how are you specifying this in the code? it should always use the provider creds first

clever-cartoon-41433

03/07/2021, 8:47 PM

const config = new loft.LoftKubeconfig("stack", {
        spaceName: ns.metadata.name,
        vclusterName: virKube.metadata.name,
    }, {dependsOn: virKube})

    return new k8s.Provider("loftK8s", {
        kubeconfig: config.data,
        cluster: config.clusterName,
        context: config.ctx,
    }, {dependsOn: config})

billowy-army-68599

03/07/2021, 8:50 PM

and

config.data

is just a standard kubeconfig string?

clever-cartoon-41433

03/07/2021, 8:50 PM

Yup

I'm using ti right now, kubectl lists secrets fine. I can double check right now that outputting it to a file actually fixes the issue like I mentioned.

billowy-army-68599

03/07/2021, 8:51 PM

huh, I have no idea why that's not working

broad-dog-22463

03/07/2021, 8:53 PM

Could be config.ctx related

What is that set to?

clever-cartoon-41433

03/07/2021, 8:53 PM

I literally grab it from the file by parsing the yaml. It doesn't work if I remove it; specifying the context was an attempted fix on my end

broad-dog-22463

03/07/2021, 8:57 PM

Do you actually need to set the cluster? Isn’t it already in the kubeconfig? I do the following elsewhere https://github.com/pulumi/examples/blob/c9558f3a236ca284933d105efad0c99ba89a6d6a/aws-go-eks/main.go#L146 Notice I only set the kubeconfig and nothing else and it works

(I’m throwing the dice here as I don’t know what happens in the loft package here)

clever-cartoon-41433

03/07/2021, 8:58 PM

The loft package is a dynamic provider I wrote 😜 Give me a few minutes to see if I actually messed something else up completely, although this is the second time I've run into this

uh oh... config.data is undefined o.o

import * as pulumi from "@pulumi/pulumi";
import * as child_process from "child_process";

export interface LoftKubeconfigOptions {
    spaceName: pulumi.Input<string> //Name of namespace where vcluster is located
    vclusterName: pulumi.Input<string> //name of vcluster itself
}

interface LoftKubeconfigDynamicProviderInputs {
    spaceName: string; //Name of namespace where vcluster is located
    vclusterName: string //name of vcluster itself
}

class LoftKubeconfigResourceProvider implements pulumi.dynamic.ResourceProvider {
    async create(inputs: LoftKubeconfigDynamicProviderInputs): Promise<pulumi.dynamic.CreateResult> {
        return {
            id: inputs.spaceName + "-" + inputs.vclusterName,
            outs: {
                data: child_process.execSync("loft use vcluster " + inputs.vclusterName + " --print --space " + inputs.spaceName + " --cluster loft-cluster --silent"),
                ...inputs
            }
        };
    }
}

export class LoftKubeconfig extends pulumi.dynamic.Resource {
    public readonly spaceName!: pulumi.Output<string> //Name of namespace where vcluster is located
    public readonly vclusterName!: pulumi.Output<string> //name of vcluster itself

    //Other outputs
    public readonly data!: pulumi.Output<string> //kubeconfig yaml file
    public readonly ctx!: pulumi.Output<string> //context in yaml file
    public readonly clusterName!: pulumi.Output<string> //context in yaml file

    constructor(name: string, args: LoftKubeconfigOptions, opts?: pulumi.CustomResourceOptions) {
        super(new LoftKubeconfigResourceProvider(), `loft:vclusterKubeConfig:${name}`, args, pulumi.mergeOptions(opts || {}, {
            additionalSecretOutputs: ["config"],
        }));
    }
}

A lot of this is boilerplate I copied over, I know nothing about typescript

sorry for bothering you guys so much btw

I'm fixing the issue with my dynamic provider rn, misunderstood what happens when you omit so many asynchronous functions from the spec

I may just develop a loft provider, as I now realize what I shouldve done is wrap the loft vcluster resource in there and write out all the methods, but atm I'm pretty lazy

It's been a couple hours and I cannot seem to figure out how I'm supposed to intialize this .data parameter, so that it's a regular Output object and not undefined. As far as I can tell, the docs talk as if this happens after the resource constructor is called. This doesn't happen though, and anytime I try to .apply() it from the newly instantiated resource pulumi crashes during the prview. Not really sure what to do but I'm trying to look through the pulumi code/docs to see if I'm missing something. I'm very new to typescript so that's probably not helping. (loft.ts):

import * as pulumi from "@pulumi/pulumi";

export interface LoftKubeconfigOptions {
    vclusterName: pulumi.Input<string> //name of vcluster itself
}

interface LoftKubeconfigDynamicProviderInputs {
    vclusterName: string //name of vcluster itself
}

interface LoftKubeconfigDynamicProviderOutputs extends LoftKubeconfigDynamicProviderInputs {
    data: string
}

class LoftKubeconfigResourceProvider implements pulumi.dynamic.ResourceProvider {
    async create(inputs: LoftKubeconfigDynamicProviderInputs): Promise<pulumi.dynamic.CreateResult> {
        const data = await require('child_process').exec("loft use vcluster " + inputs.vclusterName + " --print --silent").toString('utf8')
        const outs: LoftKubeconfigDynamicProviderOutputs = {
            vclusterName: inputs.vclusterName,
            data: data,
        }
        return {
            id: inputs.vclusterName,
            outs: outs
        };
    }

    async read(id: string, props: LoftKubeconfigDynamicProviderOutputs): Promise<pulumi.dynamic.ReadResult> {
        const data = await require('child_process').exec("loft use vcluster " + props.vclusterName + " --print --silent").toString('utf8')
        props.data = data
        return {
            id: id,
            props: props,
        };
    }
}

export class LoftKubeconfig extends pulumi.dynamic.Resource {
    public readonly vclusterName!: pulumi.Output<string> //name of vcluster itself
    public readonly data!: pulumi.Output<string> //kubeconfig yaml file

    constructor(name: string, args: LoftKubeconfigOptions, opts?: pulumi.CustomResourceOptions) {
        super(new LoftKubeconfigResourceProvider(), `loft:vclusterKubeConfig:${name}`, args, pulumi.mergeOptions(opts || {}, {
            additionalSecretOutputs: ["data"],
        }));
    }
}

(index.ts):

...
    const config12 = new loft.LoftKubeconfig("main12", {
        vclusterName: virKube.metadata.name,
    }, {dependsOn: virKube})

    config12.data.apply(v => {
        return v
    }) //Crash happens here
...

It's defining vclusterName, if that helps

This: https://github.com/pulumi/examples/blob/master/classic-azure-ts-dynamicresource/cdnCustomDomain.ts Is outdated/misleading. I found a newer example with an EKS dynamic provider that showed that you have to manually define output properties in the super() call, which the above doesn't do. Now the output is instantiated and apply() works.

The above example is what is referenced in the dynamic providers tutorial

View count: 1