pulumi-community #general

gorgeous-window-12350

05/18/2021, 5:37 PM

I have a staging and production stack managing infrastructure in the same AWS account, but I only want a single elastic container registry resource between both stacks. I can’t simply remove the ECR resource from the staging stack because a bunch of other staging resources reference it. I also tried forcing staging to use the same ECR repo name as production, but that complained that the resource already existed when I ran

pulumi up

. Any guidance here?

full-artist-27215

05/18/2021, 5:49 PM

I'm trying to build a container image using

pulumi_docker.Image

, passing

"DOCKER_BUILDKIT": "1"

in the

pulumi_docker.DockerBuild

env

argument, as documented (https://www.pulumi.com/docs/reference/pkg/docker/image/#dockerbuild) to enable BuildKit support, but my image builds are failing with errors like

Copy code

error: the --mount option requires BuildKit. Refer to <https://docs.docker.com/go/buildkit/> to learn how to build images with BuildKit enabled

which suggests that BuildKit somehow isn't being used. Has anyone else been successful with a BuildKit-enabled image build?

broad-eve-12764

05/18/2021, 6:31 PM

Hi all! I'm using

docker.RegistryImage

to have a bit more control over the docker build, following @faint-table-42725’s example in #132, but I'm not getting any logs from docker now, just:

docker:index:RegistryImage  registry-image-for-deps  creating...

Do I need to configure logging explicitly for

RegistryImage

mysterious-lighter-33699

05/18/2021, 6:48 PM

Hey folks, hope this is the right avenue for reporting what I think is a bug: In our eks node group settings, we had

ignore_changes=["scaling_config.desired_size"]

(this is in python). Unfortunately, this is not the string you can use to ignore changes to the desired_size setting. You have to use ``ignore_changes=["scalingConfig.desiredSize"]`` instead. Which we learned when our node groups all got sized to one node each after applying an unrelated update.

broad-eve-12764

05/18/2021, 7:27 PM

Running into a really weird issue where my multi-layer build caches correctly when I run

pulumi up

locally, but re-builds each layer with no caching when run in GitHub Actions. Excerpt from Pulumi `index.ts`:

Copy code

const image = repo.buildAndPushImage({
  target: 'app',
  cacheFrom: { stages: ['deps', 'files', 'app'] },
  context: '../',
})

Dockerfile is essentially:

Copy code

FROM external.image as deps
# install dependencies

FROM deps as files
# copy in some files, precompile assets

FROM files as app
# copy in other files, build app

When run locally, it builds each layer and uploads to the repo. When I run it in GitHub Actions, it does this: • builds

deps

• builds

deps

again as part of building

files

• builds

files

• builds

deps

again as part of building

app

• builds

files

again as part of building

app

• builds

app

Not sure why caching is failing to work only in CI. Any thoughts?

shy-author-33795

05/18/2021, 8:02 PM

I am also having and issue where I am able to build docker images locally with

pulumi up

but fails to build on Buildkite. To tell you more about what I have going on... I have the following `ComponentResource`:

Copy code

class ECRRepo(ComponentResource):        
    ComponentResource.__init__(
    self, "awsx:ecr:ECRRepo", name, {
    "name": name,
    "docker_build_dir": docker_build_dir,
    "docker_image": docker_image,
    "extra_protected_tags": extra_protected_tags,
    "image_tag_mutability": image_tag_mutability,
    "max_images": max_images,
    "scan_on_push": scan_on_push,
    "tags": tags
    }, opts)

Here's how I grab registry information:

Copy code

class ECRRepo(ComponentResource):
    @staticmethod
    def get_registry_info(rid):
        creds = ecr.get_credentials(registry_id=rid)
        decoded = base64.b64decode(creds.authorization_token).decode()
        parts = decoded.split(':')
        if len(parts) != 2:
            raise Exception("Invalid credentials")
        return ImageRegistry(creds.proxy_endpoint, parts[0], parts[1])

Here's how I'm creating a repo:

Copy code

def _create(self):
        repo = ecr.Repository(
            resource_name = self.name,
            image_scanning_configuration = ecr.RepositoryImageScanningConfigurationArgs(
                scan_on_push = self.scan_on_push,
            ),
            name = self.name,
            image_tag_mutability = self.image_tag_mutability,
            tags = self.tags,
            opts = ResourceOptions(parent=self)
        )

Here's how I'm building the image:

Copy code

custom_image = "{image}".format(image=self.docker_image)
directory = "{dir}".format(dir=self.docker_build_dir)

Image(
    self.name,
    image_name=pulumi.Output.concat(repo.repository_url, ":v1.0.0"),
    build=f'{directory}/{custom_image}',            
    registry=repo.registry_id.apply(ECRRepo.get_registry_info),
        opts = ResourceOptions(parent=repo)
)

many-psychiatrist-74327

05/18/2021, 8:20 PM

👋 any ideas on how to debug an error like this one?

Copy code

panic: fatal: An assertion has failed

goroutine 181 [running]:
<http://github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.failfast(...)|github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.failfast(...)>
	/Users/runner/work/pulumi/pulumi/sdk/go/common/util/contract/failfast.go:23
<http://github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.Assert(...)|github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.Assert(...)>
	/Users/runner/work/pulumi/pulumi/sdk/go/common/util/contract/assert.go:26
<http://github.com/pulumi/pulumi/pkg/v2/resource/deploy.NewDeleteReplacementStep|github.com/pulumi/pulumi/pkg/v2/resource/deploy.NewDeleteReplacementStep>(0xc000167720, 0xc001560fc0, 0xc0026ff201, 0x4d, 0xc0026ee780)
	/Users/runner/work/pulumi/pulumi/pkg/resource/deploy/step.go:294 +0x1df
<http://github.com/pulumi/pulumi/pkg/v2/resource/deploy.(*stepGenerator).generateStepsFromDiff(0xc00191ed80|github.com/pulumi/pulumi/pkg/v2/resource/deploy.(*stepGenerator).generateStepsFromDiff(0xc00191ed80>, 0x7f192ec86318, 0xc002717c20, 0xc0009bf220, 0x46, 0xc001560a20, 0xc002724360, 0xc00160de90, 0xc001540450, 0xc0027dfc80, ...)

better-shampoo-48884

05/19/2021, 6:52 AM

So.. I've spent the past few weeks refactoring my pulumi code to ComponentResource spec - and I've got one outstanding question before I embark on a new world of pain.. Right now I have one primary ComponentResource for each resource type, with the provider specific resource directly underneath. I'm planning on making another change to create a new provider-type specific ComponentResource. Currently i have ComponentResource "network" which handles virtual networks, subnets, security groups, etc. What I want to do is introduce a Component Resource inside of the "network" one which is called "azure-net" so that the "network" resource can be cloud-agnostic (all network resources take pretty much the same configuration regardless of provider - so I'll be leveraging that with sensible defaults and overrides). My question is really this: My URNs will likely change when I perform the operation, and I'm creating ~40xN resources (where N is number of datacenters/regions it should span) for each stack - is there a sensible way of reimporting created resources? Current URN:

Copy code

urn:pulumi:mystack::myproject::purpose:components:provider$purpose:components:group$purpose:components:k8s$azure-native:containerservice:ManagedCluster$random:index/randomUuid:RandomUuid::mystack-cluster-k8s-location-diskcrypto-to-kv-cert-RA

I'm guessing the

:components:k8s$azure-native:

bit is what's going to change. Initial thought: export the stack + search and replace + reimport stack? Any other ideas? Doing

pulumi import

on each one is.. just not going to work, besides taking a huge amount of time, it will just be really really really tricky to ensure I have the right URN specified for each resource.

brash-country-7541

05/19/2021, 8:22 AM

How do I get access to kubernets service as a global url? see my stackflow message https://stackoverflow.com/questions/67599082/why-do-i-not-get-acess-to-the-kubernetes-service-depoyed-on-google-cloud-by-a-gl

rough-arm-38844

05/19/2021, 8:55 AM

hello, does anyone have any information or guidance on how to use the python dynamic provisioners on a Windows machine so we can connect and perform post-provisioning activity - running powershell scripts etc. ? in particular - how do we connect? do we use WinRM? do we need to make those additional ports open on the security groups? thanks

quiet-truck-58459

05/19/2021, 12:52 PM

Has anyone been able to use

get_registry_image

from the pulumi docker package with a private image? I'm basically trying to follow the example in the docs

Copy code

import pulumi
import pulumi_docker as docker

ubuntu_registry_image = docker.get_registry_image(name="<<private image name>>")
ubuntu_remote_image = docker.RemoteImage("ubuntuRemoteImage",
    name=ubuntu_registry_image.name,
    pull_triggers=[ubuntu_registry_image.sha256_digest])

but get a Forbidden error

Copy code

Exception: invoke of docker:index/getRegistryImage:getRegistryImage failed: invocation of docker:index/getRegistryImage:getRegistryImage returned an error: invoking docker:index/getRegistryImage:getRegistryImage: Got error when attempting to fetch image version from registry: Got bad response from registry: 403 Forbidden

quiet-truck-58459

05/19/2021, 1:57 PM

To extend my question further can anyone explain why this 👇 code will not pull the latest image on every run?

Copy code

import time

import pulumi
import pulumi_docker as docker

alpine_remote_image = docker.RemoteImage("alpineRemoteImage",
    name="alpine:latest",
    pull_triggers=[str(time.time())]
)

little-market-63455

05/19/2021, 3:03 PM

Hi everyone, started recently with Pulumi and went over most of the articles I am facing a couple of problems, probably due to using TypeScript and learning it along with Pulumi at the same time With that being said, I don't think the problem is with TypeScript per say but with my understanding on how to properly handle

Promise<T>

s, `Input<T>`s and

Output<s>

1. If I am builI am building a

ComponentResource

which needs some information about an IAM user (which is already created) 2. To fetch information about that user, I can use

aws.iam.getUser

which returns a

Promise<GetUserResult>

3. Now, I don't know how to properly deal with that Promise 4. On one hand, I could make the ComponentResrouce accept

Input<GetUserResult>

and wrap my Promise with an

Output<T>

then pass it on. This should trigger dependency tracking according to my understanding. Is this really the way to go? 5. Or I could

await

the promise and get the result then pass it as a regular object but then I have to enable top-level awaits or perhaps use

then

? Sorry for the confusion, trying to learn and apply too many things at the same time and I appreciate your help.

limited-eve-38521

05/19/2021, 4:15 PM

This morning running

pulumi up

the

pulumi-resource-aws

plugin was present in the path below as expected, but got not found error for

4.5.0

. Forcing 4.3.1 solved the error below:

Copy code

Diagnostics:
  pulumi:providers:aws (default_4_5_0):
    error: could not read plugin [/Users/jeremywest/.pulumi/plugins/resource-aws-v4.5.0/pulumi-resource-aws] stdout: EOF

Deleted .pulumi folder and node_modules folder with fresh installs to try and troubleshoot but haven't looking into further since 4.3.1 doesn't have the issue.

colossal-australia-65039

05/19/2021, 7:36 PM

is it possible to get a stack reference to a stack using a different state store? e.g. two stacks in different S3 buckets

gentle-monitor-55086

05/19/2021, 8:20 PM

What's the proper / best way to prevent resources from being destroyed that i want to toggle their deployment based on a configuration boolean... Basically trying to say "only deploy to the -dev or -production api's if i've explicitly set a variable in the conf to do so". Another example was i had

"if servers < desired { // deploy ec2}"

With the issue being on the second run pulumi would see we didn't define the first set of ec2's and go "oh well i should destroy those". My issue was that i was generating unique values for the ec2, storing it in the userdata, and then also saving those off to a DB... So even using

protect

ignoreChanges

it was (is) a pain to try and avoid re-generating / saving new values into the DB and required all sorts of shims and "clever"(breaks if you breathe on it) code. I don't currently have an idea about the api and would like to do it the "right" way

ambitious-father-68746

05/19/2021, 11:03 PM

Hi, I have some resources already created, but now I feel the need to explicitly configure and define a provider and then reference it in my resources using

pulumi.ResourceOptions

. Unfortunately Pulumi wants to destroy the resources that I already have. Is there a way to move resources from the default Provider configuration to a new one?

sticky-bear-14421

05/20/2021, 6:53 AM

Hi, is it possible to set a config option to always apply options like

--suppress-outputs

to pulumi runs? Apart from the definition of an alias like pu="pulumi up --suppress-outputs <maybe other options>"

bland-army-56447

05/20/2021, 7:09 AM

I’m trying to use

tfbridge

to use an existing TF Module in Pulumi. I already “bridged” the module and generated the SDK. Then I did

go get <http://github.com/|github.com/>...

to install (?) my custom provider. But when running

pulumi up

I’m stuck with

error: no resource plugin 'lxd' found in the workspace or on your $PATH

Do I additionally have to install my resource provider through

pulumi plugin install

somehow? If yes, how do create the tarball needed for the installation? 🤔

better-shampoo-48884

05/20/2021, 7:11 AM

quik question - if I pass a

provider

to a ComponentResource, will the underlying resources implicitly use that provider? I.e. I have a ComponentResource "network" - that creates azure-vnet, azure-subnet(s) and azure-networksecuritygroups - if I pass the provider to the "network" componentresource, will the vnet, subnets, and nsg's be created with that provider?

rhythmic-activity-46295

05/20/2021, 4:50 PM

Hello, Can someone help? I want to register a class as an OutputType. used the OutputType and Constructor attributes, but it doesn't seem to make any effect, and am still getting this exception on pulumi preview: System.InvalidOperationException: {ournamespace}..SubnetStackReference is not a supported argument type. monitor.registerResourceOutputs(...).AppServiceSubnetReferences.id.snet-dev-thin-product-catalogue at Pulumi.Serialization.Serializer.SerializeAsync(String ctx, Object prop, Boolean keepResources) at Pulumi.Serialization.Serializer.SerializeDictionaryAsync(String ctx, IDictionary dictionary, Boolean keepResources) at Pulumi.Serialization.Serializer.SerializeAsync(String ctx, Object prop, Boolean keepResources) at Pulumi.Serialization.Serializer.SerializeAsync(String ctx, Object prop, Boolean keepResources) at Pulumi.Deployment.SerializeFilteredPropertiesAsync(String label, IDictionary`2 args, Predicate`1 acceptKey, Boolean keepResources) at Pulumi.Deployment.SerializeAllPropertiesAsync(String label, IDictionary`2 args, Boolean keepResources) at Pulumi.Deployment.RegisterResourceOutputsAsync(Resource resource, Output`1 outputs) at Pulumi.Deployment.Runner.<>c__DisplayClass9_0.<<WhileRunningAsync>g__HandleCompletion|0>d.MoveNext() --- End of stack trace from previous location --- at Pulumi.Deployment.Runner.WhileRunningAsync() My output: [Output] public Output<ImmutableDictionary<string,SubnetStackReference>> SubnetReferences { get; set; } The class: [Pulumi.OutputTypeAttribute] public sealed class SubnetStackReference { private readonly string _id; private readonly string _name; private readonly SubnetKind _kind; [Pulumi.OutputConstructorAttribute] public SubnetStackReference(string id, string name, SubnetKind kind) { _id = id; _name = name; _kind = kind; } public string Id => _id; public string Name => _name; public SubnetKind Kind => _kind; } Any idea on what am I missing? not even sure I can do this in the first place in C# ...?!

full-dress-10026

05/20/2021, 4:54 PM

I am using a

aws.lambda.CallbackFunction

and just added tslog to my package json. Upon adding it and calling log.info in my code, I get the following error during the pulumi preview.

Copy code

function 'callback': batchScalingManager.ts(190,31): referenced
      function 'handleBatch': batchScalingManager.ts(25,20): which captured
        variable 'log' which indirectly referenced
          function 'wrapCallSite': source-map-support.js(348,21): which referenced
            function 'isInBrowser': source-map-support.js(50,20): which could not be serialized because
              Unexpected missing variable in closure environment: window
    
    Function code:
      function isInBrowser() {
        if (environment === "browser")
          return true;
        if (environment === "node")
          return false;
      ...

Is this something that could be fixed from my end or no dice with this lib?

mysterious-lighter-33699

05/20/2021, 5:16 PM

recently, I've noticed that personal pulumi stacks, created with

pulumi stack init project/$USER --copy-config-from project/staging

"forget" configuration values after about a day. That is, pulumi up in that stack no longer can find config settings that are set for the staging stack. This happens after a few hours. Right after creating the personal stack, I can use those variables. The next workday, the variables are gone. We use an AWS SSO provider for our API credentials. Might expiration on those creds cause pulumi stack config to get dropped?

gorgeous-minister-41131

05/20/2021, 7:48 PM

anyone having problems with the LaunchTemplate resources in the aws provider? I upgraded from some 3.x version to 4.5.1 and now pulumi is complaining about my LT resources w/r/t to the invalid version. thi sis a terraform provider issue as old as the hills and the first time I ran into this in Pulumi.

Copy code

aws:ec2:LaunchTemplate (ci-kiam-server):
    error: 1 error occurred:
    	* updating urn:pulumi:ci::aws-eks-py::aws:ec2/launchTemplate:LaunchTemplate::ci-kiam-server: 1 error occurred:
    	* error modifying Launch Template (lt-0f0da90e33e334f78): InvalidParameterValue: Invalid launch template version: either '$Default', '$Latest', or a numeric version are allowed.

gorgeous-minister-41131

05/20/2021, 7:49 PM

I never changed my code, and left this:

gorgeous-minister-41131

05/20/2021, 7:49 PM

Copy code

user_data=user_data_b64,
            update_default_version=True,

gorgeous-minister-41131

05/20/2021, 8:00 PM

I raised it as an issue, since it’ seems squarely related to the terraform bug around it, https://github.com/pulumi/pulumi-aws/issues/1504, almost like pulumi-aws behavior changed and doesn’t align with what the underlying TF code is expecting for setting the default version of the launch template. I’m kinda at a loss as to what would have changed there without extensive digging, but it seems like a problem.

ambitious-salesmen-39356

05/20/2021, 9:12 PM

I'm trying to reference terraform remote state (app.terraform.io cloud backend) but I'm running into

Copy code

Preview failed: error constructing backend state manager: Remote workspace Terraform version "0.14.7" does not match local Terraform version "0.15.3"

Is it possible to configure the

NewRemoteStateReference

to use a specific terraform version in order to match our state?

echoing-zebra-28421

05/20/2021, 10:55 PM

Hi there, Anyone know how to add and read environment variable in

pulumi.yaml

? I have variables in an .env file but when I run

pulumi up

those variables are not uploaded.

colossal-australia-65039

05/20/2021, 11:58 PM

I thought Pulumi was supposed to properly recognize cloud provider secrets and encrypt them in preview/state. Or am I wrong about that? This TS code will show the value in plaintext in a preview:

Copy code

import { getSecretVersion, getSecret } from "@pulumi/aws/secretsmanager"

getSecret({ name: "somesecret" }).then((s) => getSecretVersion({ secretId: s.arn }).then((sv) => sv.secretString))