pulumi-community #general

general

calm-kitchen-4682

11/15/2022, 2:15 PM

Hey guys, I'm dealing with what's probably a simple issue, but can't tell what I'm doing wrong. I'm creating a Deployment of a Pod using the Python version of Pulumi. I'm trying to use a

nodeSelector

to restrict where the pod should be placed but I see Pulumi adding extra fields for me in the resulting YAML which leads to no node being selected during scheduling phase (please note the

node_selector

field in the Python snippet below). Does Pulumi add its own

nodegroup

selector as filtering criteria in addition to what I'm trying to filter on and if yes how do I prevent it from happening?

p_k8.apps.v1.Deployment(
        '{}-deployment-{}'.format(service, customer),
        metadata=p_k8.meta.v1.ObjectMetaArgs(
            name=app,
            namespace=name_space,
        ),
        spec=p_k8.apps.v1.DeploymentSpecArgs(
            replicas=1,
            strategy=p_k8.apps.v1.DeploymentStrategyArgs(
                type='RollingUpdate',
            ),
            selector=p_k8.meta.v1.LabelSelectorArgs(
                match_labels={
                    'app': app,
                },
            ),
            template=p_k8.core.v1.PodTemplateSpecArgs(
                metadata=p_k8.meta.v1.ObjectMetaArgs(
                    labels={
                        'app': app,
                    },
                ),
                spec=p_k8.core.v1.PodSpecArgs(
                    termination_grace_period_seconds=5,
                    image_pull_secrets=[{
                        'name': 'my-secrets',
                    }],
                    containers=[
                        p_k8.core.v1.ContainerArgs(...omitted...),
                    ],
                    node_selector={
                        '<http://kubernetes.io/os|kubernetes.io/os>': 'linux' # <--- This gets expanded into 2 selectors: a 'nodegroup' one and mine
                    },
                ),
            ),
        ))

The relevant section of the resulting yaml of the Pod which fails to be scheduled due to the extra

nodegroup

label added by Pulumi looks something like this. Why is the extra

nodegroup

label being added and how do I prevent it from happening?

delightful-computer-48592

11/15/2022, 2:47 PM

Hello 👋 I would like to know if is possible in Pulumi CLI to filter by NAME the output of the following command:

pulumi stack --show-urns

I’m getting in the output something like this:

TYPE                                                             NAME
    pulumi:pulumi:Stack                                              infra
    │  URN: urn:pulumi:xxxx
    ├─ pulumi:providers:aws                                          cluster
    │     URN: urn:pulumi:xxxx
    ├─ pulumi:providers:aws                                          default
    │     URN: urn:pulumi:xxxx
    ├─ infra:k8s-cluster-v2                                          cluster-2
    │  │  URN: urn:pulumi:xxxx

and I would like to get results only by

NAME=cluster

. Is it possible to do it without using a third-party tool like grep, sed, etc?

hundreds-jackal-10148

11/15/2022, 6:13 PM

Anyone got some pointers or experience they can share?

fast-easter-23401

11/16/2022, 1:45 AM

Anybody experiencing high latency issues?

great-jackal-59572

11/16/2022, 6:13 PM

Hi, I got a specific question about how you deal with dependencies between different features that have separate CI/CD pipelines with their own infra stacks. Particularly when it comes to the auto-naming and the random suffix that gets added onto resources. Do you simply go for the option by explicitly naming, so the random suffix isn't there? Say I have a feature A that provisions a storage account. And I might have a feature B and C that need access to the storage account & URL. Each delta infra deployment of feature A changes the resource URL via the random suffix, which might be configured for feature B and C via appsettings, but is effectively out of date with every deployment of A. What's the advice here?

little-soccer-5693

11/16/2022, 7:10 PM

what's the best way to have 'pulumi destroy' avoid destroying specific resources? e.g. i'd like to keep the aws cloudwatch log groups & streams created by the stack around subsequent to stack destruction. is 'pulumi delete' the recommended way to achieve this?

salmon-motherboard-78006

11/16/2022, 8:38 PM

hi all, how can I get the

id

of a specific Pulumi run/exeuction? Let’s say I run

pulumi up

I’m sure there is a unique ID attached to this run, how can I grab it?

rapid-eve-24229

11/17/2022, 2:17 AM

Hello All, I am trying to deploy my stack, and I keep getting this message pulumi_docker.docker.ResourceError: 'docker login' failed with exit code 1

pulumi==3.46.1
pulumi-aws==5.21.0
pulumi-docker==3.1.0
python-dotenv==0.21.0

What can I do to deploy the stack? Thanks

clever-soccer-98174

11/17/2022, 2:32 AM

Hey All, I am trying this component https://www.pulumi.com/registry/packages/aws-static-website/api-docs/website/ And have a question , when I deploy my website with example.com , it works perfectly, but I could not figure out the strategy to deploy subdomains with environments, such as app.dev.example.com , when I try, I’m getting an error that says route53 zone is not found. is this component supporting subdomains any idea how to do it?

tall-crowd-93084

11/17/2022, 11:05 AM

Hey Folks, I'd like to use pulumi github to fork a repo in an organisation. It's not immediately obvious how to do that: I want

org/repo-template

forked to

org/{name}-repo

where

name

parameterizes a load of other infra that's set up (one per customer). Questions: • Can I specify where to fork from when creating a

github.Repository

or do I need to use a template • What kind of access token do I need? • How do I specify which organisation to create the repo in?

delightful-camera-97029

11/17/2022, 11:08 AM

heyaaa im new here

delightful-camera-97029

11/17/2022, 11:08 AM

Im joining the community because of a question I have about pulumi

delightful-camera-97029

11/17/2022, 11:09 AM

I am trying to create a script for glue using glue.GetScript, the problems is I don't know what NodeType are there beside ones in the example "DataSource" "DataSink" and "ApplyMapping"

delightful-camera-97029

11/17/2022, 11:10 AM

I am trying to write a transformation result into S3

delightful-camera-97029

11/17/2022, 11:10 AM

I tried with DataType = "Bucket" but it throw err

delightful-camera-97029

11/17/2022, 11:10 AM

Im also already looking at the github, but didn't find anything useful.

delightful-camera-97029

11/17/2022, 11:11 AM

oh btw im using AWS with golang

billions-xylophone-85957

11/17/2022, 11:59 AM

is it even possible to try/catch a

new pulumiResource

careful-apartment-69053

11/17/2022, 3:36 PM

does anyone know if there are type mapping table out there between a MSFT Az defined type and a Pulumi azure-native defined type? ex:

Microsoft.Compute/virtualMachines : azure-native:compute:VirtualMachine

hallowed-printer-89159

11/17/2022, 3:42 PM

Hello All, anyone knows how to create pulumi config to gcp, for example, I need create diferents VM in diferent regions and zones, I would apreciate your help

future-france-34957

11/17/2022, 6:16 PM

How does Pulumi determine which plugin version to use? I assumed it would be the version in

requirements.txt

pulumi_aws_native>=0.41.0<1.0.0

But that doesn’t seem to be the case for

aws-native

Pulumi version used: v3.43.1

$ pulumi up
Previewing update (sitka/uat)

View Live: <https://app.pulumi.com/sitka/env-app/uat/previews/5631a648-bebb-4f25-b6bd-bd739dd68b2a>




error: could not load plugin for aws-native provider 'urn:pulumi:uat::env-app::pulumi:providers:aws-native::default': Could not automatically download and install resource plugin 'pulumi-resource-aws-native'at version v0.22.1-alpha.1661750811, install the plugin using `pulumi plugin install resource aws-native v0.22.1-alpha.1661750811`.
Underlying error: error downloading plugin aws-native to file: failed to download plugin: aws-native-0.22.1-alpha.1661750811: 403 HTTP error fetching plugin from <https://get.pulumi.com/releases/plugins/pulumi-resource-aws-native-v0.22.1-alpha.1661750811-darwin-amd64.tar.gz>

I have

aws-native

0.41.0

installed

$ pulumi plugin ls | grep 'NAME\|aws-native'
NAME        KIND      VERSION                           SIZE    INSTALLED     LAST USED
aws-native  resource  0.41.0                            92 MB   2 hours ago   30 minutes ago
aws-native  resource  0.39.0                            92 MB   3 weeks ago   3 weeks ago
aws-native  resource  0.29.0                            84 MB   2 months ago  2 months ago
aws-native  resource  0.19.1-alpha.1660828085+dfb123ac  86 MB   3 months ago  3 months ago
aws-native  resource  0.19.1-alpha.1660126749+9e28f183  84 MB   3 months ago  3 months ago
aws-native  resource  0.19.1-alpha.1659013975+9c6f4698  84 MB   3 months ago  3 months ago
aws-native  resource  0.19.1-alpha.1656019149+c2daf7c7  87 MB   4 months ago  4 months ago
aws-native  resource  0.19.0                            87 MB   4 months ago  2 months ago

Why is pulumi attempting to use

aws-native-0.22.1-alpha.1661750811

, which no longer exists, locally or on get.pulumi.com?

salmon-motherboard-78006

11/17/2022, 6:55 PM

Hi all, I’m looking to set the field

DeliveryStreamType

with the

aws

sdk, but it’s only available with

aws-native

. Any idea why? https://www.pulumi.com/registry/packages/aws-native/api-docs/kinesisfirehose/deliverystream/ CF seems to have this field: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-kinesisfirehose-deliverystream.html#cfn-kinesisfireh[…]eliverystreamtype

glamorous-australia-21342

11/17/2022, 9:38 PM

Is there any support for CloudFlare R2? https://blog.cloudflare.com/introducing-r2-object-storage/ I'm having trouble finding some information. Not seeing anything in the API docs. https://www.pulumi.com/registry/packages/cloudflare/api-docs/ Seems like Terraform might be limited as well?

sticky-bear-14421

11/18/2022, 7:52 AM

Just wanted to express my gratitude. The awesome pulumi challenge mug arrived and I made already good and extensive use of it. Drinking hot tea during the cold weather period and then … ♥️

delightful-computer-48592

11/18/2022, 2:33 PM

Hello! Is there a way to

--supress-outputs

in Pulumi GitHub actions? There is no option and I can’t find an environment variable to set either.

little-library-54601

11/18/2022, 2:40 PM

I'm getting this error in the Github action workflow that's using pulumi/actions@v3:

error: the stack is currently locked by 1 lock(s). Either wait for the other process(es) to end or delete the lock file with `pulumi cancel`.

I had to cancel a workflow run because of an error in the workflow file. I'm sure that's why the lock file is hanging around. But I can't "pulumi cancel" in the workflow itself. Can I run "pulumi cancel" from the command line to get the required effect? Or is it safe to delete the lock file from the place where it lives (azure storage blob)? Thanks.

echoing-dress-72742

11/18/2022, 2:42 PM

hey,

var resourceGroup = new Pulumi.AzureNative.Resources.ResourceGroup("rg-nicolai", new ResourceGroupArgs()
{
    ResourceGroupName = "tst-rg-nicolai",
    Location = "Norway East"

}, new CustomResourceOptions(){Provider =  azProv});

What are the differences here, I mean ResourceGroup(“what are inside here”) And compared to ResourceGroupName

few-carpenter-12885

11/18/2022, 3:56 PM

Is there a way to set an overall stack timeout? I see there is a way to set

customTimeouts

on each individual resource, but I was not able to find anything about an overall stack timeout. For my use-case, I'm wanting to run the automation API inside AWS Lambda - so I want to stay within the Lambda timeout.

chilly-plastic-75584

11/18/2022, 5:56 PM

Is there a smooth way for transferring the state of my Pulumi deployment to import into terraform. I'm expecting no, it's a manual import process, but if there's any tips on maybe turning my resource name list into equivalent terraform resource types, then I'll take it. 👋 Disclaimer: I'm a huge fan of Pulumi. I'm still using. I just want to minimize any negative experience another team that decided on Terraform has with minimizing friction for them to start managing the resources I created with Pulumi last year. Definitely don't want to be the point of friction.

fierce-dinner-64337

11/18/2022, 8:05 PM

@here Hello everyone! My team(me basically) has been tasked to deploy Azure infra using Pulumi. I understand that I need to do something like a console app that will list the resources that we need ( APIM, AKS etc etc). On my terminal I can play with some basic commands to try to deploy infra and then destroy. My plan is to keep the pulumi c# code on a different repo KYC.Pulumi My question is around the integration with Azure Devops. I've found several articles that explain how to preview, build and deploy. However, the Pupumi repo will list all infra components that I might not need for a given API. For example, I want to deploy only an App Service with Cosmos or AKS setup with SQL etc etc. My point is that I want to selectively deploy and not deploy all components listed in my pulumi repo. How do I control that? ANy ref?

Title

fierce-dinner-64337

11/18/2022, 8:05 PM

stocky-restaurant-98004

11/18/2022, 8:38 PM

I'm not sure what you mean "list all infra components", Pulumi is declarative, so if you

new

something up, it will create it (or change if necessary). If something is in Pulumi state and it's not longer in your Pulumi program, it will delete it.

If you're referring to `ComponentResource`s, I recommend you hold off on using them until you're very comfortable with the basics.

fierce-dinner-64337

11/18/2022, 8:40 PM

I need to get comfortable soon.

Let le rephrase

the pulumi console project will be responsible to provision azure stack (api management, cosmos, sql, aks bla bla)

When we do smtg like pulumi -up it will create whatever we have on that dll, right?

stocky-restaurant-98004

11/18/2022, 8:41 PM

Correct - whatever you

new

up, Pulumi will manage its lifecycle.

fierce-dinner-64337

11/18/2022, 8:42 PM

what if want to select what we create from that dll?

stocky-restaurant-98004

11/18/2022, 8:42 PM

For example?

fierce-dinner-64337

11/18/2022, 8:43 PM

Imagine that this pulumi console repo will be referenced by an x number of API's that will have different needs

for instance API A will need App services and SQL but API B will only need AKS and Cosmos, so I don't want to deploy all components that are listed on my pulumi dll's

an example is the terraform modules.

stocky-restaurant-98004

11/18/2022, 8:44 PM

Make a separate Pulumi program for each API.

fierce-dinner-64337

11/18/2022, 8:45 PM

is that the best way?

stocky-restaurant-98004

11/18/2022, 8:45 PM

The analogy of a TF module in Pulumi is a ComponentResource, but I recommend you skip them for now until you are sure you need them.

fierce-dinner-64337

11/18/2022, 8:45 PM

this is not a department with 5 engineers

stocky-restaurant-98004

11/18/2022, 8:46 PM

You can refactor to ComponentResources once you have something up and running.

fierce-dinner-64337

11/18/2022, 8:47 PM

there will be various teams that they will want to reference pulumi dll's but nobody wants to maintain 100 pulumi projects!

stocky-restaurant-98004

11/18/2022, 8:47 PM

What's the first deliverable that you need to deliver?

fierce-dinner-64337

11/18/2022, 8:48 PM

I have 2 weeks to set up infra (infra includes resource groups, AKS, KeyVault, APIM, Redis). They want to do a Poc and then make it official but the mandate is pulumi. They are 100% sure. I

(they use Azure DevOps)

I don't mind the tool, I think its great, its the deadline that make me freak out

stocky-restaurant-98004

11/18/2022, 8:50 PM

If you want to make a shared infra component library, that can sometimes be helpful, but at the same time, making an abstraction before you're ready can cause more pain the long run. Especially with infrastructure where breaking changes can be non-starters (like you can't take down the prod DB or have a virtual network get re-created).

How many app teams do you need to create infra for?

fierce-dinner-64337

11/18/2022, 8:51 PM

Our customers are those teams. We have 5.

stocky-restaurant-98004

11/18/2022, 8:52 PM

5 teams? 5 separate Pulumi programs should be manageable.

Refactor to ComponentResource if necessary, but copy/paste is not always bad. You have to really write a few programs to make sure that a ComponentResource is useful - that it's actually abstracting something.

fierce-dinner-64337

11/18/2022, 8:53 PM

ok so 5 executables for now and then refactor ( if I need to)

stocky-restaurant-98004

11/18/2022, 8:53 PM

Unless there are shared components, like an AKS cluster or something.

If there's shared stuff, like everything is deployed into the same virtual network, then put that in its own stack and add whatever stack outputs you need so the app teams can deploy into those shared resources.

But the key thing IMO is to get one team up and running end to end before you do any refactoring. Start with that.

fierce-dinner-64337

11/18/2022, 8:56 PM

Yes I understand! Thanks Josh!

stocky-restaurant-98004

11/18/2022, 8:56 PM

No problem and good luck!

fierce-dinner-64337

11/18/2022, 8:56 PM

Pulumi weekend!

stocky-restaurant-98004

11/18/2022, 8:56 PM

FYI, you can also define Azure DevOps resources with Pulumi. There's an ADO provider.

fierce-dinner-64337

11/18/2022, 8:57 PM

I found this among others https://build5nines.com/beginners-guide-to-pulumi-ci-cd-pipelines/

stocky-restaurant-98004

11/18/2022, 8:58 PM

Defining pipelines in code has been very helpful for me in the past. If all the pipelines are the same and just differ by the source repo, THAT would be a good place to try a ComponentResource. When you're ready to try them, keep them in the same repo as they are used. Publishing them to NuGet can be helpful, but it comes with considerable maintenance overhead.

But first things first: I recommend you try to get one team's infra working end-to-end from your machine, then maybe move on to creating a pipeline.

fierce-dinner-64337

11/18/2022, 9:00 PM

yes! Thanks for taking time to answer my questions!

stocky-restaurant-98004

11/18/2022, 9:01 PM

My pleasure! Check this out as well: https://www.pulumi.com/docs/guides/continuous-delivery/azure-devops/

fierce-dinner-64337

11/18/2022, 9:01 PM

👍

View count: 2