https://pulumi.com logo
Powered by
Title
b

better-shampoo-48884

06/25/2021, 12:42 PM
I know I've asked this before, but I think I have to give it another go (and I've tried looking around history as well). I'll just ask succinctly here and provide some more background in comments for those insterested: I need to rename every single resource in my stack (pulumi name, not actual deployment name) to avoid conflict. Previously I would just create one of each in a single region, but now my pulumi program needs to handle multiple regions/clusters in the same run. For example: 
new k8s.core.yaml("my-string",{..})
needs now to be `new k8s.core.yaml(
${region}-my-string
,{..)` There are ~180 resources created in my stack so far, and each of them needs a new name. Ideas?
b

billowy-army-68599

06/25/2021, 1:10 PM
Do you need to rename the resource as well?
b

better-shampoo-48884

06/25/2021, 1:20 PM
no
the deployed stuff needs to stay the same - basically i have 180 resources that I do not want touched by any changes to the stack naming 🙂
so it's just in order to prevent internal conflict.. and as this is fairly generic, I need to introduce the variable aspect to it (I can't just keep the first one as-is and the remainders with specific names)
b

billowy-army-68599

06/25/2021, 1:23 PM
you can use aliases: https://www.pulumi.com/blog/cumundi-guest-post/ or the brute force method is to import the stack state, rename the resources inside the JSON and reimport
b

better-shampoo-48884

06/25/2021, 1:25 PM
awesome - much love!
so yeah.. aliases.. and helm.. 😞
So aliases won't work. Additionally, when looking at my stack output - all k8s things generate URNs /without/ the pulumi name of the component in them - so they will naturally conflict no matter what? Am I forced to say that a pulumi program can only serve a single cluster per stack? A snippet from 
pulumi stack -u
shows the issue.. the ConfigGroup (keycloak-crd-install) is defined by me (pulumi name) but the URNs for the actual yaml (and crd) being deployed are without any naming I can influence - thus will conflict: 
kubernetes:yaml:ConfigGroup                                                              keycloak-crd-install
    │  │  │  URN: urn:pulumi:prod.k8s.devsecops::baseline-k8s::proj:k8s:foundation$kubernetes:yaml:ConfigGroup::keycloak-crd-install
    │  │  ├─ kubernetes:yaml:ConfigFile                                                            ./components/kubernetes-components/operators/keycloak-12.0.1/crds/keycloak.org_keycloakusers_crd.yaml
    │  │  │  │  URN: urn:pulumi:prod.k8s.devsecops::baseline-k8s::proj:k8s:foundation$kubernetes:yaml:ConfigGroup$kubernetes:yaml:ConfigFile::./components/kubernetes-components/operators/keycloak-12.0.1/crds/keycloak.org_keycloakusers_crd.yaml
    │  │  │  └─ kubernetes:<http://apiextensions.k8s.io/v1beta1:CustomResourceDefinition|apiextensions.k8s.io/v1beta1:CustomResourceDefinition>                   <http://keycloakusers.keycloak.org|keycloakusers.keycloak.org>
    │  │  │        URN: urn:pulumi:prod.k8s.devsecops::baseline-k8s::proj:k8s:foundation$kubernetes:yaml:ConfigGroup$kubernetes:yaml:ConfigFile$kubernetes:<http://apiextensions.k8s.io/v1beta1:CustomResourceDefinition::keycloakusers.keycloak.org|apiextensions.k8s.io/v1beta1:CustomResourceDefinition::keycloakusers.keycloak.org>
l

little-cartoon-10569

06/27/2021, 10:37 PM
I would probably create an entirely new project, import all the resources, then delete the old stack (not destroy). It would be a bit of work to pipe the stack export json through jq or similar to turn it into a big list of 
pulumi import
statements, but it should only be a couple of hours' work...
b

better-shampoo-48884

06/28/2021, 6:02 AM
my issue of the import part is knowing the ID - with kubernetes resources they seem to be fairly opaque..
At this point i'm honestly doubting if this is even possible to do - from what I see of the URNs - there is no way that looping through the creation of resources while switching providers, even with different names, will create unique URNs.
that's going to be hell when I get to multi-cluster istio setup.. which is the next step once i have multicluster baseline working as-is
(and the reason why I want multiple clusters in a single stack - I really want to control multiple clusters as a whole)
so it doesn't seem like I can simply use aliases to add scoping to my kubernetes resources.. and from what I understand, import needs an id - and ids are fairly obscure in kubernetes provider - so I don't have the foggiest of how I would even import one.. Anyone have any ideas?
So, I'm now at the point where I've given up on life, and am forced to create resources BY HAND for the first time since I started the pulumi journey.. I've: 1. Attempted using aliases, but kubernetes helm charts don't care 2. noticed that regardless I'll have URN conflicts because for some reason it doesn't seem like these things get scoped properly anyway (I can't understand how my other stack does it when this doesnt, but hey, maybe azure is different than kubernetes) 3. Created a brand new stack to try to get stuff going.. a. ..immediately hit on the secrets created by the first one in keyvault being there b. attempted to import those keys c. successfully imported those keys d. being told that those keys have to be recreated and pulumi fails to do so... As such. it will have to be by hand because this was supposed to be ready friday, and there is no longer anything I can mentally do to try to get this thing the way it nees to be.
b

billowy-army-68599

06/28/2021, 11:23 AM
I'm sorry you're having this experience, helm chart's being components makes it especially difficult. I don't have a better answer for you, but I would like to see the error around the keys failing to be imported
b

better-shampoo-48884

06/29/2021, 6:20 AM
Here's the (slightly anonymized) version of the key problem: 
Importing (prod.k8s.cust-temp-location):
     Type                                Name                                           Status
     pulumi:pulumi:Stack                 baseline-k8s-prod.k8s.cust-temp-location
     └─ base:k8s:databases               location-Database
 =      ├─ azure-native:keyvault:Secret  location-influx-pass-vault                 imported
 =      └─ azure-native:keyvault:Secret  location-influx-user-vault                 imported

Resources:
    = 2 imported
    39 unchanged

----

Updating (prod.k8s.cust-temp-location):
     Type                                Name                                           Status                   Info
     pulumi:pulumi:Stack                 baseline-k8s-prod.k8s.cust-temp-location  **failed**               1 error; 63 messages
     └─ base:k8s:databases               location-Database
 +-     └─ azure-native:keyvault:Secret  location-influx-user-vault                 **replacing failed**     [diff: -tags~properties,protect,provider]; 1 error

Diagnostics:
  azure-native:keyvault:Secret (location-influx-user-vault):
    error: cannot create already existing resource '/subscriptions/(sub)/resourceGroups/(rg)/providers/Microsoft.KeyVault/vaults/(vaultname)/secrets/influxdb-username'
For some reason it wants to recreate the key - most likely because the value of it changes, which is likely since the value it had was from a previous run in a stack I destroyed. Considering how hard it is to delete keys in vault - that was not an option. The provider from the stack had a random-looking numeric value (looking at another stack used) so I didn't feel like I could safely provide it - not that I think that was an option. Updating the value of a secret should not require recreation of that secret though, so I can't believe that the properties would cause it.
So - attempting to use the github discussions to get some traction and a bit more space to elaborate - hopefully someone would be so kind as to try to validate my approach and give some constructive criticism 🙂 https://github.com/pulumi/pulumi/discussions/7382
g

great-sunset-355

06/30/2021, 6:13 PM
@better-shampoo-48884 Thanks for looking into this, I'm trying pulumi for 2 weeks and I hit all these edge cases that were not so complicated with terraform
b

better-shampoo-48884

06/30/2021, 6:37 PM
🙂 honestly, I still feel the value of pulumi outweighs the "guarantees" of terraform - I'm already doing so much that would require many multiples of terraform states and apply's in sequence to execute. There are some rough edges though, but with great freedom comes the risk of falling off the edge.
❤️ 1
It's both a benefit and a curse to be able to have so many many ways of working around problems like these
Ok.. so I have a workaround that actually, y'know, works. If anyone needs to deploy the same kubernetes resources to multiple clusters but get 
Duplicate resource URN ...
issues - this might help. Essentially, I'm combining both `resourcePrefix`to introduce uniqueness to the naming of the resource, as well as 
transformations
to remove any instances of that uniqueness as it gets deployed. Hacky? Sure. Works? So far so good 🙂 https://github.com/pulumi/pulumi/discussions/7382 for a bit more detail.
#generalJoin Slack
Powered by