https://pulumi.com logo
#general
Title
# general
f

fast-vr-6049

12/06/2023, 7:56 PM
Hey folks! I've got a question about Stack outputs. I'm working with an Azure Container Registry Token resource, and there's a design pattern in Azure's APIs where you can't populate a secret when you create a resource, requiring an explicit API call to generate them. We do this in our code! However, there's another design pattern in their APIs, where you can't access a value of a secret after it has been created. This causes a problem where, during Pulumi runs after the first, the secret will be inaccessible, and thus always having seemed to change. There's a few ways around this: a) Use the
ignoreChanges
Custom Resource Option to ignore changes to the value, but it means that any resources that depend on the Token won't be able to get the value if anything ever changes b) Regenerate the Token every time, which works, but isn't ideal for anything that consumes the token outside of the stack (like as an output) My question: is it possible to set the Token as a (secret) Stack Output, then in future Pulumi runs, refer to that Output? I know you can reference the Outputs of another Stack using
StackReference
. Could I poke the infinite recursion bear by doing something like a
StackReference(self)
? If that pattern doesn't work, I'd also welcome suggestions for other methods.
l

little-cartoon-10569

12/06/2023, 8:06 PM
If you're using a stack reference, then the value you're referring to is coming from a different project, so the current
pulumi up
isn't touching the source of the value. You can use a stack reference to get that value any number of times, and the Azure container registry token isn't getting involved: the value is coming from the other stack's state.
f

fast-vr-6049

12/06/2023, 8:08 PM
Sadly, this is all in just the one Stack. I could try splitting this all into separate dependent Stacks, though that's not an ideal first choice
l

little-cartoon-10569

12/06/2023, 8:08 PM
You're creating a stack output and a stack reference to that output in the same stack? That's not what stack references are for.
d

dry-keyboard-94795

12/06/2023, 8:08 PM
Yes, you can use a Stack Reference to refer to outputs of the currently running stack. I used to do this to increment the image version of a deployment conditionally.
l

little-cartoon-10569

12/06/2023, 8:09 PM
You can use the value in the same stack. You don't need a stack output or reference.
Afaik, the stack reference will refer to the value in the persisted state. You will be looking at an old, cached value.
It is almost certainly the wrong thing to do.
f

fast-vr-6049

12/06/2023, 8:11 PM
@little-cartoon-10569 the problem is that I can't access this value after the Stack's first
up
command. Setting the Token's value as an output safely stores it in the Stack's state somewhere, which is why I thought using a Stack Reference would work. That old cached value is exactly what I'm hoping to retrieve
l

little-cartoon-10569

12/06/2023, 8:12 PM
It sounds like you have different deployment schedules, and thus should be using different projects.
Writing the value to the secret should happen less often than using that secret value. So the secret creation should be in a different project.
f

fast-vr-6049

12/06/2023, 8:13 PM
Is that common practice, to have a separate
Secrets
Stack?
l

little-cartoon-10569

12/06/2023, 8:13 PM
Project.
Not stack. The projects should be different. And yes, if you set up secrets only very rarely, and use them during all your "normal" deployments, then having a secrets projects is the recommended practice.
In smaller organizations, I recommend at least 3 projects: the never-change stuff (OIDC, some IAM, VPC + peering, global backup, etc.), the rarely-change stuff (clusters, secrets, per-app backup, etc.) and the normal deployments (apps etc.).
But there is no one design that fits all, so this might not work for you.
f

fast-vr-6049

12/06/2023, 8:19 PM
Thanks! Unfortunately, I'm in an (I must assume common) situation of an org that adopted Pulumi (or any IaC) far after its cloud and on-prem infra was established, so all our implementations have been self-contained up to this point. I'll think on breaking out elements into their own projects
l

little-cartoon-10569

12/06/2023, 8:20 PM
Ok. Well in the interim, I'm guessing that you put the value into the secret from configuration or similar; can you use that source to use the value, too? Then you wouldn't need to worry about continual updates to the secret.
f

fast-vr-6049

12/06/2023, 8:22 PM
Sadly not. One of the many quirks with this particular Resource is that Azure generates the secret for you, with no user input. Then it returns the value of the secret in the response to the
beginGenerateCredentialsAndWait()
invocation, then never again. After that, the only option for getting a value is to invoke
regenerateCredential()
, invalidating the old value
l

little-cartoon-10569

12/06/2023, 8:22 PM
Can you store that value is a different vault? One that fits your use case better?
f

fast-vr-6049

12/06/2023, 8:25 PM
We do store the value in an Azure Key Vault Secret, which the Stack creates after getting the value of the Token from the Azure API. The problem would then be, because the Token must be created before the Secret, we'd need the Stack to run the
GetSecret()
method on subsequent Pulumi executions, but that'd have to happen before the Secret resource is declared
l

little-cartoon-10569

12/06/2023, 8:25 PM
Yes. You need to use multiple projects to work around this.
f

fast-vr-6049

12/06/2023, 8:25 PM
We could work around this if some of the proposed work on handling circular dependencies ends up being done, but that's not an option right now
l

little-cartoon-10569

12/06/2023, 8:25 PM
You are trying to fit two deployment frequencies into a single deployment schedule.
f

fast-vr-6049

12/06/2023, 8:29 PM
Sorry, I'm not sure what you mean by that 😅
l

little-cartoon-10569

12/06/2023, 8:59 PM
I meant that the recommended practice of having the code that creates your secret and exports the stack output in a different project would solve the problem. The key indicator of the problem is that you're doing two things at once, but they need to happen at different rates. When you see that pattern, you know you need to split projects.
d

dry-keyboard-94795

12/06/2023, 9:22 PM
@fast-vr-6049 getting to the core problem you're facing. You have a Token resource, which on the first run you can use the value from. But on subsequent runs, the resource that use the value are getting updated because the value is no longer available from the Token resource. Is this the behaviour you're experiencing? Or is this theoretical behaviour you think will happen based on what the azure api does?
f

fast-vr-6049

12/06/2023, 9:29 PM
@dry-keyboard-94795 kind of in between those things. Simply creating the Token resource actually doesn't populate the secret value; you need to make a separate call to
beginGenerateCredentialsAndWait()
using the Azure ARM SDK (API here) because there's no equivalent in the Pulumi resource model. As such, that value doesn't become part of the Token resource. Because you can only call that method once, you can only have the value available to the run once, thus my dilemma
I've gotten using
regenerateCredential()
(https://learn.microsoft.com/en-us/javascript/api/%40azure/arm-containerregistry/registries?view=azure-node-latest#@azure-arm-cont[…]neratecredential) working during subsequent runs, but it runs into that less-than-ideal case of the value of the secret changing every run
d

dry-keyboard-94795

12/06/2023, 9:35 PM
Could the credentials be provided as inputs, using the
random
provider for generating a password?
f

fast-vr-6049

12/06/2023, 9:36 PM
Sadly no, you can only request that the API generate a value, no user input on what the secret is
d

dry-keyboard-94795

12/06/2023, 9:37 PM
I mean on the Token resource
f

fast-vr-6049

12/06/2023, 9:39 PM
The response object does define
Value
, but it isn't populated when the Token resource is created
d

dry-keyboard-94795

12/06/2023, 9:40 PM
Ah, true. Does the output credentials contain the password value if you supply those inputs?
f

fast-vr-6049

12/06/2023, 9:42 PM
I believe I tried that, but I don't want to rely on my fuzzy memory. I'll give it another try, with that argument populated. If that makes it generate the credential during creation, this issue will be totally resolved and I'll feel a little silly 😅
d

dry-keyboard-94795

12/06/2023, 9:44 PM
I totally get that azure can cause a great deal of stress. It's very in-line with Microsoft providing some not-great apis. Used it before, never again 🤣
f

fast-vr-6049

12/06/2023, 9:44 PM
Funnily enough, I'd still take it over AWS 😆
Ah yes, it failed, and I remember this error message
Copy code
error: Code="PasswordCannotBeAdded" Message="New passwords can be added only through 'generateCredentials'. For more information on repository permissions, please visit <https://aka.ms/acr/repo-permissions>."
d

dry-keyboard-94795

12/06/2023, 9:59 PM
Haha, why is it even part of the api then 😆 You could also use the classic provider for this specific resource: https://www.pulumi.com/registry/packages/azure/api-docs/containerservice/tokenpassword/
f

fast-vr-6049

12/06/2023, 9:59 PM
Why indeed
d

dry-keyboard-94795

12/06/2023, 10:00 PM
You can keep using the native provider for most resources, and dip into the classic provider where there's limitations. This is what I did with aws + gcp
f

fast-vr-6049

12/06/2023, 10:06 PM
Interesting! Thank you, I can give this a try
d

dry-keyboard-94795

12/06/2023, 10:06 PM
It's worth raising the missing functionality here: https://github.com/pulumi/pulumi-azure-native/issues There is an existing issue, but it got closed as it was treated more as a docs issue: https://github.com/pulumi/pulumi-azure-native/issues/2500
2 Views