Hi All. Could use some advice on secrets. I have configured my stack to use GCP KMS as the secret provider. I then proceeded to define a GCP service account and key and augmented the key resource with:

options:
      additionalSecretOutputs:
        - privateKeyData

This ensured that the corresponding output is

[secret]

and the json key is, indeed, obfuscated; however, it has only been base64 encoded as opposed to encrypted using GCP's KMS service. Clearly I've missed a step in this process and/or have too high expectations wrt pulumi's encoding scheme if the secrets provider is changed. Please can someone clarify?

Are you sure it's just base64 encoded? Secrets always look base64 encoded because the cipher text could be any bytes not just valid text. Are other secrets in the stack (such as if you do

config set --secret

) encrypted?

Def base64 encoded. Decoding yields the json.key contents intact. I haven't created any configuration secrets yet - was just looking to create one via something similar too:

pulumi config set --secret deploy_key $(pulumi stack output deploy_sa_key --show-secrets | base64 --decode --ignore-garbage)

I'd check if config values are getting encrypted properly. Secrets should only be base64 encoded if you've explictly configured your stack to use the "b64" secret provider.

My stack has been explicitly configured to use GCP KMS as per:

pulumi stack change-secrets-provider "<gcpkms://projects/*****/locations/*****/keyRings/*****/cryptoKeys/****>"

I can't think why this wouldn't work. As above, I'd try just setting a dummy config value with --secret and see if that encrypts as expected or not.

When the secrets provieder is changed for the stack two entries are added to the config file: •

secretsprovider

•

encryptedkey

My secrets provider value is correct as per the command above. fyi - creating secret config entries does work - they are definitely encrypted and not base64 encoded either. (As opposed to my situation described in my opening statement - I'll circle back to that once I've resolved this immediate issue). To test encryption via KMS I take the secret key value and run it through a gcloud kms decrypt command. It complains that the ciphertext is not valid. I suspect my expectation that the same kms key will not be able to decrypt this value and it may have something to do with the encrypted key now stored in the config file..?

hmm I wonder if additionalSecretOutputs isn't triggering an encrypt somehow. Could you raise an issue about this at github.com/pulumi/pulumi/issues so we can track it.

Right. My mistake. Context - before using pulumi I was using a proprietary python tool that used yaml as the definition language and was a wrapper to terraform.

yup not needed for Pulumi, just let the config encryption handle it

fyi - I am using a primary SA currently but plan to create a CICD workflow such that the pulumi SA will be created initially along with the key.

yup it's just list that will be default show them as [secret] unless --show-secrets is passed

Ok. Final question on this then - as opposed to using:

pulumi config set --secret deploy_key "$(pulumi stack output deploy_sa_key --show-secrets | base64 --decode --ignore-garbage)"

to generate the config secret key value Is there a way to auto-generate the config entry when the key resource is created (am using pulumi google-native yaml)

Is there a way to auto-generate the config entry when the key resource is created (am using pulumi google-native yaml)

I don't think so, the engine needs to know about the key somehow.

testing now - sec

Is that from

stack output

? Because that will also (like config get) decode for you.

No - I pipe the output from stack output to a base64 decode command

hmm ok that is odd, I don't think I'm going to have time to finish solving this over Slack today so def raise an issue at our github

How is it that both forms worked?

privateKeyData

vs

private_key_data

The value is still base64 encoded but the value does propagate. (I am assuming that the output var is updated on

pulumi up -y

and/or use of

pulumi refresh

privateKeyData

would of just been ignored, that's what I think we should warn about "hey you asked for key X to be secret but we couldn't find a key X" I'm not sure about why the value is base64 encoded but doesn't seem encrypted, but yes

up

or

refresh

should of updated the state to have the latest value for it.

Issue submitted: https://github.com/pulumi/pulumi/issues/9738

oh further oddity, I'm not sure then I don't think we do automatic case conversion but maybe I've just missed that code. We'll take a look at the issue and hopefully get the UX around this improved at the same time.

Thanks for help Fraser 👍