https://pulumi.com logo
#general
Title
# general
f

fast-sandwich-30809

03/18/2024, 12:14 AM
Hey all! Using the serialized
__provider
in the statefile for dynamic resources is causing nightmares (mostly re trying to not serialize rotating credentials). It seems the new state of the art is relying on
process.env
to pass in auth tokens. As long as my code runs before the serialized provider, this should be fine - but I'm worried about refresh operations. Does
refresh
run the program? Or does it only look at the statefile?
l

little-cartoon-10569

03/18/2024, 2:00 AM
It only looks at the state file and the cloud. It needs valid credentials to pull values from the cloud into the state file. Why does running the program affect your credentials? Your credentials will affect whether or not the program runs, but your program won't affect the credentials.
f

fast-sandwich-30809

03/18/2024, 12:46 PM
The API requires a bearer token, which is only valid for an hour. Naturally we can't embed that, but we can embed a strategy to go get it.
Reading from process.env was the solution here: https://github.com/pulumi/pulumi/issues/8265#issuecomment-1958438888 Embedding the username/password and the authentication function would also work, but then we can never change the password on the account. If refresh doesn't call my code, then I think I have three options • Make a fully-blown static provider • Embed a static token, then call my own API to do the auth • Stop doing refreshes entirely and just do
up
Update: pulling from
process.env
, even during
up
, doesn't appear to be working. We've devised an evil scheme, however, that just might work: • When our code runs, write the token to
/tmp/${provider}_${resource_name}.txt
• Read from said file in the provider (I'm guessing process.env didn't work because the provider runs in a different process) • Since we're using the automation API, we can call the subset of our code that does said writing before passing to pulumi for the
refresh
. This is extremely hacky, but avoids all embedded credentials, and we can "test" the creds before getting pulumi involved at all. (fail fast!) The only failure case I can think of is serializing
'node:fs'
. We've tried and miserably failed in the past trying to get
URL()
to work post-serialization. If we can't use node built-ins in the provider, we'll have to rely on globals, and do something with
fetch
and a local http listener think I hope that's not the case
Another update - it looks like
LocalWorkspaceOptions
takes an
envVars
option. We still need to break our pulumi programs into
preBuild
and
build
to populate tokens, but then we can use
process.env
again and avoid the
node:fs
tangle entirely!
l

little-cartoon-10569

03/18/2024, 8:09 PM
If you've got time, you could dig into another provider's time-based session token handling. I don't know how it's done internally. For AWS, the code isn't even in the Pulumi provider afaik, it's in the AWS SDK, and the Pulumi provider calls that. So you'd have to dig pretty deep to figure it out.
f

fast-sandwich-30809

03/22/2024, 5:55 PM
We got it working! Our scripts define a build function, then call our "core" which calls the automation api. By setting the process environment variables simply outside the build function, they exist for
refresh
operations. The dynamic provider then just looks for a token in the environment variables and it's good to go! We have successfully deployed + re-created + mutated custom resources 🙂