https://pulumi.com logo
Title
b

better-shampoo-48884

04/08/2021, 2:25 PM
Ok.. I've got a bit of a crazy idea for something. I'll paste more info in the thread of this message - but the gist is: I would like to generate passwords for rabbitmq (which is done with a silly sha256/512 hashing algorithm with a salt converted to base64). To do that, I'd like to use pulumi/random RandomPassword module for the passwords themselves - and put them into azure keyvault. However, I also need to pass those passwords in their rabbitmq-obfuscated-hashed form into a json object as values for rabbitmq config and put into a kubernetes secret (or more likely, throw into keyvault and use keyvault sync to create the secret). The fun begins..
So I set this up to generate the passwords themselves:
const rabbitUsers =  {
        orchestratorclient: new random.RandomPassword("orchestratorclient", {length:32}),
        orchestratoruser: new random.RandomPassword("orchestratoruser", {length:32}),
        subscriptionuser: new random.RandomPassword("subscriptionuser", {length:32}),
        orchestrator: new random.RandomPassword("orchestrator", {length:32}),
        admin: new random.RandomPassword("admin", {length:32}),
        producer: new random.RandomPassword("producer", {length:32}),
        testreader: new random.RandomPassword("testreader", {length:32})
    }
And then I have my rabbit-password-hashing-function:
const rabbitPassHasher = (pass2hash : string, salt?: Buffer) => {
        //let salt = Buffer.from([0x90,0x8D,0xC6,0x0A])
        if (salt) {
            
        } else {
            salt = randomBytes(32)
        }    
        console.log(salt)
        //let concated = Buffer.concat([salt, Buffer.from("test12")])
        let pass = Buffer.from(pass2hash)
        let concated : Buffer = Buffer.concat([salt, pass])
        let hash = createHash('sha256')
        hash.write(concated)
        let hashDigest = hash.digest()
        return Buffer.concat([salt, hashDigest]).toString('base64')    
    }

    const salt = Buffer.from([0xDE,0xAD,0xCA,0xFE])
b

brave-planet-10645

04/08/2021, 2:27 PM
You're doing
new random.RandomPassword("foo", {length: 32}).result
right?
b

better-shampoo-48884

04/08/2021, 2:27 PM
The trick comes with this...
let rabbitLoadFile = { 
        "rabbit_version": "3.7.13",
        "users": [{
            "name": "orchestratorclient",
            "password_hash": rabbitPassHasher(pulumi.interpolate`rabbitUsers.orchestratorclient.result`, salt),
            "hashing_algorithm": "rabbit_password_hashing_sha256",
            "tags": ""
        }, 
....
}
Because, of course, and as it should be - rabbitUsers.orchestratorclient.result is Output<string>
yeah was getting to that ­čśë
b

brave-planet-10645

04/08/2021, 2:27 PM
(just to interrupt you)
b

better-shampoo-48884

04/08/2021, 2:28 PM
so I've tried both the .apply and the interpolate methods - and they both turn out Output<string>.. and I'm realizing that yes - they should. it needs to be that way.
so.. my thought now is - can I extend the random module to generate both the password AND the rabbit-hashed version of the same?
b

brave-planet-10645

04/08/2021, 2:29 PM
Could you use a component resource? And the output is the rabbit-hashed version?
b

better-shampoo-48884

04/08/2021, 2:29 PM
like.. we have
.result
- if I could add
.rabbitHashedResult
to the list, that should solve it nicely
b

bored-oyster-3147

04/08/2021, 2:29 PM
can you not do:
rabbitUsers.orchestratorclient.result.apply(x => rabbitPassHasher(x, salt))
b

better-shampoo-48884

04/08/2021, 2:30 PM
i.. i.. probably..
could.
gonna test that out!
and otherwise - component resource looks like a good direction for a next step
b

bored-oyster-3147

04/08/2021, 2:31 PM
you might need to output tuple the salt, i didn't know if that was also an output or not
b

better-shampoo-48884

04/08/2021, 2:31 PM
dammit Joshua - why do you have to point out the blaringly obvious ­čśë thanks ÔŁĄ´ŞĆ
b

bored-oyster-3147

04/08/2021, 2:32 PM
haha hope it works!
b

better-shampoo-48884

04/08/2021, 2:32 PM
the salt is of no consequence (though it would be fun to also have the salt controlled by Random - but i'm going to accept a static salt in my code ;))
b

bored-oyster-3147

04/08/2021, 2:35 PM
that would give you
Output<string>
of the hash so then to put that in the JSON you would need to do like:
hash.apply(rabbitHash =>
{
    return {
        ...,
        "password_hash": rabbitHash,
        ...,
    };
});
b

better-shampoo-48884

04/09/2021, 6:03 PM
So... sorry to take so long for getting confirmation, been struggling with automation, promise leaks, and everything weird in general. Regardless! I thought I got everything nicely handled - my json object got pushed into azure as a secret, and propegated into k8s as a secret through synchronization - and what do I finally find? this:
"users": [
        {
            "name": "orchestratorclient",
            "password_hash": "Calling [toJSON] on an [Output<T>] is not supported.\n\nTo get the value of an Output as a JSON value or JSON string consider either:\n    1: o.apply(v => v.toJSON())\n    2: o.apply(v => JSON.stringify(v))\n\nSee <https://pulumi.io/help/outputs> for more details.\nThis function may throw in a future version of @pulumi/pulumi.",
            "hashing_algorithm": "rabbit_password_hashing_sha256",
            "tags": ""
        },
­čśä ­čśä
so.. Joshua was right, I'm now just trying to wrap my head around how to JSON.stringify the resulting object..
basically the object above needs to be created through this:
let rabbitLoadFile = { 
        "rabbit_version": "3.8.14",
        "users": [{
            "name": "orchestratorclient",
            "password_hash": rabbitUsers.orchestratorclient.result.apply(x => { return rabbitPassHasher(x, salt)}),
            "hashing_algorithm": "rabbit_password_hashing_sha256",
            "tags": ""
        },{
         ...
        }]
}

 const rabbitLoadSecret = new azure.keyvault.Secret(`rabbit-load-definition`, {
        resourceGroupName: environment.rg,
        vaultName: environment.vault.name,
        secretName: `rabbit-load-definition`,
        properties: {
            value: JSON.stringify(rabbitLoadFile.apply())
        }
    })
now.. since password_hash is an Output<string>..
but rabbitLoadFile is just a plain object..
how.. do I resolve all these? take one of the passwords, do an apply, output the whole dang thing as a JSON.stringify'd object as the result?
This is the part of pulumi that makes my brain shut off and go into gaga mode for some reason.. I keep doing wrong things that I know that are wrong but slightly out of desperation. Like..
value: rabbitUsers.admin.result.apply(x => { return JSON.stringify(rabbitLoadFile)})
b

bored-oyster-3147

04/09/2021, 6:15 PM
you're missing a key concept
Output<T> needs to be Output<T> all the way down. You cannot get your value you of it. (unless you were to await it, but you wouldn't do that in a traditional pulumi program)
this code is an issue:
let rabbitLoadFile = { 
        "rabbit_version": "3.8.14",
        "users": [{
            "name": "orchestratorclient",
            "password_hash": rabbitUsers.orchestratorclient.result.apply(x => { return rabbitPassHasher(x, salt)}),
            "hashing_algorithm": "rabbit_password_hashing_sha256",
            "tags": ""
        },{
         ...
        }]
}
It will never work that way, because you are setting an Output<T> on a property on the object
you need the entire object to be an Output<T>. The:
properties: {
            value: JSON.stringify(rabbitLoadFile.apply())
        }
The
value
here should accept Input<T>
b

better-shampoo-48884

04/09/2021, 6:20 PM
Right
Can I simply state that the type of the object is an Output<T> then?
I mean.. "it's never that easy". That's why I was originally thinking of extending the random generation to an extent - but I would have hit the same problem.
b

bored-oyster-3147

04/09/2021, 6:22 PM
so you need to do this:
let rabbitLoadFile = rabbitUsers.orchestratorclient.result.apply(password =>
{
    return {
        "rabbit_version": "3.8.14",
        "users": [{
            "name": "orchestratorclient",
            "password_hash": rabbitPassHasher(password, salt),
            "hashing_algorithm": "rabbit_password_hashing_sha256",
            "tags": "",
        }],
    };
});
now your load file is an Output<object>. And then:
const rabbitLoadSecret = new azure.keyvault.Secret(`rabbit-load-definition`, {
        resourceGroupName: environment.rg,
        vaultName: environment.vault.name,
        secretName: `rabbit-load-definition`,
        properties: {
            value: rabbitLoadFile.apply(x => JSON.stringify(x)),
        }
    })
b

better-shampoo-48884

04/09/2021, 6:23 PM
see - this is where I'm slightly confused. Since rabbitLoadFile is already an Output<T> in the first object - would it not be pertinent to simply JSON.stringify() that return and then use it directly with
properties.value
?
b

bored-oyster-3147

04/09/2021, 6:24 PM
rabbitLoadFile is not an
Output<T>
in your original example
b

better-shampoo-48884

04/09/2021, 6:24 PM
i.e.
value: rabbitUsers.admin.result.apply(x => { return JSON.stringify(rabbitLoadFile)})
seems to me to be identical
b

bored-oyster-3147

04/09/2021, 6:24 PM
it is an inline javascript object that you declared
b

better-shampoo-48884

04/09/2021, 6:25 PM
but in your example you're wrapping the js object inside ..client.result.apply() to force it to an output. That would be the same as my one-liner right?
b

bored-oyster-3147

04/09/2021, 6:25 PM
no
your rabbitLoadFile is
object
mine is
Output<object>
b

better-shampoo-48884

04/09/2021, 6:27 PM
let rabbitLoadFile = rabbitUsers.orchestratorclient.result.apply(password =>
{
    return JSON.stringify({
        "rabbit_version": "3.8.14",
        "users": [{
            "name": "orchestratorclient",
            "password_hash": rabbitPassHasher(password, salt),
            "hashing_algorithm": "rabbit_password_hashing_sha256",
            "tags": "",
        }],
    });
});
The above would be an Output<string> right?
b

bored-oyster-3147

04/09/2021, 6:28 PM
correct
then you could just pass
rabbitLoadFile
directly to
properties.value
assuming it takes
Input<string>
b

better-shampoo-48884

04/09/2021, 6:28 PM
ok, think I see the issue in comprehension here - I'm currently making an Output<string> of the password_hash value because I do the .result.apply() at that stage
the reason being that the object itself contains multiple users..
let rabbitLoadFile = rabbitUsers.orchestratorclient.result.apply(pass{ 
        "rabbit_version": "3.8.14",
        "users": [{
            "name": "orchestratorclient",
            "password_hash": rabbitUsers.orchestratorclient.result.apply(x => { return rabbitPassHasher(x, salt)}),
            "hashing_algorithm": "rabbit_password_hashing_sha256",
            "tags": ""
        }, {
            "name": "orchestratoruser",
            "password_hash": rabbitUsers.orchestratoruser.result.apply(x => { return rabbitPassHasher(x, salt)}),
            "hashing_algorithm": "rabbit_password_hashing_sha256",
            "tags": ""
        }, {
each of which I want a password for:
const rabbitUsers : { [index: string]: random.RandomPassword } =  {
        orchestratorclient: new random.RandomPassword("orchestratorclient", {length:32}),
        orchestratoruser: new random.RandomPassword("orchestratoruser", {length:32}),
        subscriptionuser: new random.RandomPassword("subscriptionuser", {length:32}),
        orchestrator: new random.RandomPassword("orchestrator", {length:32}),
        admin: new random.RandomPassword("admin", {length:32}),
        producer: new random.RandomPassword("producer", {length:32}),
        testreader: new random.RandomPassword("testreader", {length:32})
    }
so.. considering users is an array of Output<Object> - it would need to be an Output<Array> and so on..
b

bored-oyster-3147

04/09/2021, 6:36 PM
ah. It's probably going to be painful but this is where
output.tuple
or
output.all
comes in, so you can pass multiple
Output<T>
into an
.apply(...)
call. Alternatively you could save the
JSON.stringify(...)
until the very end and append to the
users
array. So you could do:
let rabbitLoadFile = password1.apply(pw =>
{
    return {
        "rabbit_version": "3.8.14",
        "users": [{
            "name": "orchestratorclient",
            "password_hash": rabbitPassHasher(pw, salt),
            "hashing_algorithm": "rabbit_password_hashing_sha256",
            "tags": "",
        }],
    };
});
and that would just be for the first password. You want to keep it as an
Output<object>
so you can continue to manipulate it as an object. Then all passwords after that could look like:
rabbitLoadFile = output.all(rabbitLoadFile, password2).apply((file, pw) =>
{
    // instead of adding it to the existing object
    // you may want to clone it, modify the clone, and return the clone
    file.users.add({
        "name": "orchestratorclient",
        "password_hash": rabbitPassHasher(pw, salt),
        "hashing_algorithm": "rabbit_password_hashing_sha256",
        "tags": "",
    });
});
and then finally on
properties.value
you would
rabbitLoadFile.apply(x => JSON.stringify(x))
I'm psuedo coding a bit when it comes to array manipulation idk what the exact syntax would be, but hopefully i conveyed the idea
b

better-shampoo-48884

04/09/2021, 6:38 PM
there is much contemplation in my head.. but it makes some sense yes.
just a quick thought - where did you get `output.all`from?
b

bored-oyster-3147

04/09/2021, 6:39 PM
what SDK are you in? TypeScript?
b

better-shampoo-48884

04/09/2021, 6:39 PM
yup
pulumi.output doesn't seem to have that
so guessing it's somewhere slightly different..
b

bored-oyster-3147

04/09/2021, 6:40 PM
in C# it is called
Output.Tuple(...)
b

better-shampoo-48884

04/09/2021, 6:41 PM
ah righty - looking good then.
as such - shall dive deeply into the mire of rabbits, and a few previews and ups, and with any hope, come back to report success and share the resulting code ­čśä
b

bored-oyster-3147

04/09/2021, 6:42 PM
good luck!
ill be here!
b

better-shampoo-48884

04/09/2021, 6:42 PM
thanks ­čśë
So far so good.. preview phase ended with less than 2gb of memory used for once (was 1920mb, so not too far off but hey I'll take it), and it updated the secret! ­čśä
"rabbit_version": "3.8.14",
    "users": [
        {
            "name": "orchestratorclient",
            "password_hash": "3q3K/j9AhUZPSGJYsSu7xZHu6ggA2YsaSzDkWFUO6+Ch7aWQ",
            "hashing_algorithm": "rabbit_password_hashing_sha256",
            "tags": ""
        },
        {
happy days!
Heh.. how are we with pasting 283 lines of insanity in here? ­čśë
can push it into a gist otherwise I presume
so yeah. this works. and I can live with it. THANK YOU so incredibly much, I'm definitely hoping that next time I face something like this I'll remember the lessons on Output<T>
b

bored-oyster-3147

04/09/2021, 7:54 PM
glad to hear it!