https://pulumi.com logo
Powered by
Title
w

wet-noon-14291

10/15/2020, 10:54 AM
What service principal is used when? My scenario: * I have logged in to 
az cli
using service principal 
X
since that is what will be used on the CI server to access blob storage and key vault for secrets. * In my 
Pulumi.Dev.yaml
I have service principal 
Y
defined with 
pulumi config set azure:clientId
Now when I try to run 
pulumi up
I would expect 
X
to be used to store state in blob storage as well as dealing with secrets, but service principal 
Y
to be used when creating the resources on Azure. However, 
pulumi up
fails with the message that 
X
doesn't have access to the resource group I'm trying to import. 
X
shouldn't have access, so that is ok, but why is 
X
used when I run 
pulumi up
and not 
Y
since that is what I have defined in my stack config file?
or is it so that when you are logged in the user you're logged in with will always be used no matter what you have specified in the config file? I can live with that as well, I just need to change some stuff in the pipeline if that is the case.
t

tall-librarian-49374

10/15/2020, 11:59 AM
Hmm, I would expect the CLI to be the last option https://github.com/hashicorp/go-azure-helpers/blob/4a2638291169836ade07f3cd4476035865e9d4a6/authentication/builder.go#L64
Run with verbose logging and check what’s going on?
w

wet-noon-14291

10/15/2020, 1:25 PM
Tried 
--verbose 4/5/6/7
but not that much info. Like 
pulumi up --verbose 5  -y -C <path> -s Dev
the logging doesn't even differ for me for some weird reason, I have the same on all levels.
t

tall-librarian-49374

10/15/2020, 1:35 PM
I do 
pulumi up -v=9 --logtostderr --yes &> a.txt
I see things like 
Testing if Service Principal / Client Certificate is applicable for Authentication..
there
w

wet-noon-14291

10/15/2020, 1:37 PM
I just found it, need the 
--logtostderr
What should I see if the settings from the config file are used? What I see now: 
I1015 15:39:26.015322   16120 eventsink.go:62] eventSink::Debug(<{%reset%}>2020/10/15 15:39:26 Testing if Managed Service Identity is applicable for Authentication..<{%reset%}>)
I1015 15:39:26.018086   16120 eventsink.go:59] 2020/10/15 15:39:26 Testing if Obtaining a token from the Azure CLI is applicable for Authentication..
I1015 15:39:26.018520   16120 eventsink.go:62] eventSink::Debug(<{%reset%}>2020/10/15 15:39:26 Testing if Obtaining a token from the Azure CLI is applicable for Authentication..<{%reset%}>)
I1015 15:39:26.019239   16120 eventsink.go:59] 2020/10/15 15:39:26 Using Obtaining a token from the Azure CLI for Authentication
I1015 15:39:26.019576   16120 eventsink.go:62] eventSink::Debug(<{%reset%}>2020/10/15 15:39:26 Using Obtaining a token from the Azure CLI for Authentication<{%reset%}>)
If I read this correctly it says that the CLI credentials are used.
t

tall-librarian-49374

10/15/2020, 1:43 PM
yes
Don’t you see 
Testing if Service Principal
?
w

wet-noon-14291

10/15/2020, 1:45 PM
yeah, I did find that further up: 
I1015 15:39:26.003867   16120 eventsink.go:59] 2020/10/15 15:39:26 Testing if Service Principal / Client Certificate is applicable for Authentication..
I1015 15:39:26.006003   16120 eventsink.go:62] eventSink::Debug(<{%reset%}>2020/10/15 15:39:26 Testing if Service Principal / Client Certificate is applicable for Authentication..<{%reset%}>)
I1015 15:39:26.005451   16120 step_executor.go:321] StepExecutor worker(-2): worker waiting for incoming chains
I1015 15:39:26.005478   16120 step_executor.go:321] StepExecutor worker(1): launching oneshot worker
I1015 15:39:26.007344   16120 step_executor.go:321] StepExecutor worker(1): applying step create on urn:pulumi:Dev::Elkjop.FinancialServices.Infrastructure::pulumi:providers:azure-nextgen::default_0_2_3_alpha_1602700498_177a671e (preview true)
I1015 15:39:26.007858   16120 step_executor.go:321] StepExecutor worker(1): step create on urn:pulumi:Dev::Elkjop.FinancialServices.Infrastructure::pulumi:providers:azure-nextgen::default_0_2_3_alpha_1602700498_177a671e retired
I1015 15:39:26.006941   16120 eventsink.go:59] 2020/10/15 15:39:26 Testing if Multi Tenant Service Principal / Client Secret is applicable for Authentication..
Looks like it falls through to the next method
Is there a way to check what's been read from the Pulumi.Dev.yaml file?
not sure what is going on here, but it seems like my config in 
Pulumi.Dev.yaml
isn't use...
content of the file: 
secretsprovider: azurekeyvault://<secrets>
encryptedkey: <key>
config:
  azure:clientId: <clientid>
  azure:clientSecret:
    secure: <secret>
  azure:location: westeurope
  azure:subscriptionId: <sub>
  azure:tenantId: <tenant>
I wonder if this can be the same thing I experienced earlier: https://github.com/pulumi/pulumi/issues/4840
@clever-sunset-76585, does this sound like the issue I had before you think? Was it possible to somehow check if the config was read properly?
I could probably find my way around this, but would be nice to understand what is going on and why what I have at the moment doesn't work.
t

tall-librarian-49374

10/15/2020, 7:14 PM
Is the stack 
Dev
with the capital D? Wild guess - but could it be something about that?
w

wet-noon-14291

10/15/2020, 7:15 PM
yes, it is capital 
D
I do think it is uses the yaml though, since it does update the keyvaul and that is defined in the yaml... so not sure how the capitalization might impact it
is there a way to verify that the azure config is read and used somehow?
t

tall-librarian-49374

10/15/2020, 7:25 PM
I think you can trust the log
w

wet-noon-14291

10/15/2020, 7:31 PM
but I don't anything in the log about what has been read. I mean, is the file somewhat corrupt or something that is causing it to not use the credentials in the file?
when thinking about it I also think the 
azure:location
config wasn't read either since I had to provide a location in code... but might be mistaking on that one.
another sort of related strange thing. When I try to work my way around this and login with different service principals for different environments I get a "building auth config: Authenticating using the Azure CLI is only supported as a User"... but only on Azure DevOps, authenticating with service principal locally actually worked (or at least as I see it)
ah, so the location thing is a know "issue" for azure-nextgen
does the nextgen sdk use the same azure config variables for the service principal?
t

tall-librarian-49374

10/16/2020, 7:19 AM
are you using nextgen? no, the variables need to start with 
azure-nextgen
then
azure-nextgen:clientId
w

wet-noon-14291

10/16/2020, 8:17 AM
yeah, that was what I was doing... so that probably explains it. I think the documentation needs to be updated around that. I started to think about that since the I saw a comment on it in on issue around location for nextgen. I'll try that and report back.
Did not work as expected, and I really don't understand what I'm doing wrong here. I commented on an existing issue here https://github.com/pulumi/pulumi/issues/4072#issuecomment-710055019 and would appreciate any help in getting it to work. This time it is actually for a real project 🙂
t

tall-librarian-49374

10/16/2020, 1:54 PM
@broad-dog-22463 any ideas about the setup above?
b

broad-dog-22463

10/16/2020, 1:56 PM
@wet-noon-14291 can you recap what you are trying to do for me?
w

wet-noon-14291

10/16/2020, 2:02 PM
@broad-dog-22463 sure (and I will also try to finish the blog post)... It's as simple as I try to do 
pulumi up
on on Azure devops where I have secrets in keyvault and using blob storage. I've read through the code and I don't understand why I don't go into this 
if
https://github.com/pulumi/pulumi/blob/eaac9280f5484f94213dbf54c9fd6f009d87db12/pkg/cmd/pulumi/crypto.go#L56 I definitely have a 
secretsprovider
property in my yaml. (the "official" tasks is not the answer for me because of reasons... but this shouldn't be to complicated to get working)
b

broad-dog-22463

10/16/2020, 2:03 PM
what error are you getting?
w

wet-noon-14291

10/16/2020, 2:04 PM
(the blog post is for the weekend 🙂 )
the one mentioned here: https://github.com/pulumi/pulumi/issues/4072#issuecomment-710055019 (my comment to the issue)
error: getting secrets manager: passphrase must be set with PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE environment variables
b

broad-dog-22463

10/16/2020, 2:09 PM
can you show me your Pulumi.<stack>.yaml file without any secrets in it? (just the keynames will work)
by any chance did you manually create the entries for the secrets manager in the config file?
w

wet-noon-14291

10/16/2020, 2:15 PM
Script use to create the stacks: 
#!/bin/bash -e

if [ $# -eq 0 ]; then
    echo "No arguments provided. Arguments allowed: Dev, Test, Prod"
    exit 1
fi

stacks=("$@")

script_folder=$(dirname $0)
get_creds="$script_folder/get-ops-creds.sh"

for i in "${stacks[@]}"
do
    stack=$(echo "$i" | tr '[:upper:]' '[:lower:]')
    echo "Configuring $stack"
    cmd="$get_creds $i"
    credentials=$(eval $cmd)

    client_id=$(echo $credentials | jq -r .serviceprincipalid)
    tenant_id=$(echo $credentials | jq -r .tenantid)
    client_secret=$(echo $credentials | jq -r .serviceprincipalkey)
    subscription_id=$(echo $credentials | jq -r .subscriptionid)

    pulumi stack init --secrets-provider="<azurekeyvault://elkds-pulumikv.vault.azure.net/keys/pulumisecrets>" $stack

    pulumi config set azure-nextgen:clientId $client_id -s $stack
    pulumi config set azure-nextgen:clientSecret $client_secret --secret -s $stack
    pulumi config set azure-nextgen:tenantId $tenant_id -s $stack
    pulumi config set azure-nextgen:subscriptionId $subscription_id -s $stack
    pulumi config set azure-nextgen:location westeurope -s $stack

    pulumi config set azure:clientId $client_id -s $stack
    pulumi config set azure:clientSecret $client_secret --secret -s $stack
    pulumi config set azure:tenantId $tenant_id -s $stack
    pulumi config set azure:subscriptionId $subscription_id -s $stack
    pulumi config set azure:location westeurope -s $stack

    pulumi plugin install resource azure-nextgen v0.2.3-alpha.1602700498+177a671e
done
The 
get-creds
script reads credentials from a keyvault. `Pulumi.dev.yaml`: 
secretsprovider: <azurekeyvault://keyvaultpath>
encryptedkey: <secret>
config:
  azure-nextgen:clientId: <clientid>
  azure-nextgen:clientSecret:
    secure: <secret>
  azure-nextgen:location: westeurope
  azure-nextgen:subscriptionId: <subid>
  azure-nextgen:tenantId: <tenantid>
  azure:clientId: <clientid>
  azure:clientSecret:
    secure: <secret>
  azure:location: westeurope
  azure:subscriptionId: <subid>
  azure:tenantId: <tenantid>
(It does work when I run it locally)
b

broad-dog-22463

10/16/2020, 2:17 PM
ah this is a case of it just not working in Azure Devops then?
w

wet-noon-14291

10/16/2020, 2:18 PM
Yeah. Not at the computer, had to run. I could try to logout locally and run to make sure it is closer to devops.
Will follow up more tonight since I really want this working by Monday :)
I think it is weird that I go down the passphrase route in the code. I shouldn’t end up there if I read the code correctly. Maybe the Pulumi.dev.yaml is corrupted in git, maybe having a bom thingy in the beginning.
b

broad-dog-22463

10/16/2020, 2:23 PM
yeah it may well be the case... I will be around most of the evening so ping me when you get a chance to look
w

wet-noon-14291

10/16/2020, 2:25 PM
Sure will, thanks.
@broad-dog-22463 I think I figured it out. @tall-librarian-49374 mentioned earlier that it could be the casing of "dev" vs "Dev" as a possible issue, that wasn't the case. But when I tested for it I renamed the local file to lower case, and also updated the scripts. The file in the remote repository was never changed though, so it still had upper case "Dev". That explains why it worked locally where the file had the correct casing, and remote it couldn't find the file with the correct casing and therefor couldn't find the keyvault configation.
b

broad-dog-22463

10/16/2020, 6:39 PM
ok, so there's a case insensitivity there
w

wet-noon-14291

10/16/2020, 6:45 PM
yeah, and I hadn't the configured the git repo to take casing into consideration (
git config core.ignorecase false
)
changed it now.
3 Views
#azureJoin Slack
Powered by