https://pulumi.com logo
Powered by
Title
d

damp-elephant-82829

08/31/2020, 10:04 AM
That’s really akward, @green-school-95910 do you have any hint on this? both my cloud build and my project default service acount have the role pubsub editor
g

green-school-95910

08/31/2020, 1:34 PM
I assume that error is in your application, not on pulumi right?
The deployed application needs the permission
d

damp-elephant-82829

08/31/2020, 2:16 PM
no no it’s in pulumi. Let’s summarize what I am doing: I am running a cloud build pipeline that will build some docker images and deploy an application with pulumi. I am using a pattern called the seed project pattern, where I create a new project per branch of my git repository, and I create all the resources there. It all works fine: cloud run, topics, storage buckets. The only thing I can’t create is the subscription. I assume it might have to do that I am trying to create a subscription on projectA-topicA but my pulumi up is getting runned by projectRoot-cloudBuildServiceAccount
g

green-school-95910

08/31/2020, 2:18 PM
And you are creating the project with the same service account?
d

damp-elephant-82829

08/31/2020, 2:19 PM
location = "europe-west1"

project_name = f"myproj-dev-{branch_name}"

root_project = organizations.Project.get(
    "root-project", id="myproj-qaroot"
)

organization = organizations.get_organization(organization=organization_name)


# Create an ephemeral project
project = organizations.Project(
    "branch-project",
    name=project_name,
    project_id=project_name,
    billing_account=root_project.billing_account,
    org_id=organization.org_id,
)
so it seems I need to grant to the cloud build service account either the right to crete subscription cross projects, or something like that. I tried to achieve it in two ways
Adding a topic IAM meber like so, didn’t work 
root_project_topic_iam_binding = pubsub.TopicIAMMember(
    resource_name="root-project-as-pubsub-admin",
    topic=my_topic.id,
    project=project.project_id,
    role="roles/pubsub.admin",
    member=[cloudbuild_service_accountname(root_project.number)],
)
Adding an IAM Member to root_project as roles/editor on project like so 
root_project_iam_binding = projects.IAMMember(
    resource_name="root-project-as-project-editor",
    project=project.project_id,
    role="roles/editor",
    member=cloudbuild_service_accountname(root_project.number),
)
that didn’t work either
g

green-school-95910

08/31/2020, 2:34 PM
Since apparently you are using the same 
gcp.Provider
for creating the project and the subscription (the default one) this should, in theory, work.
So... Next possible problem, do both projects have the PubSub service enabled?
d

damp-elephant-82829

08/31/2020, 2:38 PM
it might be that on the root project the pubservice is not enabled
it is certainly on the dev projct since I created a topic correctly
is there a way to get a more verbose error from google api besides a simple 403?
g

green-school-95910

08/31/2020, 2:40 PM
No, if there were you would be able to probe projects that you do not have access and get information from them
d

damp-elephant-82829

08/31/2020, 2:46 PM
ok, pubsubservice is enabled in both
other ideas?
g

green-school-95910

08/31/2020, 2:56 PM
Since you are creating the project with Pulumi, the sa of the provider should automatically be an owner of the project
Maybe the scope of cloud build is not 
cloud-platform
but a combination of smaller scopes, but I'm not finding any documentation for it
You can try something. In Pulumi you can create a service account on the new project, set it as owner or editor, create a 
gcp.Provider
using this new account and try to create a subscription with it.
Which is something I do by default, the base service account only has permission to create projects and manage iam in them.
But I never created a Subscription in Pulumi, so I'm not certain if there is something different happening here
d

damp-elephant-82829

08/31/2020, 3:00 PM
thank you, how do I change the rpovider for the following statements?
g

green-school-95910

08/31/2020, 3:02 PM
new pubsub.Subscription("name", {...}, {provider: theProviderInstance})
Oh wait, you are using Python
Let me check
pulumi.Subscription(
    ...
    opts=pulumi.ResourceOptions(
        provider=theProviderInstance,
    ),
)
https://www.pulumi.com/docs/reference/pkg/python/pulumi/#pulumi.ResourceOptions
d

damp-elephant-82829

08/31/2020, 3:05 PM
do you have by chance also a link to how to create a provider in python?
thanks
g

green-school-95910

08/31/2020, 3:08 PM
Sure: https://www.pulumi.com/docs/reference/pkg/gcp/provider/#create 
pulumi_gcp.Provider(
    credentials=serviceAccountKey,
)
If I remember correctly to get the service account key you need to: • Create the SA: https://www.pulumi.com/docs/reference/pkg/gcp/serviceaccount/account/ • Create a Key for that SA: https://www.pulumi.com/docs/reference/pkg/gcp/serviceaccount/key/ • 
serviceAccountKey = keyResource.private_key.apply(base64.b64decode)
d

damp-elephant-82829

08/31/2020, 3:13 PM
ok let’s recap: I create a SA, I create a SA key, I create a new provider, I use that provider for everything I do on the project after I added the IAMMember roles/owner to the SA
let’s try
g

green-school-95910

08/31/2020, 3:14 PM
Yep
If you wanna be fancy you can create a Component that do all that and set the provider on itself. Then you can add that componentInstance as a parent to all your other resources
I, personally, exclusively my opinion and definitely not at all a direct recommendation, think that organizing everything in components allows for insanely better readability and maintainability because then you can model your infrastructure in a file that has only you resource abstractions
d

damp-elephant-82829

08/31/2020, 3:17 PM
let me try to get it working first, then I am happy to look into components for refactoring
Can you point me to some examples on how to organize in components @green-school-95910?
g

green-school-95910

08/31/2020, 4:39 PM
Did it work?
d

damp-elephant-82829

08/31/2020, 4:47 PM
no, I wasn’t able to create a serviceAccountKey
the creation of the key fails like so
+ gcp😒erviceAccount:Key project_owner_service_account_key creating + gcp😒erviceAccount:Key project_owner_service_account_key creating error: project: required field is not set + gcp😒erviceAccount:Key project_owner_service_account_key creating failed error: project: required field is not set @ Updating....
but the serviceAccount.Key has no project input
project_owner_service_account = serviceAccount.Account(
    resource_name="ephemeral_project_owner_service_account",
    account_id="ephemeralprojectowner",
    project=ephemeral_project.project_id,
)

project_owner_service_account_key = serviceAccount.Key(
    resource_name="project_owner_service_account_key",
    service_account_id=project_owner_service_account.unique_id
)
How do you create a Key?
g

green-school-95910

08/31/2020, 4:57 PM
You need to either set 
project
on 
serviceAccount.Key
or add a parent that has the project set
I use TypeScript, here is what I use for projects, it initializes essentially everything that requires project-wide definitions.
projectDefinition.ts
d

damp-elephant-82829

08/31/2020, 5:04 PM
I just checked on the python code in github
there is nothing like a project attribute
g

green-school-95910

08/31/2020, 5:07 PM
Ooh. You are using 
service_account_id=project_owner_service_account.unique_id
it should be 
service_account_id=project_owner_service_account.name
unique_id
is the id for Pulumi
d

damp-elephant-82829

08/31/2020, 5:32 PM
Ok I was able to create the account key, now let me try to create a provider
#google-cloudJoin Slack
Powered by