https://pulumi.com logo
Powered by
#python
Title
# python
b

breezy-book-15761

06/02/2022, 9:43 AM
Hello, I'm trying to set up a gcp project from scratch, including enabling some APIs and I'm trying to set up some sort of dependency control to get it to work:
Copy code
"""A Google Cloud Python Pulumi program"""

import pulumi
import pulumi_gcp as gcp

cloud_resource_manager_api = gcp.projects.Service('crm_api', service="<http://cloudresourcemanager.googleapis.com|cloudresourcemanager.googleapis.com>")
project = gcp.organizations.get_project_output(opts=pulumi.InvokeOptions(parent=cloud_resource_manager_api))
cloud_run_api = gcp.projects.Service('cloud_run_api', disable_dependent_services=True, disable_on_destroy=True, project=project.id.apply(lambda project_id: project_id), service="<http://run.googleapis.com|run.googleapis.com>")
that doesn't work because the project depends on the crm_api to be enabled, I tried to assign crm_api as a parent to the get_project_output call as a hail mary but of course that doesn't work
I hit https://github.com/pulumi/pulumi-gcp/issues/685 and https://github.com/pulumi/pulumi-gcp/issues/675 and there's a reference to polling or sleep, not much of a clue on how to do that
I've made it work with this:
Copy code
"""A Google Cloud Python Pulumi program"""

import pulumi
import pulumi_gcp as gcp

cloud_resource_manager_api = gcp.projects.Service('crm_api', service="<http://cloudresourcemanager.googleapis.com|cloudresourcemanager.googleapis.com>")

def set_up():
    project = gcp.organizations.get_project_output(opts=pulumi.InvokeOptions(parent=cloud_resource_manager_api))
    cloud_run_api = gcp.projects.Service('cloud_run_api', disable_dependent_services=True, disable_on_destroy=True, project=project.id.apply(lambda project_id: project_id), service="<http://run.googleapis.com|run.googleapis.com>")

cloud_resource_manager_api.id.apply(lambda x: set_up())
but it feels dirty
also the summary of 
pulumi up
and 
pulumi destroy
would be pretty much useless
Copy code
"""A Google Cloud Python Pulumi program"""

import pulumi
import pulumi_gcp as gcp

cloud_resource_manager_api = gcp.projects.Service('crm_api', service="<http://cloudresourcemanager.googleapis.com|cloudresourcemanager.googleapis.com>")

def get_running_project():
    project = gcp.organizations.get_project()
    return project

project = cloud_resource_manager_api.id.apply(lambda _: get_running_project())
cloud_run_api = gcp.projects.Service('cloud_run_api', disable_dependent_services=True, disable_on_destroy=True, project=project.id, service="<http://run.googleapis.com|run.googleapis.com>")
solved! thanks! 😛
a comment if that's the pulumi way would be nice tho
p

prehistoric-activity-61023

06/02/2022, 12:59 PM
you should be able to simply use 
depends_on
that’s what I do in my gcp-project-bootstrap project
I’m trying to understand the snippets you pasted and I have no idea what kind of problem you’re trying to solve there 🤔
b

breezy-book-15761

06/02/2022, 1:33 PM
depends_on doesn't work on invokes
so I can't use it in 
gcp.organizations.get_project()
p

prehistoric-activity-61023

06/02/2022, 1:33 PM
mhm, that’s right - but why did you need invoke here?
What does 
gcp.organizations.get_project()
actually do? Get the currently active GCP project?
b

breezy-book-15761

06/02/2022, 1:34 PM
I want to have the project id and project number
yes
p

prehistoric-activity-61023

06/02/2022, 1:34 PM
if there is an active GCP project, you should be able to create a 
Service
instance without specifying 
project
I think
b

breezy-book-15761

06/02/2022, 1:35 PM
yes but for other things I built after, I do need the project id and number
p

prehistoric-activity-61023

06/02/2022, 1:35 PM
I assume you don’t want to include 
project_id
in stack values and discover it dynamically based on the env config?
b

breezy-book-15761

06/02/2022, 1:35 PM
that would work, but it would be "cheating" hahaha
what I really need is depends_on in invokes
p

prehistoric-activity-61023

06/02/2022, 1:36 PM
I don’t have this problem cause I’m creating the project itself using pulumi.
b

breezy-book-15761

06/02/2022, 1:36 PM
that's what I want to achieve eventually
is your bootstrap public?
p

prehistoric-activity-61023

06/02/2022, 1:37 PM
unfortunately it’s not but I guess I could make it a little bit more generic and share it on my Github
I’ve already copy’n’pasted to many snippets from it for other people here 😄
b

breezy-book-15761

06/02/2022, 1:37 PM
I assume you use your own credentials to set up stuff, and in the bootstrap create a service account for pulumi and take over from there?
p

prehistoric-activity-61023

06/02/2022, 1:38 PM
exactly my friend
b

breezy-book-15761

06/02/2022, 1:38 PM
I'm missing that switch step and how to do that
p

prehistoric-activity-61023

06/02/2022, 1:38 PM
I have a separate project called in my case 
gcp-project-bootstrap
that’s supposed to be run by organization administrator
b

breezy-book-15761

06/02/2022, 1:38 PM
exactly what I wanted to do later
p

prehistoric-activity-61023

06/02/2022, 1:39 PM
it creates the project, enables APIs (cause it’s a pain in the ass to create them later on and add explicitly 
depends_on
everywhere…), create some high-level IAM stuff (including some service accounts for managing it) and VPC (cause I can 😛)
then I have 
gcp-project
project that setups the rest of the infrastructure like GKE, CloudSQL etc.
b

breezy-book-15761

06/02/2022, 1:40 PM
my idea was to copy that or learn how to use the terraform bridge
p

prehistoric-activity-61023

06/02/2022, 1:40 PM
I think I even had my inspiration from it
b

breezy-book-15761

06/02/2022, 1:41 PM
but I'm new into pulumi, I have some hours of experience hahaha so step by step
p

prehistoric-activity-61023

06/02/2022, 1:41 PM
if you use pulumi classic GCP provider, you can easily transform any terraform code into pulumi (even manually)
terraform resources can give you an idea what kind of resources you should create
b

breezy-book-15761

06/02/2022, 1:41 PM
yes, that's how I came up with how the APIs are enabled
p

prehistoric-activity-61023

06/02/2022, 1:41 PM
yeah, they are called Service so you cannot find them 😄
b

breezy-book-15761

06/02/2022, 1:42 PM
hahaha
p

prehistoric-activity-61023

06/02/2022, 1:42 PM
and you have to enable them before trying to create anything meaningful in the project
I’m actually a big fan of GCP but the Service/API thing is just… meh
Copy code
project = gcp.organizations.Project(
    "project",
    name=config.project_name,
    project_id=config.project_id,
    billing_account=config.billing_account,
    auto_create_network=False,
    folder_id=config.folder_id,
)

#
# Enable Google APIs
#
apis = set(config.activate_apis)

# Ensure required APIs for VPC are enabled
apis.add("<http://compute.googleapis.com|compute.googleapis.com>")
apis.add("<http://servicenetworking.googleapis.com|servicenetworking.googleapis.com>")

enabled_apis = [enable_service_api(api, project) for api in apis]

#
# Create default svc account for compute nodes with access to shared GCR
#
default_service_account = gcp.serviceaccount.Account(
    "default",
    account_id="default",
    display_name="Default service account for compute nodes",
    project=project.project_id,
)

if config.gcr_project_id:
    gcp.projects.IAMMember(
        "sa-default-gcr-read-access",
        role="roles/storage.objectViewer",
        member=pulumi.Output.concat("serviceAccount:", default_service_account.email),
        project=config.gcr_project_id,
        opts=pulumi.ResourceOptions(parent=default_service_account),
    )

default_sa_roles = set(config.default_sa_extra_roles)

# Ensure svc account has required roles
for role in {
    "roles/logging.logWriter",
    "roles/monitoring.metricWriter",
    "roles/monitoring.viewer",
}:
    default_sa_roles.add(role)

for role in default_sa_roles:
    gcp.projects.IAMMember(
        f"sa-default-{simple_role_name(role)}",
        member=pulumi.Output.concat("serviceAccount:", default_service_account.email),
        project=project.project_id,
        role=role,
        opts=pulumi.ResourceOptions(parent=default_service_account),
    )
that’s the main part of what I wrote
I wish I could give you a ready-to-go project from Github but I’m being realistic and I know it might take me some time to publish it.
b

breezy-book-15761

06/02/2022, 1:45 PM
nah no prob, thanks
p

prehistoric-activity-61023

06/02/2022, 1:46 PM
The code above: • creates a project (without default network! so create the VPC yourself or remove 
auto_create_network=False
), • create a default svc account for nodes (so you have better control over what nodes can access) and allow it to access shared GCR project (that’s a good strategy if you’re gonna have multiple projects for prod/staging/qa environment) • activates some APIs (as you can see, it ensures that 2 hardcoded ones are enabled no matter what)
not that APIs to enable are read into python 
set
so it ensures there are no duplicated (when I manually add 
compute
and 
servicenetworking
)
have fun! 🙂
b

breezy-book-15761

06/02/2022, 1:47 PM
thanks!
and then how do you link the outputs from the bootstrap into the 
gcp-project
?
p

prehistoric-activity-61023

06/02/2022, 1:53 PM
look for StackReference in pulumi docs
b

breezy-book-15761

06/02/2022, 1:53 PM
ah gotcha
p

prehistoric-activity-61023

06/02/2022, 1:53 PM
you can basically reference outputs from a different stack - it’s a built-in Pulumi feature 🙂
b

breezy-book-15761

06/02/2022, 1:53 PM
read the whole docs, but as long as you don't use the stuff, it won't get in the brain
p

prehistoric-activity-61023

06/02/2022, 1:53 PM
that’s right
check the repo with examples
if I remember well, there’s an example of how to use them
(it’s dead simple but if that’s the missing part for you, it will do the job)
b

breezy-book-15761

06/02/2022, 1:55 PM
yeah well, right now all parts are missing parts 😛
hey @prehistoric-activity-61023 I split the bootstrap and project into two separate pulumi projects and stacks
but I'm using KMS as secret provider, and I'd like that the project that uses the bootstrap as reference doesn't have access to the secrets exported in the bootstrap
for example, I create the pulumi service account key there
but it seems pulumi expects to use the same secret provider when you use references as per https://github.com/pulumi/pulumi/issues/4665 https://github.com/pulumi/pulumi/issues/5151
is that something you encountered/solved/don't care about?
Powered by