https://pulumi.com logo
Join Slack
Powered by
Hello, I'm trying to set up a gcp project from sc...
# python
b
Hello, I'm trying to set up a gcp project from scratch, including enabling some APIs and I'm trying to set up some sort of dependency control to get it to work:
Copy code
"""A Google Cloud Python Pulumi program"""

import pulumi
import pulumi_gcp as gcp

cloud_resource_manager_api = gcp.projects.Service('crm_api', service="<http://cloudresourcemanager.googleapis.com|cloudresourcemanager.googleapis.com>")
project = gcp.organizations.get_project_output(opts=pulumi.InvokeOptions(parent=cloud_resource_manager_api))
cloud_run_api = gcp.projects.Service('cloud_run_api', disable_dependent_services=True, disable_on_destroy=True, project=project.id.apply(lambda project_id: project_id), service="<http://run.googleapis.com|run.googleapis.com>")
that doesn't work because the project depends on the crm_api to be enabled, I tried to assign crm_api as a parent to the get_project_output call as a hail mary but of course that doesn't work
I hit https://github.com/pulumi/pulumi-gcp/issues/685 and https://github.com/pulumi/pulumi-gcp/issues/675 and there's a reference to polling or sleep, not much of a clue on how to do that
I've made it work with this:
Copy code
"""A Google Cloud Python Pulumi program"""

import pulumi
import pulumi_gcp as gcp

cloud_resource_manager_api = gcp.projects.Service('crm_api', service="<http://cloudresourcemanager.googleapis.com|cloudresourcemanager.googleapis.com>")

def set_up():
    project = gcp.organizations.get_project_output(opts=pulumi.InvokeOptions(parent=cloud_resource_manager_api))
    cloud_run_api = gcp.projects.Service('cloud_run_api', disable_dependent_services=True, disable_on_destroy=True, project=project.id.apply(lambda project_id: project_id), service="<http://run.googleapis.com|run.googleapis.com>")

cloud_resource_manager_api.id.apply(lambda x: set_up())
but it feels dirty
also the summary of 
pulumi up
and 
pulumi destroy
would be pretty much useless
Copy code
"""A Google Cloud Python Pulumi program"""

import pulumi
import pulumi_gcp as gcp

cloud_resource_manager_api = gcp.projects.Service('crm_api', service="<http://cloudresourcemanager.googleapis.com|cloudresourcemanager.googleapis.com>")

def get_running_project():
    project = gcp.organizations.get_project()
    return project

project = cloud_resource_manager_api.id.apply(lambda _: get_running_project())
cloud_run_api = gcp.projects.Service('cloud_run_api', disable_dependent_services=True, disable_on_destroy=True, project=project.id, service="<http://run.googleapis.com|run.googleapis.com>")
solved! thanks! 😛
a comment if that's the pulumi way would be nice tho
p
you should be able to simply use 
depends_on
that’s what I do in my gcp-project-bootstrap project
I’m trying to understand the snippets you pasted and I have no idea what kind of problem you’re trying to solve there 🤔
b
depends_on doesn't work on invokes
so I can't use it in 
gcp.organizations.get_project()
p
mhm, that’s right - but why did you need invoke here?
What does 
gcp.organizations.get_project()
actually do? Get the currently active GCP project?
b
I want to have the project id and project number
yes
p
if there is an active GCP project, you should be able to create a 
Service
instance without specifying 
project
I think
b
yes but for other things I built after, I do need the project id and number
p
I assume you don’t want to include 
project_id
in stack values and discover it dynamically based on the env config?
b
that would work, but it would be "cheating" hahaha
what I really need is depends_on in invokes
p
I don’t have this problem cause I’m creating the project itself using pulumi.
b
that's what I want to achieve eventually
is your bootstrap public?
p
unfortunately it’s not but I guess I could make it a little bit more generic and share it on my Github
I’ve already copy’n’pasted to many snippets from it for other people here 😄
b
I assume you use your own credentials to set up stuff, and in the bootstrap create a service account for pulumi and take over from there?
p
exactly my friend
b
I'm missing that switch step and how to do that
p
I have a separate project called in my case 
gcp-project-bootstrap
that’s supposed to be run by organization administrator
b
exactly what I wanted to do later
p
it creates the project, enables APIs (cause it’s a pain in the ass to create them later on and add explicitly 
depends_on
everywhere…), create some high-level IAM stuff (including some service accounts for managing it) and VPC (cause I can 😛)
then I have 
gcp-project
project that setups the rest of the infrastructure like GKE, CloudSQL etc.
b
my idea was to copy that or learn how to use the terraform bridge
p
I think I even had my inspiration from it
b
but I'm new into pulumi, I have some hours of experience hahaha so step by step
p
if you use pulumi classic GCP provider, you can easily transform any terraform code into pulumi (even manually)
terraform resources can give you an idea what kind of resources you should create
b
yes, that's how I came up with how the APIs are enabled
p
yeah, they are called Service so you cannot find them 😄
b
hahaha
p
and you have to enable them before trying to create anything meaningful in the project
I’m actually a big fan of GCP but the Service/API thing is just… meh
Copy code
project = gcp.organizations.Project(
    "project",
    name=config.project_name,
    project_id=config.project_id,
    billing_account=config.billing_account,
    auto_create_network=False,
    folder_id=config.folder_id,
)

#
# Enable Google APIs
#
apis = set(config.activate_apis)

# Ensure required APIs for VPC are enabled
apis.add("<http://compute.googleapis.com|compute.googleapis.com>")
apis.add("<http://servicenetworking.googleapis.com|servicenetworking.googleapis.com>")

enabled_apis = [enable_service_api(api, project) for api in apis]

#
# Create default svc account for compute nodes with access to shared GCR
#
default_service_account = gcp.serviceaccount.Account(
    "default",
    account_id="default",
    display_name="Default service account for compute nodes",
    project=project.project_id,
)

if config.gcr_project_id:
    gcp.projects.IAMMember(
        "sa-default-gcr-read-access",
        role="roles/storage.objectViewer",
        member=pulumi.Output.concat("serviceAccount:", default_service_account.email),
        project=config.gcr_project_id,
        opts=pulumi.ResourceOptions(parent=default_service_account),
    )

default_sa_roles = set(config.default_sa_extra_roles)

# Ensure svc account has required roles
for role in {
    "roles/logging.logWriter",
    "roles/monitoring.metricWriter",
    "roles/monitoring.viewer",
}:
    default_sa_roles.add(role)

for role in default_sa_roles:
    gcp.projects.IAMMember(
        f"sa-default-{simple_role_name(role)}",
        member=pulumi.Output.concat("serviceAccount:", default_service_account.email),
        project=project.project_id,
        role=role,
        opts=pulumi.ResourceOptions(parent=default_service_account),
    )
that’s the main part of what I wrote
I wish I could give you a ready-to-go project from Github but I’m being realistic and I know it might take me some time to publish it.
b
nah no prob, thanks
p
The code above: • creates a project (without default network! so create the VPC yourself or remove 
auto_create_network=False
), • create a default svc account for nodes (so you have better control over what nodes can access) and allow it to access shared GCR project (that’s a good strategy if you’re gonna have multiple projects for prod/staging/qa environment) • activates some APIs (as you can see, it ensures that 2 hardcoded ones are enabled no matter what)
not that APIs to enable are read into python 
set
so it ensures there are no duplicated (when I manually add 
compute
and 
servicenetworking
)
have fun! 🙂
b
thanks!
and then how do you link the outputs from the bootstrap into the 
gcp-project
?
p
look for StackReference in pulumi docs
b
ah gotcha
p
you can basically reference outputs from a different stack - it’s a built-in Pulumi feature 🙂
b
read the whole docs, but as long as you don't use the stuff, it won't get in the brain
p
that’s right
check the repo with examples
if I remember well, there’s an example of how to use them
(it’s dead simple but if that’s the missing part for you, it will do the job)
b
yeah well, right now all parts are missing parts 😛
hey @prehistoric-activity-61023 I split the bootstrap and project into two separate pulumi projects and stacks
but I'm using KMS as secret provider, and I'd like that the project that uses the bootstrap as reference doesn't have access to the secrets exported in the bootstrap
for example, I create the pulumi service account key there
but it seems pulumi expects to use the same secret provider when you use references as per https://github.com/pulumi/pulumi/issues/4665 https://github.com/pulumi/pulumi/issues/5151
is that something you encountered/solved/don't care about?
Open in Slack
PreviousNext
Powered by