https://pulumi.com logo
Join Slack
Powered by
This message was deleted.
# google-cloud
s
This message was deleted.
p
Where does this 
GetNetwork
come from?
oh, that’s the GoogleNative plugin
a
There's an equivalent in the GCP plugin.
It would fail just the same.
p
in the “old” google plugin, 
get
function has 
opts
where you should be able to set 
dependsOn
a
Ahh, well maybe I can switch to that.
p
I’ve never used 
dependsOn
while getting existing resources but I do use that while creating some of them
and this is especially needed when you first create a service in GCP and then create some resources that require it
(so kinda similar to your case)
a
Yeah, I have to make sure APIs are enabled before trying to get stuff.
p
If the 
get
behavior is no different than the one implemented while creating new resources that should do the trick. If no, I guess it’d be worth reporting this as a bug.
Not sure if GoogleNative plugin has the same set of features. I haven’t touched it yet myself (and to be honest, I have no motivation as the standard one works just great).
a
Looks like 
dependsOn
isn't an option there either.
Just those three.
p
🤔
is that c#?
a
yep
And I guess this is what I was hoping would be possible. (part in red)
p
a
This is classic.
You can see the namespace there. 
Pulumi.Gcp.Compute
. That's the classic provider.
p
I don’t have access to any C# env right now but from the docs, the signature seems to be different. Something like:
Copy code
public static Network Get(string name, Input<string> id, NetworkState? state, CustomResourceOptions? opts = null)
a
p
the last argument, 
CustomResourceOptions
seems to have `DependsOn`: https://www.pulumi.com/docs/reference/pkg/dotnet/Pulumi/Pulumi.CustomResourceOptions.html
a
You're thinking about creating a network.
I'm using the 
GetNetwork
function
So the function used to look up an existing network.
p
Hah, I see. It seems kinda… limited and redundant?
a
Not at all.
p
Can’t you use the one I referenced above? I’m using this approach from python.
a
I don't want to manage this resource. I just want to be able to reference it. But in order to do so, I have to be able to look it up by name.
There's additional non-guessable properties that I need from it.
p
if you read it like that, you won’t manage it
in order to manage it, you’d have to import it to the pulumi state
a
Exactly. Which I don't want to do.
I don't want to manage it.
If I use what you're suggesting, I have to add it to state.
p
nope (unless I gave you a wrong link - let me check)
look up != import
it won’t add it to your state, it’s gonna be read every time you run 
pulumi up/preview
and you’re gonna get full-fledged pulumi resource of the given type
a
Oh interesting.... This is a link to the resource, but this 
Get
is a method on the type?
p
according to the docs it should be a static function
a
Yeah. Hah, amazing. I never even knew this existed, wow.
p
so I guess… try to find type 
Network
within 
Pulumi.Gcp.Compute
and see whether it has a static method called 
Get
a
Yup, it's there 🙂
For some reason I thought you were suggesting to use the resource and I couldn't figure out how that would work 😆
p
although, right now I’m not sure it’s gonna work as expected… let me quickly test sth
a
Yeah, doesn't appear to work still.
It seems to still be trying to resolve despite having a dependency on networking and compute.
p
is that the classic or native plugin?
a
All classic.
Ooh, hang on, I think my IDE grabbed from the native namespace.
Nice catch, gimme a sec.
p
I’ve seen that in one of the screenshots above 😉
a
Error 403: Compute Engine API has not been used in project ########### before or it is disabled.
Still, same error in the end. It doesn't appear to be honouring the 
DependsOn
.
This seems like it's happening during the planning phase. So it goes to read the resource anyway, even though the API is really a hard requirement before anything can be read.
At this rate, I feel like I should just set up a different stack that represents just enabling APIs and call it separately. GCP is a bit unique in this.
Although this is probably enough to be bug-worthy?
p
I think so. I’m trying to reproduce something similar on my end but I guess I’m trying to be too quick 😅
so repro steps for this would be: • create a project with default VPC • make sure Compute API is not enabled • in one stack try to: ◦ enable Compute API ◦ retrieve (already existing) default network
Can you (for tests) manually enable the API (or run the code first without the part responsible for network) and make sure that this import actually works then?
that way we’ll be able to narrow down the case to: “`get` methods of resources does not respect 
dependsOn
attribute” or sth like that
a
Sure.
Just enabling now and then I'll re-run to see if I get past the initial planning phase.
p
exactly
a
Yup, got past it now that the API is enabled.
p
Although I have to admit that Service part of GCP is kinda… demanding. In my projects, I have two separate Pulumi projects called: • gcp-project-bootstrap • gcp-project In the first one I take care about: • creating the project itself (without default VPC) • enabling required services • creating VPC • setting up some top-level IAM things The second one takes care about more specific parts like CloudSQL, GKE etc.
I’m afraid “fixing” 
get
functions might be tricky. They (most probably) are evaluated during plan phase because other changes might depend on it. If you want it to depend on some resource creation, we have chicked-and-egg problem.
a
Yup. I think the short term solution is for me to just set up a stack that represents going and flipping al the switches for the services I know that I'll end up using.
Longer term, I guess the GCP (and native?) providers could examine dependencies for instances of API-services and when that happens, defer any attempts to populate data?
I feel like this really is just a thorn with the way google APIs work and how they have to be enabled.
Most cloud providers have all APIs on at all times.
p
that’s true
anyway, create a ticket for that
a
Absolutely.
Just about to submit it.
p
I’ll try to look into it once I have some free time.
for the sake of completeness, can you elaborate why you actually needed that?
a
I'll add that to the ticket.
p
I personally think it’s good to know the motivation/backstory so the fix can be appropriately implemented or rejected (in case this is something that can be actually resolved in a different way but we didn’t think about it now).
a
For sure. Yeah, it's really just about being able to reference that 
"default"
network that GCP creates for all projects once compute is enabled.
It comes with a lot of useful stuff pre-configured and represents a much more turnkey experience.
p
from my experience, it’s better to create a VPC yourself so you have full control over it
the same goes for example with using default svc account for GCE nodes (from my point of view, default permissions are way to high)
a
Is there anywhere that defines the differences?
I feel like there were a lot of subnets defined in the default network which I just didn't want to have to go defining myself.
I'm okay with getting onboard with conventional wisdom there for sure. But I'd love to understand the differences at that point 😉
p
I don’t think it will be easy to list all the differences. I just didn’t like the fact that GCP created the VPC all by itself without me knowing anything about it. The other problem (that you actually just described) is getting reference to the default network (that’s not managed by you).
regarding “subnetworks problem”, if you take a look at VPC resource in pulumi, you’ll see an option `auto_create_subnetworks`:
Copy code
def __init__(__self__,
                 resource_name: str,
                 opts: Optional[pulumi.ResourceOptions] = None,
                 auto_create_subnetworks: Optional[pulumi.Input[bool]] = None,
                 delete_default_routes_on_create: Optional[pulumi.Input[bool]] = None,
                 description: Optional[pulumi.Input[str]] = None,
                 mtu: Optional[pulumi.Input[int]] = None,
                 name: Optional[pulumi.Input[str]] = None,
                 project: Optional[pulumi.Input[str]] = None,
                 routing_mode: Optional[pulumi.Input[str]] = None,
                 __props__=None):
so one option to overcome your problems would be to create the project from the scratch using pulumi
• you create a project with 
auto_create_network=false
• you create a vpc with most of the default values (so you don’t need to create subnetwork manually if you don’t want to) • you have a managed vpc network (so no problems with getting it)
I know it’s hell of a difference. I’m just giving you an alternative idea how to tackle it 🙂. If you can recreate the project/network, I’d go that way.
a
Oh this is greenfield 🙂
We're moving from Azure to GCP, so I wanna get it right.
p
are you going to use private subnets?
like private GKE cluster (node do not have public IP) and CloudSQL with only private network access?
a
Yeah, I have one service in Cloud Run that will need VPC access because it can't do sockets for Cloud SQL 😞
(third party self-host service, looking to kick it out of our architecture in the future...)
I've already prototyped out a terraform configuration that works though, so I make a simple subnet for that.
p
so just in case to save you some time, let me copy’n’paste one part of my component (it’s in python but you should simply translate it to C#):
Copy code
#
        # Configure NAT router
        #
        self.router = gcp.compute.Router(
            name,
            network=self.network.name,
            project=project,
            region=region,
            opts=ResourceOptions(parent=self.network),
        )
        self.nat = gcp.compute.RouterNat(
            name,
            router=self.router.name,
            nat_ip_allocate_option="AUTO_ONLY",
            source_subnetwork_ip_ranges_to_nat="ALL_SUBNETWORKS_ALL_IP_RANGES",
            log_config=gcp.compute.RouterNatLogConfigArgs(
                enable=True, filter="ERRORS_ONLY"
            ),
            project=project,
            region=region,
            opts=ResourceOptions(parent=self.network),
        )

        #
        # Create private VPC peering connection
        #
        # Some Google managed services (such as Cloud SQL) require private services access for each VPC network
        # that is used for private IP connections. This is a one-time procedure but it must be performed before
        # attempting to create dependent services.
        #
        # See: <https://cloud.google.com/sql/docs/mysql/configure-private-services-access>
        #
        ip_address_range = gcp.compute.GlobalAddress(
            self._name,
            purpose="VPC_PEERING",
            address_type="INTERNAL",
            prefix_length=16,
            network=self.network.name,
            project=project,
            opts=ResourceOptions(parent=self.network),
        )
        self.private_connection = gcp.servicenetworking.Connection(
            self._name,
            network=self.network.id,
            service="<http://servicenetworking.googleapis.com|servicenetworking.googleapis.com>",
            reserved_peering_ranges=[ip_address_range.name],
            opts=ResourceOptions(parent=self.network),
        )
a
Ah yeah, I have this all already in 
.tf
which I'm going to be transposing over.
I think every GCP deployment template project ends up with this boilerplate 😆
p
can’t post the whole Vpc component resource but just the heads-up: • private GKE cluster => you most probably will need RouterNat • private CloudSQL access => you’re gonna need VPC peering connection (people have problems with that because if you configure that using Console Web UI, most of these things are created under the hood without your knowledge)
a
Yeah, I went through the whole song and dance in terraform already. That was a week of digging 😞
p
yeah, that’s why I left an explicit comment in my repo so others can understand why it’s necessary there
a
It looks super arbitrary, yeah.
But it's totally not.
I think the one challenge waiting for me is a static outbound IP for a third party we're using. But I'll slog through that once I've got things moved over to Pulumi.
I was due to restructure my 
.tf
templates to be less monolith-y, so moving to Pulumi seemed like a smart move.
p
nowadays it’s not that bad (static outbound IP)
a
Oh, nice, well that's reassuring 🙂
p
although it might depends on… what’s gonna use this IP 😄
a
Cloud Run 😅
p
that might be… fun 😄
a
Even if it is, I actually find using a deployment templating system the best way to wrangle it.
p
anyway, I remember creating a GCE instance with some proxy in order to achieve a static outbound IP
right now, you have access to NAT Gateway as AFAIK this is what you’re looking for
you might be saved 😉
a
🙏🏻
p
my suggestion though, check these things first - you might e.g. need to manually create the subnetwork or something
in worst case, some resources in GCP have immutable attributes…
I think you can’t migrate non-private GKE cluster to a private one.
a
Oh yeah, I know... There's lots of needles to thread.
There's also a bunch of IAM stuff I have to do for cloud run. Tricky things that the webui totally hides like you mentioned.
And the loadbalancers... Oh man. That was a huge slog too. Very similar, just piecing things together.
p
Anyway, if you have a working TF example, you’re good to go. Most of the TF code can be easily translated to pulumi.
a
It totally is! 😄
I'm loving it so much, 'cause I'm basically just sprinkling C# on it all. It's a dream. And so easy to work with now with the strong typing in play.
p
my advice - if you want to do something specific using IaC on GCP, search for terraform recipes and translate them to pulumi 😄
a
✔️
p
it’s still quite hard to find some great resources referencing pulumi
(if you want to do more advanced stuff that is)
Anyway, good luck my friend and have a nice day!
a
Thank you! You have a great Friday too 🙂
5 Views
Open in Slack
PreviousNext
Powered by