https://pulumi.com logo
Powered by
#python
Title
# python
d

dazzling-angle-45051

03/23/2022, 8:36 PM
too bad Pulumi doesn't support context managers, something like this would have been awesome:
Copy code
with kubernetes.core.v1.Namespace(project):
    acme_corp.MyResource(f"{project}-{stack}-myresource")
p

prehistoric-activity-61023

03/23/2022, 8:43 PM
what I usually do in my projects is creating an implicit dependency by using Namespace object in other resources (such as Deployments, Services etc.)
Copy code
ns = kubernetes.core.v1.Namespace(..)
deployment = kubernetes.core.v1.Secret(
  ...
  metadata=kubernetes.meta.v1.ObjectMetaArgs(
    namespace=namespace.metadata.name,
    ...
  ),
)
depends_on
is a last resort option for me when the dependency cannot be detected automatically
if you want to create the namespace outside of the component resource, it might get a little bit harder
but I’ll try to declare the namespace parameter in component resource args as 
Input[str]
and pass the resource name similar to the example above
thanks to that you should achieve a very flexible solution cause the ComponentResource can be initialized with literal string value (and it won’t create any dependency implicitly so the user must ensure that the namespace exists at that point) OR with a value from pulumi.Output (and in this case, it should create a dependency graph implicitly)
d

dazzling-angle-45051

03/23/2022, 9:05 PM
I am trying to create the namespace outside of the component resource, yes
p

prehistoric-activity-61023

03/23/2022, 9:08 PM
still - can you pass the namespace as an argument to your ComponentResource?
Copy code
ns = kubernetes.core.v1.Namespace(..)

acme_corp.MyResource(
  f"{project}-{stack}-myresource",
  namespace=ns.metadata.name,
)
d

dazzling-angle-45051

03/23/2022, 9:10 PM
yes, that's what I'm trying for like two hours, but for some reason it doesn't seem to reorder the creation of resources accordingly
p

prehistoric-activity-61023

03/23/2022, 9:12 PM
hmm, that should work
can you paste more code here?
d

dazzling-angle-45051

03/23/2022, 9:12 PM
right now I'm even passing 
namespace
as a 
depends_on
to the resources that I'm creating within 
MyResource
, but still, it doesn't assign the same namespace name
p

prehistoric-activity-61023

03/23/2022, 9:13 PM
I’d need to see: 1) How you create the namespace 2) How you pass the namespace to the ComponentResource 3) Code of the ComponentResource itself or at least a 
__init__
fragment and the part where you create some another k8s resource
d

dazzling-angle-45051

03/23/2022, 9:15 PM
in `__main__.py`:
Copy code
import pulumi
import pulumi_kubernetes as kubernetes

import acme_corp

project = pulumi.get_project()
stack = pulumi.get_stack()

namespace = kubernetes.core.v1.Namespace(project)
acme_corp.MyResource(
    f"{project}-{stack}-resource", namespace=namespace
)
in `acme_corp`:
Copy code
import pulumi
import pulumi_aws as aws
import pulumi_kubernetes as kubernetes


class MyResource(pulumi.ComponentResource):
    def __init__(self, name: str, namespace, opts=None):
        super().__init__("acme_corp:MyResource", name, None, opts)

        aws.s3.Bucket(name, acl="private", opts=pulumi.ResourceOptions(parent=self))

        resource = kubernetes.yaml.ConfigFile(
            f"{name}-resource",
            "./yaml/resource.yaml",
            opts=pulumi.ResourceOptions(parent=self, depends_on=[namespace]),
        )
p

prehistoric-activity-61023

03/23/2022, 9:16 PM
I see
a couple of things
1) Just in case, you know that most probably you didn’t create namespace called the same as your project but sth like 
{project}-{random_suffix}
? If you want to fully control the name of the namespace, you have to set the 
metadata
field (see my example above).
d

dazzling-angle-45051

03/23/2022, 9:17 PM
yup
that's the point actually, my namespace is created as 
project-123
while the resource tries to be deployed in 
project-456
(that doesn't exist)
p

prehistoric-activity-61023

03/23/2022, 9:18 PM
2) You do pass the namespace object as 
depends_on
so it should actually wait until the namespace is created. However, you don’t set the metadata on the resource itself.
I see that you don’t create the resource within pulumi code but rather read it from file. Am I right?
d

dazzling-angle-45051

03/23/2022, 9:19 PM
right
p

prehistoric-activity-61023

03/23/2022, 9:19 PM
If so, I’m afraid you’ll have to use… (arr, I’m missing the name for it, wait)
transformations
can you show the content of 
./yaml/resource.yaml
?
d

dazzling-angle-45051

03/23/2022, 9:21 PM
not much to see really
p

prehistoric-activity-61023

03/23/2022, 9:21 PM
I wonder if there’s a 
metadata
field already set or not
d

dazzling-angle-45051

03/23/2022, 9:22 PM
Copy code
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: whatever
spec:
  schedule: "0 */1 * * *"
  concurrencyPolicy: Forbid
  suspend: false
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1

  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure

          containers:
          - name: whatever
            image: whatever
p

prehistoric-activity-61023

03/23/2022, 9:22 PM
ok, it’s not
so, how I see it
you create the namespace, pass it as a dependency (so the CronJob actually waits until the ns is created) but you don’t pass the namespace name at all to this CronJob
so it’s gonna be created in the default k8s namespace for the given environment/provider configuration
so if you don’t want to create this CronJob as a python object and read it from file
you have to alter (see 
transformations
) it once it’s read using 
kubernetes.yaml.ConfigFile
and add/modify 
namespace
parameter
unless, 
ConfigFile
has some dedicated support for overriding namespace (it could have, let me check)
nah, it doesn’t look like it has
d

dazzling-angle-45051

03/23/2022, 9:27 PM
Copy code
def set_namespace(obj, opts):
            obj["metadata"]["namespace"] = namespace.id

        cronjob = kubernetes.yaml.ConfigFile(
            f"{name}-cronjob",
            "./yaml/cronjob.yaml",
            opts=pulumi.ResourceOptions(parent=self, depends_on=[namespace]),
            transformations=[set_namespace],
        )
this works, but sounds super overkill
p

prehistoric-activity-61023

03/23/2022, 9:27 PM
Try to define the following function (you can skip type hints if you don’t want to use them):
Copy code
def set_namespace(name: str) -> Callable[[Any, pulumi.ResourceOptions], None]:
  def inner(obj, opts):
    obj["metadata"]["namespace"] = name
and use this transformation function while creating the resource:
Copy code
resource = kubernetes.yaml.ConfigFile(
            f"{name}-resource",
            "./yaml/resource.yaml",
            opts=pulumi.ResourceOptions(parent=self, depends_on=[namespace]),
            transformations=[set_namespace(namespace.metadata.name)],
        )
hah, I see you managed to get it yourself
the only different is that I created a more generic 
set_namespace
function so it can be reused by other modules
but the rest is identical
check whether you still need to explicitly specify 
depends_on
I’m not sure if dependency graph is altered based on 
transformations
field
d

dazzling-angle-45051

03/23/2022, 9:30 PM
what's really weird to me is that depends_on doesn't infer the namespace name to the resource
p

prehistoric-activity-61023

03/23/2022, 9:30 PM
well, it’s not that weird if you consider that EVERY pulumi resource supports this
so it’s not really related to k8s, namespaces or anything
and as I said in the beginning, it should be used only if the dependency cannot be detected automatically
namespace.metadata.name
is not a simple 
str
but rather 
pulumi.Output[str]
so it can do “some magic” to make these dependencies autodiscoverable
keep that in mind when you think about concatenating its value (https://www.pulumi.com/docs/intro/concepts/inputs-outputs/#outputs-and-strings)
I’d say that we could file a ticket to add support for overriding 
namespace
in 
kubernetes.yaml.ConfigFile
as it seems like a very common usecase and creating a transformation function for that, indeed seems like an overkill
d

dazzling-angle-45051

03/23/2022, 9:34 PM
depends_on not needed anymore
p

prehistoric-activity-61023

03/23/2022, 9:34 PM
Hah, as I expected. That’s great :)
So having a transformation function that uses an output is enough to create a dependency between the resources.
d

dazzling-angle-45051

03/23/2022, 9:35 PM
I still have a hard time to understand why the transformation is needed at all
resolved in PM, thanks!
3 Views
Powered by