https://pulumi.com logo
Title
m

mysterious-oyster-86659

04/12/2021, 5:02 PM
Hi Everyone😃 Our stack is currently written with TypeScript (TS) and have implemented Airflow in AWS. Now, we need to dynamically reference TS resources in Python and have no idea how to do that in manageable way (without just hard-coding them). Is there a way to do this? Note that this is also posted in #typescript channel too.
b

billowy-army-68599

04/12/2021, 5:04 PM
can you explain a little more? Where do you need to grab the resources from?
m

mysterious-oyster-86659

04/12/2021, 5:16 PM
Thanks for the response, @billowy-army-68599. Sure! Here's a simple one: There's an S3 bucket and Glue jobs provisioned/managed by Pulumi TS. In order for the DAG task (which is all in Python) to call on and use the TS resources, it must know said resources by
name
, a-priori. There's just not a way to do this during deployment. It seems that I must first name those TS resources and then literally copy/paste that string to the DAG task configurations. Like I said, this kind of thing can be easily hard-coded, but that'll be unruly very shortly😅
r

rich-easter-89163

04/12/2021, 10:41 PM
As airflow DAGs and pulumi programs never execute in the same environments a possible way to workaround this is storing the names in an external store so then the DAG can fetch the names in runtime.
1
w

white-balloon-205

04/13/2021, 2:52 AM