I'm looking into modifying the YAML runtime to support a

!include extra.yaml

reference type to allow for multi-file referencing. Does that conflict with any underway work? Proposal: Main entry remains the same,

main.yaml

, or

pulumi.yaml

. Within that file, a reference of

!include xyz.yaml

would cause the yaml decoder to locate the referenced file and append and overwrite values into the main.yaml. A simple overwrite-last method would be used, meaning contents of

!include 1.xyz.yaml

would be overwritten by

!include 2.zyx.yaml

if any keys or attributes match. Otherwise, unique values would just be added. Concerns: I'm trying to decide if

!include

should be a local reference level or global. Consider:

main.yaml

resources:
  s3-bucket:
    !include s3-bucket.yaml

s3-bucket.yaml

type: aws:s3:Bucket
properties:
  name: mybucket

vs:

main.yaml

resources:
  !include s3-bucket.yaml

s3-bucket.yaml

resources:
  s3-bucket:
    type: aws:s3:Bucket
    properties:
      name: mybucket

Write order:

main.yaml

resources:
  !include 1.s3-bucket.yaml
  !include 3.s3-bucket.yaml
  !include 2.s3-bucket.yaml

1.s3-bucket.yaml

resources:
  s3-bucket:
    type: aws:s3:Bucket
    properties:
      name: original-bucket
      tags:
        Application: something-useful

2.s3-bucket.yaml

resources:
  s3-bucket:
    type: aws:s3:Bucket
    properties:
      name: final-bucket
      tags:
        Owner: myself

3.s3-bucket.yaml

resources:
  s3-bucket:
    type: aws:s3:Bucket
    properties:
      name: applied-but-overwritten-bucket
      tags:
        Environment: Dev

Output:

resources:
  s3-bucket:
    type: aws:s3:Bucket
    properties:
      name: applied-but-overwritten-bucket
      tags:
        Application: something-useful
        Owner: myself
        Environment: Dev

Also, I desire the workflow of having wildcard support:

main.yaml

resources:
  !include account*.yaml

account_main.yaml

main-aws-account:
  type: aws:organizations:Account
  properties:
    name: main-aws-account

account_secondary.yaml

secondary-aws-account:
  type: aws:organizations:Account
  properties:
    name: secondary-aws-account

@fresh-scientist-56300 @quiet-wolf-18467 created a great blog article to use various other languages to create a single YAML input for Pulumi YAML from various code files utilizing the compiler property of a Pulumi.yaml file. https://www.pulumi.com/blog/extending-pulumi-languages-with-yaml-cue-jsonnet-rust/

I did read that article, but I hadn't thought about

yq

as a compile option, that's a cool idea.