Thread
#gcp
    b

    breezy-lifeguard-15721

    3 months ago
    Hey all, trying to use the flex template resource, https://www.pulumi.com/registry/packages/gcp/api-docs/dataflow/flextemplatejob/. Pulumi creates the resource just fine. It also updates the job fine the first time but the section time it gives an error:
    has terminated with state "JOB_STATE_UPDATED"
    . Dataflow will create a new jobId, seems like pulumi state keeps the old job. From what I see the provider has fixed this but still getting the above result. https://github.com/pulumi/terraform-provider-google-beta/blob/18e8f0589864f98ea7bc[…]015f6935eb64/google-beta/resource_dataflow_flex_template_job.go
    Did a bit of digging in my stack export and found the new ID is different than the ID:
    "id": "2022-06-06_23_03_18-5843176362055954902"
    "outputs": {
      "id": "2022-06-07_08_21_12-44324234441230267",
       "jobId": "2022-06-07_08_21_12-44324234441230267",
    g

    great-queen-39697

    3 months ago
    The resource docs you're referencing are from https://github.com/pulumi/pulumi-gcp instead of https://github.com/pulumi/terraform-provider-google-beta. That being said, can you share the code where you're defining the flex template job? Dataflow is a bit odd in the sense that jobs can start and stop independently of a Pulumi run, so I suspect that there's some flag that we need to set to ensure Pulumi knows that it could change outside of the defined state. I don't know it that well, but I'm willing to try to help figure it out.
    b

    breezy-lifeguard-15721

    2 months ago
    const job = new gcp.dataflow.FlexTemplateJob(
          name,
          {
            name: name,
            containerSpecGcsPath: interpolate`<gs://stagingbucket/templates/spec>`,
            onDelete: 'drain',
            project: project,
            parameters: {
              setup_file: './src/python/setup.py',
              service_account_email: serviceAccount.email,
              ...
            },
          },
          resourceOptions,
        );
    Pretty simple definition, I manually create the flex template spec with
    const template: FlexTemplate = {
          defaultEnvironment: {},
          image: imageName,
          sdkInfo: {
            language: 'PYTHON',
          },
        };
    @great-queen-39697