Hey all, trying to use the flex template resource,...
# google-cloud
Hey all, trying to use the flex template resource, https://www.pulumi.com/registry/packages/gcp/api-docs/dataflow/flextemplatejob/. Pulumi creates the resource just fine. It also updates the job fine the first time but the section time it gives an error:
has terminated with state "JOB_STATE_UPDATED"
. Dataflow will create a new jobId, seems like pulumi state keeps the old job. From what I see the provider has fixed this but still getting the above result. https://github.com/pulumi/terraform-provider-google-beta/blob/18e8f0589864f98ea7bc[…]015f6935eb64/google-beta/resource_dataflow_flex_template_job.go
Did a bit of digging in my stack export and found the new ID is different than the ID:
"id": "2022-06-06_23_03_18-5843176362055954902"
Copy code
"outputs": {
  "id": "2022-06-07_08_21_12-44324234441230267",
   "jobId": "2022-06-07_08_21_12-44324234441230267",
The resource docs you're referencing are from https://github.com/pulumi/pulumi-gcp instead of https://github.com/pulumi/terraform-provider-google-beta. That being said, can you share the code where you're defining the flex template job? Dataflow is a bit odd in the sense that jobs can start and stop independently of a Pulumi run, so I suspect that there's some flag that we need to set to ensure Pulumi knows that it could change outside of the defined state. I don't know it that well, but I'm willing to try to help figure it out.
Copy code
const job = new gcp.dataflow.FlexTemplateJob(
        name: name,
        containerSpecGcsPath: interpolate`<gs://stagingbucket/templates/spec>`,
        onDelete: 'drain',
        project: project,
        parameters: {
          setup_file: './src/python/setup.py',
          service_account_email: serviceAccount.email,
Pretty simple definition, I manually create the flex template spec with
Copy code
const template: FlexTemplate = {
      defaultEnvironment: {},
      image: imageName,
      sdkInfo: {
        language: 'PYTHON',