This message was deleted Pulumi Community #aws

Join Slack

This message was deleted.

# aws

sparse-intern-71089

03/23/2023, 4:38 PM

This message was deleted.

most-state-94104

03/23/2023, 4:40 PM

Copy code

aws.dms.ReplicationTask(
                resource_name=(
                    f"{database_name}-{instance_number}-{task_number}-{migration_type}"
                    f"-{region}-{env}"
                ),
                replication_task_id=(
                    f"{database_name}-{instance_number}-{task_number}-{migration_type}"
                    f"-{region}-{env}"
                ),
                migration_type='cdc',
                cdc_start_time=null,
                replication_instance_arn=replicationInstance.replication_instance_arn,
                source_endpoint_arn=sourceEndpoint.endpoint_arn,
                target_endpoint_arn=targetEndpoint.endpoint_arn,
                replication_task_settings=json.dumps(
                    replication_task_settings, sort_keys=True, separators=(",", ":")
                ),
                table_mappings=make_table_mappings(tables, mappings["schema"]),
                tags=tagger.create_tags(
                    f"{database_name}-{instance_number}-{task_number}-{migration_type}"
                    f"-{region}-{env}"
                ),
                opts=pulumi.ResourceOptions(
                    parent=replicationInstance,
                    depends_on=[sourceEndpoint, targetEndpoint],
                    delete_before_replace=True,
                ),
            )

steep-toddler-94095

03/24/2023, 2:59 AM

this is probably a bug. what is/are the fields that it always has a diff for?

👀 1

most-state-94104

03/24/2023, 9:38 AM

replicationTaskSettings

steep-toddler-94095

03/26/2023, 12:36 AM

what do you see in your diff? and are you sure the setting is being set in AWS on a

pulumi up

(rather than a false success like if you try to apply an IAM policy that exceeds the max length)

most-state-94104

03/27/2023, 10:46 AM

So I’m using a

pulumi preview --expect-no-changes

and this is a section of the output message:

Copy code

@ previewing update....
    aws:iam:Role oasys-dms-dev  
    aws:dms:Endpoint oasys-source-eu-west-1-dev  
    aws:iam:Role oasys-lambda-copy-object-dev  
    aws:dms:DmsSlackNotification notify-dms-events-dev  
    aws:ec2:SecurityGroup oasys-eu-west-1-dev  
    aws:iam:RolePolicy oasys-dms-dev  
    aws:dms:Endpoint oasys-target-eu-west-1-dev  
    aws:iam:RolePolicy oasys-lambda-copy-object-dev  
    aws:sns:Topic notify-dms-events-dev  
    aws:iam:Role notify-dms-events-dev  
    aws:ec2:SecurityGroupRule egress-oasys-eu-west-1-dev  
    aws:ec2:SecurityGroupRule ingress-oasys-eu-west-1-dev  
    aws:lambda:Function notify-dms-events-dev  
    aws:dms:ReplicationInstance oasys-0-eu-west-1-dev  
    aws:iam:RolePolicyAttachment notify-dms-events-dev  
    aws:sns:TopicSubscription notify-dms-events-dev  
    aws:lambda:Permission notify-dms-events-dev  
 ~  aws:dms:ReplicationTask oasys-0-2-full-load-and-cdc-eu-west-1-dev update [diff: ~replicationTaskSettings]
    aws:dms:EventSubscription replication-instance-dev  
 ~  aws:dms:ReplicationTask oasys-0-0-full-load-eu-west-1-dev update [diff: ~replicationTaskSettings]
@ previewing update....
 ~  aws:dms:ReplicationTask oasys-0-1-cdc-eu-west-1-dev update [diff: ~replicationTaskSettings]
    aws:dms:EventSubscription replication-task-dev  
@ previewing update...........
    aws:lambda:Function delius-validate-dev  
    aws:lambda:Function oasys-validate-dev  
    aws:lambda:Permission delius-validate-dev  
    aws:s3:BucketNotification mojap-land-dev  
    aws:lambda:Permission oasys-validate-dev  
@ previewing update....
error: error: no changes were expected but changes were proposed
    pulumi:pulumi:Stack pipelines-pipelines-dev  18 warnings

steep-toddler-94095

03/28/2023, 5:04 AM

i meant to ask about the detailed diff. e.g. the output of a preview with the

--diff

flag. And are you able to confirm that the value you see in AWS Console matches what Pulumi has in its statefile?

most-state-94104

03/28/2023, 8:24 AM

Understood will send later today

most-state-94104

04/12/2023, 2:23 PM

Hello @steep-toddler-94095 apologies for the delay I was busy with other items and should now have more time to look into this. Are you still free to help me or should I start a new thread?

most-state-94104

04/12/2023, 2:25 PM

I have attached the

pulumi preview --diff

just in case 🙂

most-state-94104

04/13/2023, 11:27 AM

Hello @steep-toddler-94095 I’ve also pasted the task settings for

oracle-1-0-cdc-eu-west-1-sandbox

to compare:

Copy code

{
  "Logging": {
    "EnableLogging": true,
    "EnableLogContext": false,
    "LogComponents": [
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "TRANSFORMATION"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "SOURCE_UNLOAD"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "IO"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "TARGET_LOAD"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "PERFORMANCE"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "SOURCE_CAPTURE"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "SORTER"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "REST_SERVER"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "VALIDATOR_EXT"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "TARGET_APPLY"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "TASK_MANAGER"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "TABLES_MANAGER"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "METADATA_MANAGER"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "FILE_FACTORY"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "COMMON"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "ADDONS"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "DATA_STRUCTURE"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "COMMUNICATION"
      },
      {
        "Severity": "LOGGER_SEVERITY_DEFAULT",
        "Id": "FILE_TRANSFER"
      }
    ],
    "CloudWatchLogGroup": "dms-tasks-oracle-1-eu-west-1-sandbox",
    "CloudWatchLogStream": "dms-task-MEOUNVF6YGI7FP6UVZWCTUJIGDB6ORKMJ5OQN4I"
  },
  "StreamBufferSettings": {
    "StreamBufferCount": 3,
    "CtrlStreamBufferSizeInMB": 5,
    "StreamBufferSizeInMB": 8
  },
  "ErrorBehavior": {
    "FailOnNoTablesCaptured": false,
    "ApplyErrorUpdatePolicy": "LOG_ERROR",
    "FailOnTransactionConsistencyBreached": false,
    "RecoverableErrorThrottlingMax": 1800,
    "DataErrorEscalationPolicy": "STOP_TASK",
    "ApplyErrorEscalationCount": 0,
    "RecoverableErrorStopRetryAfterThrottlingMax": true,
    "RecoverableErrorThrottling": true,
    "ApplyErrorFailOnTruncationDdl": false,
    "DataTruncationErrorPolicy": "LOG_ERROR",
    "ApplyErrorInsertPolicy": "LOG_ERROR",
    "EventErrorPolicy": "IGNORE",
    "ApplyErrorEscalationPolicy": "LOG_ERROR",
    "RecoverableErrorCount": 10,
    "DataErrorEscalationCount": 0,
    "TableErrorEscalationPolicy": "STOP_TASK",
    "RecoverableErrorInterval": 30,
    "ApplyErrorDeletePolicy": "IGNORE_RECORD",
    "TableErrorEscalationCount": 0,
    "FullLoadIgnoreConflicts": true,
    "DataErrorPolicy": "LOG_ERROR",
    "TableErrorPolicy": "STOP_TASK"
  },
  "TTSettings": {
    "TTS3Settings": null,
    "TTRecordSettings": null,
    "EnableTT": false
  },
  "FullLoadSettings": {
    "CommitRate": 10000,
    "StopTaskCachedChangesApplied": false,
    "StopTaskCachedChangesNotApplied": false,
    "MaxFullLoadSubTasks": 8,
    "TransactionConsistencyTimeout": 600,
    "CreatePkAfterFullLoad": false,
    "TargetTablePrepMode": "DROP_AND_CREATE"
  },
  "TargetMetadata": {
    "ParallelApplyBufferSize": 0,
    "ParallelApplyQueuesPerThread": 0,
    "ParallelApplyThreads": 0,
    "TargetSchema": "",
    "InlineLobMaxSize": 0,
    "ParallelLoadQueuesPerThread": 0,
    "SupportLobs": true,
    "LobChunkSize": 64,
    "TaskRecoveryTableEnabled": false,
    "ParallelLoadThreads": 0,
    "LobMaxSize": 32,
    "BatchApplyEnabled": false,
    "FullLobMode": false,
    "LimitedSizeLobMode": true,
    "LoadMaxFileSize": 0,
    "ParallelLoadBufferSize": 0
  },
  "BeforeImageSettings": null,
  "ControlTablesSettings": {
    "historyTimeslotInMinutes": 5,
    "HistoryTimeslotInMinutes": 5,
    "StatusTableEnabled": false,
    "SuspendedTablesTableEnabled": false,
    "HistoryTableEnabled": false,
    "ControlSchema": "",
    "FullLoadExceptionTableEnabled": false
  },
  "LoopbackPreventionSettings": null,
  "CharacterSetSettings": null,
  "FailTaskWhenCleanTaskResourceFailed": false,
  "ChangeProcessingTuning": {
    "StatementCacheSize": 50,
    "CommitTimeout": 1,
    "BatchApplyPreserveTransaction": true,
    "BatchApplyTimeoutMin": 1,
    "BatchSplitSize": 0,
    "BatchApplyTimeoutMax": 30,
    "MinTransactionSize": 1000,
    "MemoryKeepTime": 60,
    "BatchApplyMemoryLimit": 500,
    "MemoryLimitTotal": 1024
  },
  "ChangeProcessingDdlHandlingPolicy": {
    "HandleSourceTableDropped": true,
    "HandleSourceTableTruncated": true,
    "HandleSourceTableAltered": true
  },
  "PostProcessingRules": null
}

most-state-94104

04/13/2023, 11:54 AM

All fixed - I had to add

"EnableLogContext": false

and

"EventErrorPolicy": "IGNORE"

- thanks for the tip to look at the

diff

steep-toddler-94095

04/13/2023, 9:27 PM

Great to hear you figured it out!

8 Views

Open in Slack

Previous Next