Any ideas how I can understand what triggers this issue?

(node:5747) UnhandledPromiseRejectionWarning: Error: Program run without the Pulumi engine available; re-run using the `pulumi` CLI
    at requireTestModeEnabled (/node_modules/@pulumi/pulumi/runtime/settings.js:123:15)
    at Object.getMonitor (/node_modules/@pulumi/pulumi/runtime/settings.js:214:13)
    at /node_modules/@pulumi/pulumi/runtime/invoke.js:123:40
    at Generator.next (<anonymous>)
    at fulfilled (/node_modules/@pulumi/pulumi/runtime/invoke.js:18:58)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

Usually you would only see this error if you are running a pulumi program (ie one that makes SDK calls like

new aws.s3.Bucket(...)

) either without the CLI, or without calling

stack.up

in automation api.