Hey everyone! Running into a funny issue when using the Pulumi Automation API, described below. Background I’m using the Automation API to manage resources on Azure. I created a couple of RunCommands and Custom Script Virtual Machine Extension resources, but the

up

got interrupted (can’t recall if it was a network error or if I cancelled w/

Ctrl + C

). This, of course, left some

pending_operations

in the state that prevent the use of

up

,

down

, etc. Solution Attempt I tried to solve this the usual way, viz., by exporting the state; clearing the `pending_operations`; and re-importing it. I’m not certain if I’m doing this right, as I wasn’t able to find much documentation on this, but this is how that’s happening:

# EXPORT STATE
# ############
print('Exporting deployment...')
stack_data = stack.workspace.export_stack(stack_name)
with open('stack.json', 'w+') as output:
  output.write(str(stack_data))
print('Deployment exported.')

I then modify the

stack.json

to remove

pending_operations

, and attempt to re-import:

# IMPORT STATE
# ############
print('Importing deployment...')
with open('stack.json', 'r') as stack_data:
  deployment = auto._workspace.Deployment(deployment=stack_data.read())
  stack.workspace.import_stack(stack_name, deployment)
print('Deployment imported.')

Expected Behavior I’d expect to see output analogous to that produced by the CLI in this circumstance, namely:

# From: <https://www.pulumi.com/docs/troubleshooting/>
$ pulumi stack export | pulumi stack import
warning: removing pending operation 'creating' on '...' from snapshot
Import successful.

Issue Instead, I get the following, fairly confusing error:

File "/Users/{SCRUBBED}/env/lib/python3.10/site-packages/pulumi/automation/_cmd.py", line 78, in _run_pulumi_cmd
    raise create_command_error(result)
pulumi.automation.errors.CommandError:
 code: 255
 stdout:
 stderr: error: the stack 'dev-3' is too old to be used by this version of the Pulumi CLI

I upgraded the CLI, the

pulumi

package, and attempted re-importing an unmodified state file (i.e., export and immediately import, without making changes to

pending_operations

), to no avail. I’m not sure where to look for information on the error code or where this comes from, and I found no hints after examining the source. Any input would be much appreciated — I’d rather not tear down the stack out-of-band just to recreate it (and maybe hit the same issue anyway). Thanks in advance!

First, regarding the state/pending operations issue, I'm going to leave this note here in case you might like to vote or comment on the issue in it: https://pulumi-community.slack.com/archives/C019YSXN04B/p1646238520774459?thread_ts=1646144559.200289&cid=C019YSXN04B Regarding the version error, are you running this all on the same machine? Generally I'll find that when the up was run on a machine that was using older versions of either the CLI or the providers that it's calling originally, and then there's a mismatch of versions when it's attempting to run as the versions used are also stored in the state file. Overall, to get more debugging info, the exact flags that you want depends on whether any providers you're using are using the Terraform bridge. But here's a short snippet that should get you all of the diagnostic data that's possible if you can run it locally:

TF_LOG=TRACE pulumi up -v=11 --logflow --logtostderr 2>&1 | tee -a pulumi_log.txt

This helped a great deal, was able to solve the issue by mimicking the first Go example you linked. I’ll write up the solution step(s) a bit later, as there were a few peculiarities and opportunities for updating the docs a bit. Meantime, just wanted to say thanks, resolved!

We'd love a PR for the docs if you're so inclined, too!