Hi. The Pulumi testing documentation (https://www.pulumi.com/docs/guides/testing/) indicates it’s possible to do Unit Tests without the Pulumi CLI available. Does this include Automation API scenarios? If so, how do you set that up?

automation API operates at a layer above Pulumi, so you should be good to unit test it like any other function

I think if you want to test a Pulumi program than you would isolate that and just do it the exact same way as you did before automation api. So you wouldn’t even touch automation api to test this. Automation api will use the same Pulumi program. You’ll need to provide your mocks to simulate your resource outputs. If you want to test your automation api stack/project manipulation than it will require CLI and you’re going to want to use like a temporary local file state backend or just use the pulumi SAAS in a test project

So how would I trigger the tests? I’m just following the tutorial here: https://www.pulumi.com/docs/guides/testing/unit/ My code is a little different from the example, in that I’m testing a different resource, but the setup is the same as the tutorial. But when I run Mocha like this:

$ mocha -r ts-node/register test/flannel.test.ts

It spits out this error:

Error: Program run without the Pulumi engine available; re-run using the `pulumi` CLI

What code are you invoking inside your test? It looks like it's invoking the AutomationAPI, which would be too broad to consider a unit test (imo). Since the Pulumi programs invoked within the AutomationAPI could be massive, it's hard to see how that could be considered a behavioural unit.

Yea like I said, if you want to test code that invokes Automation API you’ll need the CLI and you’ll want to setup a temporary state backend. If you want to test a Pulumi program, meaning just the Pulumi infrastructure code, than like @little-cartoon-10569 said you may want to split it into smaller units - but even if you don’t do that, you’ll just run your tests the same way you would run any other unit tests depending on your language of choice. The various Pulumi SDKs expose test entry points that can be provided mock interfaces to simulate resource outputs. Automation API requires the CLI, but can be given a Pulumi program that you tested independently.

Actually, I was testing individual components, and still got the same error even when not using the Automation API. After going back and forth between my project and Pulumi’s example Unit Testing project (https://github.com/pulumi/examples/tree/74db62a03d013c2854d2cf933c074ea0a3bbf69d/testing-unit-ts), I found the culprit: this is broken on Pulumi versions prior to the current latest, 3.9.1. You can repro this by cloning that example Unit Testing repro, change Pulumi’s version to something like 3.9.0 or 3.8.0, and then try to run the test. It will fail with that same error:

Program run without the Pulumi engine available; re-run using the `pulumi` CLI

However, upgrading my project to 3.9.1 seems to have fixed the issue. Thanks all!