Hi guys, I am new to Pulumi and seeing some unexpected issues based on my previous experience with IaC. I am writing in Kotlin (example file in thread), but it should be the same in any language. Deploying to AWS. The basis is that I am creating a budget with 3 notifications. Before I can create the budget, I create the 3 SNS topics. Each of them has subscriptions and an attached policy. I see these 2 issues: 1. When running

pulumi up

or

pulumi preview

I see everything in the "plan" besides the policies, but when I approve after

up

, the policies show up as added. 2. I created one of the topics outside of Pulumi. In the Pulumi code I have the resource with the same name. When I run Pulumi up, it does not complain about the inability to create the topic as it already exists, but it fails when trying to create the budget instead, saying:

error: 1 error occurred:

* creating urn:pulumi:test::mimoze-kotlin-test::aws:budgets/budget:Budget::budget: 1 error occurred:        * creating Budget (<account_number>:pulumi-budget) notifications: DuplicateRecordException: Unable to create notification: { notificationType: ACTUAL, comparisionOperator: EQUAL_TO, threshold: 100.0 } for budget: pulumi-budget - the notification already exists.

so the budget isn't completely created, but the creation of the topic itself is marked as succeeded. When I do a

pulumi destroy

, pulumi will destroy all the resources, including the one which was initially created outside of it. Note that I didn't do any import of that resource manually or in code, but somehow it seems to have been imported during "up", otherwise I don't see how it would be added to the state and then later destroyed. I find it strange that that import would happen without warning.

Hey Mimozë, sorry for having this issue. Unfortunately that's a limitation in the AWS API - calling create for an SNS topic returns fine if the topic already exists. We've inherited this behaviour. It'd be helpful if you could raise an issue in https://github.com/pulumi/pulumi-aws/issues with steps on reproducing your problem. There's some work happening on https://github.com/pulumi/pulumi-aws/issues/2009, which is somewhat related and might also address your issue.

Hi Venelin, thanks for the info! I was not aware of that. I checked it and it seems to be the case that when I create another topic with the same name but no configuration, I get no error, but also the topic doesn't change at all - so configs are not overriden (removed), but we just don't get an error message. But, if we do provide some config to the topic with the same name on creation (e.g.

aws sns create-topic --name budget_exceeded_topic --tags Key=CLI,Value=True

), then I get this error message that stops the creation:

An error occurred (InvalidParameter) when calling the CreateTopic operation: Invalid parameter: Tags Reason: Topic already exists with different tags

. In the case of Pulumi, it still somehow takes over the existing topic and updates it with its specified config.

Hi Mimozë, thanks for checking. I tried to reproduce the tags issue you mentioned and I got the same issue you had with the CLI. Here's my program:

import pulumi
import pulumi_aws as aws

topic = aws.sns.Topic(
    "topic", aws.sns.TopicArgs(name="my-topic-vvm", tags={"tag1": "val"})
)


if pulumi.config.Config().get("change_tags"):
    topic_with_different_tags = aws.sns.Topic(
        "topic_with_different_tags",
        aws.sns.TopicArgs(name="my-topic-vvm", tags={"tag1": "val2"}),
    )

The steps to repro are: 1. pulumi up 2. pulumi config set change_tags true 3. pulumi up If you can reproduce the behaviour with the silent failure, I'd be very grateful if you can raise an issue with that! Edit: my repro is in Python but feel free to use whatever language you prefer!

Does it mean that this issue doesn't appear with other APIs?

I'm not sure I understand what you mean - which other APIs?

Other AWS APIs. This is a major issue if it happens with all kinds of AWS resources. We don't want resources created outside of Pulumi to be deleted or modified by Pulumi just because the name is matching.

Ah, I see. No, most other AWS resources are not affected by this. I think the behaviour is the exception but unfortunately it affects SNS topics.

Then the names wouldn't be so meaningful 😅 Any thoughts on issue 1.? Why the policies don't show up in the "plan" but then get deployed and you only see it afterwards that they were deployed?

If you let the engine auto-name the resources it uses your variable names and then adds a random suffix. So the names are still meaningful but unlikely to clash. As for issue 1 - it looks like you are defining the policy in an apply. The engine can not resolve that during preview so it only shows up when you run up https://www.pulumi.com/docs/concepts/inputs-outputs/#apply

During some program executions, apply doesn't run. For example, it won't run during a preview, when resource output values may be unknown. Therefore, you should avoid side-effects within the callbacks. For this reason, you should not allocate new resources inside of your callbacks either, as it could lead to pulumi preview being wrong.