This message was deleted.

can you share your CI config?

but no output from 

pulumi up

 is added afterwards..

The GitLab integration adds notes to the MR for

pulumi preview

only. I am curious — are you running

pulumi up

in the MR or are you running it once the MR is merged? Also I realize it isn’t the same thing but you can now subscribe to stack update (i.e.

pulumi up

) success/failure from the Pulumi Console and get email notifications. If you go to the Pulumi Console for the stack in question, you should see a “Watch” button to the right and just below the avatar button that’ll let you subscribe to updates.

Thanks for the assistance @billowy-army-68599 and @clever-sunset-76585. You can find the CI-related files below

Why does the GitLab integration not provide output of

pulumi up

in the MR? Would be highly appreciated! 🙏

Why does the GitLab integration not provide output of 

pulumi up

 in the MR?

Would be highly appreciated!

That’s good feedback! I will take that feedback back to our team.

PS: Until now we ran 

pulumi_up

 for 

only: merge_request

 and 

pulumi up

 for 

only: main, dev

 .

Do you mean

pulumi preview

for

only: merge_request

…?

We actually had a case where GitLab marked the jobs of the pipeline as successful or "passed with warnings" although it was a failure. Hence, super misleading! The fact that it was a failure and the reason for it would only be discoverable via the pulumi UI. So especially for that case, the output of

pulumi up

would be really helpful! 🙂 Getting email notifications for that are also nice, but it would be better to have all the information and links to pulumi UI on the merge request (as is already the case for

pulumi preview

).

Ah I see. I would love for you to checkout the stack email notifications in the meantime and let us know what you think!

So, we now changed the workflow such, that everything is executed as follows (all on

only: merge_requests

and

when: manual

for

pulumi up

) :

assert-formatting:
  stage: format
  before_script:
    - pip install black
  script:
    - black --check __main__.py
  only:
    - merge_requests
    - dev
    - master

pulumi-preview:
  stage: infrastructure-preview
  script:
    - ./scripts/pulumi-run.sh -o ${CI_PROJECT_ROOT_NAMESPACE} -p ${CI_PROJECT_NAME} -s ${CI_MERGE_REQUEST_TARGET_BRANCH_NAME} -c "preview --suppress-outputs" -r "eu-central-1" > pulumi-log.txt
  # Create an artifact archive with just the pulumi log file,
  # which is created using console-redirection in run-pulumi.sh.
  artifacts:
    paths:
      - pulumi-log.txt
    expire_in: 1 week
  only:
    - merge_requests

##################################
## Updating
pulumi-up:
  stage: infrastructure-update
  script:
    - ./scripts/pulumi-run.sh -o ${CI_PROJECT_ROOT_NAMESPACE} -p ${CI_PROJECT_NAME} -s ${CI_COMMIT_REF_NAME} -c "up --yes --suppress-outputs" -r ${AWS_DEFAULT_REGION}
  only:
    - merge_requests
  when: manual

And we like the process of this! However, we still don't get the

pulumi up

output (as you said), instead three times the

pulumi preview

output 🤷

However, we still don’t get the 

pulumi up

 output (as you said), instead three times the 

pulumi preview

 output

Yeah each note corresponds to a specific exceution of the

preview

command. That is, each time you push an update to that same MR, a new comment is added related to the execution triggered by the latest commit to that MR.

Yes, that is actually nice, so one can work really nicely on changes until the preview looks as desired. Also, we just tested the same problem we had before and now we do get notified that the

pulumi up

failed. And the

Update

link brings us to the respective error message. If only that error message could be also output into the MR, we would be already happy! 😊

Yes, as mentioned above we run 

pulumi up

  in 

only: merge_request

 with 

when: manual

 .

Ok yeah I was suspecting it was a typo but I guess not 🙂 Do you run a

destroy

or something if it is successful then?

Why would we run a destroy if it is successful?

If you are asking that then I take it that this is not an ephemeral update and in fact the actual infrastructure. 🙂 It is generally not recommended to run the actual update in an MR because multiple MRs can be opened. Typically, pipeline concurrency is enforced for the default/main branch pipelines which allows you to control the number of simultaneous updates in flight (note that the Pulumi service also provides concurrency control, by default to avoid corruption of resources). By nature of an MR, running only a

preview

in the MR pipeline and merging the approved changes to let the push build be the only pipeline where you apply the update.

Thanks for pointing that out @clever-sunset-76585! For sure that would be the safer way to go, with the one drawback being that we would no longer get the

pulumi up

error notification in the merge request 😕. So since it is just the two of us on our backend team, I think we can be careful about it for the time being. But you're right that this is definitely an issue that we will run into in the future as our team grows. So we'll be keen to hear about any updates from Pulumi 🙂

Yeah my suggestion would be to not do what you are doing and instead use the stack update email notification just so that when your team grows someone on your team doesn’t accidentally introduce an issue and thereby break your infrastructure 😬 Note that with MRs GitLab also allows you to control who can merge the MR eventually, which can be a good way to protect mutations to the infrastructure. By applying the update directly in the MR, you would be losing out on those protections, unfortunately.

That makes a lot of sense! I don't mean to disregard your suggestion 🙂. It's just that, at the moment (and especially since we do a lot of head-to-head trial-and-error), doing the following workflow all within a single MR:

Create issue -> Initialize MR -> Develop -> See pulumi preview -> Manually accept update -> Discover error -> Document discussion -> Develop -> Preview -> Manually accept update -> See that pulumi update was successful -> Merge

Is a more effective strategy than:

Create issue -> Initialize MR -> Develop -> See pulumi preview -> Merge ("successfully") -> Wait for email -> Discover error -> Document discussion (somewhere?) -> Reopen issue -> Initialize new MR (tenuously tied to the former) -> Develop -> See preview -> Merge -> Wait for email -> See that pulumi update was successful -> Party

Hopefully that makes our current concern a bit clearer? But of course in the future, when we are working on more than one infrastructure-related issue at a time and when the changes are more marginal and less likely to fail during the update, then the route you suggest is definitely the better option!

No hard feelings at all 🙂 I understand that each team has their own reasoning for doing things a certain way. It’ll be a good idea, though, to re-evalute your setup at a later time. That said, I see your argument for doing it the way you are currently. However, I am surprised that a

pulumi preview

in your MR pipeline isn’t giving you the failure you might see once it is merged. I recommend digging into that and look at ways to surface that in the preview. So that your MR pipeline is reliable and you have an assurance that the changes will pass when merged. On a slightly related note, you could also look into our integration test framework that could help you test your infrastructure effectively. Although depending on what you are doing in your infrastructure app it can cause your pipeline build times to become long. The integration test framework is good for setting up and tearing-down the infrastructure.