https://pulumi.com logo
Title
f

freezing-van-87649

10/01/2021, 1:42 AM
Has anyone used Pulumi with the python step function sdk? I’m jealous of the CDK step function user experience
b

billowy-army-68599

10/01/2021, 1:45 AM
I don't think we have an example, but it would be very feasible to mirror the existing SDK with automation API. Do you have a CDK example to hand?
f

freezing-van-87649

10/01/2021, 1:47 AM
Here’s the example from their docs
The main thing I’m interested in here is using a real language to generate the state machine definition vs just dealing with json blobs
b

billowy-army-68599

10/01/2021, 1:50 AM
I'm not super familiar with this particular AWS resource honestly, we do have a statemachine and activity resource: https://www.pulumi.com/docs/reference/pkg/aws/sfn/activity/ You should be able to build the JSON blobs using
JSON.stringify
on a map, right?
f

freezing-van-87649

10/01/2021, 1:54 AM
I mean you could just use maps, but it’s a big step down in expressivity CDK:
submitJob
    .next(waitX)
    .next(getStatus)
    .next(new sfn.Choice(this, 'Job Complete?')
        // Look at the "status" field
        .when(sfn.Condition.stringEquals('$.status', 'FAILED'), jobFailed)
        .when(sfn.Condition.stringEquals('$.status', 'SUCCEEDED'), finalStatus)
        .otherwise(waitX));
Python step function sdk:
lambda_state = LambdaStep(
    state_id="Convert HelloWorld to Base64",
    parameters={
        "FunctionName": "MyLambda", #replace with the name of your function
        "Payload": {
        "input": "HelloWorld"
        }
    }
)

lambda_state.add_retry(Retry(
    error_equals=["States.TaskFailed"],
    interval_seconds=15,
    max_attempts=2,
    backoff_rate=4.0
))

lambda_state.add_catch(Catch(
    error_equals=["States.TaskFailed"],
    next_step=Fail("LambdaTaskFailed")
))
For a little context on why I’m asking now: https://twitter.com/timbray/status/1443647106183401495
h

hallowed-train-1850

10/18/2021, 2:04 AM
Hi @freezing-van-87649, we’ve used this pattern. We use the Workflow.definition.to_json() method and pass it as an input to the pulumi state machine constructor. This gets us the expressivity and the pulumi state management that the python step functions sdk doesn’t provide.
👍 1
Happy to answer specific questions if you have any.
b

billowy-army-68599

10/18/2021, 2:06 AM
this sounds awesome @hallowed-train-1850! do you have an example, even stripped down that you could share?
h

hallowed-train-1850

10/18/2021, 2:33 AM
Stripping down something quickly. Might not compile as is but should be pretty similar. Unfortunately this is for a customer with a workflow under NDA so I’ll have to make it generic:
from stepfunctions.steps import LambdaStep, ChoiceRule, Choice
from stepfunctions.steps.states import Choice, Chain, Retry, Succeed, Fail

from modules.pulumi.statemachine import StepFunctionStateMachine
from modules.pulumi.workflow import LambdaWorkflow

    do_X = LambdaStep(
        state_id="DoX",
        parameters={
            "FunctionName": "$.do_Z",
            "Payload.$": "$"
        },
        output_path="$.Payload"
    )

    do_X.add_retry(Retry(
        error_equals=["Lambda.ServiceException","Lambda.AWSLambdaException","Lambda.SdkClientException"],
        interval_seconds=2,
        max_attempts=6,
        backoff_rate=2
    ))

    does_y_hold_true = Choice(
        state_id="DoesYHoldTrue?",
    )

    do_Z = LambdaStep(
        state_id="DoZ",
        parameters={
            "FunctionName": "$.do_Z",
            "Payload.$": "$"
        },
        output_path="$.Payload"
    )

    do_Z.add_retry(Retry(
        error_equals=["Lambda.ServiceException","Lambda.AWSLambdaException","Lambda.SdkClientException"],
        interval_seconds=2,
        max_attempts=6,
        backoff_rate=2
    ))

    success = Succeed(
        state_id="Succeed"
    )

    fail = Fail(
        state_id="Fail"
    )

    do_X.next(
        next_step=does_y_hold_true
    )

    does_y_hold_true.add_choice(
        ChoiceRule.StringEquals(
            variable="$.yHoldsTrue",
            value="true"
        ),
        next_step=do_Z
    )
    
    terms_and_conditions_accepted.default_choice(fail)
    do_Z.next(success)

    workflow = Workflow(
        workflow_name,
        role=account_role,
        definition=Chain([do_X])

from pulumi_aws import iam, provider, Provider, sfn

    stateMachine = sfn.StateMachine(
            f'{name}StateMachine',
            type="STANDARD",
            role_arn=stateMachineRole.arn,
            definition=workflow.definition.to_json(pretty=True)
        )
sorry about the indentation mess. I chopped bits and pieces and pasted them into the slack editor.
(@billowy-army-68599)
b

billowy-army-68599

10/18/2021, 2:38 AM
that's amazing, thank you!
h

hallowed-train-1850

10/18/2021, 2:41 AM
Definitely prefer the api from the cdk here. If I had more time I would have built out a custom state machine builder / tried to port the cdk’s construct to pulumi but alas 🤷‍♂️
f

freezing-van-87649

10/18/2021, 2:45 AM
That’s awesome, thanks @hallowed-train-1850. Definitely along the lines of what I was thinking, glad to know it’s viable