# Building a Step Functions Workflow With SAM, AppSync, and Python [https://github.com/trey-rosius/sam_stepfunctions](https://github.com/trey-rosius/sam_stepfunctions) Hey!!! How you doing? In this post, we’ll look at how to build a workflow, using SAM as IaC, Appsync, and python. # Prerequisites - Install AWS Cli ([https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html)) - Install AWS SAM CLI([https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)) - AWS Account - Any IDE of your choice. I use PyCharm - Install python 3.8 ### Assumption We'll assume you've used SAM at the basic level before. If I’m wrong, I apologize. Please level up with these articles: - [https://phatrabbitapps.com/building-modern-serverless-apis-with-aws-dynamodb-lambda-and-api-gatewaypart-3](https://phatrabbitapps.com/building-modern-serverless-apis-with-aws-dynamodb-lambda-and-api-gatewaypart-3) - https://github.com/aws/aws-sam-cli-app-templates ## What are we trying to solve? So while building out a bigger system(Apartment Complex Management System), I encountered an interesting problem. I’ll assume that most of us have reserved or booked either an apartment or hotel or flight online. For this scenario, let’s go with apartments. So when you reserve an apartment, here’s a breakdown in the simplest form of the series of steps that occur after that: - The apartment is marked as reserved, probably with a status change. Let’s say the apartment status changes from **vacant** to reserved. - This apartment is made unavailable for reservation by others for a particular period of time. - The client is required to make payment within that period of time - If payment isn’t made within that time, the reservation is canceled, and the apartment status changes back from reserved to **vacant.** - If payment is made, then the apartment status changes from reserved to occupied/paid Building out this business logic using custom code is possible, but tedious and inefficient. ### Why? You might ask Because as developers, and good ones at that, we always have to be on the lookout for tools that’ll help us carry out tasks in an efficient and productive manner. Remember, code is a liability. The series of steps outlined above serves as a good use case for AWS Step Functions. - The sequence of a service interaction is important - State has to be managed with AWS service calls - Decision trees, retries, and error-handling logic are required. # Solutions Architecture ![https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/sol_arch.png](https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/sol_arch.png) Let me quarterback this entire architecture for you, please. Here’s what’s happening: 1. A frontend application sends a mutation to Appsync. 2. A Lambda resolver is invoked by Appsync based on that mutation. 3. Lambda gets the input from the mutation and starts a step functions workflow based on the input. We’ll use Flutter and Amplify to build out the front-end application in the next tutorial. ### Create And Initialize a SAM Application Open any Terminal/Command line interface, type in the command ``` sam init ``` And then follow the instructions as seen in these screenshots: ![https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/a.png](https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/a.png) Choose python 3.8 as your runtime environment ![https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/c.png](https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/c.png) I gave the project name `samWorkshopApp` ![https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/d.png](https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/d.png) Once your application has been created, open it up in your IDE, and let’s proceed. For reference, I’m using Pycharm. Activate your **virtualenv** like this on mac or linux machines: `source .venv/bin/activate` If you are a Windows platform, you would activate the **virtualenv** like this: `.venv\Scripts\activate.bat` Once the **virtualenv** is activated, you can install the required dependencies. From the root directory of the project, install all dependencies in `requirements.txt` by running the command `pip install -r requirements.txt` Initially, here’s what my folder structure looks like: ![https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/e.png](https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/e.png) There are a couple of changes we are about to make: 1. Inside the `functions` directory, delete all folders, and then create a folder called lambda. 2. Delete everything inside the `statemachine` folder, then create a file inside that same folder called `booking_step_functions.asl.json. This file would contain the state machine definition for our workflow. We visually defined this workflow in part 1 of this series. Copy the ASL(Amazon States Language) for the workflow below and paste it inside the file we’ve created above: ```json { "Comment":"A description of my state machine", "StartAt":"Change Apartment Status", "States":{ "Change Apartment Status":{ "Type":"Task", "Resource":"arn:aws:states:::dynamodb:updateItem", "Parameters":{ "TableName":"apartment_workshop_db", "Key":{ "Id":{ "S.$":"$.input.apartmentId" } }, "UpdateExpression":"SET #apartmentStatus = :status", "ExpressionAttributeNames":{ "#apartmentStatus":"status" }, "ExpressionAttributeValues":{ ":status":{ "S.$":"$.input.status" } }, "ConditionExpression":"attribute_exists(Id)" }, "Catch":[ { "ErrorEquals":[ "States.TaskFailed" ], "Comment":"Apartment Doesn't Exist", "Next":"Fail", "ResultPath":"$.error" } ], "Next":"Wait", "ResultPath":"$.updateItem" }, "Wait":{ "Type":"Wait", "Seconds":5, "Next":"Get Apartment Status" }, "Get Apartment Status":{ "Type":"Task", "Resource":"arn:aws:states:::dynamodb:getItem", "Parameters":{ "TableName":"apartment_workshop_db", "Key":{ "Id":{ "S.$":"$.input.apartmentId" } } }, "ResultPath":"$.getItem", "Next":"Has Client Made Payment ?" }, "Has Client Made Payment ?":{ "Type":"Choice", "Choices":[ { "And":[ { "Variable":"$.getItem.Item.status.S", "StringEquals":"paid" }, { "Variable":"$.getItem.Item.Id.S", "StringEquals":"1234567" } ], "Next":"Payment Was made." } ], "Default":"Payment Wasn't Made, revert." }, "Payment Was made.":{ "Type":"Pass", "End":true }, "Payment Wasn't Made, revert.":{ "Type":"Task", "Resource":"arn:aws:states:::dynamodb:updateItem", "Parameters":{ "TableName":"apartment_workshop_db", "Key":{ "Id":{ "S":"1234567" } }, "UpdateExpression":"SET #apartmentStatus = :status", "ExpressionAttributeNames":{ "#apartmentStatus":"status" }, "ExpressionAttributeValues":{ ":status":{ "S":"vacant" } } }, "End":true }, "Fail":{ "Type":"Fail", "Error":"Apartment Doesn't Exist", "Cause":"Update Condition Failed" } } } ``` Now, my folder structure looks like this: ![https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/f.png](https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/f.png) ## Create GraphQL API Remember, we have to create a GraphQL API, attach a schema and a database, and connect a Lambda resolver to it. This Lambda would be responsible for invoking the step functions workflow. Open up the `template.yaml` and add this GraphQl API and API key to the resources section ```yaml SamStepFunctionsApi: Type: "AWS::AppSync::GraphQLApi" Properties: Name: SamStepFunctionsApi AuthenticationType: "API_KEY" XrayEnabled: true LogConfig: CloudWatchLogsRoleArn: !GetAtt RoleAppSyncCloudWatch.Arn ExcludeVerboseContent: FALSE FieldLogLevel: ALL ``` We want to see a stream of logs in CloudWatch from AppSync, so let’s create and assign a CloudWatch role to the GraphQL API: ```yaml RoleAppSyncCloudWatch: Type: AWS::IAM::Role Properties: ManagedPolicyArns: - "arn:aws:iam::aws:policy/service-role/AWSAppSyncPushToCloudWatchLogs" AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - sts:AssumeRole Principal: Service: - appsync.amazonaws.com ``` ## GraphQL Schema A GraphQL API always works with a GraphQL schema. In the root directory, create a folder called ‘**graphql**’; inside that folder, create a file called `schema.graphql`. Type in the following ****‘**graphql**’ schema into the file: ```graphql type StepFunctions { id: String! arn: String! } type Query { getStepFunctions: [ StepFunctions! ] } input StepFunctionsInput { id:ID! arn: String! } type Mutation { addStepFunction(input: StepFunctionsInput!): StepFunctions } schema { query: Query mutation: Mutation } ``` This schema has a single mutation `addStepFunction` that sends an input(`id` and `arn`) to a Lambda resolver. The Lambda resolver uses this input to start a step functions execution. Let’s define the schema in `templates.yaml` under resources. ```yaml SamStepFunctionsApiSchema: Type: "AWS::AppSync::GraphQLSchema" Properties: ApiId: !GetAtt SamStepFunctionsApi.ApiId DefinitionS3Location: 'graphql/schema.graphql' ``` ## Create Lambda Function Let’s create a lambda function that we’ll attach to a data source and then attach that data source to an AppSync resolver Inside the `functions/lambda` folder, create a file called `app.py` and type in the following code: ```python import json def lambda_handler(event, context): print("Lambda function invoked") print(json.dumps(event)) print(json.dumps(event["arguments"]['input'])) return {"id": event["arguments"]['input']['id'], "arn": event["arguments"]['input']['arn']} ``` For now, this lambda function simply takes an input(`id` and `arn`) and outputs(`id` and `arn`). Later on, we’ll use this lambda function to start the step functions workflow. Let’s define the Lambda function in `template.yaml` alongside its `role`: ```yaml SamStepFunctionFunction: Type: AWS::Serverless::Function Properties: CodeUri: functions/lambda/ Handler: app.lambda_handler Role: !GetAtt lambdaStepFunctionRole.Arn Runtime: python3.8 Architectures: - x86_64 ``` ```yaml lambdaStepFunctionRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Action: - "sts:AssumeRole" Effect: "Allow" Principal: Service: - "lambda.amazonaws.com" ``` ## Lambda Datasource For this application, we’ll be using Lambda as our data source. Inside `template.yaml` add the following code below resources: ```yaml SamStepFunctionDataSource: Type: "AWS::AppSync::DataSource" Properties: ApiId: !GetAtt SamStepFunctionsApi.ApiId Name: "SamStepFunctionsLambdaDirectResolver" Type: "AWS_LAMBDA" ServiceRoleArn: !GetAtt AppSyncServiceRole.Arn LambdaConfig: LambdaFunctionArn: !GetAtt SamStepFunctionFunction.Arn ``` Since this data source would have to call AppSync, we attach an AppSync service role to it: ```yaml AppSyncServiceRole: Type: "AWS::IAM::Role" Properties: AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: - "appsync.amazonaws.com" Action: - "sts:AssumeRole" ``` ### Create Direct Lambda Resolver Now, we have to create a direct Lambda resolver connecting the mutation in our schema to the Lambda data source we created above. Under resources in `template.yaml`, type in: ```yaml CreateAddStepFunctionsResolver: Type: "AWS::AppSync::Resolver" Properties: ApiId: !GetAtt SamStepFunctionsApi.ApiId TypeName: "Mutation" FieldName: "addStepFunction" DataSourceName: !GetAtt SamStepFunctionDataSource.Name ``` ## Database In our workflow, we save apartment attributes to a database. Let’s go ahead and create the database. It’s a DynamoDB with a single primary key of ID. ```yaml SamStepFunctionsTable: Type: AWS::Serverless::SimpleTable # More info about SimpleTable Resource: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-resource-simpletable.html Properties: PrimaryKey: Name: Id Type: String ProvisionedThroughput: ReadCapacityUnits: 1 WriteCapacityUnits: 1 ``` ## State Machine A couple of steps above, we saved the step functions workflow in a file called `booking_step_functions.asl.json`. We have to create a state machine resource in `template.yml`, link to that file, do some variable substitutions like the DB name, and also provide DynamoDB read and write policies for the `update` and `get` item DynamoDB methods. So let’s go ahead and define the step machine resource under ‘**Resources**’ in `template.yml`: ```yaml SamStepFunctionStateMachine: Type: AWS::Serverless::StateMachine # More info about State Machine Resource: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-resource-statemachine.html Properties: DefinitionUri: statemachine/booking_step_function.asl.json DefinitionSubstitutions: DDBUpdateItem: !Sub arn:${AWS::Partition}:states:::dynamodb:updateItem DDBGetItem: !Sub arn:${AWS::Partition}:states:::dynamodb:getItem DDBTable: !Ref SamStepFunctionsTable Policies: # Find out more about SAM policy templates: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-policy-templates.html - DynamoDBWritePolicy: TableName: !Ref SamStepFunctionsTable - DynamoDBReadPolicy: TableName: !Ref SamStepFunctionsTable ``` After all variable substitutions, the `booking_step_function.asl.json` file looks like this now: ```json { "Comment":"This state machine updates the status of a booked transaction in the DB, waits for payment to be made, and then updates again or passes", "StartAt":"Change Apartment Status", "States":{ "Change Apartment Status":{ "Type":"Task", "Resource":"${DDBUpdateItem}", "Parameters":{ "TableName":"${DDBTable}", "Key":{ "Id":{ "S.$":"$.details.accountId" } }, "ConditionExpression":"attribute_exists(Id)", "UpdateExpression":"SET bookedStatus = :bookedStatus", "ExpressionAttributeValues":{ ":bookedStatus":{ "S.$":"$.details.bookedStatus" } } }, "Next":"Wait", "ResultPath":"$.updateResult", "Catch":[ { "ErrorEquals":[ "States.ALL" ], "Comment":"Items Doesn't Exist", "Next":"Fail", "ResultPath":"$.updateError" } ] }, "Wait":{ "Type":"Wait", "Seconds":60, "Next":"Get Booking Status" }, "Get Booking Status":{ "Type":"Task", "Resource":"${DDBGetItem}", "Parameters":{ "TableName":"${DDBTable}", "Key":{ "id":{ "S.$":"$.details.accountId" } } }, "Next":"Has the Apartment been Paid ?", "ResultPath":"$.getItem", "Catch":[ { "ErrorEquals":[ "States.ALL" ], "Comment":"Couldn't find item", "Next":"Fail" } ] }, "Has the Apartment been Paid ?":{ "Type":"Choice", "Choices":[ { "And":[ { "Variable":"$.getItem.Item.Id.S", "StringEquals":"1234567" }, { "Variable":"$.getItem.Item.bookedStatus.S", "StringEquals":"Paid" } ], "Next":"Apartment Paid" } ], "Default":"Not Paid(Revert Apartment Status)" }, "Not Paid(Revert Apartment Status)":{ "Type":"Task", "Resource":"${DDBUpdateItem}", "Parameters":{ "TableName":"${DDBTable}", "Key":{ "Id":{ "S.$":"$.getItem.Item.Id.S" } }, "UpdateExpression":"SET bookedStatus = :bookedStatus", "ExpressionAttributeValues":{ ":bookedStatus":{ "S":"PENDING" } } }, "End":true, "ResultPath":"$.notPaid" }, "Fail":{ "Type":"Fail" }, "Apartment Paid":{ "End":true, "Type":"Pass" } } } ``` Please Grab the complete code [HERE](https://github.com/trey-rosius/sam_stepfunctions). ## Invoke Step Functions From Lambda Navigate to `functions/lambda/app.py` and type in this code: ```python import json import boto3 step_function_client = boto3.client("stepfunctions") def lambda_handler(event, context): print("Lambda function invoked") print(json.dumps(event)) print(json.dumps(event["arguments"]['input'])) step_function_client.start_execution( stateMachineArn=event["arguments"]['input']['arn'], name=event["arguments"]['input']['id'], input= "{\"details\":{\"accountId\":\"1234567\",\"bookedStatus\":\"Booked\"}}", ) return {"id": event["arguments"]['input']['id'], "arn": event["arguments"]['input']['arn']} ``` We import the step functions class from the boto3 client and use it to start a step functions execution by passing in the StateMachineArn we get from deploying the project, a unique name for the state machine execution, and the state machine input. ## Deploy Deploy the app to your AWS account using `sam build` and `sam deploy` Once deployment is successful, grab the step functions arn and proceed to test in AppSync. ## Testing Sign in to your AWS console and search for AppSync. Open up AppSync and click on your newly deployed AppSync project. ![https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/g.png](https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/g.png) ![https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/j.png](https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/j.png) ![https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/i.png](https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/i.png) ## Conclusion In this post, we built a step functions workflow using Appsync, SAM, and Python. This workflow mimics a real-life scenario of booking/reserving an apartment. - We saw how to invoke Step functions from a Lambda. - We covered defining a state machine in a yaml file with variable substitution. - We broke down how to use IaC to create Applications with Step Functions. In the next post, we’ll invoke the step functions workflow from a mobile frontend application built with amplify and flutter. Stay tuned! ## More Screenshots ![https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/apartment_studio.jpeg](https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/apartment_studio.jpeg) ![https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/stepfunctions_graph.png](https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/stepfunctions_graph.png) ![https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/success.png](https://raw.githubusercontent.com/trey-rosius/sam_stepfunctions/master/assets/success.png)