[[AWS Fargate|Fargate]] integrates pretty tightly with [[AWS Step Functions|Step Functions]] and is a great pairing if you have a workload which requires running an on-demand job for which [[AWS Lambda|Lambda]] would not be a good fit (e.g. it's long-running job). You can launch a Fargate task from a StateMachine state ([see docs](https://docs.aws.amazon.com/step-functions/latest/dg/connect-ecs.html)) without going via Lambda. The nice thing about using StepFunctions with containers is that it gives you automatic error and retry handling if your container fails. It also allows you to add additional logic for your use case using Lambda functions (which tend to be easier to author and deploy) rather than having to add changes to your container. A general principle I like to follow is to keep the responsibility of the container to a minimum. ## Passing parameters into your container In order to pass parameters into your container, there are two ways to do this. - command line arguments top the container entrypoint - environment variables The following example shows how these could be supplied to your state machine (see the `ContainerOverrides` section): ```yml RunScreamingFrogContainer: Type: Task Resource: arn:aws:states:::ecs:runTask.sync Parameters: LaunchType: FARGATE Cluster: ${cf:${self:custom.fargateClusterStackName}.Cluster} TaskDefinition: !Ref ScreamingFrogTask NetworkConfiguration: AwsvpcConfiguration: Subnets: - ${cf:${self:custom.vpcStackName}.SubnetAPublic} - ${cf:${self:custom.vpcStackName}.SubnetBPublic} AssignPublicIp: ENABLED SecurityGroups: - !Ref ServiceSecurityGroup Overrides: ContainerOverrides: - Name: app 'Command.: '$.screamingFrogInput.commands' Environment: - Name: OUTPUT_FOLDER 'Value.: '$.screamingFrogInput.outputFolder' - Name: SF_LICENSE_USER 'Value.: '$.screamingFrogInput.license.user' - Name: SF_LICENSE_KEY 'Value.: '$.screamingFrogInput.license.key' ``` ## Resource list Here is the full set of AWS resources that are needed when for this use case: - [[AWS ECS]] `TaskDefinition` and associated [[AWS IAM|IAM]] role - [[AWS Step Functions|Step Functions]] `StateMachine` - [[AWS VPC|VPC]] & 2+ subnets - `SecurityGroup` - [[AWS CloudWatch|CloudWatch]] `LogGroup` for storing the container logs Optional resources: - [[AWS EFS|EFS]] `FileSystem`, `AccessPoint` and `MountTarget`s - if your container needs to read or write data from a mounted volume that you need to access once it terminates Note that you don't need [[AWS ECS]] `Service` or `LoadBalancer` resources when running a Fargate task on-demand, these are only needed for always-on containers. ## [[AWS Wishlist]] - [ ] Fargate to allow auto-mounting of files from [[AWS S3|S3]]. Currently only supports mounting from [[AWS EFS]] which involves more moving parts. See [this Twitter discussion](https://twitter.com/paulswail/status/1433087076128464902) --- ## References - [Cloudonaut Fargate CloudFormation templates](https://templates.cloudonaut.io/en/stable/fargate/) - robust [[AWS CloudFormation|CloudFormation]] templates you can use to create the resources listed above. - [Mount Your AWS EFS Volume Into AWS Lambda With the Serverless Framework](https://medium.com/swlh/mount-your-aws-efs-volume-into-aws-lambda-with-the-serverless-framework-470b1c6b1b2d)