Amazon Elastic Container Service (ECS) is a fully managed container orchestration service that allows you to run applications on Docker containers. This is the second part of a series on automating the creation of an AWS cluster using CloudFormation. This tutorial brings all the parts together. At the end of this tutorial, you will have a complete ECS cluster created and configured using CloudFormation.

In the previous tutorials, I explained the series’ goal and how to create VPC, SG, Load Balancer, and IAM roles. I highly recommend you read those tutorials before starting this one because the resources created in this tutorial will use resources from previous tutorials.

Create AWS ECS Cluster using CloudFormation. Part I: VPCCreate AWS ECS Cluster using CloudFormation. Part II: Security Groups, IAM Roles, and Load Balancer

Just as a short summarisation, I will create an ECS cluster during this series that will look like this:

ECS cluster

Finally, we will define the cluster using the following template:

AWSTemplateFormatVersion: ‘2010-09-09’
Description: >
This template defines:
– An empty ECS CLuster
– Capacity provider
– Autoscailing Group
– Instance profile
– Launch template
Parameters:
EnvironmentName:
Description: An environment name that will be prefixed to resource names
Type: String
Default: dev
InstanceType:
Type: String
Default: t2.small
Description: Class of EC2 instance used to host containers.
AllowedValues:
– t2.micro
– t2.small
DesiredCapacity:
Type: Number
Default: ‘1’
Description: Number of EC2 instances to launch in your ECS cluster.
MinSize:
Type: Number
Default: ‘1’
Description: Minimum number of EC2 instances that can be launched in your ECS cluster.
MaxSize:
Type: Number
Default: ‘1’
Description: Maximum number of EC2 instances that can be launched in your ECS cluster.
ECSAMI:
Description: The Amazon Machine Image ID used for the cluster
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
Default: /aws/service/ecs/optimized-ami/amazon-linux-2023/recommended/image_id

Resources:
# ECS Resources
ECSCluster:
Type: AWS::ECS::Cluster
Properties:
ClusterSettings:
– Name: containerInsights
Value: enabled
Tags:
– Key: Name
Value: !Ref EnvironmentName

# Autoscaling group. This launches the actual EC2 instances that will register
# themselves as members of the cluster, and run the docker containers.
ECSAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
DeletionPolicy: Delete
Properties:
VPCZoneIdentifier: !Split [“,”, !ImportValue ECSClusterVPCPrivateSubnets] LaunchTemplate:
LaunchTemplateId: !Ref ContainerInstances
Version: !GetAtt ContainerInstances.LatestVersionNumber
MinSize: !Ref MimSize
MaxSize: !Ref MaxSize
DesiredCapacity: !Ref DesiredCapacity
Tags:
– Key: Name
PropagateAtLaunch: true
Value: !Ref EnvironmentName

# The config for each instance that is added to the cluster
ContainerInstances:
Type: AWS::EC2::LaunchTemplate
Properties:
LaunchTemplateData:
ImageId: !Ref ECSAMI
InstanceType: !Ref InstanceType
IamInstanceProfile:
Name: !Ref EC2InstanceProfile
SecurityGroupIds:
– !ImportValue ECSClusterSecurityGroup
UserData:
# This injected configuration file is how the EC2 instance
# knows which ECS cluster on your AWS account it should be joining
Fn::Base64:
!Sub |
#!/bin/bash
echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
# Disable IMDSv1, and require IMDSv2
MetadataOptions:
HttpEndpoint: enabled
HttpTokens: required

EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles:
– !ImportValue ECSClusterIAMRole

# Create an ECS capacity provider to attach the ASG to the ECS cluster
# so that it autoscales as we launch more containers
CapacityProvider:
Type: AWS::ECS::CapacityProvider
Properties:
AutoScalingGroupProvider:
AutoScalingGroupArn: !Ref ECSAutoScalingGroup
ManagedScaling:
InstanceWarmupPeriod: 60
MinimumScalingStepSize: 1
MaximumScalingStepSize: 100
Status: ENABLED
# Percentage of cluster reservation to try to maintain
TargetCapacity: 100

# Create a cluster capacity provider assocation so that the cluster
# will use the capacity provider
CapacityProviderAssociation:
Type: AWS::ECS::ClusterCapacityProviderAssociations
Properties:
CapacityProviders:
– !Ref CapacityProvider
Cluster: !Ref ECSCluster
DefaultCapacityProviderStrategy:
– Base: 0
CapacityProvider: !Ref CapacityProvider
Weight: 1
Outputs:
ClusterName:
Description: The ECS cluster into which to launch resources
Value: !Ref ECSCluster

In the parameters section, I’ve defined the environment name and the instance type that will be used for EC2 instances. For the testing, I’ve defined the cheapest instances; you can always modify them according to your requirements. How to work with parameters you can find here:

AWS CloudFormation Parameters

I’ve extracted values such as a cluster’s minimum, maximum, and desired capacity so they can be changed easily. The EC2 image ID is also defined in the Parameter section and will be automatically retrieved. It can also be overridden.

The Resource section defines the Cluster and the AutoScaling group, which is responsible for launching EC2 instances and registering as a member of a cluster.

For the configuration of EC2 instances, I’ve defined a LaunchTemplate. This resource will use the security group defined in previous tutorials. User data contains a special code that will be added to each instance so the instance will know the cluster name it has to register itself.

For each instance, we have to create an Instance profile that will use the AIM role created in a previous tutorial.

The capacity provider will attach Auto Scaling Group to a cluster, which will then handle scaling out events when services running on the cluster require more computing power or scaling in events when demand for computing power goes down.

The diagram will look like this:

The complete code can be found here:

GitHub – polovyivan/aws-cloudformation-ecs-cluster

Execute template

Let’s execute the template and create resources defined in it. Go to AWS Console, search for CloudFormation, and on the CloudFormation page, click Create a stack with new resources. Make sure all resources from previous tutorials are created. Select the template

Fill out the name, select the parameters, and press Next, Next. I changed the default parameters that define cluster size to 3 so that the instance will be launched in each AZ. For tests, you can keep it 1, which will cost you less. Mark the checkbox I acknowledge that AWS CloudFormation might create IAM resources and press submit.

The stack is created after a couple of minutes. This template contains many resources, and their creation may take a while, so be patient.

We can check a cluster on the ECS Cluster page.

Click on it and check infrastructure:

Create a Task Definition

Go to ECS and select Task definition and press Create.

Fill up the name and select Launch Type as EC2 instances.

Set a network as default, and select IAM roles. In a real live project, you will have two separate roles for task and task execution, but for sick of simplicity, I will use the same. Set the memory. The memory should be less than the memory of the instance. Otherwise, it won’t work. If you end up in a situation where your Service is stuck in a provisioning state, check the memory of your Task definition and lower it if it is more than the EC2 instance has available.

Configure a container. Leave the Host port as zero; it will be automatically selected by the Load Balancer. For this test, we will use a Nginx image.

Uncheck Use log collection option. In a real live example, you will need this to be enabled, but this will require more stuff to be done, such as the right AIM permission and logging configuration. For this example, I will keep it simple. Press Create and select Create a task.

Deploy a Service

Go to your cluster, and on the service tab, press Create.

The environment section is left as is.

Select the application type as a Service, select the previously created task in a Family drop-down box, and give the service a name. Then, configure three replicas, one for each instance.

Select the previously created load balancer and configure the listener for a port different than 80 because this port was taken by the default listener defined in the template.

For the rest of the settings, leave it as is, and press Create.

At first, the service will be in a provisioning state, but after a few minutes, it will change to a Running status.

All three instances are in a running state.

Now, let’s test it. Click on the service.

Go to the Listener protocol port.

Press on a Load Balancer name.

and copy DNS

Past the DNS to your browser and append a port number configured during a service deployment. Press Enter, and you will see a default page of Nginx. Congratulations you get here, and now you have a complete ECS cluster!!!

The final diagram looks like this:

To delete a service, you can go to CloudFormation, select the service, and click the Delete button.

In the same way, you can delete all stacks in the order they were created.

Conclusion

In this tutorial, we finally created an ECS cluster and deployed a service. As you can see, CloudFormation simplifies a lot a deployment of multiple resources. Now, templates can be reused. This is the last part of the series but not the final one. In the next one, I will show you how to deploy everything together using nested stacks, so stay tuned.

Create AWS ECS Cluster using CloudFormation. Part IV: Nested Sacks

Thank you for reading! Please like and follow. If you have any questions or suggestions, please feel free to write in the comment section or reach out to me on my LinkedIn account.

Create AWS ECS Cluster using CloudFormation. Part III: ECS Cluster was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.

​ Level Up Coding – Medium

about Infinite Loop Digital

We support businesses by identifying requirements and helping clients integrate AI seamlessly into their operations.

Gartner
Gartner Digital Workplace Summit Generative Al

GenAI sessions:

  • 4 Use Cases for Generative AI and ChatGPT in the Digital Workplace
  • How the Power of Generative AI Will Transform Knowledge Management
  • The Perils and Promises of Microsoft 365 Copilot
  • How to Be the Generative AI Champion Your CIO and Organization Need
  • How to Shift Organizational Culture Today to Embrace Generative AI Tomorrow
  • Mitigate the Risks of Generative AI by Enhancing Your Information Governance
  • Cultivate Essential Skills for Collaborating With Artificial Intelligence
  • Ask the Expert: Microsoft 365 Copilot
  • Generative AI Across Digital Workplace Markets
10 – 11 June 2024

London, U.K.