Everyone is talking about high available systems. Today we are going to build high available design in Amazon Web Services. To provision resources I am going to use Terraform. I hope you have prior experience working with Terraform. In this example I am not going to talk about database layer, I am planning to extend this post with db layer in future.  First, we’ll look at what are the resources we are going to provision during this example.

Resources We Are Going To Provision

  • VPC
  • Subnets
  • Nat Gateway
  • EC2 Instances
  • Application Load Balancers
  • Security Groups
  • Network Access Control Lists (NACL)

Architecture

Architecture

Create the VPC and Subnets

Firstly, we are creating the subnet in the us-east-1 regions. Lets see the terraform code for creating the VPC.

provider.tf

    provider "aws" {
      version = "~> 2.0"
      region  = "us-east-1"
    }

variables.tf

    variable "cidr_vpc" {
      description = "CIDR for our VPC"
      default     = "10.1.0.0/16"
    }

    variable "environment" {
      description = "Environment of the resources"
      default     = "Dev"
    }

    variable "cidr_public_subnet_a" {
      description = "Subnet fot the public subnet"
      default     = "10.1.0.0/24"
    }

    variable "cidr_public_subnet_b" {
      description = "Subnet fot the public subnet"
      default     = "10.1.1.0/24"
    }

    variable "cidr_app_subnet_a" {
      description = "Subnet fot the public subnet"
      default     = "10.1.2.0/24"
    }

    variable "cidr_app_subnet_b" {
      description = "Subnet fot the public subnet"
      default     = "10.1.3.0/24"
    }

    variable "az_a" {
      description = "Availablilty zone for the subnet"
      default     = "us-east-1a"
    }

    variable "az_b" {
      description = "Availablilty zone for the subnet"
      default     = "us-east-1b"
    }

Throughout my example I am using above variables to provision resources. Those variable contains what are the availability zones and the cidr ranges for my example.

vpc.tf

    resource "aws_vpc" "wordpress_vpc" {
      cidr_block           = "${var.cidr_vpc}"
      enable_dns_support   = true
      enable_dns_hostnames = true
      tags = {
        Name        = "wordpress-vpc",
        Environment = "${var.environment}"
      }
    }

I created a vpc named wordpress-vpc, and I am using 10.1.0.0/16 cidr range for the vpc.

Public Subnet and Private Subnet

“If a subnet is associated with a route table that has a route to an internet gateway, it’s known as a public subnet. If a subnet is associated with a route table that does not have a route to an internet gateway, it’s known as a private subnet.”

I am creating two public subnets in two different AZs  (us-east-1a,us-east-1b) for bastion host and to other public resources like NAT gateway. And I make sure that public IP will be assign for the instances which will be launched in this subnet* (map_public_ip_on_launch = true)*.

    resource "aws_subnet" "public_subnet_a" {
      vpc_id                  = "${aws_vpc.wordpress_vpc.id}"
      cidr_block              = "${var.cidr_public_subnet_a}"
      map_public_ip_on_launch = "true"
      availability_zone       = "${var.az_a}"

      tags = {
        Name        = "public-a",
        Environment = "${var.environment}"
      }

      depends_on = ["aws_vpc.wordpress_vpc"]
    }

    resource "aws_subnet" "public_subnet_b" {
      vpc_id                  = "${aws_vpc.wordpress_vpc.id}"
      cidr_block              = "${var.cidr_public_subnet_b}"
      map_public_ip_on_launch = "true"
      availability_zone       = "${var.az_b}"

      tags = {
        Name        = "public-b",
        Environment = "${var.environment}"
      }
      depends_on = ["aws_vpc.wordpress_vpc"]
    }

Then create two other subnets to host application (web servers). These will be private subnets, where instances deployed here does not get a public IP address.

    resource "aws_subnet" "app_subnet_a" {

      vpc_id     = "${aws_vpc.wordpress_vpc.id}"
      cidr_block = "${var.cidr_app_subnet_a}"
      availability_zone = "${var.az_b}"

      tags = {
        Name        = "app-a",
        Environment = "${var.environment}"
      }
      depends_on = ["aws_vpc.wordpress_vpc"]
    }


    resource "aws_subnet" "app_subnet_b" {

      vpc_id     = "${aws_vpc.wordpress_vpc.id}"
      cidr_block = "${var.cidr_app_subnet_b}"
      availability_zone = "${var.az_b}"

      tags = {
        Name        = "app-b",
        Environment = "${var.environment}"
      }
      depends_on = ["aws_vpc.wordpress_vpc"]
    }

Now we have already created subnets which are required for us to provision EC2 instances. Now lets create a Internet Gateway (IGW) so that instances in our VPC can communicate with public endpoints.

    resource "aws_internet_gateway" "wordpress_igateway" {
      vpc_id = "${aws_vpc.wordpress_vpc.id}"

      tags = {
        Name        = "wordpress-igateway"
        Environment = "${var.environment}"
      }

      depends_on = ["aws_vpc.wordpress_vpc"]
    }

Then we need to create a route-table and a route to IGW. Here we adding a route to 0.0.0.0/0 traffic to go via IGW. And attached that route to the created route table.

    resource "aws_route_table" "rtb_public" {

      vpc_id = "${aws_vpc.wordpress_vpc.id}"
      tags = {
        Name        = "wordpress-public-routetable"
        Environment = "${var.environment}"
      }
      depends_on = ["aws_vpc.wordpress_vpc"]
    }

    # Create a rout in the roite table, to access public via internet gateway
    resource "aws_route" "route_igw" {
      route_table_id         = "${aws_route_table.rtb_public.id}"
      destination_cidr_block = "0.0.0.0/0"
      gateway_id             = "${aws_internet_gateway.wordpress_igateway.id}"

      depends_on = ["aws_internet_gateway.wordpress_igateway"]
    }

So now our route table and route is crated, now we need to mention which subnets should use this route table. As per our architecture we want our both public subnets to associate with IGW.  Therefore we are attaching

    resource "aws_route_table_association" "rta_subnet_association_puba" {
      subnet_id      = "${aws_subnet.public_subnet_a.id}"
      route_table_id = "${aws_route_table.rtb_public.id}"

      depends_on = ["aws_route_table.rtb_public"]
    }

    resource "aws_route_table_association" "rta_subnet_association_pubb" {
      subnet_id      = "${aws_subnet.public_subnet_b.id}"
      route_table_id = "${aws_route_table.rtb_public.id}"

      depends_on = ["aws_route_table.rtb_public"]
    }

Create bastion server to connect to the environment.

We are using a bastion server, so we can ssh and troubleshoot the vms that deployed in private subnets.

Create a key pair so I can use my ssh key to, ssh to the EC2 instances.

    resource "aws_key_pair" "myec2key" {
      key_name   = "prageeshaPublicKey"
      public_key = "${file("/home/prageesha/.ssh/id_rsa.pub")}"
    }

Create a security group to allow connection to port 22 from the anywhere. In production deployment make sure its only allow your own cidr blocks.

    resource "aws_security_group" "sg_22" {

      name   = "sg_22"
      vpc_id = "${aws_vpc.wordpress_vpc.id}"

      ingress {
        from_port   = 22
        to_port     = 22
        protocol    = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
      }


      egress {
        from_port   = "0"
        to_port     = "0"
        protocol    = "-1"
        cidr_blocks = ["0.0.0.0/0"]
      }

      tags = {
        Name        = "sg-22"
        Environment = "${var.environment}"
      }

    }

Create NACL, inbound and outbound rules for allowing traffic to port 22

    resource "aws_network_acl" "wordpress_public_a" {
      vpc_id = "${aws_vpc.wordpress_vpc.id}"

      subnet_ids = ["${aws_subnet.public_subnet_a.id}"]

      tags = {
        Name        = "acl-wordpress-public-a"
        Environment = "${var.environment}"
      }
    }


    resource "aws_network_acl_rule" "inbound_rule_22" {
      network_acl_id = "${aws_network_acl.wordpress_public_a.id}"
      rule_number    = 200
      egress         = false
      protocol       = "-1"
      rule_action    = "allow"
      # Opening to 0.0.0.0/0 can lead to security vulnerabilities.
      cidr_block = "0.0.0.0/0"
      from_port  = 0
      to_port    = 0
    }

    resource "aws_network_acl_rule" "outbound_rule_22" {
      network_acl_id = "${aws_network_acl.wordpress_public_a.id}"
      rule_number    = 200
      egress         = true
      protocol       = "-1"
      rule_action    = "allow"
      # Opening to 0.0.0.0/0 can lead to security vulnerabilities.
      cidr_block = "0.0.0.0/0"
      from_port  = 0
      to_port    = 0
    }

EC2 instance for bastion server.

    resource "aws_instance" "wordpress_bastion" {
      ami                    = "ami-0a887e401f7654935"
      instance_type          = "t2.micro"
      subnet_id              = "${aws_subnet.public_subnet_a.id}"
      vpc_security_group_ids = ["${aws_security_group.sg_22.id}"]
      key_name               = "${aws_key_pair.myec2key.key_name}"

      tags = {
        Name        = "wordpress-bastion"
        Environment = "${var.environment}"
      }

    }

Create NAT gateway

Our web server instances are will be deployed in private subnet where you dont have internet connectivity. In order to communicate with public for example to do yum updates or install any packages we use NAT gateways. Since we are deploying to 2 availability zones make sure you deploy 2 NAT gateways in each AZ for high availability.

NAT Gateway for public-a subnet

    resource "aws_eip" "eip_public_a" {
      vpc = true
    }
    resource "aws_nat_gateway" "gw_public_a" {
      allocation_id = "${aws_eip.eip_public_a.id}"
      subnet_id     = "${aws_subnet.public_subnet_a.id}"

      tags = {
        Name = "wordpress-nat-public-a"
      }
    }

Create route table and add routes to NAT gateway and attach the public-a subnet to that route table.

    resource "aws_route_table" "rtb_appa" {

      vpc_id = "${aws_vpc.wordpress_vpc.id}"
      tags = {
        Name        = "wordpress-appa-routetable"
        Environment = "${var.environment}"
      }

    }


    resource "aws_route" "route_appa_nat" {
      route_table_id         = "${aws_route_table.rtb_appa.id}"
      destination_cidr_block = "0.0.0.0/0"
      nat_gateway_id         = "${aws_nat_gateway.gw_public_a.id}"
    }


    resource "aws_route_table_association" "rta_subnet_association_appa" {
      subnet_id      = "${aws_subnet.app_subnet_a.id}"
      route_table_id = "${aws_route_table.rtb_appa.id}"
    }

Similarly we create a another NAT gateway and route table for public-b subnet.

    resource "aws_eip" "eip_public_b" {
      vpc = true
    }
    resource "aws_nat_gateway" "gw_public_b" {
      allocation_id = "${aws_eip.eip_public_b.id}"
      subnet_id     = "${aws_subnet.public_subnet_b.id}"

      tags = {
        Name = "wordpress-nat-public-b"
      }
    }

    resource "aws_route_table" "rtb_appb" {

      vpc_id = "${aws_vpc.wordpress_vpc.id}"
      tags = {
        Name        = "wordpress-appb-routetable"
        Environment = "${var.environment}"
      }

    }

    resource "aws_route" "route_appb_nat" {
      route_table_id         = "${aws_route_table.rtb_appb.id}"
      destination_cidr_block = "0.0.0.0/0"
      nat_gateway_id         = "${aws_nat_gateway.gw_public_b.id}"
    }


    resource "aws_route_table_association" "rta_subnet_association_appb" {
      subnet_id      = "${aws_subnet.app_subnet_b.id}"
      route_table_id = "${aws_route_table.rtb_appb.id}"
    }

Creating web servers

create a security group and allow traffic to the EC2 instances

    resource "aws_security_group" "sg_wordpress" {

      name   = "sg_wordpress"
      vpc_id = "${aws_vpc.wordpress_vpc.id}"
      tags = {
        Name        = "sg-wordpress"
        Environment = "${var.environment}"
      }

    }

    resource "aws_security_group_rule" "allow_all" {
      type              = "ingress"
      cidr_blocks       = ["10.1.0.0/24"]
      to_port           = 0
      from_port         = 0
      protocol          = "-1"
      security_group_id = "${aws_security_group.sg_wordpress.id}"
    }

    resource "aws_security_group_rule" "outbound_allow_all" {
      type = "egress"

      cidr_blocks       = ["0.0.0.0/0"]
      to_port           = 0
      from_port         = 0
      protocol          = "-1"
      security_group_id = "${aws_security_group.sg_wordpress.id}"
    }

Create EC2 instances for web server and install apache2 on those VMs

    resource "aws_instance" "wordpress_a" {
      ami                    = "ami-0a887e401f7654935"
      instance_type          = "t2.micro"
      subnet_id              = "${aws_subnet.app_subnet_a.id}"
      vpc_security_group_ids = ["${aws_security_group.sg_wordpress.id}"]
      key_name               = "${aws_key_pair.myec2key.key_name}"

      user_data = <<-EOF
                  #!/bin/bash
                  sudo yum update -y
                  sudo yum install -y httpd
                  sudo service httpd start
                  sudo echo "<html> <h1> Server A </h1> </html>" > /var/www/html/index.html
                 EOF

      tags = {
        Name        = "prageesha-wordpress-a"
        Environment = "${var.environment}"
      }

    }


    resource "aws_instance" "wordpress_b" {
      ami                    = "ami-0a887e401f7654935"
      instance_type          = "t2.micro"
      subnet_id              = "${aws_subnet.app_subnet_b.id}"
      vpc_security_group_ids = ["${aws_security_group.sg_wordpress.id}"]
      key_name               = "${aws_key_pair.myec2key.key_name}"

      user_data = <<-EOF
                  #!/bin/bash
                  sudo yum update -y
                  sudo yum install -y httpd
                  sudo service httpd start
                  sudo echo "<html> <h1> Server B </h1> </html>" > /var/www/html/index.html
                 EOF

      tags = {
        Name        = "prageesha-wordpress-b"
        Environment = "${var.environment}"
      }

    }

Creating an Application Load Balancer for web servers.

Create a security group to allow traffic to port 80. (For this demo we are not using SSL so only port 80 will be exposed)

    resource "aws_security_group" "sg_application_lb" {

      name   = "sg_application_lb"
      vpc_id = "${aws_vpc.wordpress_vpc.id}"

      ingress {
        # TLS (change to whatever ports you need)
        from_port = 80
        to_port   = 80
        protocol  = "tcp"
        # Please restrict your ingress to only necessary IPs and ports.
        # Opening to 0.0.0.0/0 can lead to security vulnerabilities.
        cidr_blocks = ["0.0.0.0/0"]
      }


      egress {
        from_port   = "0"
        to_port     = "0"
        protocol    = "-1"
        cidr_blocks = ["0.0.0.0/0"]
      }

      tags = {
        Environment = "${var.environment}"
      }

    }

Create a aws loadbalcer in two public subnets.

    resource "aws_lb" "lb_wordpress" {
      name               = "wordpress-elb"
      internal           = false
      load_balancer_type = "application"
      subnets            = ["${aws_subnet.public_subnet_a.id}", "${aws_subnet.public_subnet_b.id}"]
      security_groups    = ["${aws_security_group.sg_application_lb.id}"]
      enable_deletion_protection = false
      tags = {
        Environment = "${var.environment}"
      }
    }

Create a load-balancer listener  that listen on port 80 and forwarded it to our target group.

    resource "aws_lb_listener" "front_end" {
      load_balancer_arn = "${aws_lb.lb_wordpress.arn}"
      port              = "80"
      protocol          = "HTTP"

      default_action {
        type             = "forward"
        target_group_arn = "${aws_lb_target_group.wordpress_vms.arn}"
      }
    }

Create a target group.

    resource "aws_lb_target_group" "wordpress_vms" {
      name     = "tf-wordpress-lb-tg"
      port     = 80
      protocol = "HTTP"
      vpc_id   = "${aws_vpc.wordpress_vpc.id}"
    }

Add our web servers to the target group.

    resource "aws_lb_target_group_attachment" "wordpressa_tg_attachment" {
      target_group_arn = "${aws_lb_target_group.wordpress_vms.arn}"
      target_id        = "${aws_instance.wordpress_a.id}"
      port             = 80
    }


    resource "aws_lb_target_group_attachment" "wordpressb_tg_attachment" {
      target_group_arn = "${aws_lb_target_group.wordpress_vms.arn}"
      target_id        = "${aws_instance.wordpress_b.id}"
      port             = 80
    }

When you create these Terraform resources and do apply you will get a dns name for your load balancer. Using that endpoint you can access your web server. You can notice that traffic will be routed to Webserver A and Webserver B

Terraform scripts for this post