Amazon EC2 & Scalr – Roles, Instances, Regions, Availibility Zones & ELB

In this blog I will briefly describe some terminologies that you will frequently encounter when deploying an application on Amazon EC2 and while using Scalr for application management in Amazon EC2. I will then describe how these objects work together in harmony. To gain a better understanding of their working we need to first understand what they mean.

Role

A role is a machine image and as the name indicates, it serves a specific purpose of an application in the cloud. Typically a role is an abstraction of an instance (defined next). A role helps in defining a template which consists of a set of installations, needed to fulfil a specific function of an application. For example, a typical application will have web servers, cache servers and data servers. All three can serve as a role (Apache2 + PHP + APC Role, Memcached Role and MySQL Role). Roles are generally assigned to a farm (a set of instances working together to accomplish a task) and have their own security groups. For example, an application role can be made public while caching and database roles should be kept private and internal to the network.

Instances

Unlike a Role, that does not have any physical existence, an Instance is a physical existence of a Role. There can be multiple instances running for a particular role. Roles are templates and Instances are actual implementations of those templates.

Regions

Amazon EC2 infrastructure is spread across the globle in different regions. These regions are geographically seperated and provide an opportunity to run an application in different regions thus making an application fault tolerant. Also application can serve requests to the clients from the closest region. Regions are completely isloated from each other. Following are the regions available in Amazon EC2:

US East (Northern Virginia)

US West (Oregon)

US West (Northern California)

EU (Ireland)

Asia Pacific (Singapore)

Asia Pacific (Tokyo)

South America (Sao Paulo)

Availibility Zones

Availibility Zones are locations within a Region where instances can run. They help in making instances in a region failure proof. We can run instances in a region in one or more availibility zones or distribute the instances equally among the availibility zones. Availibility Zones inside are region are connected to each other.

Elastic Load Balancer

Elastic Load Balance or ELB, as the name indicates distributes incoming traffic among many instances in availability zone or many availibility zones. ELB also checks for unheathy instances in an availability zone and routes the incoming traffic to healthy instances. ELB supports sticky sessions and has the ability to terminate SSL at the balancer level so the application servers do not need to perform SSL decryption. When you launch an ELB in a region, make sure that it routes the traffic to the availability zones that carry instances. By default an ELB will distribute traffic among all availability zones inside a region. Be sure to select only the availablity zones that carry instances, otherwise the application will face timeouts.

Architecture for Scaleable Resource Discovery (Part II)

This is the second part of this post. In Part I, I explained the problem, analysis, architecture and algorithms that were devised to solve the presented problem. In this part I will be presenting a strategy to test the architecture using Amazon EC2. The strategy includes the implementation of architecture using EC2 and then implementation of testing clusters to simulate thousands of requests per second to test the architecture.

Simulation Strategy

Region Design

In order to go about simulating our algorithm, we chose to use Amazon’s Elastic Compute Cloud (EC2) as a base framework for simulating regional data replication and distribution. Amazon’s EC2 service is already broken into seven regions (known as Availability Zones) which would allow us to partially test our geographically distributed Regional Minimum Spanning Tree building algorithm. Distributing data accesses across multiple servers within a cluster or ring can be handled by the Elastic Load Balancing (ELB) feature which also detects and reroutes traffic from unhealthy instances of the data to healthy instances until the unhealthy instances can be restored. The CloudWatch feature would allow us to monitor just how efficient our design is performing. The following diagram shows our planned usage of the Amazon EC2 platform in order to simulate our regional architecture:

Request for a resource when generated by the client is routed to an appropriate Resource Region (present in one Amazon EC2 Region). The request is routed to the region closest to the client's location. This is done automatically by Amazon. The request first lands on the Elastic Load Balancer (ELB) of the Resource Region. The ELB sends the resource request to one of the region servers. This forwarding of requests is based on the ELB's request distribution algorithm. The region servers are auto-scaled which means that the number of servers will increase or decrease as the resource requests increase or decrease. There can be different auto-scaling criteria like bandwidth, requests per second, idle CPU time, etc. Once the request reaches the region server, the server determines the cluster to which the request should be forwarded (based on the resource type). The region server then forwards the request to the ELB of the selected cluster. The cluster ELB then routes the request to one of the cluster servers. The cluster servers are auto-scaled just like the region servers. When the request is received by the cluster server, the cluster server uses consistent hashing to figure out what ring cluster to forward the request to. After identifying the ring cluster, the request is forwarded to the ring cluster ELB. This ELB then forwards the request to any MySQL ring server. MySQL ring server has a web service that takes the resource ID and looks for the resource in the database. It then returns the appropriate response (resourceLocation or notFound). Unlike the other servers (region and cluster), the ring servers are not auto-scaled. For ease of deployment and re-use, we will use server templates for Region Server, Cluster Server, Ring Server. The templates will contain all of the necessary server configuration. All we need to do to add an additional pre-configured server is spawn

another instance of a template. We can also clone the Ring Cluster (Farm Cloning) to create a new Ring Cluster when we need more rings, in the case of new resources being added to the system.

Security

All of the servers will be closed to public access. Only for the reason of region synchronization will the cluster servers be allowed to connect to the cluster server of the neighbor region.

Testing Clusters

Now that the regional architecture is set up, we also need to simulate millions of users accessing the data concurrently across all seven regions of the EC2 platform. The following diagram is an example of how we plan on testing our region architecture described above:

We will create a separate instance of the EC2 platform which will send requests to our regional architecture at a predefined rate. These instances will be known as the Tester instances. The Tester instances will take advantage of predefined tools and use our Tester template to spawn multiple instances of a tester for each region, which will allow us to scale up the number of requests and analyze the network traffic and bandwidth used. This approach allows us to create as many or as few requests per second as we want for testing purposes.