Sitemap

Deep dive into AWS for developers | Part1 — IAM & EC2

13 min readApr 17, 2021

--

AWS has been an revolutionary in powering the IT companies with least possible workforce. AWS is an cloud offering from Amazon, which helps its users to provide the servers and services on demand.

Press enter or click to view image in full size

AWS Regions : AWS has regions all around the world. There are many upcoming regions, marked in orange colour in below map. A Region is nothing, but a cluster of data-centres. Many AWS services are region-scoped, i.e. if we use the same service in more than 1 regions, our data may not be replicated. Naming conventions used for regions are like “us-east-2”, “ap-east-1”, “eu-central-1”, etc.

Press enter or click to view image in full size

AWS Availability Zones (AZ):- Each AWS regions has many availability-zones. Usually its 3, Minimum, it can be 2 and maximum it can be 6. For e.g. For Sydney region, following are the AZs.

Each AZ is one or more discrete data-centres with redundant power, connectivity and networking. Consider each AZ as something like Multiple computers being placed in one big room. Every AZ is geographically separated from each other, so that these are isolated from each other’s disasters. Each AZ (within a single given region) is connected to each other with high bandwidth and ultra-low latency networking.

Next, Please note that, there are many AWS services like IAM, which is linked to Global AWS. On the contrary, services like EC2, etc. are region specific i.e. linked to every region separately.

Press enter or click to view image in full size

IAM :- It stands for ‘Identity & Access Management’. IAM is at centre of AWS security with Users, Groups & Roles playing important roles. ‘Root’ user should never be used except for the initial setup. Using root account, users must be created with proper permissions. Policies are written into JSON. Each IAM-User represents one-actual physical-person. Users are grouped together into various Groups depending upon the permissions that needs to be allotted to the Users. Roles are for the purpose of Internal usage within AWS-resources. Roles are usually assigned to machines. Each IAM-Role per application. Policies dictates, what each of these individual entities can actually do.

  • IAM has a global-view and its applicable for all the regions.
  • Permissions are governed by policies. IAM also provides pre-defined managed policies.
  • It’s in best interest that, Users should be given minimal permissions that they need to perform their jobs. (Least Privilege Principle).
  • IAM credentials should never be shared and most-importantly, IAM credentials should never be written into the code.
Press enter or click to view image in full size

Let’s now do the setup of MFA for root account. I am going ahead with “Google Duo Mobile” android app, there are many options being listed there at AWS site.

Press enter or click to view image in full size

As a next step, we shall be creating our very first IAM user. As recommended above, ‘root’ user is not suggested to be used, therefore I have created an IAM user with name ‘aditya’ for myself as below :-

Press enter or click to view image in full size

We would like to give ‘AdministratorAccess’ policy to this IAM User.

Press enter or click to view image in full size

Here is the final screen, before AWS account is to be created.

Press enter or click to view image in full size

Finally, here is our first IAM user been created. Don’t forget to download the access-key, secret-key & password.

Press enter or click to view image in full size

As a next step, we shall be creating our very first IAM group. As recommended above, groups are used for grouping the same type of users together.

Press enter or click to view image in full size

Next, lets associate the ‘Administrator’ policy with this group.

Press enter or click to view image in full size

Here, is the review screen for group-creation :-

Press enter or click to view image in full size

Let’s now add users to this newly formed Group.

Press enter or click to view image in full size

Finally, here is the summary of the group that we have recently created.

Press enter or click to view image in full size

Please note that, If we assign a particular policy with a group, the same gets activated for all the users within that group OR All the users under a particular-group would automatically inherit all permissions from the group. Thus, In case we want to remove the ‘Administrator’ policy with the ‘aditya’ IAM user as we created above, still there shall be no effect on the same, because this user shall be inheriting the ‘Administrator’ permissions from the group to which, its associated with.

Press enter or click to view image in full size

As a next step, as we learnt that, we shall not be using the root account to login to system, let’s customise the sign-in url for IAM users :-

Press enter or click to view image in full size

We have created following-alias for the root-account, which we shall be using in order to login through IAM users.

Press enter or click to view image in full size
Press enter or click to view image in full size

Further, it would ask us to reset the password, as soon as we login through our newly created IAM user. Pl note that, we shall be using the user-name & password, from the excel sheet, that we recently downloaded.

Press enter or click to view image in full size
Press enter or click to view image in full size

Conclusively, we saw that, how we created an IAM user and logged-in with this user into AWS console.

***************************************************************

EC2 :- Amazon EC2 is the first and foremost starting point to understand, what does AWS has to offer and how does the cloud VM actually works ?

Press enter or click to view image in full size

The first thing that we have to choose for is : AMI i.e. Amazon Machine Image. It is the software that shall be launched on the server we are about to spin. There are multiple types available out here. We shall go with ‘Amazon Linux AMI’ which comes pre-loaded with lot of Amazon related stuff.

Press enter or click to view image in full size

Although Amazon provides lots of options of AMIs to choose from, usually, companies would go for their own AMI, because of lot of reasons like pre-installed packages into the AMI.

Press enter or click to view image in full size

The ‘type’ dictates : how much powerful machine you want, how many number of vCPUs you want along with the Internal-storage in gigs. For now, we shall go with free-tier eligible server option which is ‘t2.micro’.

Press enter or click to view image in full size

In this step, we basically customise our requirement for this new instance to be launched. For now, we shall go with default settings.

Press enter or click to view image in full size

For any machine to booted, it needs disk for the Operating-System to be housed. That’s where we are here : Adding Storage to our new instance. We proceed with default storage volume.

Press enter or click to view image in full size

Next, A security group is a set of firewall rules that control the traffic for our instance. On this page, we can add rules to allow specific traffic to reach to our instance. For example, if we want to set up a web server and allow Internet traffic to reach to our instance, we shall add corresponding rules, that allow unrestricted access to the HTTP and HTTPS ports.

Press enter or click to view image in full size

A key pair consists of a public key that AWS stores, and a private key file that we(owner of the EC2 instance) store. Together, these keys, allow us to connect to our instance securely. For Linux AMIs, the private key file allows us to securely SSH into our instance. We can name our key-pair anything. I am naming it to be: ‘Learning-EC2-AWS’.

Press enter or click to view image in full size

Below is how our instance looks :-

Press enter or click to view image in full size

Let’s now access our newly launched EC2 instance using the private key, we just downloaded from previous step. Wait, we got this permission-error of 0644 for the private key. Let’s understand, what is this. Permission ‘644’ means that, User can RW, Group can R, Others can R.

Press enter or click to view image in full size
Press enter or click to view image in full size

So, to fix this, we change the permissions of this file to 0400. Let’s understand, what permissions it means now. Permission ‘400’ means that, User can R, Group & Others can do nothing.

Press enter or click to view image in full size

Now, we are able to log-into the instance as below :-

Press enter or click to view image in full size

Introduction to AWS Security-Groups :- As of now, we had allowed the inbound-traffic at port no 22. Pl note that, SSH works on TCP protocol. In case, we remove the Inbound-Rule, then no traffic shall be accepted on port no 22 and hence we shall be unable to do SSH onto our instance.

Press enter or click to view image in full size

With following configuration for “Inbound-Rules”, we meant that, SSH type of traffic shall be acceptable on port no. 22 from anywhere in the world to this instance.

Press enter or click to view image in full size
Press enter or click to view image in full size
Press enter or click to view image in full size

Security-Groups are really firewalls sitting outside the EC2-instances i.e. they can make sure what traffic is allowed IN and OUT of the instance. If, cause of some rule, the particular-traffic is not supported, simply EC2 wouldn’t even see that. For e.g. Say we had only allowed traffic from a particular source, and In case, the traffic is being sent through some another IP, that would not at all land at the EC2-instance.

Press enter or click to view image in full size

Let’s see, some more important aspects regarding Security-Groups :-

Press enter or click to view image in full size

Introduction to Public & Private IPs :- Two servers with public IP can very well communicate with each other openly.

Similarly, computers within a private network of any company can talk to each other internally only.

In AWS parlance, if servers/computers of some company wants to talk to servers/computers of the other company, then that communication has to happen via the Internet-Gateways only.

Press enter or click to view image in full size
Press enter or click to view image in full size

Concept of Elastic IPs :- Whenever any instance is stopped & started, the public-IP allocated to it may change. There comes the need of Elastic-IP.

Press enter or click to view image in full size
Press enter or click to view image in full size

Usage of Elastic-IP is quite an uncommon pattern, as it reflects poor architectural decisions. There is also a limit attached to the number of Elastic IPs. Please note that, When we stop & restart an EC2 instance, the private-IP doesn’t changes, only the public IP changes.

Press enter or click to view image in full size

Below is how our EC2 instance looks like, after we attached the Elastic IP with it.

Press enter or click to view image in full size

UseCase : Installation of ‘Apache’ webserver on the EC2 instance :-

  • First we update all our libraries at the EC2 instance :-

[ec2-user@ip-172–31–4–134 ~]$ yum update -y

  • Next, we install the ‘httpd’ server :-

[root@ip-172–31–4–134 ec2-user]# yum install -y httpd.x86_64

  • Next, we start the web-server :-

[root@ip-172–31–4–134 ec2-user]# systemctl start httpd.service

  • Next, we make sure that, service keeps running, even across instance-restarts.

[root@ip-172–31–4–134 ec2-user]# systemctl enable httpd.service

Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.

[root@ip-172–31–4–134 ec2-user]#

  • Next, let’s write some static content to the file, that web-server shall be returning upon being invoked.

[root@ip-172–31–4–134 html]# pwd

/var/www/html

[root@ip-172–31–4–134 html]# ls -lhrt

total 4.0K

-rw-r — r — 1 root root 94 Apr 17 12:11 index.html

[root@ip-172–31–4–134 html]#

  • Now, if we want to access this web-server through web-browser, then it would not be allowed to happen, because our security-group only allows the inbound traffic on port 22 at this instance. So, let’s go ahead and allow the http based traffic on port no. 22 as well by modifying the security-groups as below :-
Press enter or click to view image in full size

Now, if we invoke the web-server running on port-80, through the public IP of our instance, we do get the static-content on the screen, as shown in this snapshot.

Press enter or click to view image in full size

Please note that, all of the above stuff that we just did manually, can all be automated through the use of “EC2-User-data-Script” and therefore, these scripts would automatically gets executed at the time of server boot time.

Press enter or click to view image in full size

Types of EC2 instances :-

In case we don’t know, what is going to be the compute-power required for our instance, but we know that we have a heavy workload, then we can use : ‘Convertible Reserved Instances’.

Press enter or click to view image in full size
Press enter or click to view image in full size

“On-Demand instances” are most costly stuff and hence suitable for only short-term workloads, where application’s behaviour is un-predictable. Analogous to : Unplanned-Hotel-stay, you come and pay-in-full and get a room.

Press enter or click to view image in full size

“Reserved-Instances” are more suitable for hosting the Database-server, since DB servers are expected to run for years and years without interruption. Analogous to : You plan in advance for a long stay in hotel, so you get a discount.

Press enter or click to view image in full size

“Spot-instances” are quite suitable for some ad-hoc jobs like data-analysis and batch-jobs. Absolutely NO for hosting DB servers. Analogous to : Hotel has some empty rooms and it starts process of bidding and highest bidder gets the room, but you may get kicked out anytime.

Press enter or click to view image in full size

“Dedicated-Hosts” are quite suitable for companies, who have stringent regulatory requirements. Analogous to : Reserving the entire building of Hotel for an upcoming stay.

Press enter or click to view image in full size

“Dedicated-instances” are equivalent to dedicated-hosts, but these may share hardware with other VMs into the same account.

Press enter or click to view image in full size

Here, is the price-comparison for this instance compared amongst multiple types of instances.

Press enter or click to view image in full size

EC2 pricing depends upon plethora of factors including region, type, OS, etc. It costs, only when instance is in RUNNING state.

Press enter or click to view image in full size

As a rule of thumb, ‘R’ types of instances comes with lot of RAM and similarly ‘C’ types of instances comes with lot og Compute-power. ‘M’ instances are good at everything.

Press enter or click to view image in full size

Elastic Network Interface :- This is nothing but the “Virtual Network Card”, which can be associated with any EC2 instance independently. If some ENI is moved from one EC2 instance to another, then private IP shall also be changed for those instances.

Press enter or click to view image in full size

See here, for example, we have an EC2 instance and it has both private & public IP. It also has an attached : Network-Interface i.e. “eth0”. This is nothing, but the ENI only. Please note that, each EC2 instance gets its own ENI. Primary ENI is denoted by ‘eth0’ usually.

Press enter or click to view image in full size

Each ENI has its private IP, associated exclusively for this ENI.

Press enter or click to view image in full size

Let’s now create a new ENI and associate that with our EC2 instance. Please note that, the ENI has to be necessarily into the same Availability-Zone, as that of EC2 instance to which we want to associate it.

Press enter or click to view image in full size

We now can observe that, our EC2 instance has 2 ENIs (i.e. 2 Network Interface Cards) associated with it. Pl note that, these ENIs are very much transferable from one instance to another.

Press enter or click to view image in full size

The attachment of multiple ENIs (i.e. Virtual Network Cards) to an EC2 instance can be very well checked from within the instance as well.

Press enter or click to view image in full size

That’s all in this part. We shall explore more about AWS in upcoming parts.

References :-

--

--

aditya goel
aditya goel

Written by aditya goel

Software Engineer for Big Data distributed systems