AWS – Storage




  • A critical component of cloud computing is cloud storage that holds the information used by applications.

AWS offers a complete range of cloud storage services to support both application and archival compliance requirements.

Amazon S3 Basics

  • In Amazon S3, data is stored as objects within buckets.
  • The file can be uploaded in order to store object to a bucket.
  • Permissions can be set on the object as well as any metadata in order to upload a file.
  • Buckets acts as containers for objects and one can have one or more buckets.
  • One can control access, view access logs and its objects, and choose the geographical region where S3 will store each bucket and their contents.

Click on Get started button to begin.

Amazon S3 storage

Storage is of two types:

Block storage and file storage

Block storage : raw storage device level, manages data as a set of numbered , fixed size blocks.

File storage : higher level – operating system level and manages data as a named hierarchy of files and folders.


Object  storage

  • In Amazon S3 , instead of managing data as blocks or files using SCSI, CIFS or NFS protocols , data is managed as objects using an API built on standard HTTP levels.
  • Each Amazon S3 object contains both data and metadata.
  • Objects reside in containers called buckets.
  • Identification of each object by a unique user specified key ( file- name).
  • 1 object = 0 byte – 5TB
  • 1 bucket = unlimited number of objects.

Amazon S3 operations

  • S3 API is intentionally simple .
  • It has list of common operations :

create/ delete a bucket

Write an object

Read an object

Delete an object

List keys in a bucket

Object life cycle management

  • Data – natural lifecycle : “hot” – frequently accessed data,

“Warm”- less frequently accessed data

“ cold” – long term backup and archive

Using Amazon S3 lifecycle configuration rules, can reduce storage costs by automatically transitioning data from one storage class to another or eventually deleting data after a period of time.

Amazon S3 key features

  • Simplicity
  • Durability
  • Scalability
  • Security
  • Broad integration with other AWS services.
  • Cloud data migration options
  • Enterprise- class storage management

Sign up for Amazon S3

  • AWS account is needed to use Amazon S3.
  • To sign up for Amazon S3

○Go to and choose Get started with Amazon S3.

○Follow the on screen instructions

  • Create a bucket
  • Add an object to a bucket
  • View an object
  • Delete an object

Sign up for Amazon S3

Follow practical exercises

  • Ex 2.1 : to create S3 bucket
  • Ex 2.2 : to add and make an object public
  • Ex 2.3 : to enable version of an object
  • Ex 2.4 : to enable life cycle management
  • Ex 2.5 : to enable static web hosting on a bucket
  • Ex 2.6 : to enable web hosting.

Amazon S3 storage management

  • It allow customers to take a data driven approach to storage optimization, compliance and management efficiency.
  • These features work together to help improve workload performance, streamline business process workflows .
  • It enable more intelligent storage tiering to optimize storage costs and performance.

Storage management

  • S3 object tagging : you can manage and control access for Amazon S3 objects. S3 object tags are key value pairs applied to S3 objects which can be created, updated or deleted at any time during the lifetime of an object.
  • S3 inventory : speed up business workflows and big data jobs . it supports CSV format.
  • S3 analytics – storage class analysis : you can monitor the access frequency of the objects within S3 bucket in order to transition less frequently accessed storage to help you transition the right objects to S3 standard-IA.
  • S3 analytics – storage class analytics : with this, you can monitor the access frequency of the objects within S3 bucket in order to transition less frequently accessed storage to a lower cost storage class.
  • S3 cloudwatch metrics : it helps to improve end user experience by providing end user monitoring and alarming on a host of different metrics. 1 min metrics are available at bucket level.
  • Data lifecycle management : assigns and change cost performance as data evolves.
  • Cross region replication : it replicates every object uploaded to your source bucket to a destination bucket in AWS region.

Data durability and reliability

  • It provides highly durable storage infrastructure designed for mission – critical and primary data storage.’

Standard is :

  • Backed with the Amazon S3 service level agreement for availability.
  • Designed for 99.99999% durability and 99.99999% availability of objects over a given period.
  • Designed to sustain concurrent loss of data.Standard – infrequent access  is :
    • Backed with the Amazon S3 service level agreement for availability.
    • Designed for 99.99999% durability and 99.99999% availability of objects over a given period.
    • Designed to sustain concurrent loss of data.

    Amazon Glacier is :

    • Designed for 99.99999% durability of objects over a given period.
    • Designed to sustain concurrent loss of data.

Transferring large amounts of data

Amazon has a set of tools that makes migrating data into cloud faster , including ways to optimize or replace your network , and ways to integrate existing workflows with S3.

S3 transfer acceleration : it is designed to maximize transfer speeds to Amazon S3 buckets over long distances. It works by carrying HTTP and HTTPS traffic over a highly optimized network bridge that runs between AWS edge location nearest clients and S3 bucket.

AWS snowball, snowball edge and snowmobile : used for large scale data transfers including high network costs, long transfer times & security concerns.

AWS storage gateway : data or storage systems that exist on – premises can be easily linked to Amazon S3 using AWS storage gateway.

3rd party partner integration : a number of ISV partners are integrated with Amazon S3 for simplified data transfers and retrievals.

Security and Access management

Data stored in Amazon S3 is secure by default, only bucket and object owners have access to S3 resources they create. It supports multiple access control mechanisms as well as encryption for both secure transit and secure storage at rest.

Flexible access control mechanisms : it provides four different access control mechanisms : AWS IAM (identity and access management ) policies , ACLs (Access control lists ) , bucket policies and query string authentication .

VPC endpoints : they are easy to configure and provide reliable connectivity to Amazon S3 without requiring an internet gateway or a NAT (network address translation ) instance. With these endpoints , the data between an Amazon VPC and Amazon S3 is transferred within Amazon network , helping protect your instances from internet traffic.

Encryption : when Amazon S3 SSE encrypts data at rest, it uses Advanced encryption standard (AES) 256 bit encryption keys . for server side encryption keys are:

  • SSE-S3, SSE- C, SSE- KMS

Audit logs : it also supports logging of requests made against your S3 resources. These server access logs capture all requests made against a bucket or the objects in it and can be used for auditing purposes.

Versioning : it provides further protection with versioning capability. You can use versioning to preserve, retrieve and restore every version of every object in s3 bucket.

Multifactor authentication delete : this is an additional security. It requires the use of MFA device to delete objects stored in S3 bucket.

Time limited access to objects : it supports query string authentication which allows you to provide a URL that is valid only  for a length of time that you define. This time limited URL can be useful for scenarios such as software downloads or other applications where you want to restrict the length of time users have access to an object.

Elastic Block Store (EBS)

  • Amazon EBS provides block level storage volumes for use with EC2 instances.
  • EBS volumes are highly available and reliable storage volumes.
  • These can be attached to any running instance that is in the same availability zone.
  • EBS volumes can be launched as encrypted volumes for simplified data encryption.
  • There is no need to build, manage, and secure key management infrastructure.
  • When an encrypted EBS is created and attached to a supported instance type, data stored at rest on the volume, disk I/O and snapshots created from the volume are all encrypted.
  • AWS key Management Service (AWS KMS) master keys are used by EBS to create encrypted volumes and any snapshots from encrypted volumes.
  • A default master key is created automatically when an encrypted EBS volume is created for the first time.
  • This key is used unless Customer Master Key (CMK) is selected that has been created separately using AWS key Management service.

Features of EBS

  • One can Create EBS General Purpose SSD (gp2), Provisioned IOPS SSD (io1), Throughput Optimized HDD (st1), and Cold HDD (sc1) volumes up to 16 TiB in size.
  • With General Purpose SSD (gp2) volumes, the base performance of 3 IOPS/GiB, with the ability to burst to 3,000 IOPS for extended periods of time has been expected.
  • Volume performance monitoring.

Benefits of EBS volumes

  • Data availability
  • Data persistence
  • Data encryption
  • Snapshots
  • Flexibility

EBS volume types

  • Solid-state Drives (SSD)
  • Hard disk Drives (HDD)

Creating an EBS volume

  • Open console
  • Select the region from the navigator bar
  • Choose VOLUMES under ELASTIC BLOCK STORE in the navigation bar
  • Above the upper pane, choose Create Volume.
  • In the Create Volume dialog box, for Volume Type, choose General Purpose SSD (GP2), Provisioned IOPS SSD (IO1), Throughput Optimized HDD (ST1), Cold HDD (SC1), or Magnetic.
  • For Size, enter the size of the volume, in GiB.


Attaching an EBS volume to instance

  • Open console
  • Choose volumes in the navigation bar
  • Select a volume and choose Actions, Attach Volume
  • In the Attach Volume dialog box, start typing the name or ID of the instance to attach the volume to for Instance, and select it from the list of suggestion options
  • You can keep the suggested device name, or enter a different supported device name.
  • Choose Attach.
  • Connect to your instance and make the volume available.

Amazon EBS–Optimized Instances

  • An Amazon EBS–optimized instance uses an optimized configuration stack and provides additional, dedicated capacity for Amazon EBS I/O.
  • This optimization provides the best performance for your EBS volumes by minimizing contention between Amazon EBS I/O and other traffic from your instance.
  • EBS–optimized instances deliver dedicated bandwidth to Amazon EBS, with options between 500 Mbps and 12,000 Mbps, depending on the instance type you use.
  • When attached to an EBS–optimized instance, General Purpose SSD (gp2) volumes are designed to deliver within 10% of their baseline and burst performance 99% of the time in a given year.
  • Provisioned IOPS SSD (io1) volumes are designed to deliver within 10% of their provisioned performance 99.9% of the time in a given year.
    • Enabling EBS Optimization at Launch

    ○Open the Amazon EC2 console.

    ○Click Launch Instance. In Step 1: Choose an Amazon Machine Image (AMI), select an AMI.

    ○In Step 2: Choose an Instance Type, select an instance type that is listed as supporting EBS optimization.

    ○In Step 3: Configure Instance Details, complete the fields that you need and select Launch as EBS-optimized instance.

    ○Follow the directions to complete the wizard and launch your instance.

    • Modifying EBS Optimization for a Running Instance

    ○Open the Amazon EC2 console.

    ○In the navigation pane, click Instances, and select the instance.

    ○Click Actions, select Instance State, and then click Stop.

    ○In the confirmation dialog box, click Yes, Stop.

    ○With the instance still selected, click Actions, select Instance Settings, and then click Change Instance Type.

    ○Click Actions, select Instance State, and then click Start.

Amazon EBS Encryption

  • Amazon EBS encryption offers you a simple encryption solution for your EBS volumes without the need for you to build, maintain, and secure your own key management infrastructure.
  • The following types of data are encrypted:

○Data at rest inside the volume

○All data moving between the volume and the instance

○All snapshots created from the volume

  • The first time you create an encrypted volume in a region, a default CMK is created for you automatically.
  • This key is used for Amazon EBS encryption unless you select a CMK that you created separately using AWS KMS.
    • Snapshots that are taken and volumes that are created from encrypted volumes are automatically encrypted.
    • You can share an encrypted snapshot with specific accounts if you take the following steps:

    ○Use a custom CMK, not your default CMK, to encrypt your volume.

    ○Give the specific accounts access to the custom CMK.

    ○Create the snapshot.

    ○Give the specific accounts access to the snapshot.

Initializing Amazon EBS Volumes

  • New EBS volumes receive their maximum performance the moment that they are available and do not require initialization.
  • Storage blocks on volumes that were restored from snapshots must be initialized before you can access the block.
  • This preliminary action takes time and can cause a significant increase in the latency of an I/O operation the first time each block is accessed.

Benchmark EBS Volumes

  • Launch an EBS-optimized instance.
  • Create new EBS volumes.
  • Attach the volumes to your EBS-optimized instance.
  • Configure and mount the block device.
  • Install a tool to benchmark I/O performance.
  • Benchmark the I/O performance of your volumes.
  • Delete your volumes and terminate your instance so that you don’t continue to incur charges.

Amazon CloudWatch Events for Amazon EBS

  • Amazon EBS emits notifications based on Amazon CloudWatch Events for a variety of snapshot and encryption status changes.
  • With CloudWatch Events, you can establish rules that trigger programmatic actions in response to a change in snapshot or encryption key state.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s