AWS

Command Completion

AWS Services

  1. Computing

    • EC2
    • Lambda
    • Elastic Beanstalk

  2. Networking

    • VPC
    • Route 53

  3. Storage

    • S3
    • Cloudfront
    • Glacier
    • EFS

  4. Security

    • IAM
    • KMS

Common Acronyms

AWS Global Infrastructure

Types of Cloud Computing

Regions

AWS Points of Presence (Edge Locations)

AWS IAM

EC2

EC2, Amazon's Elastic Compute Cloud, is a virtual server that can perform computations remotely. The compute capacity is easy to resize, and you only pay for the computing capacity that is used.

Recently, AWS announced support for Mac EC2 instances. What's more, these instances aren't limited to using the computer exclusively through the console. You can even connect to your instance using VNC.

AWS CLI

aws configure --profile tommy

The AWS Console will check for these variables in your shell environment:

  1. AWS_ACCESS_KEY_ID otherwise specified in ~/.aws/credentials or inside ~/.aws/config as aws_access_key_id
  2. AWS_SECRET_ACCESS_KEY otherwise specified in ~/.aws/credentials or inside ~/.aws/config as aws_secret_access_key
  3. AWS_SESSION_TOKEN otherwise specified in ~/.aws/credentials or inside ~/.aws/config as aws_session_token
  4. AWS_PROFILE otherwise specified with aws --profile tommy
  5. AWS_DEFAULT_REGION otherwise specified with aws --region us-east-1 or inside ~/.aws/config as aws_default_region
  6. AWS_DEFAULT_OUTPUT otherwise specified with aws --output json or inside ~/.aws/config as aws_default_output

--query

When the AWS CLI returns output, it's typically formatted as JSON. You can use --query supplied with JMESPath, a query language for JSON.

AWS SageMaker

AWS SageMaker allows you to make cloud-hosted Jupyter notebooks, which can easily be connected to S3 buckets and EC2 instances available on your account.

You can use Amazon's SDK for Python, known as boto3 to perform operations between AWS services within a python script, such as a Jupyter notebook.

# Import the AWS SDK boto3
import boto3
s3 = boto3.resource('s3')

# Print all of the available S3 buckets
for bucket in s3.buckets.all():
  print(bucket.name)

# Specify the name of the S3 bucket
bucket = s3.Bucket('example')

# List all of the objects in a bucket
for obj in bucket.objects.all():
  print(obj.key)

# Download the S3 file, and save it to the Jupyter notebook
bucket.download_file('/s3bucket/path/to/sample.json', '/path/to/sample.json')

# Open the file inside the Jupyter notebook
my_file = open('/path/to/sample.json')
import json
my_object = json.load(my_file)

# View properties of the object
print(my_object)

IAM

Vocabulary:

IAM Policy Structure has a few key components:

By default, all permissions are denied. It must be specifically allowed. If the action you are trying to perform is being denied, it could be a result of the policy's surrounding any of the above components. Maybe the current ARN doesn't have permission for that action, or it would if a different condition was in place.

Types of Policies:

IAM on the Command-Line

AWS S3

AWS Lambda

EC2

Configure

Documentation

It's worth noting that you can specify which SSO profile name to use in two different ways:

  1. By passing a name to the --profile option, (e.g. --profile tommy)

  2. By assigning a name to the environment variable AWS_DEFAULT_PROFILE

Cloud9

AWS has an in-browser IDE called Cloud9, which you can power using an existing EC2 instance. Supposedly it supports pair programming as well.

Organizations

API Gateway

SQS

Roles

Assume into a role using the role name and account number: [^1] [1]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-console.html

https://signin.aws.amazon.com/switchrole?roleName=ROLE_NAMEYOURROLE&account=ACCOUNT_ID

AWS ECS

AWS Batch

Creating a Batch Job

First, you'll need to create the "Compute Environment"

To have Amazon manage everything, use Fargate.

Note: the following are the types of compute environments for AWS Batch that you have to choose from:

Next, you will create the "Job Queue"

After that, you will need to register the job definition.

Deleting a Batch Job

Source of learning: https://ec2spotworkshops.com/rendering-with-batch/cleanup.html

When deleting AWS Batch components, the order matters; a compute environment cannot be deleted if it is associated to a valid queue, so we have to start by deleting the queue.

Also, you have to disable a job queue before you are able to delete it. Same goes for a compute environment.

To disable the AWS Batch components:

aws batch update-job-queue --job-queue JOB_QUEUE_NAME --state DISABLED && \
aws batch update-compute-environment --compute-environment COMPUTE_ENV_ARN --state DISABLED && \

The previous operation may take up to 2 minutes. Job queues and compute environments cannot be deleted while being modified, so running the commands below while the compute environments and job queue are being disabled might result in an error with the message "resource is being modified"

To delete the AWS Batch components:

aws batch delete-job-queue --job-queue JOB_QUEUE_NAME
aws batch delete-compute-environment --compute-environment COMPUTE_ENV_ARN

To deregister the AWS Batch components, deregister the job definition:

aws batch deregister-job-definition --job-definition "${JOB_DEFINITION_ARN}"

If you need a list of job definitions on your account, you can run the following command:

aws batch describe-job-definitions --status INACTIVE \
| jq --raw-output '.jobDefinitions[].jobDefinitionName' \
| sort \
| uniq

Don't forget to remove the image from Amazon ECR:

aws ecr batch-delete-image --repository-name "${RepositoryName}" --image-ids imageTag=latest