If the CloudFormation create has an error, the CloudFormation doesn’t roll back so it has to be deleted and recreated. To avoid multiple initial create/delete, the best approach is to use a null CloudFormation. An example is below:
AWSTemplateFormatVersion: 2010-09-09 Description: CloudFormation Starter Conditions: Never: !Equals [ a, b ] Resources: NullResource: Type: Custom::Null Condition: Never
While the actual application code for many lambda functions might be small, in many (data science) usecases libraries used and to be packaged together push over the deployment package limit of 50 MB compressed. Container packaging to the rescue.
A sample Dockerfile installing SageMaker Python SDK
ARG WORK_DIR="/home/app/" FROM public.ecr.aws/lambda/python:3.8 ARG WORK_DIR RUN pip install wheel sagemaker COPY main.py . WORKDIR ${WORK_DIR} CMD ["main.handler"] and corresponding main.py
import sagemaker import sys def handler(event, context): print(sys.
Many AWS customers skip initial planning steps and don’t establish infrastructure-as-code scripting practices initially necessitating querying and cleaning up resources to avoid billing charges.
AWS tooling AWS Cost Explorer AWS Billing Console AWS Tag Editor select all regions, all resource types and uncheck tags The open source community developed a number of tools
List all AWS resources aws_list_all command line awsls AWSRetriever desktop aws-inventory command line and GUI Filtered delete aws-nuke extensive filtering cloud-nuke filtering by region, age, resourcetype is available
For a number of uses including generating IAM roles and various policies, lambda code might require access to current account id and region. Here is a code snippet on how to acquire those values:
import json import logging import boto3 logger = logging.getLogger() logger.setLevel(logging.INFO) def lambda_handler(event, context): logger.info(json.dumps(event)) account_id = boto3.client('sts').get_caller_identity().get('Account') # use a client which is region based logs_client = boto3.client('logs') region_name = logs_client.meta.region_name logger.info(f"{account_id}:{region_name}")
EC2 Instance Connect is somewhat overlooked functionality improving security of EC2 logins. During configuration, the default instance user is assigned a public key so it’s private pair can be used to connect to the instance. The private key tends to be shared within support teams and logins can no longer be attributed to an individual. EC2 instance Connect runs and logs connect commands as the individual user and obeys user’s permission.
While I found a number of examples for generating signed upload S3 URLs, there didn’t seem to be examples with the basics.
After substituting the name for your bucket, file name and expiry desired, run below code to generate the URL:
import boto3 if __name__ == "__main__": s3_client = boto3.client('s3') response = s3_client.generate_presigned_url( ClientMethod='put_object', Params={"Bucket": "mybucket", "Key": "file.pdf"}, ExpiresIn=48*60*60, HttpMethod="PUT") print(response) To upload from command line run below (substituting URL from previous section):
When AWS infrastructure configured in “traditional” compute/storage/network style, identifying, referencing and patching AMIs in all regions in use is crucial. Cloudformation has a way to redirect AMI references through SSM Parameter Store.
This represents a tradeoff, as recreating the Cloudformation stack might pickup the next (patched) AMI hence it is no longer immutable. But resulting state is similar to externally patched Linux/Windows images which also cannot be recreated by simply redeploying Cloudformation.
Many AWS customers take advantage of AWS Organizations to organize and secure their workloads. In many cases, users login into their master account and configure permissions allowing to switch to member accounts in the Console. The same permissions can be used for AWS CLI.
In this example below, a single AWS Access Key has to be generated (and rotated) in the master account and it can be used to switch to test/2222222 and production/3333333 accounts using the CrossAccountAccessRole already configured for switching in the Console.
When looking into AWS SimpleDB, a quick search didn’t return any short Python Boto3 examples. So I decided to post one. (As with any services you to subscribe to, running this code below might cost you money …)
from __future__ import print_function import boto3 def quote(string): return string.replace("'", "''").replace('"', '""').replace('`', '``') def put_attributes(sdb, domain, id, color): response = sdb.put_attributes( DomainName=domain, ItemName=id, Attributes=[ { 'Name': 'color', 'Value': color, 'Replace': True }, ], ) print(response) if __name__ == "__main__": domain = "TEST_DOMAIN" sdb = boto3.