Developing on AWS - Tips and Tricks

This is my list of hints and tips for this course. It’s markdown so you can save it, access it or store it anywhere. I might also give you other links that are course specific. I’ll add specific answers to questions I get during the course. I’ll share it with everyone.

Your Instructors

Ian Falconer https://www.linkedin.com/in/leftbrainstuff/

Administrivia

We need to jump through some hoops to get access to the labs, notes and my hints and tips. Be consistent with the email address you use for all sites. There are three seperate sites you need to access and one bitly link which is this page:

Join or login to https://www.aws.training/ to ensure your training and certifications are captured. No we don’t spam you or sell your details.
Access Qwiklab (yes it is spelt INCORRECTLY)
- aws.qwiklabs.com for the labs in this class
- run.qwiklabs.com for outside of the class or to do other labs at your own pace. NOTE: Some are free others require course credits. Also check out the AWS Professional Developer Series of MOOCs on edX https://www.edx.org/aws-developer-professional-series
Access the course notes and slides. You’ll receive two emails. One confirming your attendance at this course and with the following links. The download link seems broken. You can download apps for phones, tablets and laptops. Or use your browser.
www.vitalsource.com look for a signup link and download link. Or just go to https://evantage.gilmoreglobal.com/#/user/signin
Once you’ve logged into Vitalsource (aka Bookshelf, Gilmore, eVantage) you can redeem your unique course materials code (in a seperate email) and update your book list. You should see a lab guide and student guide for Developing on AWS, version 3.1 . The student guide is the powerpoint decks and notes and the lab guide is the step by step instructions for the labs. The lab guide is included in the labs so this document is somewhat redundant. You can download the Vitalsource Bookshelf app for Windows, Mac, IoS and Android at https://support.vitalsource.com/hc/en-us/articles/201344733-Bookshelf-Download-Page
You can print the student and lab guides to pdf from the app.
Solutions and AWS CloudFormation templates for the labs can be downloaded from https://bitly.com/2MPJQWE

Group Exercises (Instructor will decide if and when we tackle these)

Cool links

James Hamilton, AWS SVP, talks about our infrastructure. This should be the first AWS video you watch. Here’s the youtube video. https://www.youtube.com/watch?v=JIQETrFC_SQ There are Youtube videos from more recent ReInvents with some updates too. Here is James in 2016. It’s titled as Global Innovation at Scale. https://www.youtube.com/watch?v=uj7Ting6Ckk
AWS re:Invent 2017: Scaling Up to Your First 10 Million Users (ARC201). This is like the Tech Essentials course in a single video. Well worth a watch. https://www.youtube.com/watch?v=w95murBkYmU
A Day in the Life of a Billion Packets" - [http://www.youtube.com/watch?v=Zd5hsL-JNY4 and the sequel Another Day, Another Billion Packets" - [https://www.youtube.com/watch?v=3qln2u1Vr2E
Amazon EC2 Instance Types explained in neat tabular comparisons. https://aws.amazon.com/ec2/instance-types/ . Also here’s a third party site that has a table that lets you sort on memory, network performance, cost and instance type. You can also quickly compare costs here too. https://ec2instances.info/ . Here’s a stackoverflow thread on non AWS benchmarks of different instance types. https://stackoverflow.com/questions/20663619/what-does-amazon-aws-mean-by-network-performance
An external post about S3 data leaks. The AWS Shared Responsibility Model is key to avoiding this misconfiguration. https://aws.amazon.com/compliance/shared-responsibility-model/ Engage with your AWS Solution Architects to get your security right. CIA CIO John Edwards has publicly stated that “it’s the best decision we’ve ever made” and “It’s the most innovative thing we’ve ever done” in reference with the CIA’s partnership with AWS. Here are the links. https://fcw.com/articles/2017/06/14/cia-cloud-aws.aspx and https://www.cio.com/article/2375269/hybrid-cloud/cia-off-and-running-with-amazon-web-services.html
Latency between AWS regions. Lot’s of good empirical data points. Note these are averages over a 24 hour period. https://www.cloudping.co/
Latency - Lots of interesting links here. http://highscalability.com/latency-everywhere-and-it-costs-you-sales-how-crush-it
List of CIDR ranges of AWS regions http://ec2-reachability.amazonaws.com/
How we handle prime day (DynamoDB) https://www.youtube.com/watch?v=83-IWlvJ__8
New frameworks and automation. A very powerful combination. Chalice python based microservices framework using Lambda, API Gateway and IAM. http://chalice.readthedocs.io/en/latest/ Very fast, very lightweight and very extensible. Also look at SAM, the AWS Serverless Application Model and AWS Amplify.
Benchmark tests of EC2 versus other bare metal and cloud servers. AWS keeps innovating relentlessly. https://www.phoronix.com/scan.php?page=article&item=cloud-cpu-36#=1
Here’s a Lambda deep dive which more clearly explains some of the questions around managing state, retries, testing and handling large data sets with Lambda. https://www.youtube.com/watch?v=dB4zJk_fqrU
AWS Open Guide on GitHub is a good summary of AWS Documentation https://github.com/open-guides/ We have also open sourced our documentation at https://github.com/awsdocs and the full list of AWS services with their documentation links is at https://docs.aws.amazon.com/index.html#lang/en_us
More open source updates at https://aws.amazon.com/opensource/?opensource-all.sort-by=item.additionalFields.startDate&opensource-all.sort-order=asc Don’t miss the quickstart, awsome days and online builder events listed at the open source page. They are a great way to build and learn.
You can check the overall health and availability of AWS globally at the Service Health Dashboard (SHD) https://status.aws.amazon.com/ You can also use the AWS Health API to programatically check for service health at https://docs.aws.amazon.com/health/latest/ug/getting-started-api.html
Netflix have some cool tools that they’ve open sourced. https://netflix.github.io/
All AWS quickstarts (aka reference architectures) now in github https://github.com/aws-quickstart
Adrian Cockcroft AWS VP of Cloud Architecture Strategy and former CTO of Netflix. Here’s his Youtube playlist with talks about DevOps, migrations, Netflix lessons learned and digital transformation topics. https://www.youtube.com/playlist?list=PL_KXMLr8jNTnwkzV7SePa0jHFUG2qn0MA
The Network is Reliable and other fallacies. Key performance and reliability concerns of distributed systems. https://blog.acolyer.org/2014/12/18/the-network-is-reliable/
Architecture reviews are important. The cloud design principles, here is the 2011 AWS Whitepaper https://media.amazonwebservices.com/AWS_Cloud_Best_Practices.pdf and the Well Architected Review are key inputs. https://aws.amazon.com/architecture/well-architected/
AWS General Reference. https://docs.aws.amazon.com/general/latest/gr/Welcome.html This document is a key reference when architecting and designing AWS solutions.
All AWS documentation in one place and neatly categorized. https://docs.aws.amazon.com/index.html?nc2=h_ql_doc#lang/en_us
Two interesting solutions for transcribing podcasts and creating subtitles
- Implementing Serverless Video Subtitles https://aws.amazon.com/blogs/compute/implementing-serverless-video-subtitles/
- Discovering and indexing podcast episodes using Amazon Transcribe and Amazon Comprehend https://aws.amazon.com/blogs/machine-learning/discovering-and-indexing-podcast-episodes-using-amazon-transcribe-and-amazon-comprehend/
Here is a full serverless backend and front end app in github, https://github.com/aws-samples/aws-serverless-airline-booking and the full build on Twitch https://pages.awscloud.com/GLOBAL-devstrategy-OE-BuildOnServerless-2019-reg-event.html
A great way to understand how Cloudformation can build, update and delete immutable or mutable environments is to reverse engineer AWS Quickstarts (gold standard reference architectures). Check out https://aws.amazon.com/quickstart/saas/identity-with-cognito/ for the deployment guide and https://github.com/aws-quickstart/saas-identity-cognito for all the Cloudformation templates.
Building mobile apps fast on AWS re Real-Time Offline Ready Chat App written with GraphQL, AWS AppSync, & AWS Amplify https://github.com/aws-samples/aws-appsync-chat

Want to build a full stack web / mobile app with front end and backend, CICD pipeline and tests. Watch on twitch https://pages.awscloud.com/GLOBAL-devstrategy-OE-BuildOnServerless-2019-reg-event.html and grab the code from github https://github.com/aws-samples/aws-serverless-airline-booking

Ian’s list of links for weekly review of all stuff AWS. (trying to keep up with the firehose)

Search for AWS

and Deep Dive or Ninja and you’ll find lots of great videos or slideshares.
AWS is continually sharing more good stuff on github. https://github.com/aws
AWS What’s New: https://aws.amazon.com/new/
And my favourite the weekly AWS Podcast https://aws.amazon.com/podcasts/aws-podcast/
The first place to start on any of AWS’ more than 125 services is the the faq pages for each product (this is where I start reading) https://aws.amazon.com/faqs/

https://aws.amazon.com/podcasts/aws-podcast/ and all the faq pages for each product (this is where I start reading)

AWS has released a number of webinars and now has a monthly cadence https://aws.amazon.com/about-aws/events/monthlywebinarseries/

AWS Answers is now available to the public. It contains some interesting links. https://aws.amazon.com/answers/

Get to know your Technical Account Manager (TAM) The TAMs provide support for your applications running on AWS. They can help you prepare for major events like testing and scaling. They can also help troubleshoot and provide visibility into AWS infrastructure metrics for troubleshooting. https://aws.amazon.com/premiumsupport/faqs/

AWS Glossary contains service names and nomenclature https://docs.aws.amazon.com/general/latest/gr/glos-chap.html

Best Practice

http://www.opendatacenteralliance.org/docs/DevCloudCapApp.pdf (This pdf has some good information about developing cloud applications)
Learn from others. An personal reflection on how best to utilize AWS https://wblinks.com/notes/aws-tips-i-wish-id-known-before-i-started/
Amazon SNS Subscription Filter Policies allows you to build logic into your SNS messages and subsequently handle the response based on a filter policy. https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html􏰀
A Simple Serverless Test Harness using AWS Lambda (circa 2015) but still relevant today. https://aws.amazon.com/blogs/compute/serverless-testing-with-aws-lambda/ There is also a Lambda blueprint available.
Identity management is a complex topic, challenging to implement and maintain. Deep dive on the AWS Identity and Access Management User Guide https://github.com/awsdocs/iam-user-guide/blob/master/doc_source/index.md and check out the example policies.

Compute and Containers

James Hamilton on AWS new Arm Gravitron and Inferentia chips. https://perspectives.mvdirona.com/2018/11/aws-inferentia-machine-learning-processor/
Popquiz - How many EC2 options does AWS provide? Hint - https://perspectives.mvdirona.com/2018/11/aws-inferentia-machine-learning-processor/

Here are some links to running Springboot Microservices on AWS - Deploy Spring Boot App to AWS Fargate https://dzone.com/articles/deploy-spring-boot-app-to-aws-fargate and note the link to a github repo - Heres a serverless (Lambda and API Gateway) example https://www.rowellbelen.com/serverless-microservices-with-spring-boot-and-spring-data/ - Heres justification for running on ElasticBeanstalk https://stackoverflow.com/questions/48934158/spring-boot-cloud-microservices-on-aws - Some AWS guidance on Spring Boot - Elastic Beanstalk is called out specifically in the Spring Boot documentation https://docs.spring.io/spring-boot/docs/1.5.10.RELEASE/reference/htmlsingle/#production-ready-metricsWe - (AWS) have made a small library that makes Spring Boot AWS Lambda friendly. It’s called the Java Serverless Container. It supports Lambda, API Gateway, Load Balancing and Route 53 https://github.com/awslabs/aws-serverless-java-container - Spring has another module (which they maintain) that allows you to consume some AWS services: https://spring.io/projects/spring-cloud-aws which supports SQS, SNS, Elasticache, RDS and Cloudformation

Lambda Execution Context explained. https://docs.aws.amazon.com/lambda/latest/dg/running-lambda-code.html and here’s an AWS Xray link that describes how you can decipher how long your Lambda function spends initializing and running your handler function. https://docs.aws.amazon.com/lambda/latest/dg/lambda-x-ray.html
Understanding container reuse in Lambda https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/
Best practice for Lambdas. Especially Java based. https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html and https://docs.aws.amazon.com/lambda/latest/dg/java-programming-model.html
Lambda cold start comparison https://medium.com/@nathan.malishev/lambda-cold-starts-language-comparison-%EF%B8%8F-a4f4b5f16a62 and this also links to the cheekily titled I’m afraid you’re thinking about AWS Lambda cold starts all wrong https://hackernoon.com/im-afraid-you-re-thinking-about-aws-lambda-cold-starts-all-wrong-7d907f278a4f
Phillip Gerbe’s posts on EC2 autoscaling
- https://garbe.io/blog/2016/10/17/docker-on-ecs-scale-your-ecs-cluster-automatically/
- https://garbe.io/blog/2017/04/12/a-better-solution-to-ecs-autoscaling/

Running Containers on AWS https://aws.amazon.com/containers/services/ Start here to see all the container options on AWS. Also check out the publically available AWS Container Roadmap on github https://github.com/aws/containers-roadmap

Instrumenting Kubernetes for Observability using AWS X-Ray and Amazon CloudWatch https://github.com/aws-samples/reinvent2018-dev303-code
Arun Gupta talking about Corretto and OpenJDK https://www.infoq.com/news/2019/03/amazon-releases-corretto-8?utm_campaign=infoq_content&utm_source=twitter&utm_medium=feed&utm_term=java
A deep dive on Fargate. Lot’s of feature updates around container orchestration so watch the AWS what’s new for the latest. Here’s a slide share that deep dives on Fargate. https://de.slideshare.net/AmazonWebServices/deep-dive-into-aws-fargate Also see https://docs.aws.amazon.com/AmazonECS/latest/developerguide/AWS_Fargate.html
Deep dive on EC2 instance metadata https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html Here’s a QandA from Github with documentation links to how IAM roles work with EC2 instances. https://github.com/ex-aws/ex_aws/issues/30 NOTE be wary of inadvertently exposing credentials when using HTTP proxies, HTML/CSS validator services, and XML processors that support XML inclusion.
Self paced 10 hr (2+4+4) Kubernetes workshop. AWS Workshop for Kubernetes https://github.com/aws-samples/aws-workshop-for-kubernetes
Another nice EKS workshop: https://eksworkshop.com/introduction/ .This workshop covers k8 basics and some cool things such as the k8 dashboard, helm etc.. :-) https://ecsworkshop.com/ and https://eksworkshop.com/

ETL

Build and automate a serverless data lake using an AWS Glue trigger for the Data Catalog and ETL jobs https://aws.amazon.com/blogs/big-data/build-and-automate-a-serverless-data-lake-using-an-aws-glue-trigger-for-the-data-catalog-and-etl-jobs/

Creating a Simple “Fetch & Run” AWS Batch Job https://aws.amazon.com/blogs/compute/creating-a-simple-fetch-and-run-aws-batch-job/ This blog post shows you how to use AWS Batch to run a container and customer bash script for ETL processing. Also eheckout ENTRYPOINT in the AWS Documentation in Tutorial: Using the Array Job Index to Control Job Differentiation https://docs.aws.amazon.com/batch/latest/userguide/array_index_example.html

Networking links

Latency http://highscalability.com/latency-everywhere-and-it-costs-you-sales-how-crush-it
List of CIDR ranges of AWS regions http://ec2-reachability.amazonaws.com/
Create private links to AWS using Privatelink https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html
Task Networking in AWS Fargate (How containers can talk to each other over localhost, how we can create microservices networking and using ALB for abstractin scaling from routing) https://aws.amazon.com/blogs/compute/task-networking-in-aws-fargate/
Network (aka VPC) Deep Dive from 2015 https://s3-eu-west-1.amazonaws.com/awssummit2015/Slides+and+recordings/Tech+track/Deep+Dive+-+Amazon+VPC.pdf A good first start as you begin to deep dive on AWS networking. NOTE: there have been many enhancements to networking performance and reducing complexity of large enterprise networks on AWS over the last few years. ENAs, Transit VPCs, etc.
A technical comparison between API Gateway vs Application Load Balancer–Technical Details from serverless-training.com https://serverless-training.com/articles/api-gateway-vs-application-load-balancer-technical-details/ Includes working examples. There’s also a Reddit titled Invoking Lambda with ALB vs API Gateway.. https://www.reddit.com/r/aws/comments/a1mirw/invoking_lambda_with_alb_vs_api_gateway/
VPC Traffic Mirroring – Capture & Inspect Network Traffic gives you another, perhaps more flexible approach for monitoring network traffic to and from ENIs. https://aws.amazon.com/blogs/aws/new-vpc-traffic-mirroring/

Database, Caching and Storage links

AWS provides many specialist database options https://aws.amazon.com/products/databases/
AWS blogs are a great reference. Here’s the database blog link https://aws.amazon.com/blogs/database/
24 Jul 2017 S3 Rate Request Performance Increase announcement https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-s3-announces-increased-request-rate-performance/ and notice the exponetial scaling possible with multiple prefixes. https://docs.aws.amazon.com/AmazonS3/latest/dev/request-rate-perf-considerations.html but if using sse-kms this service will a limiting factor. https://docs.aws.amazon.com/kms/latest/developerguide/limits.html#requests-per-second-table
EFS Performance Deep Dive. The documentation is very informative. Read in this order
Announcing the New Amazon DynamoDB Key Diagnostics Library | https://aws.amazon.com/about-aws/whats-new/2018/12/announcing-the-new-amazon-dynamodb-key-diagnostics-library/
DynamoDB Point in Time Restore
- Start here https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/PointInTimeRecovery_Howitworks.html max 4 concurrent restores any time in the last 35 days. There are post restore actions which may need to be manually completed.
- Now consider your specific restore case https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/backuprestore_HowItWorks.html#backuprestore_HowItWorks-restore
  - Backups are asynchronous and available for restore in minutes
  - You need to rebuild things like Cloudwatch alarms, tags, Auto Scaling, etc
  - Restores don’t impact throughput or API performance but you can only write to the restored table once it is active
  - It can take up to 20 minutes to restore a table (or longer if your data is skewed). For partitions with billions of items full table restore should take less than 10 hours. Assuming even data distribution across partitions
Elasticache comparison Memcached and Redis https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/SelectEngine.html
Loss of caching nodes could be due to underlying hardware failure, reboots or restarts during a maintenance window or even from the loss of an AZ. https://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/FaultTolerance.html
AWS CLI S3 documentation https://docs.aws.amazon.com/cli/latest/topic/s3-config.html#addressing-style
Everything you ever wanted to know about the Amazon DynamoDB console but were afraid to ask: A detailed walkthrough https://aws.amazon.com/blogs/database/everything-you-ever-wanted-to-know-about-the-amazon-dynamodb-console-but-were-afraid-to-ask-a-detailed-walkthrough/
Amazon S3 Path Deprecation Plan – The Rest of the Story https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/
Rick Houlihan on AWS re:Invent 2018: Building with AWS Databases: Match Your Workload to the Right Database (DAT301) https://www.youtube.com/watch?v=hwnNbLXN4vA&t=19s and here is a discussion about the Rick’s PIE theorem as being more useful in architecting cloud hyper scale databases. It’s titled ‘Why the PIE theorem is more relevant than the CAP theorem’ https://www.alexdebrie.com/posts/choosing-a-database-with-pie/
Amazon S3 bucket and object permissions are independent. https://docs.aws.amazon.com/AmazonS3/latest/user-guide/set-permissions.html
Can you have S3 objects with an infinite number of versions? Well now and here’s why. That would also be not cost optimized. https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectVersioning.html
Updates or Puts to DynamoDB table not showing up in your GSI? This can happen if your write is missing attributes that the GSI is expecting to be projected. See Global Secondary Indexes documentation (first section) for an explanation. https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSI.html
AWS Big Data course is a broad look at big data services. The Data Warehousing on AWS Course is a deep dive on AWS Redshift and other big data services with a focus on tuning and optimization. https://aws.amazon.com/training/course-descriptions/data-warehousing/
Just announced NoSQL Workbench for Amazon DynamoDB – Available in Preview . The workbench allows you design and test your NoSQL data schemas and build them in DynamoDB in a Mac or Windows GUI. https://aws.amazon.com/blogs/aws/nosql-workbench-for-amazon-dynamodb-available-in-preview/
If your NoSQL documents are too big for DynamoDB (> 400 kb) then Amazon DocumentDB, which is MongoDB compartible, is an option. https://aws.amazon.com/documentdb/faqs/

IAM

Configuring the AWS CLI https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html#config-settings-and-precedence
IAM example policies https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_examples.html and policy evaluation logic https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_evaluation-logic.html
Deep dive on EC2 instance metadata https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html Here’s a QandA from Github with documentation links to how IAM roles work with EC2 instances. https://github.com/ex-aws/ex_aws/issues/30 NOTE be wary of inadvertently exposing credentials when using HTTP proxies, HTML/CSS validator services, and XML processors that support XML inclusion.
IAM Policy Evaluation Logic https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_evaluation-logic.html and Testing IAM Policies with the IAM Policy Simulator https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_testing-policies.html

Streaming

FluentD versus Flume https://www.slideshare.net/treasure-data/fluentd-loves-mongodb-at-mongosv-july172012/37-Fluentd_vs_Flume_Easy_to and https://www.slant.co/versus/959/960/~fluentd_vs_flume
2014 Kinesis deep dive https://www.youtube.com/watch?v=8u9wIC1xNt8&feature=youtu.be&t=104.+%28AWS+21%29+AWS.+Developing+on+AWS+2.4+%28EN%29%3A+Instructor+Guide.+AWS%2FGilmore.+VitalBook+file and something newer https://www.youtube.com/watch?v=IXcs_e0oTKE
Kinesis Firehose buffer time configuration https://aws.amazon.com/kinesis/data-firehose/faqs/
Waiting for a stream to become active and adding some robustness when creating a stream. https://docs.aws.amazon.com/streams/latest/dev/kinesis-using-sdk-java-create-stream.html#kinesis-using-sdk-java-create-the-stream
For Kinesis Firehose I suggest getting familiar with the documentation. This is a fully managed service and it is designed to be easy to use and highly scalable. This service, like most, has service specific service limits that you need to design to exploit.
- Kinesis Firehose Developer Guide https://docs.aws.amazon.com/kinesis/?id=docs_gateway specifically
- What Is Amazon Kinesis Data Firehose? https://docs.aws.amazon.com/firehose/latest/dev/what-is-this-service.html
- Amazon Kinesis Data Firehose Limits in the developer guide have specific guidance. For example Kinesis Firehose supports Classic Load Balancer and not ALB or NLB (yet?), Redshift clusters must be in a publically accessible cluster and buffering limits vary between Redshift and Elasticsearch. There are other service specific service limits but I’ll let you find those yourself.

Queues and Messaging

SQS and SNS SLAs. Personal experience demonstrates that SQS is a very reliable service. I can’t recall any customer or personal issues with SQS or SNS over 9+ years. However this anecdotal information is useless for others. Here’s a stackoverflow thread on this topic presumably contributed to by ex AWS folks. https://stackoverflow.com/questions/30750033/amazon-sns-delivery-retry-policies-for-sqs
as of Jan 2019 SQS and SNS have a 99.9% SLA https://aws.amazon.com/messaging/sla/

Serverless

Lambda videos to watch
- 2014 ReInvent Lambda. Some early best practices https://www.youtube.com/watch?v=UFj27laTWQA
- Then for a more recent update on Lambda best practices and architectures watch AWS re:Invent 2017: Serverless Architectural Patterns and Best Practices (ARC401) https://www.youtube.com/watch?v=Xi_WrinvTnM
- 2016 Lambda mobile support. https://www.youtube.com/watch?v=copO_JQQsBs
For a deeper dive into Lambda
- AWS re:Invent 2018: A Serverless Journey: AWS Lambda Under the Hood (SRV409-R1) https://www.youtube.com/watch?v=QdzV04T_kec
- Lambda Internals: Exploring AWS Lambda (2 parts) https://epsagon.com/blog/lambda-internals-exploring-aws-lambda/ and https://epsagon.com/blog/lambda-internals-part-two/
- I’m afraid you’re thinking about AWS Lambda cold starts all wrong https://hackernoon.com/im-afraid-you-re-thinking-about-aws-lambda-cold-starts-all-wrong-7d907f278a4f
It’s key to use the principles of loose coupling with Lambda functions. Error handling, wait dependencies are better handled with logic outside of individual Lambda functions. Consider using AWS Step Functions to coordinate your Lambdas and business logic. Some examples to help introduce good patterns are:
- Error Processor Sample Application for AWS Lambda https://docs.aws.amazon.com/lambda/latest/dg/sample-errorprocessor.html?icmpid=docs_lambda_landingpage
- Discovering and indexing podcast episodes using Amazon Transcribe and Amazon Comprehend https://aws.amazon.com/blogs/machine-learning/discovering-and-indexing-podcast-episodes-using-amazon-transcribe-and-amazon-comprehend/
- Here is a hack to suspend a Lambda function and then resume. This is definitely an anti pattern and better handled using AWS Step Functions https://medium.com/@galbashan1/aws-lambda-internals-part-2-going-deeper-1e12b9d2515f . While you could do this it is not elegant and could potentially impact the service more broadly. I provide this to highlight the ‘different thinking’ needed to efficiently ’exploit’ AWS services. Loose coupling is a key architectural principle in architecting for cloud and is definitely the approach you should take for serverless and ephemeral functions.
Firecracker Announcement (circa Nov 2018) Firecracker – Lightweight Virtualization for Serverless Computing (Secure and fast microVMs for serverless computing and containers). Think of firecracker as next generation ‘fabric’ to replace legacy compute underlying containers, Lambda and edge computing. Firecracker also brings first class security to containers. https://aws.amazon.com/blogs/aws/firecracker-lightweight-virtualization-for-serverless-computing/ and here’s the github page https://firecracker-microvm.github.io/ and the Firecracker documentation https://github.com/firecracker-microvm/firecracker
Best Practices for Working with AWS Lambda Functions https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html
Best Practices for Amazon SQS ( SQS and SNS are key services for building loosely coupled architectures) https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/working-with-messages.html

Security

Neat graphic and detailed description of signature version 4 signing of authenticating requests. https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html information about signature version 2 (generally deprecated and less preferred than signature version 4) https://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html
Deep Dive on the Nitro hypervisor and the security benefits of loosely coupling the hypervisor (or more correctly the compute management system and control plane) in a video titled AWS Live re:Inforce - Security Benefits of the EC2 Nitro Architecture https://www.youtube.com/watch?v=t_9CASbagag And Nitro is also described in detail in Amazon EC2 High Memory instances for SAP HANA: simple, flexible, powerful https://aws.amazon.com/blogs/awsforsap/amazon-ec2-high-memory-instances-for-sap-hana-simple-flexible-powerful/
AWS Key Management Service Cryptographic Details explains many of the encryption details about the myriad of encryption options available on AWS. https://d0.awsstatic.com/whitepapers/KMS-Cryptographic-Details.pdf

Security bulletins published by AWS https://aws.amazon.com/security/security-bulletins/

AWS SDKs

Start with the Python SDK (boto3) to understand how you can interact with AWS service APIs (client SDK calls), your AWS resources directly (using resource SDK calls) and using AWS sessions with client and resource SDK Calls - Python Boto3 documentation
Difference in boto3 between resource, client, and session? which is a brief introduction from Stack Overflow https://stackoverflow.com/questions/42809096/difference-in-boto3-between-resource-client-and-session
http://www.oznetnerd.com/python-demystifying-aws-boto3/ which is a longer and more descriptive introduction to client, resource and session SDK calls http://www.oznetnerd.com/python-demystifying-aws-boto3/
2014 ReInvent talk on boto3 https://www.youtube.com/watch?v=Cb2czfCV4Dg

Self paced Learning and Building

Read the service FAQ pages, http://aws.amazon.com/faqs/, and documentation for each of the services. Just search for AWS + + documentation in any search engine. You can keep the documentation as pdf, html online or even in your Kindle. You can also git clone the documentation for most services.
Find and build interesting AWS and partner solutions you find the in AWS Blog https://aws.amazon.com/blogs/ . Any post you find with a yellow launch button will build that solution using Cloudformation.
AWS free digital training is mostly 100 level but we also have over 40 hours of Machine Learning training available for free. You can search by topic, role or level. https://www.aws.training/LearningLibrary?src=courses
You can also take AWS Qwiklabs Labs for free at https://aws.amazon.com/training/self-paced-labs/
Get a sandbox or personal account. There are free tiers for many services. https://aws.amazon.com/free/
http://run.qwiklabs.com and complete quests and labs. These enhance your familiarity with AWS services without you having to use your own account. Some labs are free. Others will require you to redeem Qwiklab credits. Reach out to your training manager or AWS account manager. Also check out the Exam guides for SA, SysOps and Advanced Networking https://www.amazon.com/Certified-Advanced-Networking-Official-Study/dp/1119439833/ref=sr_1_1?s=books&ie=UTF8&qid=1519925473&sr=1-1&keywords=advanced+networking
Search github, https://github.com/aws , and the AWS blogs, https://aws.amazon.com/blogs/ , for solutions that interest you. Look for posts with a launch button. These will build a complete environment using Cloudformation. Retrieve the Cloudformation templates either from the built environment in your account or from Github. You can reverse engineer or use these templates as scaffolds for your own use.
Visit Stackoverflow and the AWS discussion forum to pose questions or to contribute to answers about AWS
You can also take the AWS edX ‘Amazon SageMaker: Simplifying Machine Learning Application Development’ Check out https://www.edx.org/course/simplifying-machine-learning-app-development-with-amazon-sagemaker to enroll.
There are many other self paced labs and solutions you can build on AWS. Try:
- Build a Serverless Web Application https://aws.amazon.com/getting-started/projects/build-serverless-web-app-lambda-apigateway-s3-dynamodb-cognito/
- How about AWS Developer Center https://aws.amazon.com/developer/ where you can build the Mythical Misfits app in your choice of programming language.
Now go build stuff…

Continue reading articles in my Amazon Web Services series

Developing on AWS