Elastic Beanstack can be used to automate deployment and scaling .
Wednesday, April 29, 2020
Amazon Aurora and Dynamo DB
DynamoDB will throttle or deny requests that exceed the table's provisioned throughput capacity .
Provisioned Throughput Capacity
Read/Write Operations per second provisioned for your DynamoDB table .
1 Read /Write Unit is 4kb item .
Provisioned Throughput Capacity
Read/Write Operations per second provisioned for your DynamoDB table .
1 Read /Write Unit is 4kb item .
AI
Workflow of a Machine Learning Project .
Example : Amazon Echo
1.collect data
2.Train Model : Iterate many times until good enough .
3. Deploy Model : get data back , maintain update model .
Workflow of a Data Science Project .
1.Collect Data
2. Analyze Data : Iterate many times to get good insight
3. Suggest hypothesis /action : Deploy changes / reanalyze new Data periodically .
Example : Amazon Echo
1.collect data
2.Train Model : Iterate many times until good enough .
3. Deploy Model : get data back , maintain update model .
Workflow of a Data Science Project .
1.Collect Data
2. Analyze Data : Iterate many times to get good insight
3. Suggest hypothesis /action : Deploy changes / reanalyze new Data periodically .
Monday, April 27, 2020
Aws database services
Aws database migration service
Redshirt data warehouse solution.
Petabyte scale .
Wide column storage
Graph database.
Neptune
RDS database snapshot is placed to S3.
This is an increment backup.
Aurora : reduce cost by 90%
Dynamo db runs on top of s3
Redshirt data warehouse solution.
Petabyte scale .
Wide column storage
Graph database.
Neptune
RDS database snapshot is placed to S3.
This is an increment backup.
Aurora : reduce cost by 90%
Dynamo db runs on top of s3
Amazon S3
Simple Storage Service
- Files can be stored and retrieved from anywhere on the web .
- Used for live streaming . Netflix, Hotstar use it for hosting.
- Used for content distribution , big data analytics , backup etc .
- Files are stored in buckets , a max of 100 buckets can be created by a developer .
- Replica of the file is available in 3 different Availability Zones , all in the same region . AWS also supports cross region replication which can be configured manually .
- Supports periodic automatic archival/deletion , using lifecycle rules.
Storage class
Standard available in 3 availability zone
Glacier : cold data.
Archival data ..
Glacier deep
Long data retention
Requester pays
The one who download pays
Object lifecycle management
Move file automatically to glacier after 50 days
Cannot move back from glacier to S3 via lifecycle
Retreival options
EFS : multiple EC2 share same storage .
Nfs : network file system.
Snowball.
Hardware device to migrate a data .
Like a briefcase .
Device will be shipped.
Aws elastic block storage
IaaS
Infrastructure as a Service :
provides access to virtual or physical hardware .
EBS
Elastic Block Storage
- Similar to a hard drive
- Block storage for EC2 and other services
- Automatic replication within its AZ
- Snapshot backup to Amazon S3
- Restore a snapshot to a new EBS volume
- Multiple volumes can be attached a single EC2
- Volumes can be shared between multiple EC2
SSD
HDD
Amazon ECS
ECS supports docker
Eliminate need to install operate your cluster.
Fleet of docker container
Fargate : managed ECS .
Next step to ECS.
Container as a Service
Eliminate need to install operate your cluster.
Fleet of docker container
Fargate : managed ECS .
Next step to ECS.
Container as a Service
Subscribe to:
Posts (Atom)
System Design : Live Streaming and Video On Demand
1. Problem Statement Design an AWS-based live and VOD-to-live HDR streaming platform that can ingest real-time broadcast feeds as well a...
-
id to load is required for loading is a common error you may face while runtime . I have faced this error while a bean call . A session ...
-
Error Blog This blog is about the error Error: Objects are not valid as a React child Problem Statement error on start of...
-
Hi Guys , This blog is about the error " Target container is not a DOM element. " The full error in the console log is ...
