Hands-on Lab – AWS Scalability & High Availability (TicketHub)

1

🟢 Easy · Classification

Classify requirements: Scalability (S), High Availability (H), or Both (S+H)

The TicketHub leadership handed you 8 requirements from different departments. For each one, mark it S, H, or S+H, and add a one-line justification.

// 8 requirements 1. Handle 200,000 users hitting "Buy" at the same second. 2. When one AWS data centre is lost, tickets still sell. 3. Never sell the same seat to two people, even during a spike. 4. Bring up new servers automatically within 3 minutes when traffic rises. 5. Fans already browsing should not lose their cart on a deploy. 6. Infrastructure cost should not double when traffic peaks for 5 minutes. 7. Payment data must not be lost if a disk fails. 8. A single unhealthy instance must be removed from rotation automatically.

💡 Hint

Ask yourself: is this about how much load the system can handle, or about staying alive when something fails? Some requirements will be both — for example, automatic replacement of an unhealthy instance is HA, but if it's driven by a load signal, it can be S+H.

📘 Solution

1 → S: Extreme load → horizontal scaling + queue + caching. Failure-resistance isn't the focus.
2 → H: Surviving the loss of a whole AZ → spread the stack across ≥ 2 AZs.
3 → S+H: Correct inventory under high concurrency requires scalable transactional consistency. If the DB goes down mid-sale, you must not oversell — so it also touches HA.
4 → S: Auto Scaling Group reacting to demand. Fast response matters more than fault tolerance here.
5 → H: Zero-downtime deploys — rolling update + Connection Draining + session externalization.
6 → S: Cost efficiency of scaling. This is about the economics of horizontal scale, not fault tolerance.
7 → H: Durability — Multi-AZ RDS with synchronous replication on write.
8 → S+H: Automatic instance replacement is HA. If it also happens because of a load-driven health signal, it helps scale as well.

🎯 Key takeaway: Scalability = capacity. High Availability = uninterrupted service when things fail. A system that scales beautifully can still go dark in 10 seconds if all its instances live in one AZ.

2

🟢 Easy · Service selection

Choose the right Elastic Load Balancer for each workload

TicketHub has 3 distinct traffic paths. For each, pick ALB, NLB, or GWLB and give a one-sentence reason.

// Three workloads A. Public website + REST API (HTTP/HTTPS), needs to route /api/v1/* to one group and /admin/* to another, with WAF in front. B. Internal gRPC service used by the payment microservice. It is TCP-based, needs single-digit-millisecond latency, and the upstream partner whitelists by static IP. C. A third-party DDoS scrubbing appliance must inspect all ingress traffic inline (bump-in-the-wire) before it enters the VPC.

💡 Hint

Map each workload to an OSI layer: HTTP/HTTPS = Layer 7 → ALB. Raw TCP/UDP, static IP, ultra-low latency = Layer 4 → NLB. Inline security appliance = Layer 3 + GENEVE → GWLB.

📘 Solution

A → ALB (Application Load Balancer)
Layer 7, understands HTTP/HTTPS. Only ALB supports path-based routing (/api/v1/* vs /admin/*) plus native WAF attachment.

B → NLB (Network Load Balancer)
Layer 4 (TCP). Perfect for gRPC over TCP with static Elastic IPs, and gives sub-millisecond latency by operating at the connection level instead of parsing HTTP.

C → GWLB (Gateway Load Balancer)
Layer 3 with the GENEVE protocol (port 6081). Designed for bump-in-the-wire insertion of third-party security appliances without changing source/destination IPs.

🎯 Common trap: Some people pick NLB for A "because it's faster". But NLB cannot route by path — it would send every request to the same target group. Choosing the wrong layer means choosing the wrong tool.

3

🟢 Easy · Security

Fix the Security Group misconfiguration

TicketHub is moving from a single EC2 with an Elastic IP to: Internet → ALB → EC2 backend. The team has already written Security Groups, but there are three mistakes. Identify them and rewrite the rules.

// Current configuration — find the 3 mistakes sg-alb (attached to ALB): Inbound : HTTP(80) from 0.0.0.0/0 ✓ Inbound : HTTPS(443) from 0.0.0.0/0 ✓ Outbound: all to 0.0.0.0/0 ✓ sg-app (attached to EC2 backend): Inbound : HTTP(80) from 0.0.0.0/0 ?? Inbound : SSH(22) from 0.0.0.0/0 ?? Outbound: all to 0.0.0.0/0 ✓

💡 Hint

Security Groups support referencing another SG as the source. For EC2 backends, HTTP should come from the ALB — not from the whole internet. SSH should never be open to 0.0.0.0/0.

📘 Solution

The three mistakes:

sg-app opens HTTP(80) to 0.0.0.0/0 — anyone can bypass the ALB and hit the backend directly.
sg-app opens SSH(22) to 0.0.0.0/0 — a classic attack vector.
The ALB has no explicit outbound rule to the backend — actually the default allows all outbound, but for least privilege you should restrict it to the app SG.

// Correct configuration sg-alb (attached to ALB): Inbound : HTTP(80) from 0.0.0.0/0 # redirect 80 → 443 Inbound : HTTPS(443) from 0.0.0.0/0 # public traffic Outbound: TCP(8080) to sg-app # least privilege sg-app (attached to EC2 backend): Inbound : TCP(8080) from sg-alb # ONLY from the ALB Inbound : SSH(22) from sg-bastion # only via bastion host Outbound: all to 0.0.0.0/0 # to call APIs, package repos…

🎯 Golden rule: Never expose the EC2 backend directly to the internet. Reference the ALB's Security Group (source-group) rather than a CIDR block — the ALB's internal IP can change, but its SG-ID never does. This keeps WAF, access logs and rate limiting effective.

4

🟢 Easy · Architecture

Design a minimal Multi-AZ layout for TicketHub

Draw (or describe in words) the minimum architecture that gives TicketHub High Availability in the region ap-southeast-1. Clearly list: number of AZs, subnets, subnet types (public/private), and where each component sits.

// Constraints - Region: ap-southeast-1 has 3 AZs available: 1a, 1b, 1c - Budget: tight. Use the minimum number of AZs that still gives HA. - Components: ALB, EC2 app tier, RDS MySQL (write-heavy) - Must be reachable on HTTPS from the internet - Must allow EC2 to install packages / call AWS APIs

💡 Hint

Minimum HA is 2 AZs. Each AZ needs a public subnet (ALB + NAT) and a private subnet (EC2 + RDS). Don't put EC2 in public subnets.

📘 Solution

VPC 10.0.0.0/16 │ ├── AZ ap-southeast-1a │ ├── Public Subnet 10.0.1.0/24 → ALB node A + NAT Gateway A │ └── Private Subnet 10.0.11.0/24 → EC2 app A + RDS primary │ └── AZ ap-southeast-1b ├── Public Subnet 10.0.2.0/24 → ALB node B + NAT Gateway B └── Private Subnet 10.0.12.0/24 → EC2 app B + RDS standby

✅ ALB spans both public subnets — one LB node per AZ, always reachable even when one AZ is lost.
✅ EC2 app tier in private subnets, managed by an Auto Scaling Group spanning both subnets.
✅ RDS MySQL Multi-AZ with a synchronous standby in the other AZ, automatic failover on primary failure.
✅ NAT Gateway: one per AZ is the HA-correct choice. For cost optimization you can start with one and accept the AZ-level risk — call this out explicitly.
✅ No third AZ needed. Two AZs already provide HA at minimum cost.

🎯 Remember: Multi-AZ protects against a single datacenter failure. It does not protect against a whole-region failure — that requires Multi-Region, which is far more expensive and usually reserved for global products.

5

🟡 Medium · Health check

Design the Target Group health check endpoint

The engineering team has three candidate endpoints. Pick the one you would attach to the ALB's Target Group, and explain why the other two should not be used.

// Candidate endpoints (Node.js / Express) GET /ping → app.get('/ping', (req,res) => res.send('ok')) Returns 200 instantly. No dependency checks. GET /health → checks: DB SELECT 1, Redis PING, SQS reachable Returns 200 if all OK, 503 otherwise. Typical latency: 8 ms. GET /events → returns the full list of upcoming concerts Heavy query joining 4 tables, average latency 1.1 s.

💡 Hint

A good health check must satisfy two things at once: it must be meaningful (reflect real ability to serve) and cheap (safe to run every 10–15 seconds per instance).

📘 Solution

✅ Choose: GET /health
It validates the critical dependencies. If the DB is gone, the app cannot serve requests even though the Node.js process is alive — this endpoint captures that, while staying under 10 ms so it can run frequently.

❌ Do not choose: GET /ping
It always returns 200 — even if the DB is down and the app is effectively dead. This is the "is the port open?" anti-pattern. The LB would keep sending traffic to a broken instance.

❌ Do not choose: GET /events
Too heavy (1.1 s per call × N instances every 10–15 s). It hammers the DB and can produce false positives when the DB is momentarily slow — the LB would then remove healthy instances and cause an outage during the flash sale.

🎯 Suggested parameters:

Path: /health
Interval: 15 s · Timeout: 5 s
Healthy threshold: 2 · Unhealthy threshold: 3
Matcher: 200-299

6

🟡 Medium · Session

Handle cart sessions during horizontal scaling

TicketHub currently stores each user's cart in the Node.js process memory (variable carts = {}). Once you move to multiple instances behind an ALB, users will lose their carts on every refresh. Propose 3 solutions, rank them, and pick the best one for the flash-sale scenario.

// Current state - Carts stored in RAM of each Node.js process - ALB round-robin, no stickiness - 6 backend instances at peak - On scale-in, the instance is terminated → cart lost - Cart must survive for at least 30 minutes // Constraints - Do not use Sticky Sessions (worried about imbalance at 200k users) - Cart contents are small (< 5 KB per user) - Must be fast: cart read happens on every page view

💡 Hint

Principle: any "state" on an instance is a scaling obstacle. Move it out so the instance becomes stateless. For very fast reads, a cache is the natural home.

📘 Solution

🥇 Recommended: ElastiCache Redis (Multi-AZ)
Store each cart as a Redis hash keyed by cart:{userId} with a TTL of 30 min. Sub-millisecond reads, native replication, automatic failover, and every instance sees the same cart. Also usable as a rate limiter at the API gateway for the flash sale.

🥈 Alternative: DynamoDB with TTL
Put cart items in a DynamoDB table with a 30-minute TTL. Fully managed, HA by default, pay-per-request so cost is trivial at low traffic. Slightly higher latency (single-digit ms) than Redis, but avoids running a cache cluster.

🥉 Alternative: JWT cart token
Sign the cart as a JWT and keep it in a cookie. No server state at all. Downside: the cart becomes hard to revoke, and large carts bloat every request. Usually a bad fit for a flash sale where cart abandonment and abuse are real concerns.

⚠️ Not recommended: Sticky Sessions
Only a band-aid. At scale-in, pinned users lose their cart, and load imbalance forces you to over-provision EC2 — you'll pay more than you would for Redis.

🎯 Decision: For a flash sale with 200k users, pick Redis. Its latency and atomic operations (DECR, SETNX) are exactly what you need for cart updates and stock counters during the drop.

7

🟡 Medium · SSL/TLS

SSL for multiple domains on a single ALB

TicketHub uses three domains — tickethub.vn, api.tickethub.vn, and admin.tickethub.vn. All go through the same ALB on port 443, each with its own ACM certificate. Describe the configuration and explain the role of SNI.

// Technical constraints - An ALB can have only ONE listener per (protocol, port) pair - 3 certificates are already issued in ACM (must be in the same region as the ALB) - HTTP (80) must permanently redirect to HTTPS (443) - /api/* must hit the API target group - /admin/* must hit the admin target group AND require Cognito authentication - Everything else hits the web target group

💡 Hint

If your plan is "create three listeners on 443", stop — an ALB forbids that. Think "one listener, many certificates", and let SNI decide.

📘 Solution

// Configuration Listener :80 → Rule 1: redirect 301 → https://#{host}:443#{path} Listener :443 → Certificates: cert-tickethub.vn (default) cert-api.tickethub.vn cert-admin.tickethub.vn Default action: forward → tg-web Rules on :443 (evaluated top-down) Rule 1: IF host = api.tickethub.vn THEN forward → tg-api Rule 2: IF host = admin.tickethub.vn THEN authenticate (Cognito) → forward → tg-admin Rule 3: ELSE → tg-web

How SNI works: When a browser connects to api.tickethub.vn:443, it includes that hostname in the ClientHello — the first message of the TLS handshake. The ALB reads the SNI field, matches it against the certificate list attached to the listener, and returns the correct one. Without SNI, only one certificate could be served per listener, and the other two domains would fail with a certificate mismatch.

🎯 Key points:

SNI is enabled by default on ALB and NLB — you don't turn it on, you just add certificates.
Certificates must live in the same region as the ALB (Singapore here).
Host-based rules + path-based rules can be combined on the same listener for a clean multi-tenant setup.
One ALB for 3 domains saves money versus three separate ALBs, with zero loss of security.

8

🟡 Medium · Auto Scaling

Pick the right scaling metric and policy for two very different workloads

TicketHub runs two Auto Scaling Groups. Choose the best metric and the policy type (Target Tracking / Step / Simple) for each.

// Workload A — Web API behind ALB - Handles page views and API calls - Request/response mainly waits on DB and Redis I/O - Baseline: 3,000 rpm; flash sale spike: 1,200,000 rpm for 5 minutes - Metric available: ALB RequestCountPerTarget // Workload B — Reservation worker - Consumes messages from SQS ("lock seat N for user X") - Each message takes 40–120 ms of CPU - Queue can grow from 0 to 300,000 messages within 30 seconds - Current ASG: min 2, max 40 // Constraints (both) - Scale out fast, scale in slowly (avoid flapping) - Do not exceed $1,500/month

💡 Hint

For request-driven web apps, CPU lags behind the real signal. For queue workers, CPU may be low even when the backlog is huge. The right metric is the one closest to the work.

📘 Solution

Workload A — Web API

Metric: ALB RequestCountPerTarget. CPU is a poor signal because the app mostly waits on I/O. A spike from 3k → 1.2M rpm will not appear on CPU until well after the LB is saturated.
Policy: Target Tracking on RequestCountPerTarget, target ≈ 800–1,000 requests per target per minute.
Cooldown: ScaleOutCooldown = 30–60 s (fast), ScaleInCooldown = 300 s (slow).
Extra: Because the spike is only 5 minutes, add a Scheduled Scaling action for the known drop time so capacity is already warm before traffic arrives.

Workload B — Reservation worker

Metric: SQS ApproximateNumberOfMessagesVisible divided by running instances (backlog-per-instance). CPU can lag and doesn't reflect pending work.
Policy: Target Tracking on backlog-per-instance, target ≈ 100 messages per instance. Use Step Scaling instead if you want a stronger reaction once the queue crosses hard thresholds (e.g., add 10 instances if backlog > 1,000).
Cooldown: Similar — quick scale-out to drain the queue, slow scale-in to avoid interrupting seat-lock operations.

🎯 Notes for a flash sale:

Set MinCapacity high (e.g., 20 for web, 10 for worker) on drop day.
Pre-warm AMIs and use EC2 Warm Pools so new instances are ready in seconds, not minutes.
Set MaxCapacity generously — spot instances can backfill if you're cost-sensitive.

9

🟡 Medium · Configuration

Set Health Check Grace Period and Deregistration Delay

Measurements from the current staging environment. Propose specific numeric values for Health Check Grace Period and Deregistration Delay, and explain the arithmetic.

// Measured values (P95) Node.js app startup time : 70 seconds Seat-lock transaction (typical) : 250 ms Seat-lock transaction (slowest) : 4 seconds Checkout / payment call (slowest) : 22 seconds Health check interval : 15 seconds Healthy threshold : 2 consecutive checks Unhealthy threshold : 3 consecutive checks // Goals - Do not terminate instances prematurely during startup - Do not cut off an in-flight payment when scaling in - Still react quickly when an instance truly fails

💡 Hint

Grace Period ≥ (startup time) + (interval × healthy threshold) + buffer. Deregistration Delay ≥ (slowest in-flight request) + buffer.

📘 Solution

Health Check Grace Period → 120 seconds
Arithmetic: startup 70 s + (15 s × 2 healthy checks) = 100 s. Add ~20 s buffer for a cold network and slow disk attach → 120 s. If you set it to 60 s, the ASG will terminate every new instance before it can become healthy, creating an endless launch–fail–terminate loop.

Deregistration Delay → 30 seconds
The slowest in-flight request is a payment call at 22 s. Set 30 s to allow that plus a 8 s safety margin. Setting it to the default 300 s would make scale-in glacially slow; setting it to 5 s would cut payments mid-flight and produce 502/504s.

🎯 Practical checklist:

Add a /health endpoint that verifies the DB and Redis connection — otherwise your grace period is meaningless.
Ensure the app handles SIGTERM: stop accepting new requests and finish active ones before shutdown.
Test the sequence by triggering a scale-in on staging and watching ALB access logs for 5xx during the draining window.

10

🔴 Hard · Troubleshooting

Diagnose 502/504 errors during a rolling deploy 5 minutes before the drop

TicketHub now runs on an ALB + ASG. The team deploys a new version 5 minutes before a smaller concert drop. Immediately, users see a wave of 502 and 504 errors that lasts for about 3 minutes. Diagnose 4 plausible causes and the fix for each.

// Observed symptoms - Errors start exactly at deploy (rolling update) - 502 Bad Gateway : ~35% of requests - 504 Gateway Timeout : ~20% of requests - Recovery after ~3 minutes - No application-level errors in logs (server-side) // Configuration right now Health check grace period : 60 s (default not changed) Deregistration delay : 20 s (team tuned this from default) Rolling update settings : MaxBatchSize=4, MinInstancesInService=desired-1 App startup time : 70 s (P95) Slowest request path : checkout → 22 s

💡 Hint

502 = backend gone or connection dropped mid-flight. 504 = backend too slow, LB gave up. Compare each configured timer against the measured reality above — several are shorter than they should be.

📘 Solution

Cause 1: Deregistration delay (20 s) is shorter than the slowest request (22 s)
Checkout requests that take 22 s get cut off at second 20 → 502 for the user.
Fix: raise deregistration_delay.timeout_seconds to 30–45 s.

Cause 2: Grace period (60 s) is shorter than startup (70 s)
New instances need 70 s, but ASG stops ignoring health checks after 60 s. It sees an unhealthy instance and terminates it → the ASG thrashes, effective capacity drops, and the remaining instances get overloaded → 504.
Fix: raise health_check_grace_period to 120 s.

Cause 3: Rolling update is too aggressive
MaxBatchSize=4 with MinInstancesInService=desired-1 means 4 instances are being replaced while only one stays in service. During a drop, that's not enough capacity to absorb traffic → 504.
Fix: use MaxBatchSize=1 and MinInstancesInService=desired-2 (or higher); add PauseTime=120 s. Better still: do not deploy 5 minutes before a drop.

Cause 4: Node.js cold start / JIT warm-up
Right after the first health check passes, an instance is technically healthy but has cold caches and uncompiled hot paths. The first few hundred requests are slow → 504.
Fix: add a warm-up script that calls key routes right after /health passes, or use ALB slow start (if available) so traffic ramps over 30–60 s.

🎯 Systematic debugging order for any deploy-time 502/504:

Compare deregistration delay against the slowest in-flight request.
Compare grace period against app startup time + healthy threshold × interval.
Check rolling update parameters (batch size, min in service, pause time).
Look for cold start effects and add warm-up / slow start.

11

🔴 Hard · Cost & Design

"Sticky Sessions vs. Redis for carts" — pick one and defend it with numbers

The finance team asks: "Why spend $50/month on Redis when sticky sessions are free?" Answer with numbers.

// Option A — Enable Sticky Sessions on the ALB - stickiness.type = lb_cookie - stickiness.lb_cookie.duration = 3600 s - No code change needed - Expected effect: -25% DB reads because users hit the same instance - Risk: uneven load across instances // Option B — Move carts to ElastiCache Redis (Multi-AZ) - cache.t3.medium × 2 (for HA) ≈ $110/month - Code change: 1–2 dev-days - Expected effect: -25% DB reads, plus stateless instances - Also reusable for rate-limiting and seat-lock counters during the drop // Numbers you can use - Current EC2 spend : $700/month - Peak instances (no sticky): 20 - Observed imbalance with sticky: top instance carries 3× the average - Cost per m5.large : $0.096/hour = ~$70/month - Budget ceiling : $1,500/month

💡 Hint

"Free" solutions often hide cost elsewhere. If one instance carries 3× the average, the fleet has to be sized for that outlier — you pay for idle capacity on every other node.

📘 Solution

Option A — the "free" sticky session

Imbalance: top instance = 3× average. To keep the top instance under 70% CPU, you must size the whole fleet for 3× — roughly 10 extra m5.large over 20 instances.
Extra monthly cost: 10 × $70 ≈ $700/month — but only during peak hours. Averaged at ~40% peak hours: ≈ $280/month.
Add the loss of zero-downtime deploys: every scale-in drops sessions for pinned users, and users abandon carts. A single lost ticket sale costs more than Redis for the month.
Verdict: sticky sessions here cost 5× more than Redis, and produce a worse UX.

Option B — Redis in Multi-AZ

Fixed cost: ~$110/month for two nodes (primary + replica for HA).
Instances become stateless → fleet can be sized to actual load, saving ≈ $280/month in EC2.
Redis is also a rate limiter at 200k requests and a seat-lock counter using atomic DECR — one tool, three jobs.
One-time engineering cost: 1–2 dev-days to change the session middleware. This is an investment, not an expense.

🎯 Final recommendation: Choose Redis. Net monthly saving ≈ $170 (EC2 savings – Redis cost) before counting the revenue protected by zero-downtime deploys and correct carts. When someone says a solution is "free", ask them to size the fleet — the hidden cost usually shows up right there.

12

🔴 Hard · Capstone design

Design the full architecture for the next flash sale

Put it all together. Produce the complete target architecture for TicketHub's next Blackpink-scale drop, containing an ASCII diagram, a list of AWS services used and why, and 3 residual risks.

// Combined requirements 1. Survive 200,000 concurrent users at the exact moment of the drop 2. Never fully go down if one AZ fails 3. Deploy a new version with zero user-visible errors 4. Do not oversell: a seat must never be reserved by 2 users at once 5. Same ALB must serve tickethub.vn, api.tickethub.vn and admin.tickethub.vn with their own certs 6. Cart must survive for 30 minutes, across scale-in 7. Ticket PDF generation must happen asynchronously (worker) 8. Stay under $1,500/month when idle traffic is normal // Budget reality - Current spend : $300/month - Allowed spend : $1,500/month - Prefer elastic costs (pay only when scaling)

💡 Hint

Split the system into three tiers and make every tier multi-AZ: Web tier (ALB + ASG), App tier (worker + SQS), Data tier (RDS Multi-AZ + ElastiCache + S3). Add CloudFront at the edge so static assets (concert photos, JS bundles) never reach your app tier.

📘 Solution

🗺️ Reference architecture

Users │ Route 53 (3 records, latency + health checks) │ CloudFront ← static assets from S3 (photos, JS) │ AWS WAF (rate-limit per IP, bot rules) │ ┌─────────┴─────────┐ │ ALB │ Listener 80 → redirect 443 │ 3 certs via SNI │ Listener 443 with host/path rules │ /api → tg-api │ │ /admin → tg-adm │ └─────────┬─────────┘ │ ┌─────────────────────┴─────────────────────┐ │ Web ASG │ │ (EC2 t3.medium, 6–60 instances) │ │ Target Tracking: ALB req/target │ │ │ ┌────────┴────────┐ ┌────────┴────────┐ │ AZ 1a │ │ AZ 1b │ │ Private subnet │ │ Private subnet │ │ EC2 web × N │ │ EC2 web × N │ └────────┬────────┘ └────────┬────────┘ │ │ └───────────────┬─────────────────────────┘ │ ┌───────────────┴───────────────┐ │ Data Tier │ │ RDS MySQL Multi-AZ (writer) │ │ ElastiCache Redis (Multi-AZ) │ ← cart, session, seat-lock │ S3 buckets (tickets, media) │ └───────────────────────────────┘ Async pipeline (order fulfilment) Web ASG → SQS "ticket-orders" (standard, with DLQ) │ ▼ Worker ASG (EC2 c5.large, 2–40 instances) │ Target Tracking: SQS backlog per instance │ ├── reserve seat atomically (Redis DECR + RDS transaction) ├── generate PDF via headless Chrome ├── upload PDF to S3 └── send email via SES

📦 AWS services and why

🌐 Route 53 — DNS with health checks so users are routed to a healthy ALB endpoint.

⚡ CloudFront — keeps 80% of image/JS traffic off your app tier.

🛡️ AWS WAF — blocks bots and scalper scripts at Layer 7 before they hit the ALB.

⚖️ ALB — Layer 7 routing, SNI, host and path rules; single entry point.

🖥️ EC2 + ASG (web) — target tracking on ALB RequestCountPerTarget; warm pool for instant boots.

📬 SQS — decouples order intake from PDF/email generation; a DLQ protects against poison messages.

🖨️ EC2 + ASG (worker) — target tracking on SQS backlog per instance.

🗄️ RDS MySQL Multi-AZ — strong transactional guarantees for seat locking.

🔴 ElastiCache Redis (HA) — cart, session, atomic counters; sub-millisecond reads.

🪣 S3 — ticket PDFs and media; versioning + lifecycle to Glacier.

📊 CloudWatch — alarms on ALB 5xx rate, SQS backlog, RDS connections, Redis evictions.

🔐 ACM — 3 certificates attached to the single 443 listener.

⚠️ 3 residual risks to watch

Risk 1 — RDS writer is the true bottleneck at 200k users. Even with Multi-AZ, all seat-lock writes go to one writer. Use Redis atomic DECR to gate traffic before writes reach RDS, add read replicas for read-heavy queries, and consider Aurora if you need faster failover or a global writer.

Risk 2 — Cold capacity at the moment of the drop. Auto Scaling is reactive; a sudden spike can overwhelm the system in the first 60 seconds. Mitigate with Scheduled Scaling to pre-warm the ASG before the drop, a Warm Pool for instant scale-out, and elevated MinCapacity on drop day.

Risk 3 — Cross-AZ and NAT data transfer costs. A multi-AZ architecture exchanges traffic between AZs (EC2 ↔ RDS, EC2 ↔ Redis). Monitor Cost Explorer weekly, use S3 Gateway VPC Endpoints to avoid NAT charges, and keep chatty services (e.g., Redis) in the same AZ as the client where possible.

🎯 Grading rubric:

Every tier spans ≥ 2 AZs (no single-AZ silos).
No obvious single point of failure — a lost AZ must degrade performance, not kill the service.
Self-healing exists (ASG + health checks + RDS Multi-AZ + DLQ for SQS).
Scaling uses the right metric for each workload (requests for web, backlog for workers).
Users are protected during deploys (Connection Draining + safe rolling update + warm-up).
Cost model is elastic: pay for spikes, not for peak capacity 24/7.

Redesigning TicketHub for the Next Ticket Drop

General Context

Classify requirements: Scalability (S), High Availability (H), or Both (S+H)

Choose the right Elastic Load Balancer for each workload

Fix the Security Group misconfiguration

Design a minimal Multi-AZ layout for TicketHub

Design the Target Group health check endpoint

Handle cart sessions during horizontal scaling

SSL for multiple domains on a single ALB

Pick the right scaling metric and policy for two very different workloads

Set Health Check Grace Period and Deregistration Delay

Diagnose 502/504 errors during a rolling deploy 5 minutes before the drop

"Sticky Sessions vs. Redis for carts" — pick one and defend it with numbers

Design the full architecture for the next flash sale