Lose a Region and Stay Online: Multi-Region Failover

What you will build

A two-region architecture with a database that follows you, traffic that moves automatically, and numbers that prove it works.

Starting point: you only need a Vultr account. This challenge is self-contained, so you can do it without having done any other.

The requirements

• The same app running in two different Vultr regions
• A primary database with a replica in the second region, and a tested promotion
• DNS-based failover with health checks and a short TTL
• A write workload running during the drill so data loss is measurable
• An outage simulated by cutting the primary region's traffic or powering it off, not by stopping the app politely

The drill

1. Start a script that writes a numbered record every second and reads it back
2. Cut off the primary region
3. Record when the first user-visible error happens and when service returns
4. Promote the replica and confirm the last record that survived
5. Fail back and describe what you had to reconcile

while true; do curl -s -o /dev/null -w '%{http_code} %{time_total}\n' https://app.example.com/health; sleep 1; done

How you will be judged

• Measured RTO (time to recover) and RPO (data lost), not estimates
• Everything defined as code so a third region could be added
• An honest list of what is still a single point of failure

Destroy everything you created when you are done so you are not billed for it.