In recent years, resilience has gone from being a “nice to have” to a critical requirement for any digital platform. Regional failures, although rare, exist, as do problems stemming from human error, and when they do occur, the impact on the business can be huge if you are not prepared.
In this article I want to share a recent and successful project in which, from Unikal Tech Partners, we automated the complete recovery of an AWS environment deployed in a primary region (Region A) to a secondary region (Region B), using Infrastructure as Code (IaC) and native AWS services.
Disaster Recovery on AWS: Goals, Challenges, and Real-World Environment
The main objective set was clear: to recover the platform quickly, repeatably and without manual intervention. Even in the face of a serious failure of an entire region.
The main challenge was the following: regional recovery without improvisation.
The original environment in Region A mainly included the following elements:
- Applications deployed on EC2 behind Application Load Balancers
- Managed databases in Amazon RDS
- Object Storage in Amazon S3
- Queuing (Amazon SQS) and Notification Services (Amazon SNS)
- Network configuration with VPCs, subnets, gateways, and security rules
- Security and compliance services due to the fact that it is a High ENS (National Security Scheme) certified environment
Logically, as it is a critical productive environment, critical dependencies between services were taken into account. One of the main premises set by the client was the following:
“If the region becomes unavailable, we don’t want to rebuild the environment by hand.”
Main challenges in multi-region disaster recovery
The main challenges faced by the company’s CIO were the following:
- Reduce the actual RTO (Recovery Time Objective), since it was not possible to meet the required RTO if the current methodology of work followed in disaster recovery was continued
- Minimize human errors in a crisis scenario, either by not having the resources available with the necessary knowledge to rebuild the environment or by making mistakes in a crisis situation in which the business is pressing for an immediate solution.
- Ensure that the infrastructure in Region B was identical and consistent, as the SLAs committed to their customers did not allow the ecosystem to suffer a degradation of service. If it occurs, economic penalties would be applied.
- To be able to test the recovery plan without affecting production, on a regular basis and with guarantees that the results are realistic.
- To be able to adapt the recovery plan to the changes that the productive ecosystem may undergo, in an easy and controlled way, guaranteeing that the environment deployed in Region B will always be identical to the environment in Region A.
Automated Disaster Recovery Strategy with IaC on AWS
Among the different options we have when carrying out a Disaster Recovery, we opt for an active/passive multi-region strategy, where Region B remains prepared to raise the entire environment on demand. Despite the criticality of the environment, taking into account the trade-off between RTO, RPO and recurring costs, active-active modalities were discarded.
The pillars of the solution were:
1. Infrastructure as Code as the basis of everything
The entire infrastructure was defined as code using Terraform (although the approach is equally valid with AWS CloudFormation):
- VPC, subnets, route tables
- Security Groups and NACLs
- Load Balancers and Target Groups
- EC2, Auto Scaling Groups
- RDS and dependencies
- IAM Roles and Policies
- ENS High Security and Compliance Configurations and Services
As a main premise, we set the following: nothing is created manually. If it’s not in the code, it doesn’t exist.
This allowed us to:
- Replicate your environment in any region
- Versioning changes
- Execute reproducible and auditable deployments
2. Data synchronization and preparation
For data, we take different approaches depending on the service:
1. Amazon S3
Due to the high volumes of data stored in S3, it was impossible to restore the buckets within the RTO, so we opted to:
- We activate cross-region replication
- Versioning enabled for added protection
- Buckets in Region B were always ready
2. Amazon RDS
Since we did not have an active-active solution in which the databases were permanently up, the methodology used was as follows:
- Using Automatic Snapshots
- Copying snapshots to Region B
- IaC definition to restore RDS instances from the last available snapshot.
3. EC2
- Automated creation and copying of AMIs to Region B
- AMIs were used as the basis for Auto Scaling Groups
3. Failover automation
One of the key points of the project was that the DR did not depend on executing manual commands. We create an automated pipeline that:
- Detects the recovery scenario
- Run the full deployment in Region B from IaC
- Restore databases from the latest snapshots
- Lift instances and balancers
- Run basic health validations
The entire process could be started with a single controlled action.
4. Traffic management and DNS
For routing:
- We use Amazon Route 53
- DNS records ready to point to Region B
- TTLs tuned to reduce impact on change
In a regional failure scenario, traffic switching is done in a fast and controlled manner.
Real results of automated disaster recovery
Thanks to this approach, the client was able to:
- Recover the entire environment in Region B in minutes
- Dramatically reduce RTO vs. manual deployment
- Eliminate human error at critical moments
- Test the DR plan regularly and safely
- Having living documentation: the code itself is the documentation
In addition, the use of IaC made it possible to optimize costs, since Region B only consumes minimal resources (storage and backup) until the recovery plan is activated.
5 Key Lessons in Disaster Recovery Projects on AWS
Some key takeaways from the project:
- If it’s not automated, it’s not a real DR
- Infrastructure as Code isn’t just for deployments; it is a tool for resilience.
- Testing DR is just as important as designing it
- An outdated DR is not a useful DR
- AWS provides all the necessary services, but the value is in how they are integrated
Conclusion
Disaster recovery should not be a document forgotten in a drawer. It must be a living, tested, and automated process. AWS, combined with Infrastructure as Code, enables you to build high-availability and regional recovery solutions in an elegant, secure, and efficient way.
If your platform still relies on manual steps to recover from a major failure, it’s probably not as prepared as you think. We invite you, from Unikal Tech Partners, to review your Disaster Recovery Plan in AWS and analyze if it really meets the SLAs set by business.
