When the legacy object storage became unreliable and began affecting API availability, moving 2.5 TiB and 12M+ objects to Amazon S3, from an EU region to APAC, became a priority.
Context
When the legacy object storage became unreliable and began affecting API availability, moving 2.5 TiB and 12M+ objects to Amazon S3, from an EU region to APAC, became a priority.
My role
I planned and executed the migration.
Approach & trade-offs
Before committing to a method, I tested a relay approach and proved it wasn't viable. The migration then used S3 Batch Operations, so the bulk copy ran over the AWS backbone.
Data was verified folder by folder rather than trusted in aggregate, and a delta-sync cutover plan covered objects written during the migration window.
Results
- 2.5 TiBmigrated
- 12M+objects
- 0copy failures
2.5 TiB and more than 12 million objects were copied with zero copy failures, verified folder by folder. I wrote up the details on dev.to: How I Migrate DigitalOcean Spaces to S3: 12 Million Objects, Two Hops, Zero Failures.
Lessons
- Prove the transfer path before scaling it: ruling out the relay approach early saved time.
- Verify at a granularity you can act on. Folder-level checks make gaps visible and fixable.
- Plan the cutover (delta sync) as carefully as the bulk copy.
More case studies
- Edge Security at Scale: Leading a DDoS Response and Hardening the Edge
- Cloud Cost Optimization at FundedNext: 65% Lower Cloud Costs Without Disrupting the Business
Related article: How I Migrate DigitalOcean Spaces to S3 (dev.to)