Skip to content
Private S3 Artifact Store

Private S3 Artifact Store

Publish a private object in Sydney, replicate it to Auckland, and curl it from a VPC in each Region through an S3 gateway endpoint.
Get Started

Why this lab

Section titled “Why this lab”

You want the same private object available in Sydney and Auckland. A Pulp-style HTTP repo can do that too, but you end up running a server and owning the network path. Here S3 does the heavy lifting: publish once in Sydney, let replication
Cross-Region Replication. S3 copies objects (and versions) from a primary bucket to a replica in another Region.
 copy to Auckland, and have each consumer VPC read its local bucket through a gateway VPC endpoint
S3 gateway VPC endpoint. Route-table target that keeps S3 traffic on the AWS network; no hourly charge for the gateway itself.
.

Once the object has replicated, Auckland keeps reading Auckland if Sydney is down (and the other way around). That is not full multi-Region HA
High availability. Design goal of staying usable through failures; multi-Region HA usually needs failover routing, not only a replica bucket. though. You still publish to Sydney, replication is async so RPO
Recovery point objective. How much data you can afford to lose after a failure, measured as time. Asynchronous replication means RPO is greater than zero. is not zero, and anything in a failed Region is still down.

Pulp-style HTTP repo versus regional S3 Same job on both sides: Sydney and Auckland consumers. Left keeps one HTTP repo and a network path that reaches both Regions. Right publishes once to Sydney S3, replicates to Auckland, and each consumer reads the local bucket through a gateway VPCE. HTTP repo (Pulp-style) Icon-Resource/General-Resource/Res_Client_48_Light Publisher Pulp / HTTP repo one Region Icon-Architecture/64/Arch_AWS-Transit-Gateway_64 Network path you operate this too Icon-Resource/General-Resource/Res_Client_48_Light Consumer Sydney Icon-Resource/General-Resource/Res_Client_48_Light Consumer Auckland This lab (S3) Icon-Resource/General-Resource/Res_Client_48_Light Publisher Icon-Architecture/64/Arch_Amazon-Simple-Storage-Service_64 S3 primary Sydney Icon-Resource/Storage/Res_Amazon-Simple-Storage-Service_S3-Replication_48 Replication Icon-Architecture/64/Arch_Amazon-Simple-Storage-Service_64 S3 replica Auckland Icon-Resource/Networking-and-Content-Delivery/Res_Amazon-VPC_Endpoints_48 Gateway Syd VPCE Icon-Resource/General-Resource/Res_Client_48_Light Consumer Sydney Icon-Resource/Networking-and-Content-Delivery/Res_Amazon-VPC_Endpoints_48 Gateway Akl VPCE Icon-Resource/General-Resource/Res_Client_48_Light Consumer Auckland

What you will do

Section titled “What you will do”

Buckets + replication

Primary in ap-southeast-2, replica in ap-southeast-6, filtered to a demo/ prefix.

Consumer VPCs

An S3 gateway endpoint
S3 gateway VPC endpoint. Route-table target that keeps S3 traffic on the AWS network; no hourly charge for the gateway itself.
 per VPC, locked with aws:SourceVpce
Bucket policy condition key aws:SourceVpce. Limits which VPC endpoint IDs can access the bucket..

Publish and prove

Upload in Sydney, wait for the replica, curl from an SSM-managed EC2.

Tear down

Empty the versioned buckets and delete the lab with demo.sh down.

Lab architecture: private S3 artifact store Publish to an S3 primary bucket in Sydney; CRR copies objects to an Auckland replica. Consumer VPCs in each Region read via regional S3 gateway VPC endpoints. Sydney primary → Auckland replica, regional S3 gateways ap-southeast-2 (Sydney) ap-southeast-6 (Auckland) publish S3 primary bucket Replication demo/ S3 replica bucket Gateway VPCE S3 Consumer VPC Gateway VPCE S3 Consumer VPC
Expectation Notes
Same object key Different regional hostnames
Cross-region reads Via the replica, not cross-Region PrivateLink
Cost while running S3 + replication transfer + probe EC2 / SSM endpoints

Read in this order

Section titled “Read in this order”
  1. Architecture
  2. Lab findings: New Zealand
  3. Gateway VPCE
  4. Replication Syd to Akl
  5. Deploy and operate

References

Section titled “References”