Neon is Generally Available! Serverless Postgres with branching to boost your development velocity.Read more
Community

Moving scale-to-zero workloads from Aurora Serverless V1 to Neon

Scale to zero is dead, long live scale to zero!

Post image

Amazon recently announced that Aurora Serverless V1 will shut down at the end of the year, and all users need to migrate to V2. The elephant in the room is that unlike V1, there is no scale to zero in Aurora Serverless V2.

If you’re unfamiliar, Neon is serverless Postgres with performant scale to zero. Our cold-start time is much faster, with a P95 of 500ms. This makes it extremely efficient for any Postgres workload that may have periods of inactivity. We will discuss the V1/V2/Neon comparison and migration steps below.

Because it cannot scale to zero, an Aurora V2 instance costs $43.8/month at a bare minimum. That’s assuming an instance with zero load running at the V2 minimum of 0.5 ACUs/hr. AWS charges $0.12 per ACU-hour so the calculation is ($0.12 x 0.5 ACU x 730 hrs = $43.8/month).

$43.8 doesn’t sound too bad in isolation. What will really hurt V1 users is a “lift and shift” transition of a mature V1 architecture where idle instances were free to the same architecture on V2, where they’re not.

“I learned the expensive way that Aurora Serverless v2 does not, in fact, scale down to 0 when I created three instances for a few days when I was testing out configuration changes. It ended up costing us $300.”  

superdeeda on HN

To illustrate the cost incurred migrating from V1 to V2 let’s go through a typical V1 scenario.

Example scenario

A team’s dev/test/scratch infra on V1

Say you have an active development team of five engineers working on a data-intensive product. At any given time that you check in on your AWS account, you see 25 scale-to-zero V1 instances, but when you look at the total hours they’re running, it’s about 1000 ACUs per month.

Total InstancesACU Hours UsedCost
AWS Aurora Serverless V1251000$120

That costs you about $120 (1000 hrs x $0.12) in compute. That’s very reasonable for the dev/test/scratch workloads of a 5 person team.

Moving from V1 to V2

When moved over to V2, the mere existence of 25 instances would cost you $1,095 per month in compute. (25 instances x 730hrs x 0.5ACU/hrs x $0.12) The 1000 hours where the instances are actually getting used might register you another $100-$500/month, varying based on the max scale limits you set.

Total InstancesACU Hours UsedCost
AWS Aurora Serverless V1251000$120
AWS Aurora Serverless V2259,125 idling + [variable] active$1,095+

In summary: Moving dev/test/scratch to V2 without changing anything would 10x your infra costs.

Nobody wants to pay 10x more for the same service, what’s the alternative? You could re-architect your infra usage to create and destroy instances on demand, but that’s going to take valuable engineering time, and we all know the pain of creating an instance and waiting minutes for it to come online.

Moving from V1 to Neon

Neon charges based on similar metrics of compute, storage, bandwidth, (Neon actually runs on AWS behind the scenes). And as in AWS, most of the Neon bill typically comes from compute. Neon charges $0.102 per CU/hr (our equivalent to ACU/hr)

The logical equivalent of an instance is a Project on Neon, so 25 instances would migrate to 25 projects. By default, all projects will autosuspend when idle, so the actual hours used would look very similar to the hours used on V1.

Total InstancesACU Hours UsedCost
AWS Aurora Serverless V1251000$120
AWS Aurora Serverless V2259,125 idling + [variable] active$1,095+
Neon25 projects1000$102

In summary: A workload that took 1000 ACU/hrs on V1 and cost $120 in compute would cost $102 on Neon.

Keep in mind that Neon is not exactly the same as Aurora Serverless V1

  • Faster cold starts – 500ms P95 start time on Neon, vs 20-60s on V1
  • Branching – You can create a branch of a Neon database just like you create branches of code in git. 
  • Full Postgres – You have access to all the amazing features of Postgres: Extensions, triggers, CDC, etc..

As in all technical decisions, though, it’s about trade-offs. Here are some of the hurdles to overcome when moving from V1 to Neon:

  • Requires engineering effort – The engineering hours are the biggest cost of any migration effort.
  • Outside your AWS VPC – this is probably the biggest one, but for dev/test/scratch this might be workable.

How to migrate to Neon

It’s possible to move your relational Postgres data from AWS Aurora to Neon Postgres using the AWS Database Migration Service (DMS). Using a replication server, DMS will copy the data from your Aurora source database in AWS to your target Neon PostgreSQL database. This process, controlled by a migration task, will maintain the integrity of your database structure with schema conversion and table mapping during replication.

Conclusion

You shouldn’t be penalized because you took advantage of scale to zero in Aurora Serverless V1. If you have the kinds of workloads that don’t need to run 24/7, you shouldn’t have to pay for a dedicated vm to sit around waiting for work just because AWS decided not to build scale to zero into Serverless V2.

If you’re interested in mapping out whether your V1 workload would fit on Neon, and what the migration and cost would look like, sign up for a free account and try it out for yourself, or just get in touch with our team here.