App Engine Charges $6,500 to Update a Listproperty on 14.1 Million Entities

This is an extreme example of App Engine’s surprising new price structure.

It’s worth tearing apart a little bit. To App Engine, an entity with a ListProperty of length N is equivalent to N separate entities. In this case, on average, each list has 8 items; that’s 112.8 million writes. In addition, the ListProperty is indexed; indexes need to be written, too. The math explodes to 6.5 billion write operations which indeed costs $6,500.

Is the cost reasonable, then? Absolutely not. The ListProperty in question contains geobox string prefixes; this is effectively a workaround for the fact that App Engine’s datastore isn’t terribly geo-friendly. The same 14.1 million entities in a PostGIS database might take a little time and a fair amount of I/O to update but even if you were (say) using Amazon EBS, you’d probably spend a couple dollars at most on fees.

Reading the Hacker News comments, you might think that this is operator error or even poor architecture decision-making on the part of the app’s developers. But that’s not the case.

The problem with App Engine’s recent price changes isn’t that things are more expensive (although they are); the problem is that the axes along which charges are made have substantially changed. A similar write operation might have cost tens of dollars under the old scheme.

App Engine’s original premise was that if you developed against its bespoke APIs and adhered to its quirky methodology, then your app would scale at low engineering and monetary cost. The pricing changes are a sign, perhaps, that App Engine couldn’t deliver on its promise.