Dave Peckdavepeck.org

First Thoughts on the App Engine Blobstore

December 15, 2009

App Engine SDK 1.3.0 is here, and with it a major new feature: the blobstore. City-Go-Round has encountered early growing pains related to latency when serving screenshots from the datastore — just the sort of thing we need a blobstore for!

The basic story is simple: you call an API that manufactures a POST URL. App Engine processes the POST, stripping out all file content and creating blobs (which are opaque) and BlobInfo objects (which are read-only datastore entities.) App Engine then calls your handler, substituting blob keys for the original file content in the POST data. When you’re ready to serve a blob, you simply include the appropriate key in a response header; App Engine traps for this and serves your data as fast as it can.

This is about what I’d expect, with two exceptions:

  1. There is no (straightforward) way to access blob contents from within your handlers. On the one hand, this protects naive developers who might attempt to serve blob bytes directly. On the other hand, this prevents developers from performing custom processing.

    In order to get at blob bytes (for example, to resize an image) it seems that you must URL fetch the blob, perform your transform, and URL POST back. This doesn’t seem as straightforward as it could be; it also doesn’t seem to be optimal from a performance perspective — though this is hard to judge without knowledge of the blobstore’s internal architecture.

  2. I’m uncertain how cleanly the POST/redirect model will integrate with Django’s forms model. It should be easy to create a new AppEngineBlobStoreField for forms, but the catch comes when some other part of the form fails validation. By the time your form handler is invoked, blobs have been created and stored. If the form does not pass validation, you’ll have to manually remove the blobs and try again. (Or do something nicer, like keep the blobs around across multiple presentations of the form.)

All in all, this appears to be a promising start for the service. I look forward to getting those pesky screenshots out of City-Go-Round’s datastore and into its newly minted blobstore!