Cocoa, Sync, and Appengine

The latest edition of Daniel Jalkut and Manton Reece’s excellent Core Intuition podcast discusses “sync” as a general infrastructure problem.

Apple has a great sync story called MobileMe. It allows mail, calendar appointments, contacts, and a few other things to automatically sync via the cloud. Alas, Apple offers no public synchronization API. Apple’s home-grown apps sync up over MobileMe; Cocoa developers are left stranded, forced to “roll their own” sync solution.

Many Sync, Much Headache

As a result, the Mac software world is littered with disjoint sync strategies. Often, these hand-rolled strategies aren’t so great. To pick one example: Cultured Code makes a beautiful task management tool called Things which runs on both OSX and the iPhone. I use Things every day to manage my tasks and its sync behavior is my nemesis. In order to sync, I’ve got to remember to open up Things Touch before I leave home. Only if my phone and laptop are on the same network — and only if they’re both powered up and running Things — will my data get synchronized. If I run out the door for a meeting and forget to synchronize, I’m sunk.

I can understand why Cultured Code designed their sync system this way. It makes elegant use of Apple’s local area networking technologies and side-steps the problem of needing to maintain an intermediate service to pass data between app instances.

If you’re a die-hard Cocoa developer, the thought of building a scalable and reliable web back-end might just be daunting — daunting enough to cut it out entirely. I’ve noticed a strong resistance in the Cocoa community to building such things.

Think About Money

Enter AppEngine. AppEngine makes scalable + reliable back-end development extremely cost effective. You write some code; you deploy it in Google’s data centers; with little to no work on your part, you get a back-end that can handle far more traffic than you’ll ever see, and which will exhibit far better uptime than you could likely arrange for elsewhere. Maintaining AppEngine applications is nothing at all like maintaining custom web deployments. You upload your code and check the AppEngine dashboard every once in a while to see how things are running. There is no need to play system administrator; no need to patch systems or tweak network configurations, etc.

Best of all, AppEngine has a freemium pricing model. Below a certain amount of usage, AppEngine is entirely free. After that, you pay as you go. For an indie OSX developer, this means that when you’ve just launched your product, your back-end fees will be $0 — a price that’s hard to argue with. My guess is that for many products — even moderately successful products — the monthly cost will remain $0.

If your product really takes off, back-end fees will scale with your user base. If you price your product carefully, you should have no problem covering your monthly AppEngine costs for many years to come. For example, let’s say Cultured Code stored all of my task data inside an AppEngine instance. My data weighs in at a hefty 13MB, which would cost Cultured Code exactly $0.02 per year to host. I think they can recover those costs, even if I remain a user for the next decade!

Think About Features

It’s not just economics: from a feature and technology standpoint, AppEngine dovetails well with the Cocoa sync problem. In order to make scalability as simple as possible, AppEngine is a severely restrictive platform. Your AppEngine code must be strictly request/response. Individual responses must happen rapidly. You can’t use a relational database, but you can use a highly scalable property-value store. All transactions must run over HTTP.

It turns out that these restrictions are no obstacle to building a solid sync service; if anything, they’ll help you design it right the first time. It makes sense, especially in the iPhone world, to communicate with your back-end via HTTP (and in all probability, NSURLConnection.) It’s simple in both Python and Objective-C to work with JSON, so it should be simple to get your data transported across with a minimum of fuss. Once your data reaches AppEngine, you can dump it into the property store as a blob. There’s probably no reason for your back-end to need to know about the structure and format of that data.

What’s best is that your AppEngine back-end will always be there, ready to sync user content as soon as your front-end software requests it. And when iPhone OS 3.0 and push notification go mainstream, AppEngine will be the perfect environment from which to manage them, too.

The Missing Piece

All that’s missing, of course, is a library to actually perform the synchronization. This seems like an Open Source effort just waiting to happen. There would have to be three pieces:

1. A Python/AppEngine back-end that developers can easily deploy with little or no configuration on their part
2. A Objective-C front-end that would make it extremely easy to communicate NSData and various NSCollection objects with the AppEngine back-end. You would of course want to tag this data with unique IDs and you’ll need hooks for performing custom conflict resolution, but it’s probably not a terribly complex API.
3. A one-time-only mechanism for pairing disparate instances of the application together. An easy solution would be to let this take place over the user’s LAN, round-tripping the necessary tokens through the back-end service after pairing is complete. A more involved solution would be to actually have user accounts on your back-end service. I imagine the easy solution is good for 85% of the applications out there — it would certainly be sufficient for Things!

So there you have it. I admit this is nowhere near fully baked yet, but I believe that if you let your back-end have no knowledge of your data (beyond “here it is”) you can go a long way without too much complexity. It seems that Cocoa+AppEngine+sync is a beautiful world indeed. Anyone up for rolling up their sleeves and writing some code?

With apologies to Cultured Code, who make great products that I use every day and heartily recommend!