Rob Sayre’s Mozilla Blog

Over on Sam’s blog, I see a little debate about Joe Gregorio’s How To Do RESTful Partial Updates. Sam is correct that Joe’s technique is not RESTful. I’ll explain why below, and present an alternative technique. I don’t think Joe’s technique will melt the Internet, so its unRESTful design is not a big deal to me.

The key thing that makes the technique unRESTful is the special meaning an “update URI” gives the PUT method and its payload. The payload is really just a change list disguised as an Atom entry. If you make deletions explicit, it’s just a list of paths and values. Why don’t we run with that? I’ll use JSON for my payload, because I have cool scripts.

Let’s say I have a JSON file at http://www.example.com/my/stuff/thing.json, and I send a GET request for it from JavaScript.

var serverCopy = loadJSONDoc("http://www.example.com/my/stuff/thing");

Which gets me traffic like this:

HTTP/1.1 200 OK
Date: Fri, 15 Feb 2008 17:17:11 GMT
Content-Length: 184
Content-Type: application/json
ETag: "anEtag12345"

{
  "x": 42,
  "a": 1,
  "b":
  {
    "c": 2,
    "d":
    {
      "e": 3,
      "f": 4
    },
    "g": 5
  },
  "h": 6.6,
  "i": [7, 8, 9],
  "j": 10,
  "k": { "m": 11 },
  "n": 66
}

Now, I can make some updates.

var localCopy = clone(serverCopy);
localCopy.x = 43;
localCopy.new = 11;
localCopy.b.new2 = 22;
delete localCopy.b.d.f;
delete localCopy.h;
localCopy.i.push(99);
localCopy.n = {"new3": 77}

Some fields have been changed, while others are unchanged. If the changed fields are small in comparison to the unchanged ones, it might be nice to do a partial update. For instance, the value of “a” is small in this example, but it could be the text of War and Peace. Here’s the new state of localCopy:

{
  "x": 43,             /* edited */
  "a": 1,
  "new": 11,           /* created */
  "b":
  {
    "c": 2,
    "new2": 22,        /* created */
    "d":
    {
      "e": 3,
      /*"f": 4*/       /* removed */
    },
    "g": 55,           /* edited  */
  },
  /* "h": 6.6, */      /* removed */
  "i": [7, 8, 9, 99],  /* added array element */
  "j": 10,
  "k": 42,             /* replaced object with primitive */
  "n": { "new3": 77 }  /* replaced primitive with object */
}

To find out what’s changed we’ll use the detectUpdates function from sync.js.

var updates = detectUpdates(serverCopy, localCopy);

The updates object is a JSON array.

[
 {"action":"edit", "path":["x"], "value":43},
 {"action":"remove", "path":["b", "d", "f"]},
 {"action":"create", "path":["b", "new2"], "value":22},
 {"action":"remove", "path":["h"]},
 {"action":"create", "path":["i", "3"], "value":99},
 {"action":"edit", "path":["n"], "value":{}},
 {"action":"create", "path":["n", "new3"], "value":77},
 {"action":"create", "path":["new"], "value":11}
]

The code in sync.js can take sync lists like this and merge an arbitrary number of them together, given a conflict resolution policy. But conflicts have a decent chance of being avoided, because we’ll get by OK if someone else edited “a” on the server before we send our changes back, since we didn’t touch it. Coarse-grained PUT updates would fail.

Let’s say I have function patchJSON that takes a URI and list of changes, and sends it to the server as below. Notice that the Content-Length is longer than the original. This is just an example–work with me here.

PATCH /my/stuff/thing HTTP/1.1
Host: example.com
Content-Length: 394
Content-Type: x-application/json-sync

[
 {"action":"edit", "path":["x"], "value":43},
 {"action":"remove", "path":["b", "d", "f"]},
 {"action":"create", "path":["b", "new2"], "value":22},
 {"action":"remove", "path":["h"]},
 {"action":"create", "path":["i", "3"], "value":99},
 {"action":"edit", "path":["n"], "value":{}},
 {"action":"create", "path":["n", "new3"], "value":77},
 {"action":"create", "path":["new"], "value":11}
]

Some people would prefer to use PATCH here, but I don’t think it matters. Update: In the comments, James Snell points out why POST is a bad idea. PATCH it is.

This is RESTful because I can send my change list to any old server, without the fear that it will apply them. For instance, I could save them somewhere else on my server.

PUT /my/stuff/edits/thing.json HTTP/1.1
Host: example.com
Content-Length: 394
Content-Type: x-application/json-sync

[
 {"action":"edit", "path":["x"], "value":43},
 {"action":"remove", "path":["b", "d", "f"]},
 {"action":"create", "path":["b", "new2"], "value":22},
 {"action":"remove", "path":["h"]},
 {"action":"create", "path":["i", "3"], "value":99},
 {"action":"edit", "path":["n"], "value":{}},
 {"action":"create", "path":["n", "new3"], "value":77},
 {"action":"create", "path":["new"], "value":11}
]

PUT means PUT. Even the format in Joe’s article could work this way, if it had a new root element and media type. It would still be hard to tell where deletions were supposed happen, though.

A funny thing about this little JSON sync format I came up with (but see Norman Ramsey reference in the file), is that it maps pretty well to path segments. Our patches could become batches. That would mean

http://www.example.com/my/stuff/thing/a

would return “1″, and

http://www.example.com/my/stuff/

might return a JSON object with a “thing” key, and other keys would have other objects. You could do this for blog posts, for example.

Hierarchical stuff like this gets a little tricky as it interacts with ETags, but there’s no requirment to manage a really deep, explicit URI space. There are clever ways to encode these dependencies, if you do want that.