When I was first working on HolyBible.com, I struggled for quite a while to wrap my head around the right way to structure its API—and in truth, I actually didn’t come up with what I would call the right solution. I came up with a working solution, and the site performs all right, most of the time. However, our goal as developers shouldn’t be “all right, most of the time.” It should be “really well, all the time.” A big part of what I did wrong came from the bad advice I found in reading up on the issue along the way. This is my shot at helping you, dear reader, avoid making the same mistake.
When building a client-side application, we need to get the data for each view so that we can render it. In the case of HolyBible.com, that means everything from actual Bible text to study Bible notes, about pages, etc. The question is how to do this: we need to be able to load an actual page from our server, and we need a way to request data (rather than whole pages) from the server.
(More experienced developers already know where this is going: that last sentence there has the key to this whole thing. I know. But the internet doesn’t. I learned this the hard way.)
Here’s the mistake I made: I built the Bible data API as (essentially) a single endpoint. When I went looking for advice on how to build this in Angular and Node/Express, every single tutorial or blog post I found outlined the same basic solution: routes for your data endpoints, and catch-all route that returns the basic frame page for everything else. So, for HolyBible.com, that would come out with route matchers for e.g.
/data/gen.1.1, and for any other specific routes needed (for other views, static resources, etc.), with a default behavior of just dropping a static, basically empty template at the catchall
* route. Then, once the application has loaded, it can inspect the URL and load the relevant data.
This works. It’s exactly what I did on HolyBible.com, in fact. But it’s slow.
Don’t get me wrong: the time until the initial page load is actually relatively quick (though I plan to improve it substantially over the next couple months). The real problem is that the initial page load doesn’t include any content.
Don’t write one API. Write two. They should be structured nearly identically, but one of them will be a page API endpoint, and one will be a data API endpoint. In the context of HolyBible.com, here’s how that would play out.1 One endpoint would be based purely on the standard URL, something like
holybible.com/jhn.3.16. The other would be to retrieve a set of data associated with a given address, like
holybible.com/data/jhn.3.16. This is only a little different from the approach suggested above, but that small difference matters—in fact, it matters a lot.
Instead of having the
/jhn.3.16 route get handled by a catchall
/data/<reference> endpoint. So, for example, if there is a navigation control on the page (as on HolyBible.com and indeed most sites), clicking to navigate to Job 14 could, instead of requesting
/job.14.4, fetch the data from the other endpoint by running an AJAX request to
The backend thus supplies both a
/<resource> and a
/data/<resource> route. This might seem redundant, but we’ve just seen why it isn’t, Moreover, if you have any logic that needs to be in place—in our example here, a Bible reference parser, for example, to decide what content should be supplied—you can easily reuse it between the two routes. The differences is simply in the form of the data returned: is it a fully-rendered template, or just the data?
This approach has two big advantages over the catch-all approach that was frequently recommended in e.g. Angular SPA tutorials I read.
It’s not often an approach gives you progressive enhancement and actually increases the performance of an application, but this one does. Better yet, you can apply this in just about any framework: it’s equally applicable to AngularJS with ExpressJS, Backbone with Rails, Ember with Django, Aurelia with Phoenix, or any other combination you come up with.
Note: this is not the actual API structure of HolyBible.com, or even particularly close to it. Remember, I learned everything I’m writing here by doing it wrong.↩
Or possibly a section which constitutes a semantic block of data. I have some thoughts on chunking Bible data semantically rather than by chapter and verse for this kind of thing. That’s another post for another day, though.↩