Don’t Just Relax; Slouch: A JS Client for CouchDB that Does the Heavy Lifting

With the recent release of CouchDB 2, the CouchDB ecosystem is on fire! There is now an out-of-the-box way of setting up a multi-master cluster of nodes that can scale to handle a ton of data. And, tools like PouchDB can allow your app to talk directly to CouchDB to enable your apps to be offline-first. It’s an exciting time to be using CouchDB at the data layer of your app!

Even if you focus on a frontend-heavy design, chances are, you’re going to have to do a lot of coding on your backend to communicate with your CouchDB instance, including subscribing to real-time changes.

When we first started working on Quizster, a digital dropbox and grading system, we relied heavily on the awesome nano client. Through a lot of trial-and-error though, we found that a lot of messy logic, mostly to handle large data sets and transient database errors, was starting to clutter the application layer. We were also left wanting something flatter than nano that more closely mimicked the raw CouchDB RESTful API. We didn’t want to have to bind the client to a specific database, as in practice, this leads to a lot of overhead. We also wanted promises to be a native part of our client as we consider them essential to keeping our code manageable.

So, Slouch was born.

Slouch provides all the typical features that you’d expect to find in a CouchDB client and a lot more. Let’s take a look at some of the cool stuff.

Fault Tolerance

Slouch attempts to retry your requests in the event that the connection to your database is temporarily lost. Moreover, it retries through max_dbs_open errors and other transient errors to prevent your application from crashing every time there is a hiccup on the database.

StreamIterators

These days, working with large data sets is practically a requirement. Sure, you can paginate your data, but CouchDB supports streams and StreamIterators make working with these streams a breeze.

StreamIterators are available for all requests that can return a large number of docs, i.e. db.changes(), db.view(), doc.all(), system.updates()

StreamIterators work with promises and reduce your code by introducing a simple and powerful pattern.

Example 1 — Sequentially process all docs in a DB

slouch.doc.all('mydb', { include_docs: true }).each(function (item) {

// If we return a promise then the all() iterator won't move on  
// to the next item until the promise resolves. This allows us  
// to iterate through a large number of docs without consuming a  
// lot of memory. It also allows us to control the flow of the  
// docs and process them sequentially so that we don't end up  
// thrashing the processor with too many concurrent promises.  
return Promise.resolve('foo => ' + item.foo);  

}).then(function () {

// Done iterating through all docs  

}).catch (function (err) {

// An error occurred  

});

Example 2 — Process a max of 5 docs concurrently

var Throttler = require('squadron').Throttler;var throttler = new Throttler(5);

slouch.doc.all('mydb', { include_docs: true }).each(function (item) {

return Promise.resolve('foo => ' + item.foo);  

}, throttler).then(function () {

// Done iterating through all docs  

});

Helper functions

The CouchDB conflict policy is a very powerful convention and is at the heart of CouchDB and its offline capabilities. Sometimes however, conflicts don’t matter to you and it’s nice to have functions to ensure that your logic persists through these conflicts.

upsert

Sometimes you just want to force a creation or update even if there is a conflict:

slouch.doc.upsert('mydb', { _id: '1', foo: 'bar' });

getMergeUpsert

getMergeUpsert allows you to make partial updates without regard to conflicts:

// Create a docslouch.doc.create('mydb', { _id: '1', foo: 'bar' }).then(function (doc) {

// Add the \`yar\` attr to the doc and ignore any conflicts  
return slouch.doc.getMergeUpsert('mydb',  
  { \_id: '1', yar: 'nar' });

});

Slouch and Beyond

This just scratches the surface of what you can do with Slouch. We’ve found Slouch to work very well in production for Quizster and using it has greatly simplified our application layer as a lot of the boilerplate code is now at the Slouch layer.

Our next step is to use Slouch to open source our Spiegel layer so that it is a little easier to implement scaling replication and change listening for CouchDB.

About the Author

Geoff Cox is the Co-Founder of Quizster, a digital dropbox and grading system. Quizster, uses a full stack of JS and runs CouchDB and PouchDB at the data layer.