What you must know about the NodeJS Callback Pattern?

NodeJS is single-threaded. And yet, it also happens to be asynchronous.

This is made largely possible due to the adoption of callback design pattern in NodeJS.

In this post, we will take a look at a number of important concepts:

• What is the NodeJS Callback Pattern?
• How callbacks actually work in NodeJS?
• Common conventions around the use of callbacks in NodeJS

In case you want to understand the need of callbacks in the first place, I highly recommend checking out our post on the NodeJS Reactor Pattern.

What are Callbacks in NodeJS?

The NodeJS Event Loop relies heavily on handlers for performing its magic.

Handlers are also known as callbacks and they give NodeJS its distinctive programming style.

Stating thing more formally, callbacks are basically functions that are invoked to propagate the result of an operation.

“What operation?” you ask.

Any asynchronous operation. For example, reading a file from the file system. Or waiting for the response of an API request.

Callbacks make it possible for us to deal with the uncertainty associated with asynchronous operations.

Luckily, JavaScript happens to be a great language for callbacks.

“Why so?” you may ask.

Because in JavaScript, functions are first class objects. You can do a lot with functions in JavaScript such as:

• assign functions to variables
• pass functions as arguments
• return function from another function invocation
• store functions in other data structures

Together with the presence of closures, JavaScript provides an ideal environment for implementing callbacks.

Synchronous Continuation-Passing Style Callbacks

To dig further into the concept of callbacks, let’s first take a look at the common life of a synchronous function:

function sum(a, b) {
   return a + b;
}

Nothing special here. The result of sum() is passed back to the caller using the return instruction. This is the typical synchronous approach. Things look pretty direct in this approach.

Now let’s look at the equivalent implementation of the sum() function with a callback.

function sum(a, b, callback) {
   callback(a + b);
}

To make things clear, the above function is also still synchronous. In other words, it will return a value only when the callback completes its execution. However, it uses something known as the continuation-passing style.

In this continuation-passing style, a callback is a function that is passed as an argument to another function and it is invoked with the result when the operation completes. In our example, the operation to be completed is the sum of a and b. The result of this operation is passed to the callback.

“So, how is this function used?” you ask.

See the below code:

console.log('Before');
sum(3, 5, function(result) {
   console.log('Sum: ' + result);
});
console.log('After');

If we run this code, we get the below output. Everything happens synchronously.

Before
Sum: 8
After

Asynchronous Continuation-Passing Style Callbacks

Of course, the above example might seem like jumping through hoops un-necessarily.

What’s the point of passing the result of the sum operation to the callback when we already had the result?

This is a valid question considering that the sum was a synchronous operation.

But what if the sum() function was asynchronous?

In that case, we can’t rely on receiving the result instantly. It might arrive some time in the future.

But we do want to perform some operation when the result becomes available such as printing it to the console.

Callbacks shine in this scenario.

Check out the below code:

function sumAsync(a, b, callback) {
  setTimeout(function() {
      callback(a + b);
  }, 1000)
}

As you can see, using the setTimeout() makes the function asynchronous.

We can now run the same testing code once more.

console.log('Before');
sumAsync(3, 5, function(result) {
   console.log('Sum: ' + result);
});
console.log('After');

The response will be as below:

Before
After
Sum: 8

Since setTimeout() has triggered an asynchronous operation, the program will not wait for the callback to be executed. It will return immediately giving control back to the sumAsync() function and then back to the caller. Ultimately, the control goes returns to the event loop.

Check out the below illustration that depicts how callbacks actually work for our example code.

The event loop is free to process other events from the event queue. When the asynchronous sum operation completes, the execution is resumed from the callback and the result is printed to the console.

In case you want more details about how the event loop leverages callbacks, I strongly recommend you to read more about the various phases of event loop.

A synchronous function blocks until it completes its operations. An asynchronous function returns immediately and the result is passed to a handler or callback at a later cycle of the event loop.

Are Callbacks always Asynchronous?

To make things clear, presence of a callback argument does not always mean the function is asynchronous or is using a continuation-passing style. There are also non continuation-passing style callbacks.

Check out the below code:

let result = [3, 6, 9].map(function(element) {
    return element - 1;
}

Here, the callback is used to iterate over the array elements and not to pass the result of some operation. In fact, the result is returned synchronously.

In most library functions, the intent of a callback is usually stated in the API documentation.

Of course, sometimes developers can also makes mistakes.

For example, we can build a callback function that behaves synchronously in some cases but asynchronously in some other cases. This can lead to extremely frustrating, pull-you-hair-out kind of bugs. More on that in a future post about unleashing Zalgo in our code.

NodeJS Callback Conventions

In NodeJS, callbacks follow a set of specific conventions.

Many of these conventions have propagated from the NodeJS core API. But even userland modules and applications follow these conventions.

To maintain a healthy level of code consistency, it is always good to adhere to these conventions as much as possible.

Let us look at the important ones.

Callbacks Come Last

If a function accepts a callback as input, the callback should be passed as the last argument. See the below example:

fs.readFile(filename, [options], callback)

This example belongs to the NodeJS core API to access the filesystem. The callback is always placed in the last position. Even the optional arguments (if any) come before the callback.

This convention makes the function call more readable in case we define the callback in place like in the below example:

sumAsync(3, 5, function(result) {
   console.log('Sum: ' + result);
});

Error Comes First

In continuation-passing callbacks, errors are also propagated along with the result.

The convention is that any error produced by a CPS function is always passed as the first argument of the callback. The actual result comes second.

In other words, NodeJS prefers error-first callbacks to maintain a consistent API signature across the ecosystem.

Check out the below example:

fs.readFile('foo.txt', 'utf8', function(err, data) {
   if(err)
     handleError(err);
   else
     processData(data);
});

If the operation succeeds without errors, the first argument will be null or undefined and we can handle things accordingly. It is always good practice to check for errors before doing something with the result.

Propagating Errors to the Callback

In synchronous function, we propagate error using the throw command. This causes the error to jump up in the call stack until it’s caught.

However, in the case of asynchronous callbacks, proper error propagation means passing the error to the next callback in the chain.

Check out the below snippet.

var fs = require('fs');
function readJSONFromFile(filename, callback) {
     fs.readFile(filename, 'utf8', function(err, data) {
       var parsedData;
       if(err)
         //pass error to the next callback
         return callback(err);
       try {
         //parse the file contents
         parsedData = JSON.parse(data);
       } catch(err) {
         //pass the error to the next callback
         return callback(err);
       }
       //no errors, propagate just the data. error is set to null
       callback(null, parsedData);
    }); 
};

You can notice the difference in the way we trigger callbacks in different conditions. If there is an error, we propagate the error object to the callback. However, when there is no error, we set the error object to null and pass the valid result.

Conclusion

The NodeJS callback pattern is a great example of how NodeJS leverages the power of JavaScript to overcome its limitations.

Callbacks are central to how NodeJS works internally.

The Reactor Pattern makes use of callbacks to enable handling of asynchronous operations in NodeJS. Without callbacks, NodeJS would not be able to support concurrency.