Scheduled tasks are hard to debug. Inherent to their asynchronous nature, bugs in scheduled tasks strike later, anything that can help prevent that behavior and curb failures ahead of time are always good to have.

Unit testing is one of the effective tools to challenge this behavior. The question we have an answer for is How to test scheduled tasks in isolation. This article introduces some techniques to do that. Using modularization techniques on scheduled background tasks, we will shift focus to making chunks of code-block accessible to testing tools.

In this article we will talk about:

• How to define a job(task)
• How to trigger a job(task)
• How to modularize tasks for testability
• How to modularize tasks for reusability
• How to modularize tasks for composability
• How to expose task scheduling via a RESTful API
• Alternatives to the agenda scheduling model

Even though this blog post was designed to offer complementary materials to those who bought my Testing nodejs Applications book, the content can help any software developer to tuneup working environment. You use this link to buy the book.

Show me the code

The following example shows how Job trigger can be used under an expressjs route:

//jobs/email.js
var email = require('some-lib-to-send-emails'); 
var User = require('./models/user.js');

module.exports = function(agenda) {
  agenda.define('registration email', function(job, done) {
    User.findById(job.attrs.data.userId, function(err, user) {
       if(err) return done(err);
       	var message = ['Thanks for registering ', user.name, 'more content'].join('');
      	return email(user.email, message, done);
     });
  });
  agenda.define('reset password', function(job, done) {/* ... more code*/});
  // More email related jobs
};

//route.js
//lib/controllers/user-controller.js
var app = express(),
    User = require('../models/user-model'),
    agenda = require('../worker.js');

app.post('/users', function(req, res, next) {
  var user = new User(req.body);
  user.save(function(err) {
    if(err) return next(err);
    //@todo - Schedule an email to be sent before expiration time
    //@todo - Schedule an email to be sent 24 hours
    agenda.now('registration email', { userId: user.primary() });
    return res.status(201).json(user);
  });
});

Example:

What can possibly go wrong?

When trying to figure out how to approach modularization of nodejs background jobs, the following points may be quite a challenge on their own:

• abstraction, and/or injecting, background job library into an existing application
• abstraction or making schedule jobs outside the application.

The following sections will explore more on making points stated above work.

How to define a job

agenda library comes with an expressive API. The interface provides two sets of utilities, one of which is .define(), and does the task definition chore. The following example illustrates this idea.

agenda.define('registration email', 
  function(metadata, done) {

});

How to trigger a job

As stated earlier, the agenda library comes with an interface to trigger a job or schedule an already defined job. The following example illustrates this idea.

agenda.now('registration email', {userId: userId});
agenda.every('3 minutes', 'delete old users');
agenda.every('1 hour', 'print analytics report');

How to modularize tasks for reusability

There is a striking similarity between event handling and task definition.

That similarity raises a whole new set of challenges, one of which turns out to be a tight coupling between task definition and the library that is expected to execute those jobs.

The refactoring technique we have been using all along is handy in the current context as well. We have to eject job definition from agenda library constructs. The next step in refactoring iteration is to inject agenda object as a dependency, whenever it is needed.

The modularization cannot end at this point, we also need to export individual jobs (task handlers) and expose those exported modules via an index file.

How to modularize tasks for testability

There challenges when mocking any object that applies to agenda instance as well.

Implementation of jobs(or task handlers) will be lost, as soon as a stub/fake is provided. The arguments stating that stubs will play well are valid, as long as independent jobs(task handlers) are tested in isolation.

To avoid the need to mock the agenda object in multiple instances, loading agenda from a dedicated library provides quite a good solution to this issue.

How to modularize tasks for composability

In these modularization series, we focused on one perspective. There is no restriction to turn the tables and see things from an opposite vantage point. We can take agenda as an injectable object. The classic approach is the one used with injecting(or mounting) app instances in a set of reusable routes(RESTful APIs).

How to expose task scheduling via a RESTful API

One of the reasons to opt for agenda for background task processing is its ability to persist jobs in a database, and resume pending jobs even after a database server shutdown, crash, or data migration from one instance to the next.

This makes it easy to integrate job processing in regular RESTful APIs. We have to remember that background tasks are mainly designed to run like cronjobs.

Alternatives to agenda scheduling model

In this article we approached job scheduling from a library perspective, agenda. agenda is certainly one of the multiple other solutions in the wild, for instance, cronjobs.

Another viable alternative is tapping into system-based solutions such as monit for Linux and systemctl for macOS.

There is a discussion on how to use nodejs to execute monit tasks in this blog and monit service poll time.

Modularization of Scheduled Tasks

Modularization of scheduled tasks requires 2 essential steps, as for any other module. The first step is to make sure the job definition and job trigger(invocation) is exportable, the same way independent functions do. The second step is to provide access to it, via index.

The next two steps help to achieve these two objectives. Before we dive into it, it worth clarifying a couple of points.

• Tasks can be scheduled from dedicated libraries, cronjobs, and software such as monit.
• There are a lot of libraries to choose from such as bull and bee or kue. agenda is chosen for clarification purposes.
• Task invocation can be triggered from the socket, routes, and agenda handlers
• Example of delayed tasks is sending an email at a given time, deleting inactive accounts, data backup, etc.

agenda uses mongodb to store job descriptions. Good choice in case the project under consideration relies on mongodb for data persistence.Example Project Structure

Conclusion

Modularization is key when crafting re-usable composable software. Scheduled tasks are not an exception to this rule. Background jobs modularization brings elegance to the codebase, reduces copy/paste instances, improves performance and testability.

In this article, we revisited how to increase background jobs more testable, by leveraging key modularization techniques. There are additional complimentary materials in the “Testing nodejs applications” book.

References

• Testing nodejs Applications book
• Export this: Interface Design Patterns for nodejs Modules ~ GoodEggs Blog

tags: #snippets #modularization #scheduled-jobs #nodejs