How to Stub mongoose methods and Mock Document Objects

How to Mock mongodb database access functions

From the cost perspective, the least database read/write operations the better. Not all test cases are created equal, and with intensive read/write capabilities comes big accountability. This blog is an expose of some techniques to mock database access without compromising the quality of test results.

In this article we will talk about:

Even though this blogpost was designed to offer complementary materials to those who bought my Testing nodejs Applications book, the content can help any software developer to tuneup working environment. You use this link to buy the book. Testing nodejs Applications Book Cover

The mongoose model comes with helpers baked in. The example below, implicitly make functions such as save(), find() update() available by default. The trouble starts settling in when we realize that some functions are used with an instance, for example new User().save(). And others are made available on the class declaration instead, for example: User.find() etc. From this vantage point, the reality of not applying the same techniques while stubbing starts settling in.

Another real struggle is to be able to figure out how to stub custom helpers. In our example, are statics.findByName() and methods.addEmail(). To better understand how to stub those two categories of function, we should start by understanding how they are unique in their own ways, and how they stack up against the instance and class functions mentioned above.

Show me the code

We can think of database access from two perspectives. The first is a set of functions designed to extend mongoose utilities via statics and methods properties. The second is from a usage perspective, or other entities using a mongoose model method to talk to the database.

Difference between static vs method

Let's look at both scenarios, first when our functions are expected to extend mongoose capabilities


var UserSchema = new mongoose.Schema({name: String});
UserScheme.statics.findByName(function(name, next){
    //gives: access to Compiled Model
    return this.where({'name': name}).exect(next);
});

UserSchema.methods.addEmail(function(email, next){
    //works: with retires un-compiled model
    return this.model('User').find({ type: this.type }, cb);
});

//exporting the model 
module.exports = mongoose.model('User', UserSchema);

Example: mongoose Model definition example in core/user/model

And next, when our functions are expected to leverage existing capabilities

const Contact = require('core/contact/model');
function addContact(params, next){	
    return new Contact(params).save((error, contact) => {	
        if(error) return next(error);	
        return next(null, contact);	
    });	
}

function findContact(id, next){	
    return Contact.findById(id, (error, contact) => {	
        if(error) return next(error);	    
        return next(null, contact);	    
    });	
}

function findContacts(params, next){ 
    return Contact.find(params, (error, contacts) => {	
        if(error) return next(error);	
        return next(null, contacts);
    });
});

Example: mongoose Model usage example in core/contact/model

What can possibly go wrong?

The following points may be a challenge when testing the model layer:

Tools

It is feasible to replace database access functions with fakes that emulate similar corresponding actions. Two libraries that come to mind when doing this are sinon and sinon-mongoose.

How to apply same techniques to test SQL based alternative to mongoose such as knex?

There is a feature-complete wrapper of most mongoose utilities: mockgoose.

Database drop-in replacements

Replacing a database with a drop-in-replacement for testing purposes makes it possible to avoid a database spin-up server altogether. The following example highlights good practices when testing with a live database(local development).

var mongoose = require('mongoose');
describe('User', function(){
  
  before(function(){
    mongoose.connect(process.env.CONNECTION_URL);
  });

  after(function(){
    mongoose.connection.close(); 
    mongoose.disconnect(); 
  });
  
  //Do the tests here. 
});

This approach is not convenient, since every time we need to test another model, we will need to spin up a database. read/write operations are also expensive. mockgoose and mongodb-memory-server provide alternatives to mock the whole database infrastructure.

For sake of simplicity, we are going to avoid that, and rely on test doubles instead.

Mocking database access functions

Functions that access or change database state, can be replaced by fakes/stubs capable of supply|emulate output identical to data coming from a real database system.

There are a couple of solutions that can be used, one of them is based on test double libraries such as sinon.


//cb will be the callback to simulate callback function ~ it takes the control from where the stub left off. That means that after executing a stub, the regular callback will execute just as in regular circumstances 
function cb(fn, params){
 return fn.apply(this, arguments);
 //check if params are the one that has to apply instead and apply it.
}

//Model should be an actual model, eg: User|Contact|Address, etc
saveStub = sinon.stub(Contact.prototype, 'save', cb);
findStub = sinon.stub(Contact, 'find', cb);
findByIdStub = sinon.stub(Contact, 'findById', cb);

Stub mongoose with sinon-mongoose

The following is the order in which libraries are loaded to stub the entire mongoose library.

First, we will need to replace the default promise with Promise A+, or another promise library of your choice. Second, we will need to replace mongoose with sinon-mongoose. And the trick is completed.

// Using sinon-as-promised with custom promise 
var sinon = require('sinon'),
    Promise = require('promise'),
    require('sinon-as-promised')(Promise);

// Adding sinon-mongoose to mongoose 
var mongoose = require('mongoose'),
    require('sinon-mongoose');

The promise is not the only kid on the block, in the promised land. The mongoose documentation showcase how to replace the default library, with BYOL (bring your own library).

In next example, we to replace default mongoose promise library with bluebird. Another promise library that made rounds in nodejs and JavaScript community.


var bluebird = require('bluebird'),
 mongoose = require('mongoose'),
 mongoose.Promise = bluebird,
 uri = 'mongodb://localhost:27017/mongoose_test',
 options = { promiseLibrary: bluebird },
 db = mongoose.createConnection(uri, options);

That is good as far as information goes, but not necessarily helping our current task at hand.

Mocking Library: — with callbacks

//in model/user.js
var UserSchema = new mongoose.Schema({name: String});
mongoose.model('User', UserSchema);


//in test.spec.js
describe('User', function(){
    before(function(){
        //model is declared in model/user.js
        this.User = mongoose.model('user');
        this.UserMock = sinon.mock(this.User);
    });
    after(function(){
        this.User.restore();
    });

    it('#save()', function(){
        var self = this,
            user = {name: 'Max Zuckerberg'},
            results = Object.assign({}, user, _id: '11122233aabb');
        //yields works for callbacks
        //.chain('sort').withArgs('-date')
        this.UserMock
            .expects('save')
            .withArgs(user)
            .yields(null, results);  

        new this.User(user).save(function(err, user){
            //add all assertions here. 
            self.UserMock.verify();//verifying 
            self.UserMock.restore();//restore 
        });
    });
    
});

Mock works on an actual object, i.e instance of a model. save() is defined on Document, and not the model object itself. This explains why to we spy on the prototype: sinon.stub(User.prototype, 'save', cb).

Without the mock, it becomes impossible to chain extra function such as .exec(), .stream() etc. A double stub may be an answer to be able to test such edge cases. A quick example is provided to give an idea what we mean by double stub.

var results = { ... }//mock of user data
sinon
 .stub(User.prototype, 'save', cb)
 .returns({
     exec: sinon.stub().yields(null, results)
 });

Alternatively we can use .create() instead. But that may be a little too late, in case the application adopted save() and used in multiple instances. Needless to mention that .create() looks a little bit off. Same this in this this instance, but requires far more changes.

It is also possible to rely on libraries such as factory girl, as explained in this SO answer

Mocking Library: — with promises


//in User.js
MongooseModel
 .find()
 .limit(10)
 .sort('-date')
 .exec()
 .then(result =>  result);

//in user.model.spec.js 
require('sinon'),
require('mongoose'),
require('sinon-mongoose'),
require('sinon-as-promised');

//describe 
describe('User', function(){
    it('works', function(){
        sinon.mock(MongooseModel)
        .expects('find').withArgs(10)
        .chain('limit').withArgs(10)
        .chain('sort').withArgs('-date')
        .chain('exec')
        .resolves('SOME_VALUE'); 
        //.yields(null, 'SOME_VALUES')       
    });
});

Mocking Library: — with streams

The following is a drop-in-replacement of usage of a model paired with a stream.


//code example  
UserModelMock
 .find()
 .stream()
 .pipe(new Transformer())
 .pipe(res);


// in tests ~ using the double mock technique ~ return a readableStream
sinon.stub(Model, 'find').returns({
  stream: sinon.stub().yields(null , readableStreamMock)
});
//readableStreamMock has to have generated data for testing purposes. 

//in tests ~ create writable stream compatible with response object somehow 
 writableStream.on('data|end|close|finish', function(){
  expect(Model.find.called, 'find() has been called');
});

The technique to test the streams has been intensively covered in the – Testing nodejs Applications book. The ideas in this code sample, are rough and still have loopholes that are covered in the said book.

Final notes

Models should be created once, across all tests.

The error: OverwriteModelError: Cannot overwrite 'Activity' model once compiled. means one of the following occurred: – 1. got the caps wrong while importing models. => import User from 'model/user – 2. got wrong definitions of models => var userSchema = new Schema({}); module.exports = mongoose.model('user', userSchema) <=== new schema and not just schema(this was my case) – 3. got models twice(two times recompilation) => module.exports = mongoose.model.User || mongoose.model('user', userSchema);

There are more answers on this subject on StackOverflow

Conclusion

In this article, we established the cost of hitting the database every time a unit test runs, and how to avoid worst-case scenario by mocking out the most expensive parts of the model layer.

We also reviewed how to reduce such costs by strategically stubbing read/write operations to make tests fast, without losing test effectiveness.

Testing tends to be more of art, than a science, practice makes perfect. There are additional complimentary materials in the “Testing nodejs applications” book.

References