How to modularize socket.io/expressjs application
WebSocket protocol is an extension to the HTTP protocol that makes near real-time communication magic a reality. Adding this capability to an already complex application does not make large-scale applications any easier to work with. Using modularization techniques to decoupling the real-time portion of the application makes maintenance a little easier. The question is How do we get there?. This article applies modularization techniques to achieve that.
There is a wide variety of choice to choose from when it comes to WebSocket implementation in
nodejsecosystem. For simplicity, this blog post will provide examples usingsocket.io, but ideas expressed in this blog are applicable to any othernodejsWebSocket implementation.
In this article we will talk about:
- How to modularize WebSocket for reusability
- How to modularize WebSocket for testability
- How to modularize WebSocket for composability
- The need for a store manager in a
nodejsWebSocket application - How to integrate
redisin anodejsWebSocket application - How to modularize
redisin anodejsWebSocket application - How to share session between an HTTP server and WebSocket server
Even though this blog post was designed to offer complementary materials to those who bought my Testing
nodejsApplications book, the content can help any software developer to tuneup working environment. You use this link to buy the book.
Show me the code
//in server.js
var express = require('express');
var app = express();
...
app.get('/users/:id', function(req, res, next){
User.findById(req.params.id, function(error, user){
if(error) return next(error);
return res.status(200).json(user);
});
});
...
var server = require('http').createServer(app);
server.listen(app.get('port'), () => console.log(`Listening on ${ process.env.PORT || 8080 }`));
var wss = require('socket.io')(server);
//Handling realtime data
wss.on('connection|connect', (socket, event) => {
socket.on('error', () => {});
socket.on('pong', () => {this.isAlive = true;});
socket.on('disconnect', () => {});
socket.on('message', () => {});
});
What can possibly go wrong?
The following points may be a challenge when modularizing WebSocket nodejs applications:
- WebSocket handlers are tightly coupled to the rest of the application. The challenge is how to reverse that.
- How to modularize for optimal code reuse and easy testability
The following sections will explore more on making points stated above work.
How to modularize WebSocket for reusability
When looking at the WebSocket handlers, there is something that strikes our eyes. Every handler has a signature that looks like any event handler common in the JavaScript ecosystem. We also realize that handlers are tightly coupled to the WebSocket object. To break the coupling, we can apply one technique: eject handlers from WebSocket, inject WebSocket and Socket objects whenever possible(composition).
How to modularize WebSocket for testability
As noted earlier, the WebSocket event handlers are tightly coupled to the WebSocket object. Mocking the WebSocket object comes with a hefty price: to lose implementation of the handlers. To avoid that, we can tap into two techniques: eject handlers, and loading the WebSocket via a utility library.
How to modularize WebSocket for composability
It is possible to shift the perspective on the way the application is adding WebSocket support. The question we can also ask is: Is it possible to restructure our code, in such a way that it requires only one line of code to wipe out the WebSocket support?. An alternative question would be: Is it possible to add WebSocket support to the base application, only using one line of code?. To answer these two questions, we will tap into a technique similar to the one we used to mount app instance to a set of routers(API for example)
The need of a store manager in a nodejs WebSocket application
JavaScript, for that matter nodejs, is a single-threaded programming language.
However, that does not mean that parallel computing is not feasible. The threading model can be replaced with a process-based model when it comes to parallel computing. This enhancement comes with an additional challenge: How to make it possible for processes to communicate or share data, especially when processes are running on two separate CPUs.
The answer is using a third-party process/es that handles inter-process communications. Key stores are good examples that make the magic possible.
How to integrate redis in a nodejs WebSocket application
redis comes with an expressive API that makes it easy to integrate with an existing nodejs application.
It makes sense to question the approach used while adding this capability to the application. In the following example, any message received on the wire will be logged into the shared redis key store.
All subscribed message listeners will then be notified about an incoming message. In the event there is a response to send back, the same approach will be followed, and the listener will be responsible to send the message again down the wire. This process may be repetitive, but it is one of the good ways to handle this kind of scenario.
There is an entire blog dedicated to modularizing
redisclients here
How to modularize redis in a nodejs WebSocket application
The example of integration with redis in nodejs application is tightly coupled to redis event handlers. Ejecting handlers can be a good starting point. Grouping ejected handlers in a module can follow suit. The next step in modularization can be composing(inject redis) on the resulting modules when needed.
How to share sessions between the HTTP server and WebSocket server.
If we look closer, especially when dealing with namespaces, we find a similarity between HTTP requests(handled by express in our example) and WebSocket messages(handled by socket.io in our example). For applications that require authentication, or any other type of session on the server-side, it would be not necessary to have one authentication per protocol. To solve this problem, we will rely on a middleware that passes session data between two protocols.
Modularization reduces the complexity associated with large scale node js applications in general. We assume thatsocket.io/expressjs` applications won't be an exception in the current context. In a real-time context, we focus on making most parts accessible to be used by other components and tests.
Express routes use socket.io instance to deliver some messages
Structure of a socket/socket.io enabled application looks like following:
//module/socket.js
//server or express app instance
module.exports = function(server){
var io = socket();
io = io.listen(server);
io.on('connect', fn);
io.on('disconnect',fn);
};
//in server.js
var express = require('express');
var app = express();
var server = require('http').createServer('app');
//Application app.js|server.js initialization, etc.
require('module/socket.js')(server);
For socket.io app to use same Express server instance or sharing route instance with socket.io server
//routes.js - has all routes initializations
var route = require('express').Router();
module.exports = function(){
route.all('',function(req, res, next){
res.send();
next();
});
};
//socket.js - has socket communication code
var io = require('socket.io');
module.exports = function(server){
//server will be provided by the calling application
//server = require('http').createServer(app);
io = io.listen(server);
return io;
};
Socket Session sharing
Sharing session between socket.io and Express application
//@link http://stackoverflow.com/a/25618636/132610
//Sharing session data between `socket.io` and Express
sio.use(function(socket, next) {
sessionMiddleware(socket.request, socket.request.res, next);
});
Conclusion
Modularization is a key strategy in crafting re-usable composable software. Modularization brings not only elegance but makes copy/paste detectors happy, and at the same time improves both performance and testability.
In this article, we revisited how to aggregate WebSocket code into composable and testable modules. The need to group related tasks into modules involves the ability to add support of Pub/Sub on demand and using various solutions as project requirements evolve. There are additional complimentary materials in the “Testing nodejs applications” book.
References + Reading List
- Testing
nodejsApplications book redispubsublibrary ~ A Github cloned library- Building a Chat Server with node and
redis– tests ~ Matthew Daly Blog - High Volume, low latency difficulties
nodejs/pubsub/redis~ StackOverflow Question - examples using
redis-storewithsocket.io~ StackOverflow Question - Using
redisas PubSub oversocket.io~ StackOverflow Question socket.io-redis~ Github Librarysocket.iowithin concurrent processes ~ Tolle Iv Blognodejs,expressjsandsocket.ioapplication ~ DeveloperIQ Blog- Modularize Your Chat App(or How to Write a nodejs Express App in More Than One File) ~ Philosophy is Thinking Medium
bacon.js+nodejs+mongodb: Functional Reactive Programming on the Server ~ Carbon Five Blog- Modularizing
socket.iowithexpressjs@4~ StackOverflow Answer