Integration of redis
in nodejs
applications
The reactive aspect of nodejs
applications is synonymous with the nodejs
runtime itself. Even though the real-time aspect may be attributed to WebSocket implementation, the realtime reactive aspect of nodejs
applications heavily rely on pub/sub mechanisms — most of the time backed by datastore engines like redis
. This article explores how to integrate redis
datastore into a nodejs
application.
In this article we will talk about:
redis
support with and withoutexpressjs
redis
support with and without WebSocket push mechanism- Alternatives to
redis
innodejs
world and beyond
Even though this blog post was designed to offer complementary materials to those who bought my Testing
nodejs
Applications book, the content can help any software developer to tuneup working environment. You use this link to buy the book.
Show me the code
The following code sample showcase how nodejs
/redis
integration can be achieved. It demonstrates that it is still possible to share sessions via a middleware between socket.io
and expressjs
.
var app = express();
var server = Server(app);
var sio = require("socket.io")(server),
redis = require('redis'),
rhost = process.env.REDIS_HOST,
rport = process.env.REDIS_PORT,
pub = redis.createClient(rport, rhost),
sub = redis.createClient(rport, rhost);
function middleware(req, res, next){
//session initialization thing
next();
}
//socket.io/expressjs session sharing middleware
sio.use(function(socket, next){
middleware(socket.request, socket.request.res, next);
});
//express uses middleware for session management
app.use(middleware);
//somewhere
sio.sockets.on("connection", function(socket) {
//socket.request.session
//Now it's available from `socket.io` sockets too! Win!
socket.on('message', (event) => {
var payload = JSON.parse(event.payload || event),
user = socket.handshake.user || false;
//except when coming from pub
pub.publish(payload.conversation, payload));
});
//redis listener
sub.on('message', function(channel, event) {
var payload = JSON.parse(event.payload || event),
user = socket.handshake.user || false;
sio.sockets.in(payload.conversation).emit('message', payload);
});
Example: excerpt source: StackOverflow
What can possibly go wrong?
When trying to figure out how to approach redis
datastore integration into nodejs
application for inter-process communication and real-time feature, the following points may be a challenge:
- How to decouple the WebSocket events from the
redis
specific (pub
/sub
) events. We should be able to decouple, but still provide an environment where interoperability is possible at any time. - How to make integration modular, testable, and overall friendly to the rest of the application ecosystem
When testing this implementation, we should expect the additional challenge to emerge:
- The
redis
client instances (pub
/sub
) are created as soon as the library loads, and aredis
server should be up and running by that time. The issue is when testing the application, there should be no server or any other system dependency hindering the application from being tested. - getting rid of
redis
server with a drop-in-replacement, orstubs/mocks
, is more of a dream than reality ~ hard but feasible.
There is additional information in mocking and stubbing
redis
data store in “How to Mockredis
datastore” article.
Conclusion
In this article, we revisited how to enhance nodejs
application with redis
based pub/sub mechanism, critical to having a reactive real-time experience. The use of WebSocket and Pub/Sub powered by a key/value data store was especially the main focus of this article. There are additional complimentary materials in the “Testing nodejs
applications” book.
References
- Testing
nodejs
Applications book - Sharing session between
expressjs
andsocket.io
~ StackOverflow Question - examples using
redis
withsocket.io
~ StackOverflow Question - Using
redis
asPubSub
oversocket.io
~ StackOVerflow Question socket.io
–redis
redis
pubsub
library ~ Github Cloned Library- Building a Chat Server with node and
redis
– tests ~ Matthew Daly Blog - High Volume, low latency difficulties
nodejs
/pubsub
/redis
~ StackOverflow Question