nodejs monorepos

This blog is at the same time a reflection taking the monorepos route, from decision-making perspective and implementation side. This document takes two approaches to the problem.

First, identify aspects that have to be containerized, test the containers both in development and production mode.

Second, create the actual codebase built atop containers, to test and deliver code to production. It will be better if the CI/CD is included in this package.

This article is under active development, more information is going to be added frequently, as I find free time.

In this article we will talk about:

Even though this blog post was designed to offer complementary materials to those who bought my Testing nodejs Applications book, the content can help any software developer to tuneup working environment. You use this link to buy the book. Testing nodejs Applications Book Cover

The monorepo project layout architecture

The monorepo project layout architecture is ideal when planning to share Web UI/SDKs/API and backend code.

What are key differences between a monorepo and monolith

How is monorepo different from git-submodules

Tools to manage monorepos

npm is not well-positioned to manage monorepos, as we write this article. There are however other tools that help to achieve that. Here is a non exhaustive list:

Sharing code between repos in the same monorepos, without needing npm

The package.json's private: true property makes sure npm doesn't search npm, but relies on git/github. One extra mile when using monorepos, is to share the code using tar files.

It is possible to use tar files. That will require reach release to have its own tar file that is deployable and reachable for a particular endpoint.

{
    dependencies: {
      "common-package": "file:../common-package", //OR
      "tar-common-package": "github.com/../../common-package.tar.gz",
    }
  }

github.com makes it possible to run installations(packages) from its servers.

Sharing code via npm without opening an npmjs.com account

This section introduces two concepts, with direct dependency on each other. The first concept deals with using git/github as an npm hub. This concept makes it possible to avoid opening an account on hosting services such as npmjs, while being able to keep private packages. The second concept makes sure private packages are installable using npm install just like regular packages.

Alternatively, there is a new infrastructure that github.com rolled out. Those are packages that can be installed using npm installer. We will see how to operate these as well.

How to manage authentication keys

One of the problems sharing a large codebase relates to security. How is it possible to share authentication keys, such as database passwords, without compromising the overall security of the application?

Containerization with Docker

Docker makes it possible to run a stack of applications, regardless of the system the application is developed on. Docker makes it possible to simulate with success the application behavior once the application finally hits the production servers.

Container Orchestration with kubernetes

If Docker symbolizes Containerization, kubernetes is aligning itself as the best container orchestration resource. This guide provides resources to get started, and the basic designs that are commonly used in the MEAN stack world.

Installation can be done via MacPorts or Homebrew. It is always possible to use binaries as well.

Containerized database

This section is an exploration of the implementation of a clustered database. We will see what it takes to deploy a containerized database, how to add upgrade new engines, how to backup and migrate data, how to migrate to new models.

How to deploy monorepo apps

In a multi-documents context, how to deploy one section to one platform and another section to another platform?

Every single build has to have a corresponding individual deploy script. Push to deploy would be a challenge, unless there is an alternative to selectively detect which part has to go were. Else, all servers running an instance of code, have to have a copy of the full monorepo code.

Conclusion

In this article, we reviewed what it takes, and reasons to move to a monorepo architecture. We also revisited the monorepo coupled with the containerization technique to deliver a better developer experience. There are additional complimentary materials in the “Testing nodejs applications” book.

References

#nodejs #monorepos #multirepos #monolyths #microservice