This blog is at the same time a reflection taking the monorepos route, from decision-making perspective and implementation side. This document takes two approaches to the problem.

First, identify aspects that have to be containerized, test the containers both in development and production mode.

Second, create the actual codebase built atop containers, to test and deliver code to production. It will be better if the CI/CD is included in this package.

This article is under active development, more information is going to be added frequently, as I find free time.

In this article we will talk about:

• What is the structure of a typical monorepo
• What are the tools used to manage daily development activities
  
• Compared to multirepo project layout, What are the main components of a monorepo that do not exist in a multirepo, and vice versa
• How do packages relate to the actual application from the content perspective
• What are the key differences between a monorepo and monolith
• What are key differences between a monorepo and a multi-repos
• How is monorepo different from git-submodules
• Is it possible to leverage git submodule add <url> projects/url to compose multiple projects into one independent project
• What are the best strategies for transitioning from multi-repos to monorepo
• What are the best strategies for transitioning from monolith to monorepo
• How do frontend/backend/widgets code repositories fit into monorepo project layout architecture
• How to deploy monorepo projects to different platforms (frontend and widgets to CDN, backend to backend servers)
• How to share the core packages amongst monorepo components, without using npm
• How packaging monorepo works
• Do monorepo allow to deploy and install private packages
• How to automate versioning in a monorepo context
• How to automate change-log in a monorepo context
• How to automate release notes in a monorepo context
• How to automate deployment in a monorepo context
• How to manage deployment keys in a monorepo context

Even though this blog post was designed to offer complementary materials to those who bought my Testing nodejs Applications book, the content can help any software developer to tuneup working environment. You use this link to buy the book.

The monorepo project layout architecture

The monorepo project layout architecture is ideal when planning to share Web UI/SDKs/API and backend code.

• Develop and Deploy your own React monorepo app in under 2 hours, using Lerna, Travis and Now.

• Building Large Scale React Applications in a monorepo architecture

What are key differences between a monorepo and monolith

• Misconceptions about monorepos: monorepo != monolith ~ Nrwl Blog
• monorepo and npm ~ npm Blog

How is monorepo different from git-submodules

• monorepos in git ~ Atlassian Tutorials
• Configuring npm for use with GitHub Package Registry ~ Github Articles
• Why Babel Uses monorepo ~ Github Babel Design Docs

Tools to manage monorepos

npm is not well-positioned to manage monorepos, as we write this article. There are however other tools that help to achieve that. Here is a non exhaustive list:

• learna ~ community-backed
• rush ~ Microsoft backed

• yarn ~ Facebook backed. Uses workspaces, but in reality, they are monorepo components as well.

• How to successfully manage a large scale JavaScript monorepo aka megarepo ~ Jonathan Creamer Blog

Sharing code between repos in the same monorepos, without needing npm

The package.json's private: true property makes sure npm doesn't search npm, but relies on git/github. One extra mile when using monorepos, is to share the code using tar files.

It is possible to use tar files. That will require reach release to have its own tar file that is deployable and reachable for a particular endpoint.

{
    dependencies: {
      "common-package": "file:../common-package", //OR
      "tar-common-package": "github.com/../../common-package.tar.gz",
    }
  }

github.com makes it possible to run installations(packages) from its servers.

• JavaScript monorepo ~ Using learna

Sharing code via npm without opening an npmjs.com account

This section introduces two concepts, with direct dependency on each other. The first concept deals with using git/github as an npm hub. This concept makes it possible to avoid opening an account on hosting services such as npmjs, while being able to keep private packages. The second concept makes sure private packages are installable using npm install just like regular packages.

Alternatively, there is a new infrastructure that github.com rolled out. Those are packages that can be installed using npm installer. We will see how to operate these as well.

How to manage authentication keys

One of the problems sharing a large codebase relates to security. How is it possible to share authentication keys, such as database passwords, without compromising the overall security of the application?

• Where to store the private keys
• Using services such as git-secret
• Using services such as git-crypt
• Safely Storing Secrets in Git
• Best practices for securely storing API keys

Containerization with Docker

Docker makes it possible to run a stack of applications, regardless of the system the application is developed on. Docker makes it possible to simulate with success the application behavior once the application finally hits the production servers.

• Building Efficient Dockerfile ~ BitJudo blog

Container Orchestration with kubernetes

If Docker symbolizes Containerization, kubernetes is aligning itself as the best container orchestration resource. This guide provides resources to get started, and the basic designs that are commonly used in the MEAN stack world.

Installation can be done via MacPorts or Homebrew. It is always possible to use binaries as well.

• How to install kubernetes on MacoOS

Containerized database

This section is an exploration of the implementation of a clustered database. We will see what it takes to deploy a containerized database, how to add upgrade new engines, how to backup and migrate data, how to migrate to new models.

• mongodb-enterprise-kubernetes

How to deploy monorepo apps

In a multi-documents context, how to deploy one section to one platform and another section to another platform?

Every single build has to have a corresponding individual deploy script. Push to deploy would be a challenge, unless there is an alternative to selectively detect which part has to go were. Else, all servers running an instance of code, have to have a copy of the full monorepo code.

• How to deploy multiple apps in a monorepo with Heroku
• Continuous integration in projects using monorepos
• DevOps ~ using ansible
• How to set up CI/CD Pipeline for a node.js app with Jenkins ~ Moshe Ezderman – Medium

Conclusion

In this article, we reviewed what it takes, and reasons to move to a monorepo architecture. We also revisited the monorepo coupled with the containerization technique to deliver a better developer experience. There are additional complimentary materials in the “Testing nodejs applications” book.

References

• Testing nodejs Applications book
• monorepos in the wild ~ @maoberlehner Medium
• We went monorepo ~ Invisible Reports Medium

#nodejs #monorepos #multirepos #monolyths #microservice