Simple Engineering

Jan2020

This article revisits essentials on how to install upstart an event based daemon for starting/stopping tasks on development and production servers.

This article has complementary materials to the Testing nodejs Applications book. However, the article is designed to help both those who already bought the book, as well as the wide audience of software developers to setup working environment. Testing Nodejs Applications Book Cover You can grab a copy of this book on this link

There are a plethora of task execution solutions, for instance systemd and init, rather complex to work with. That makes upstart a good alternative to such tools.

In this article you will learn about:

  • Tools available for task execution
  • How to install upstart task execution
  • How to write basic upstart task

Installing upstart on Linux

It is always a good idea to update the system before start working. There is no exception, even when a daily task updates automatically binaries. That can be achieved on Ubuntu and Aptitude enabled systems as following:

$ apt-get update # Fetch list of available updates
$ apt-get upgrade # Upgrades current packages
$ apt-get dist-upgrade # Installs only new updates

Example: updating aptitude binaries

At this point most of packages should be installed or upgraded. Except Packages whose PPA have been removed or not available in the registry. Installing software can be done by installing binaries, or using Ubuntu package manager.

Installing a upstart on Linux using apt

Installing upstart on macOS

upstart is a utility designed mainly for Linux systems. However, macOS has its equivalent, launchctl designed to stop/stop processes prior/after the system restarts.

Installing upstart on a Windows machine

Whereas macOS systems and Linux are quite relax when it comes to working with system processes, Windows is a beast on its own way. upstart was built for *nix systems but there is no equivalent on Windows systems: Service Control Manager. It basically has the same ability to check and restart processes that are failing.

Automated upgrades

Before we dive into automatic upgrades, we should consider nuances associated to managing a mongodb instance. The updates fall into two major, quite interesting, categories: patch updates and version upgrades.

Following the SemVer ~ aka Semantic Versioning standard, it is recommended that the only pair minor versions be considered for version upgrades. This is because minor versions, as well as major versions, are subject to introducing breaking changes or incompatibility between two versions. On the other hand, patches do not introduce breaking changes. Those can therefore be automated.

In case of a critical infrastructure piece of processes state management calibre, we expect breaking changes when a new version introduces a configuration setting is added, or dropped between two successive versions. Upstart provides backward compatibility, so chances for breaking changes between two minor versions is really minimal.

We should highlight that it is always better to upgrade at deployment time. The process is even easier in containerized context. We should also automate only patches, to avoid to miss security patches.

In the context of Linux, we will use the unattended-upgrades package to do the work.

$ apt-get install unattended-upgrades apticron

Example: install unattended-upgrades

Two things to fine-tune to make this solution work are: to enable a blacklist of packages we do not to automatically update, and two, to enable particular packages we would love to update on a periodical basis. That is compiled in the following shell scripts.

Unattended-Upgrade::Allowed-Origins {
//  "${distro_id}:${distro_codename}";
    "${distro_id}:${distro_codename}-security"; # upgrading security patches only 
//   "${distro_id}:${distro_codename}-updates";  
//  "${distro_id}:${distro_codename}-proposed";
//  "${distro_id}:${distro_codename}-backports";
};

Unattended-Upgrade::Package-Blacklist {
    "vim";
};

Example: fine-tune the blacklist and whitelist in /etc/apt/apt.conf.d/50unattended-upgrades

The next step is necessary to make sure unattended-upgrades download, install and cleanups tasks have a default period: once, twice a day or a week.

APT::Periodic::Update-Package-Lists "1";            # Updates package list once a day
APT::Periodic::Download-Upgradeable-Packages "1";   # download upgrade candidates once a day
APT::Periodic::AutocleanInterval "7";               # clean week worth of unused packages once a week
APT::Periodic::Unattended-Upgrade "1";              # install downloaded packages once a day

Example: tuning the tasks parameter /etc/apt/apt.conf.d/20auto-upgrades

This approach works on Linux(Ubuntu), especially deployed in production, but not Windows nor macOS. The last issue, is to be able to report problems when an update fails, so that a human can intervene whenever possible. That is where the second tool apticron in first paragraph intervenes. To make it work, we will specify which email to send messages to, and that will be all.

EMAIL="<email>@<host.tld>"

Example: tuning reporting tasks email parameter /etc/apticron/apticron.conf

Conclusion

In this article we revisited ways to install upstart on various platforms. Even though configuration was beyond the scope of this article, we managed to get everyday quick refreshers out.

References

#nodejs #homebrew #UnattendedUpgrades #nginx #y2020 #Jan2020 #HowTo #ConfiguringNodejsApplications #tdd #TestingNodejsApplications

This article revisits essentials on how to install monit monitoring system on production servers.

This article has complementary materials to the Testing nodejs Applications book. However, the article is designed to help both those who already bought the book, as well as the wide audience of software developers to setup working environment. Testing Nodejs Applications Book Cover You can grab a copy of this book on this link

There are a plethora of monitoring and logging solutions around the internet. This article will not focus on any of those, rather provide alternatives using tools already available in Linux/UNIX environments, that may achieve near same capabilities as any of those solutions.

In this article you will learn about:

  • Difference between logging and monitoring
  • Tools available for logging
  • Tools available for monitoring
  • How to install monitoring and logging tools
  • How to connect end-to-end reporting for faster response times.

Installing monit on Linux

It is always a good idea to update the system before start working. There is no exception, even when a daily task updates automatically binaries. That can be achieved on Ubuntu and Aptitude enabled systems as following:

$ apt-get update # Fetch list of available updates
$ apt-get upgrade # Upgrades current packages
$ apt-get dist-upgrade # Installs only new updates

Example: updating aptitude binaries

At this point most of packages should be installed or upgraded. Except Packages whose PPA have been removed or not available in the registry. Installing software can be done by installing binaries, or using Ubuntu package manager.

Installing a monit on Linux using apt

Installing monit on macOS

In case homebrew is not already available on your mac, this is how to get one up and running. On its own, homebrew depends on ruby runtime to be available.

homebrew is a package manager and software installation tool that makes most developer tools installation a breeze.

$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Example: installation instruction as provided by brew.sh

Generally speaking, this is how to install/uninstall things with brew

$ brew install wget 
$ brew uninstall wget 

Example: installing/uninstalling wget binaries using homebrew

We have to to stress on the fact that Homebrew installs packages to their own directory and then symlinks their files into /usr/local.

It is always a good idea to update the system before start working. And that, even when we have a daily task that automatically updates the system for us. macOS can use homebrew package manager on maintenance matters. To update/upgrade or check outdated packages, following commands would help.

$ brew outdated                   # lists all outdated packages
$ brew cleanup -n                 # visualize the list of things are going to be cleaned up.

$ brew upgrade                    # Upgrades all things on the system
$ brew update                     # Updates all outdated + brew itself
$ brew update <formula>           # Updates one formula

$ brew install <formula@version>    # Installs <formula> at a particular version.
$ brew tap <formular@version>/brew  # Installs <formular> from third party repository

# untap/re-tap a repo when previous installation failed
$ brew untap <formular> && brew tap <formula>   
$ brew services start <formular>@<version>

Example: key commands to work with homebrew cli

For more informations, visit: Homebrew ~ FAQ.

Installing a monit on a macOS using homebrew

It is hard to deny the supremacy of monit on *NIX systems, and that doesn't exclude macOS systems. Installation of monit on macOS using homebrew aligns with homebrew installation guidelines. From above templates, the next example displays how easy it is to have monit up and running.

$ brew install monit        # Installation of latest monit
$ brew services start monit # Starting latest monit as a service 

Example: installing monit using homebrew

Installing monit on a Windows machine

Whereas macOS systems and Linux are quite relax when it comes to interacting with processes, Windows is a beast on its own way. monit was built for *nix systems but there is no equivalent on Windows systems: Service Control Manager. It basically has the same ability to check and restart processes that are failing.

Automated upgrades

Following the SemVer ~ aka Semantic Versioning standard, it is not recommended to consider minor/major versions for automated upgrades. One of the reasons being that these versions are subject to introducing breaking changes or incompatibility between two versions. On the other hand, patches are less susceptible to introduce breaking changes, whence ideal candidates for automated upgrades. Another among other reasons, being that security fixes are released as patches to a minor version.

In case of a critical infrastructure piece that is monitoring, we expect breaking changes when a new version introduces a configuration setting is added, or dropped between two successive versions. Monit is a well thought software that provides backward compatibility, so chances for breaking changes between two minor versions is really minimal.

We should highlight that it is always better to upgrade at deployment time. The process is even easier in containerized context. We should also automate only patches, to avoid to miss security patches.

In the context of Linux, we will use the unattended-upgrades package to do the work.

$ apt-get install unattended-upgrades apticron

Example: install unattended-upgrades

Two things to fine-tune to make this solution work are: to enable a blacklist of packages we do not to automatically update, and two, to enable particular packages we would love to update on a periodical basis. That is compiled in the following shell scripts.

Unattended-Upgrade::Allowed-Origins {
//  "${distro_id}:${distro_codename}";
    "${distro_id}:${distro_codename}-security"; # upgrading security patches only 
//   "${distro_id}:${distro_codename}-updates";  
//  "${distro_id}:${distro_codename}-proposed";
//  "${distro_id}:${distro_codename}-backports";
};

Unattended-Upgrade::Package-Blacklist {
    "vim";
};

Example: fine-tune the blacklist and whitelist in /etc/apt/apt.conf.d/50unattended-upgrades

The next step is necessary to make sure unattended-upgrades download, install and cleanups tasks have a default period: once, twice a day or a week.

APT::Periodic::Update-Package-Lists "1";            # Updates package list once a day
APT::Periodic::Download-Upgradeable-Packages "1";   # download upgrade candidates once a day
APT::Periodic::AutocleanInterval "7";               # clean week worth of unused packages once a week
APT::Periodic::Unattended-Upgrade "1";              # install downloaded packages once a day

Example: tuning the tasks parameter /etc/apt/apt.conf.d/20auto-upgrades

This approach works on Linux(Ubuntu), especially deployed in production, but not Windows nor macOS. The last issue, is to be able to report problems when an update fails, so that a human can intervene whenever possible. That is where the second tool apticron in first paragraph intervenes. To make it work, we will specify which email to send messages to, and that will be all.

EMAIL="<email>@<host.tld>"

Example: tuning reporting tasks email parameter /etc/apticron/apticron.conf

Conclusion

In this article we revisited ways to install monit on various platforms. Even though configuration was beyond the scope of this article, we managed to get everyday quick refreshers out.

Reading list and References

#nodejs #homebrew #UnattendedUpgrades #monit #y2020 ,#Jan2020 #HowTo #ConfiguringNodejsApplications #tdd #TestingNodejsApplications

This article revisits essentials on how to install nginx non blocking single threaded multipurpose web server on development and production servers.

This article has complementary materials to the Testing nodejs Applications book. However, the article is designed to help both those who already bought the book, as well as the wide audience of software developers to setup working environment. Testing Nodejs Applications Book Cover You can grab a copy of this book on this link

Installing nginx on Linux

It is always a good idea to update the system before start working. There is no exception, even when a daily task updates automatically binaries. That can be achieved on Ubuntu and Aptitude enabled systems as following:

$ apt-get update # Fetch list of available updates
$ apt-get upgrade # Upgrades current packages
$ apt-get dist-upgrade # Installs only new updates

Example: updating aptitude binaries

At this point most of packages should be installed or upgraded. Except Packages whose PPA have been removed or not available in the registry. Installing software can be done by installing binaries, or using Ubuntu package manager.

Installing a nginx on Linux using apt

Updating/Upgrading or first install of nginx server can be achieved with by the following commands.

$ sudo add-apt-repository ppa:nginx/stable
$ sudo apt-get update 
$ sudo apt-get install - nginx 

# To restart the service:
$ sudo service nginx restart 

Example: updating PPA and installing nginx binaries

Adding nginx PPA in first step is only required for first installs, on a system that does not have the PPA available in the system database.

Installing nginx on macOS

In case homebrew is not already available on your mac, this is how to get one up and running. On its own, homebrew depends on ruby runtime to be available.

homebrew is a package manager and software installation tool that makes most developer tools installation a breeze.

$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Example: installation instruction as provided by brew.sh

Generally speaking, this is how to install/uninstall things with brew

$ brew install wget 
$ brew uninstall wget 

Example: installing/uninstalling wget binaries using homebrew

We have to to stress on the fact that Homebrew installs packages to their own directory and then symlinks their files into /usr/local.

It is always a good idea to update the system before start working. And that, even when we have a daily task that automatically updates the system for us. macOS can use homebrew package manager on maintenance matters. To update/upgrade or check outdated packages, following commands would help.

$ brew outdated                   # lists all outdated packages
$ brew cleanup -n                 # visualize the list of things are going to be cleaned up.

$ brew upgrade                    # Upgrades all things on the system
$ brew update                     # Updates all outdated + brew itself
$ brew update <formula>           # Updates one formula

$ brew install <formula@version>    # Installs <formula> at a particular version.
$ brew tap <formular@version>/brew  # Installs <formular> from third party repository

# untap/re-tap a repo when previous installation failed
$ brew untap <formular> && brew tap <formula>   
$ brew services start <formular>@<version>

Example: key commands to work with homebrew cli

For more informations, visit: Homebrew ~ FAQ.

Installing a nginx on a Mac using homebrew

$ brew install nginx@1.17.8  # as in <formula>@<version>

Example: installing nginx using homebrew

Installing nginx on a Windows machine

MacOs comes with Python and Ruby already enabled, these two languages are somehow required to run successfully a nodejs environment on a Mac. This is an easy target as nginx gives windows binaries that we can download and install on a couple of clicks.

Automated upgrades

Before we dive into automatic upgrades, we should consider nuances associated to managing an nginx deployment. The updates fall into two major, quite interesting, categories: patch updates and version upgrades.

Following the SemVer ~ aka Semantic Versioning standard, it is not recommended to consider minor/major versions for automated upgrades. One of the reasons being that these versions are subject to introducing breaking changes or incompatibility between two versions. On the other hand, patches are less susceptible to introduce breaking changes, whence ideal candidates for automated upgrades. Another among other reasons, being that security fixes are released as patches to a minor version.

In case of a WebServer, breaking changes may be introduced when a critical configuration setting is added, or dropped between two successive versions.

We should highlight that it is always better to upgrade at deployment time. The process is even easier in containerized context. We should also automate only patches, to avoid to miss security patches.

In the context of Linux, we will use the unattended-upgrades package to do the work.

$ apt-get install unattended-upgrades apticron

Example: install unattended-upgrades

Two things to fine-tune to make this solution work are: to enable a blacklist of packages we do not to automatically update, and two, to enable particular packages we would love to update on a periodical basis. That is compiled in the following shell scripts.

Unattended-Upgrade::Allowed-Origins {
//  "${distro_id}:${distro_codename}";
    "${distro_id}:${distro_codename}-security"; # upgrading security patches only 
//   "${distro_id}:${distro_codename}-updates";  
//  "${distro_id}:${distro_codename}-proposed";
//  "${distro_id}:${distro_codename}-backports";
};

Unattended-Upgrade::Package-Blacklist {
    "vim";
};

Example: fine-tune the blacklist and whitelist in /etc/apt/apt.conf.d/50unattended-upgrades

The next step is necessary to make sure unattended-upgrades download, install and cleanups tasks have a default period: once, twice a day or a week.

APT::Periodic::Update-Package-Lists "1";            # Updates package list once a day
APT::Periodic::Download-Upgradeable-Packages "1";   # download upgrade candidates once a day
APT::Periodic::AutocleanInterval "7";               # clean week worth of unused packages once a week
APT::Periodic::Unattended-Upgrade "1";              # install downloaded packages once a day

Example: tuning the tasks parameter /etc/apt/apt.conf.d/20auto-upgrades

This approach works on Linux(Ubuntu), especially deployed in production, but not Windows nor macOS. The last issue, is to be able to report problems when an update fails, so that a human can intervene whenever possible. That is where the second tool apticron in first paragraph intervenes. To make it work, we will specify which email to send messages to, and that will be all.

EMAIL="<email>@<host.tld>"

Example: tuning reporting tasks email parameter /etc/apticron/apticron.conf

Conclusion

In this article we revisited ways to install nginx on various platforms. Even though configuration was beyond the scope of this article, we managed to get everyday quick refreshers out.

Reading list

#nodejs #homebrew #UnattendedUpgrades #nginx #y2020 #Jan2020 #HowTo #ConfiguringNodejsApplications #tdd #TestingNodejsApplications

This article revisits essentials on how to install mongodb, one of leading noSQL databases on development and production servers.

This article has complementary materials to the Testing nodejs Applications book. However, the article is designed to help those who bought the book to setup their working environment, as well as the wide audience of software developers who need same information.

In this article we will talk about:

__

  • How to install mongodb on Linux, macOS and Windows.
  • How to stop/start automatically prior/after system restarts

We will not talk about:

__

  • How to configure mongodb for development and production, as that is subject of another article worth visiting.
  • How to manage mongodb in a production environment, either in a containerized or standalone contexts.
  • How to load mongodb on Docker and Kubernetes

Installing mongodb on Linux

It is always a good idea to update the system before start working. There is no exception, even when a daily task updates automatically binaries. That can be achieved on Ubuntu and Aptitude enabled systems as following:

$ apt-get update        # Fetch list of available updates
$ apt-get upgrade       # Upgrades current packages
$ apt-get dist-upgrade  # Installs only new updates

Example: updating aptitude binaries

At this point most of packages should be installed or upgraded. Except Packages whose PPA have been removed or not available in the registry. Installing software can be done by installing binaries, or using Ubuntu package manager.

Installing a mongodb on Linux using apt

Updating/Upgrading or first time fresh install of mongodb can follow next scripts.

sudo may be skipped if the current user has permission to write and execute programs

# Add public key used by the aptitude for further updates
# gnupg should be available in the system
$ apt-get install gnupg 
$ wget -qO - https://www.mongodb.org/static/pgp/server-3.6.asc | sudo apt-key add - 

# create and add list for mongodb (version 3.6, but variations can differ from version to version, the same applies to architecture)
$ echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list

# Updating libraries and make the actual install 
$ sudo apt-get update
$ sudo apt-get install -y mongodb-org

# To install specific version(3.6.17 in our example) of mongodb, the following command helps
$ sudo apt-get install -y mongodb-org=3.6.17 mongodb-org-server=3.6.17 mongodb-org-shell=3.6.17 mongodb-org-mongos=3.6.17 mongodb-org-tools=3.6.17

Example: adding mongodb PPA binaries and installing a particular binary version

It is always a good idea to upgrade often. Breaking changes happen on major/minor binary updates, but less likely on patch upgrades. The versions goes by pair numbers, so 3.2, 3.4, 3.6 etc. The transition that skips two version may be catastrophic. For example upgrades from any 3.x to 3.6, for this to work, there should be upgraded to an intermediate update from 3.x to 3.4, after which the update from 3.4 to 3.6 becomes possible.

# Part 1
$ apt-cache policy mongodb-org          # Checking installed MongoDB version 
$ apt-get install -y mongodb-org=3.4    # Installing 3.4 MongoDB version 

# Part 2   
# Running mongodb
$ sudo killall mongod && sleep 3 && sudo service mongod start
$ sudo service mongodb start           

# Part 3 
$ mongo                                 # Accessing to mongo CLI

# Compatible Mode
$ > db.adminCommand( { setFeatureCompatibilityVersion: "3.4" } )  
$ > exit

# Part 3 
$ sudo apt-get install -y mongodb-org=3.6   # Upgrading to latest 3.6 version 
# Restart Server + As in Part 2.

Example: updating mongodb binaries and upgrading to a version

Installing mongodb on macOS

In case homebrew is not already available on your mac, this is how to get one up and running. On its own, homebrew depends on ruby runtime to be available.

homebrew is a package manager and software installation tool that makes most developer tools installation a breeze. We should also highlight that homebrew requires xcode to be available on the system.

$ /usr/bin/ruby -e \
    "$(curl -fsSL https://raw.githubusercontent.com \
    /Homebrew/install/master/install)"

Example: installation instruction as provided by brew.sh

Generally speaking, this is how to install and uninstall things with brew

$ brew install wget 
$ brew uninstall wget 

Example: installing/uninstalling wget binaries using homebrew

We have to to stress on the fact that Homebrew installs packages to their own directory and then symlinks their files into /usr/local.

It is always a good idea to update the system before start working. And that, even when we have a daily task that automatically updates the system for us. macOS can use homebrew package manager on maintenance matters. To update/upgrade or check outdated packages, following commands would help.

$ brew outdated                   # lists all outdated packages
$ brew cleanup -n                 # visualize the list of things are going to be cleaned up.

$ brew upgrade                    # Upgrades all things on the system
$ brew update                     # Updates all outdated + brew itself
$ brew update <formula>           # Updates one formula

$ brew install <formula@version>    # Installs <formula> at a particular version.
$ brew tap <formular@version>/brew  # Installs <formular> from third party repository

# untap/re-tap a repo when previous installation failed
$ brew untap <formular> && brew tap <formula>   
$ brew services start <formular>@<version>

Example: key commands to work with homebrew cli

For more informations, visit: Homebrew ~ FAQ.

Installing a mongodb on a Mac using homebrew

$ brew tap mongodb/brew 
$ brew install mongodb-community@3.6
$ brew services start mongodb-community@3.6 # start mongodb as a mac service 

Example: Install and running mongodb as a macOS service

Caveats ~ We have extra steps to make in order to start/stop automatically when the system goes up/down. This step is vital when doing development on macOS , which does not necessarily needs Linux production bound task runners.

# To have launchd start mongodb at login:
$ ln -s /usr/local/opt/mongodb/*.plist ~/Library/LaunchAgents/

# Then to load mongodb now:
$ launchctl load -w ~/Library/LaunchAgents/homebrew.mxcl.mongodb.plist

# To unregister and stop the service, use the following command
$ launchctl unload -w ~/Library/LaunchAgents/homebrew.mxcl.mongodb.plist

# When not want/need launchctl this command works fine
$ mongod 

Example: Stop/Start when the system stops/starts

Installing mongodb on a Windows machine

Whereas MacOs, and most Linux distributions, come with Python and Ruby already enabled, It takes extra mile for Windows to make those two languages available. We have to stress on the fact that those two languages are somehow required to deploy a mongodb environment on most platforms, especially when working with nodejs.

The bright side of this story is that mongodb provides windows binaries that we can downloaded and installed in a couple of clicks.

Automated upgrades

Before we dive into automatic upgrades, we should consider nuances associated to managing a mongodb instance. The updates fall into two major, quite interesting, categories: patch updates and version upgrades.

Following the SemVer ~ aka Semantic Versioning standard, it is not recommended to consider minor/major versions for automated upgrades. One of the reasons being that these versions are subject to introducing breaking changes or incompatibility between two versions. On the other hand, patches are less susceptible to introduce breaking changes, whence ideal candidates for automated upgrades. Another among other reasons, being that security fixes are released as patches to a minor version.

We should highlight that it is always better to upgrade at deployment time. The process is even easier in containerized context. We should also automate only patches, to avoid to miss security patches.

In the context of Linux, we will use the unattended-upgrades package to do the work.

$ apt-get install unattended-upgrades apticron

Example: install unattended-upgrades

Two things to fine-tune to make this solution work are: to enable a blacklist of packages we do not to automatically update, and two, to enable particular packages we would love to update on a periodical basis. That is compiled in the following shell scripts.

Unattended-Upgrade::Allowed-Origins {
//  "${distro_id}:${distro_codename}";
    "${distro_id}:${distro_codename}-security"; # upgrading security patches only 
//   "${distro_id}:${distro_codename}-updates";  
//  "${distro_id}:${distro_codename}-proposed";
//  "${distro_id}:${distro_codename}-backports";
};

Unattended-Upgrade::Package-Blacklist {
    "vim";
};

Example: fine-tune the blacklist and whitelist in /etc/apt/apt.conf.d/50unattended-upgrades

The next step is necessary to make sure unattended-upgrades download, install and cleanups tasks have a default period: once, twice a day or a week.

APT::Periodic::Update-Package-Lists "1";            # Updates package list once a day
APT::Periodic::Download-Upgradeable-Packages "1";   # download upgrade candidates once a day
APT::Periodic::AutocleanInterval "7";               # clean week worth of unused packages once a week
APT::Periodic::Unattended-Upgrade "1";              # install downloaded packages once a day

Example: tuning the tasks parameter /etc/apt/apt.conf.d/20auto-upgrades

This approach works on Linux(Ubuntu), especially deployed in production, but not Windows nor macOS. The last issue, is to be able to report problems when an update fails, so that a human can intervene whenever possible. That is where the second tool apticron in first paragraph intervenes. To make it work, we will specify which email to send messages to, and that will be all.

EMAIL="<email>@<host.tld>"

Example: tuning reporting tasks email parameter /etc/apticron/apticron.conf

Conclusion

In this article we revisited ways to install mongodb on various platforms. Even though configuration was beyond the scope of this article, we managed to get everyday quick refreshers out in the article. There are areas we wish we added more coverage, probably not on the first version of this article.

Some of other places where people are wondering how to install mongodb on various platforms include, but not limited to Quora, StackOverflow and Reddit.

Reading list

#nodejs #homebrew #UnattendedUpgrades #mongodb #y2020 #Jan2020 #HowTo #ConfiguringNodejsApplications #tdd #TestingNodejsApplications

This article revisits essentials on how to install redis key/value data store on development and production servers.

This article has complementary materials to the Testing nodejs Applications book. However, the article is designed to help both those who already bought the book, as well as the wide audience of software developers to setup working environment. Testing Nodejs Applications Book Cover You can grab a copy of this book on this link

Installing redis on Linux

It is always a good idea to update the system before start working. There is no exception, even when a daily task updates automatically binaries. That can be achieved on Ubuntu and Aptitude enabled systems as following:

$ apt-get update # Fetch list of available updates
$ apt-get upgrade # Upgrades current packages
$ apt-get dist-upgrade # Installs only new updates

Example: updating aptitude binaries

At this point most of packages should be installed or upgraded. Except Packages whose PPA have been removed or not available in the registry. Installing software can be done by installing binaries, or using Ubuntu package manager.

Installing a redis on Linux using apt

Updating redis – In case the above global update didn't take effect. The -y option passes YES, so there are no prompt for Y/N? while installing libraries.

# Installing binaries 
$ curl -O http://download.redis.io/redis-stable.tar.gz
$ tar xzvf redis-stable.tar.gz && cd redis-stable 
$ make && make install
# Configure redis - for first time installs 

# Install via PPA 
$ apt-get install -y python-software-properties #provides access to add-apt-repository 
$ apt-get install -y software-properties-common python-software-properties

# @link https://packages.ubuntu.com/bionic/redis 
$ add-apt-repository -y ppa:bionic/redis # Alternatively rwky/redis
$ apt-get update
$ apt-get install -y redis-server

# Starting Redis for development on Mac
$ redis-server /usr/local/etc/redis.conf 

Example: installing redis binaries with aptitude

Installing redis on a Mac system

In case homebrew is not already available on your mac, this is how to get one up and running. On its own, homebrew depends on ruby runtime to be available.

homebrew is a package manager and software installation tool that makes most developer tools installation a breeze.

$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Example: installation instruction as provided by brew.sh

Generally speaking, this is how to install/uninstall things with brew

$ brew install wget 
$ brew uninstall wget 

Example: installing/uninstalling wget binaries using homebrew

We have to to stress on the fact that Homebrew installs packages to their own directory and then symlinks their files into /usr/local.

It is always a good idea to update the system before start working. And that, even when we have a daily task that automatically updates the system for us. macOS can use homebrew package manager on maintenance matters. To update/upgrade or check outdated packages, following commands would help.

$ brew outdated                   # lists all outdated packages
$ brew cleanup -n                 # visualize the list of things are going to be cleaned up.

$ brew upgrade                    # Upgrades all things on the system
$ brew update                     # Updates all outdated + brew itself
$ brew update <formula>           # Updates one formula

$ brew install <formula@version>    # Installs <formula> at a particular version.
$ brew tap <formular@version>/brew  # Installs <formular> from third party repository

# untap/re-tap a repo when previous installation failed
$ brew untap <formular> && brew tap <formula>   
$ brew services start <formular>@<version>

Example: key commands to work with homebrew cli

For more informations, visit: Homebrew ~ FAQ.

Installing a redis on a macOS using curl

Installing redis via a curl command is not that different as on Linux system. The following instructions can accomplish the installation.

# Installing binaries 
$ curl -O http://download.redis.io/redis-stable.tar.gz
$ tar xzvf redis-stable.tar.gz && cd redis-stable 
$ make && make install
# Configure redis - for first time installs 

Example: install redis binaries using curl and make

Installing a redis on a Mac using homebrew

$ brew install redis        # Installation following <formula>@<version> template 
$ brew services start redis # Starting redis as a service  

# Alternatively start as usual 
$ redis-server /usr/local/etc/redis.conf
# Running on port: 6379

Example: install redis binaries using homebrew

Installing redis on a Windows machine

MacOs comes with Python and Ruby already enabled, these two languages are somehow required to run successfully a nodejs environment. This is an easy target as redis gives windows binaries that we can download and install on a couple of clicks.

Automated upgrades

Before we dive into automatic upgrades, we should consider nuances associated to managing a mongodb instance. The updates fall into two major, quite interesting, categories: patch updates and version upgrades.

Following the SemVer ~ aka Semantic Versioning standard, it is recommended that the only pair minor versions be considered for version upgrades. This is because minor versions, as well as major versions, are subject to introducing breaking changes or incompatibility between two versions. On the other hand, patches do not introduce breaking changes. Those can therefore be automated.

We should highlight that it is always better to upgrade at deployment time. The process is even easier in containerized context. We should also automate only patches, to avoid to miss security patches.

In the context of Linux, we will use the unattended-upgrades package to do the work.

$ apt-get install unattended-upgrades apticron

Example: install unattended-upgrades

Two things to fine-tune to make this solution work are: to enable a blacklist of packages we do not to automatically update, and two, to enable particular packages we would love to update on a periodical basis. That is compiled in the following shell scripts.

Unattended-Upgrade::Allowed-Origins {
//  "${distro_id}:${distro_codename}";
    "${distro_id}:${distro_codename}-security"; # upgrading security patches only 
//   "${distro_id}:${distro_codename}-updates";  
//  "${distro_id}:${distro_codename}-proposed";
//  "${distro_id}:${distro_codename}-backports";
};

Unattended-Upgrade::Package-Blacklist {
    "vim";
};

Example: fine-tune the blacklist and whitelist in /etc/apt/apt.conf.d/50unattended-upgrades

The next step is necessary to make sure unattended-upgrades download, install and cleanups tasks have a default period: once, twice a day or a week.

APT::Periodic::Update-Package-Lists "1";            # Updates package list once a day
APT::Periodic::Download-Upgradeable-Packages "1";   # download upgrade candidates once a day
APT::Periodic::AutocleanInterval "7";               # clean week worth of unused packages once a week
APT::Periodic::Unattended-Upgrade "1";              # install downloaded packages once a day

Example: tuning the tasks parameter /etc/apt/apt.conf.d/20auto-upgrades

This approach works on Linux(Ubuntu), especially deployed in production, but not Windows nor macOS. The last issue, is to be able to report problems when an update fails, so that a human can intervene whenever possible. That is where the second tool apticron in first paragraph intervenes. To make it work, we will specify which email to send messages to, and that will be all.

EMAIL="<email>@<host.tld>"

Example: tuning reporting tasks email parameter /etc/apticron/apticron.conf

Conclusion

In this article we revisited ways to install redis on various platforms. Even though configuration was beyond the scope of this article, we managed to get everyday quick refreshers out.

Reading list

#nodejs #homebrew #UnattendedUpgrades #nvm #n #y2020 #Jan2020 #HowTo #ConfiguringNodejsApplications #tdd #TestingNodejsApplications

This article revisits essentials on how to install a nodejs environment, on both development and production servers.

This article has complementary materials to the Testing nodejs Applications book. However, the article is designed to help both those who already bought the book, as well as the wide audience of software developers to setup working environment. Testing Nodejs Applications Book Cover You can grab a copy of this book on this link

Installing nodejs on Linux

When running aptitude, or similar package management tools, installing nodejs becomes quite like installing any other program. The following commands are the ones most people tend to use.

Installing a nodejs on Linux using apt

Challenge using this strategy, is to make sure we have nodejs in our PPA database.

# Adding and using nodejs from individual PPAs 
$ sudo add-apt-repository ppa:chris-lea/nodejs
$ sudo apt-get update
$ sudo apt-get install -y nodejs


# Removing Chriss Lea PPA before using NodeSource PPA 
# @link https://github.com/nodesource/distributions#manual-installation
$ sudo add-apt-repository -y -r ppa:chris-lea/nodejs
$ sudo rm -f /etc/apt/sources.list.d/chris-lea-node_js-*.list
$ sudo rm -f /etc/apt/sources.list.d/chris-lea-node_js-*.list.save


# Adding and using NodeSource distribution 
# @link https://github.com/nodesource/distributions 
$ curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
# Alternatively 
$ curl -sL https://deb.nodesource.com/setup | sudo bash -
$ sudo apt-get install -y nodejs

Example: install nodejs binaries aptitude

Once we have nodejs to be recognized by aptitude, most operations becomes a little more familiar. Normally running apt-get upgrade should upgrade all binaries including nodejs. We should mention that updating nodejs also updates npm.

Installing a nodejs on Linux using nvm

When we want to test an application against a particular version of nodejs, the environment we work in should be able to manage multiple versions of nodejs. That is where tools such as nvm come into play.

To make the tool work, we will have to have nodejs installed on our work environment. The second step will be to download and install nvm to our work environment. After installation, we can use tools provided by the nvm to install and toggle a specific version.

$ curl https://raw.githubusercontent.com/creationix/nvm/master/install.sh | bash
# Or 
$ wget https://raw.githubusercontent.com/creationix/nvm/master/install.sh | bash
# Install a particular nodejs version
$ nvm install v<Major>.<Minor>.<Patch>
# toggle a particular nodejs version
$ nvm use  v<Major>.<Minor>.<Patch>
# execture code with a particular nodejs version 
$ nvm run  v<Major>.<Minor>.<Patch> /path/to/server.js
# uninstall a particular nodejs version
$ nvm uninstall  v<Major>.<Minor>.<Patch>

Example: working with nvm tool

An approach a bit similar to this is available for yet another nodejs version manager known under pseudonym n

Installing nodejs on macOS

It is possible to download and install nodejs manually, with ready to use installer. However, when wanting to tuning nodejs environment at our taste, messing with methods and other tools described below becomes a must.

In case homebrew is not already available on your mac, this is how to get one up and running. On its own, homebrew depends on ruby runtime to be available.

homebrew is a package manager and software installation tool that makes most developer tools installation on macOS a breeze. We should also highlight that homebrew requires xcode to be available on the system.

$ /usr/bin/ruby -e \
    "$(curl -fsSL https://raw.githubusercontent.com \
    /Homebrew/install/master/install)"

Example: installation instruction as provided by brew.sh

Generally speaking, this is how to install and uninstall things with brew

$ brew install wget 
$ brew uninstall wget 

Example: installing/uninstalling wget binaries using homebrew

We have to to stress on the fact that Homebrew installs packages to their own directory and then symlinks their files into /usr/local.

It is always a good idea to update the system before start working. And that, even when we have a daily task that automatically updates the system for us. macOS can use homebrew package manager on maintenance matters. To update/upgrade or check outdated packages, following commands would help.

$ brew outdated                   # lists all outdated packages
$ brew cleanup -n                 # visualize the list of things are going to be cleaned up.

$ brew upgrade                    # Upgrades all things on the system
$ brew update                     # Updates all outdated + brew itself
$ brew update <formula>           # Updates one formula

$ brew install <formula@version>    # Installs <formula> at a particular version.
$ brew tap <formular@version>/brew  # Installs <formular> from third party repository

# untap/re-tap a repo when previous installation failed
$ brew untap <formular> && brew tap <formula>   
$ brew services start <formular>@<version>

Example: key commands to work with homebrew cli

For more informations, visit: Homebrew ~ FAQ.

Installing a nodejs on a macOS using curl

nodejs installation guide suggests this alternative for the MacOs. However, we have to customize a download directory, the version, and installation directory.

$ curl "https://nodejs.org/dist/latest/node-${VERSION:-$(wget \
    -qO- https://nodejs.org/dist/latest/ | \
    sed -nE 's|.*>node-(.*)\.pkg</a>.*|\1|p')}.pkg" > \
    "$HOME/Downloads/node-latest.pkg" && sudo installer \
    -store -pkg "$HOME/Downloads/node-latest.pkg" -target \
    "/"

Example: installation instruction as provided by nodejs

Using homebrew, as demo-ed in next paragraph, may be the most popular approach to install nodejs version on a macOS machine.

Installing a nodejs on a Mac using homebrew

In case homebrew is not already available on your mac, this is how to get one up and running. On its own, homebrew depends on ruby runtime to be available.

homebrew is a package manager and software installation tool that makes most developer tools installation a breeze.

$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Example: installation instruction as provided by brew.sh

In case the homebrew is already available, it is a good practice to update it before adding new software. In the next code sample, we install some tools such as wget, sometimes useful to download binaries.

how to install things with brew

$ brew install nodejs  

Example: install nodejs binaries using homebrew

After installation, add binary to the PATH [ /usr/local/share/npm/bin ], $ nano /etc/paths, append the path /usr/local/share/npm/bin at the end of the file.

homebrew installs packages to their own directory and then symlinks their files into /usr/local.

Installing nodejs on a Windows machine

MacOs and most Linux distributions come with Python and Ruby already enabled. These two languages are somehow required to run successfully a nodejs environment on any server. This is an easy target as nodejs gives windows binaries that we can download and install on a couple of clicks.

Automated upgrades

Before we dive into automatic upgrades, we should consider nuances associated to managing a nodejs instance. The updates fall into two major, quite interesting, categories: patch updates and version upgrades.

Following the SemVer ~ aka Semantic Versioning standard, it is not recommended to consider minor/major versions for automated upgrades. One of the reasons being that these versions are subject to introducing breaking changes or incompatibility between two versions. On the other hand, patches are less susceptible to introduce breaking changes, whence ideal candidates for automated upgrades. Another among other reasons, being that security fixes are released as patches to a minor version.

We should highlight that it is always better to upgrade at deployment time. The process is even easier in containerized context. We should also automate only patches, to avoid to miss security patches.

In the context of Linux, we will use the unattended-upgrades package to do the work.

$ apt-get install unattended-upgrades apticron

Example: install unattended-upgrades

Two things to fine-tune to make this solution work are: to enable a blacklist of packages we do not to automatically update, and two, to enable particular packages we would love to update on a periodical basis. That is compiled in the following shell scripts.

Unattended-Upgrade::Allowed-Origins {
//  "${distro_id}:${distro_codename}";
    "${distro_id}:${distro_codename}-security"; # upgrading security patches only 
//   "${distro_id}:${distro_codename}-updates";  
//  "${distro_id}:${distro_codename}-proposed";
//  "${distro_id}:${distro_codename}-backports";
};

Unattended-Upgrade::Package-Blacklist {
    "vim";
};

Example: fine-tune the blacklist and whitelist in /etc/apt/apt.conf.d/50unattended-upgrades

The next step is necessary to make sure unattended-upgrades download, install and cleanups tasks have a default period: once, twice a day or a week.

APT::Periodic::Update-Package-Lists "1";            # Updates package list once a day
APT::Periodic::Download-Upgradeable-Packages "1";   # download upgrade candidates once a day
APT::Periodic::AutocleanInterval "7";               # clean week worth of unused packages once a week
APT::Periodic::Unattended-Upgrade "1";              # install downloaded packages once a day

Example: tuning the tasks parameter /etc/apt/apt.conf.d/20auto-upgrades

This approach works on Linux(Ubuntu), especially deployed in production, but not Windows nor macOS. The last issue, is to be able to report problems when an update fails, so that a human can intervene whenever possible. That is where the second tool apticron in first paragraph intervenes. To make it work, we will specify which email to send messages to, and that will be all.

EMAIL="<email>@<host.tld>"

Example: tuning reporting tasks email parameter /etc/apticron/apticron.conf

Conclusion

In this article we revisited ways to install nodejs on various platforms. Even though configuration was beyond the scope of this article, we managed to get everyday quick refreshers out.

Reading list

#nodejs #homebrew #UnattendedUpgrades #nvm #n #y2020 #Jan2020 #HowTo #ConfiguringNodejsApplications #tdd #TestingNodejsApplications

Configuration is at forefront of any applications. This article discusses a couple strategies to go about nodejs configurations, and some tools available that can be leveraged to that end.

Techniques explained in this blog, are also available with more details in “Configurations” chapter of the “Testing nodejs Applications” book. You can grab a copy of this book on this link. Testing Nodejs Applications Book Cover

In this article you will learn about:

  • Differentiation of configuration layers
  • Overview of tools that help to manage configuration files
  • Overview of basic configurations for a production ready nodejs application
  • Better manage configuration files, storing and provisioning production secret keys
  • Monitoring, failover, server and SSL certificate tools
  • Reducing configuration code change when new versions of system applications are released

Layers of configuration of nodejs applications

Although this blog article provides overview of tools and configurations, it leaves modularization of configurations in a nodejs setting to another blog post: “Modularize nodejs configurations”.

From production readiness perspective, there are two distinctive layers of application configurations, or at least in context of this blog post.

The first layer consists of configurations required by the system that is going to be hosting nodejs application. Database server settings, monitoring tools, SSH keys, and other third party programs running on the hosting entity, are few examples that fall under this category. We will refer to these as system variables/settings.

The second layer consists of configuration that nodejs application needs to execute intrinsic business logic. They will be referred to as environment variables/settings. Third party issued secret keys or server port number configurations, fall under this category. Most of cases, you will find such configurations in static variables found in the application.

This blog will be about working with the first layer: system settings.

For disambiguation, the system is a computing entity composed of software(an operating system, etc) and hardware(virtual or physical).

Managing system configuration variables

Since environment variable layer of the configuration is technically embedded within the code that uses it by default, changes in configurations are in sync with the code, and vice-versa.

Unlike the environment variables, system variables are not managed the same way nodejs applications they run is managed. Just because our application's new version saw some changes in environment settings, does not mean that the nginx server has its own settings changed as well. From another perspective, just because nginx latest version saw some changes in its settings, does not necessarily mean that our nodejs application environment settings have to change as well.

The problem we constantly face, is to figure out how to manage changes in configurations as the code evolves, and as the underlying system software evolves.

Things become a bit complicated, at least to manage when third party software(database, monitoring tools) code change also involves configuration changes. We will have to be informed about changes at hand, which is not always evident as those changes are released at will of the vendors, and not necessarily communicated to us in realtime. Next, we will have to figure out where every single configuration is located on our system, the apply new modifications. Additional complexity comes in when new changes become incompatible with our current version of nodejs application code, or when rollbacks are not avoidable.

The nodejs application code is not always in sync with the system that hosts it. This is where configuration management(aka CM) tools shine. Passing around both system and environment configuration variable values is a risky business, security-wise. This is where configuration provisioning tools come in handy.

Provisioning secrets at deployment time

In teams that have CI/CD implemented, every programmer has ability to deploy latest code version to production. With great powers comes great responsibilities. Making sensitive data accessible to a larger audience, comes with an increased security risk – for instance leaking secret keys to the public.

The challenge relies on how to approach configuration data management, as a part of a software, giving access to developers ability to work with code, but limiting access to production configuration secrets.

The keyword is in provisioning production secrets at deployment time, and as a part of delivery step, and let any developer have own development secrets. This way, one compromised developer account cannot lead to organization wide data breach.

Example of tools that makes provisioning secrets possible: SecretHub, Kubernetes, HashiCorp Vault, etc.

Reducing configuration changes when new system applications rollout

The 12 App Factor suggests to manage configuration as code. That makes it fast to deploy application anywhere, with less code change upon receiving new code releases.

In most of applications that are not containerized, configurations can be stored on a file server, for example at /etc/config/[app-name]/config.ext. This works on a smaller scale. You will always realize that it becomes a problem to set up a new developer and production machines. But having such a convention in place, reduces the pain.

In case of managing multiple instances of same application, it is better to move this configuration inside the code, at least at build time, ideally at root: [app-root]/config/config.ext. At deployment time, there will be an additional symlinking step, to make sure the new deployment points to the right configuration files.

Configure nginx to serve nodejs application

nginx is a very good alternative to Apache server. Its non blocking, single threaded model makes it a perfect match to proxy nodejs applications. It is also possible to configure it as a load balancer.

The location of nginx configuration files depend on Operating System distribution, the application is hosted on. In our context, we assume that our operating system is Linux/Unix and nginx is installed and configured at /etc/nginx.

Some other possible places are /usr/local/nginx, /usr/local/etc/nginx or any other location depending on how the operating system manages its filesystem. The paths above are of course on Linux or Unix distributions.

We recommend reading “How to install nodejs and “How to install nginx for more in-depth information that may not be found in current blog post.

The magic happens at the upstream nodeapps section. This configuration plays the gateway role, and makes public a server that was otherwise private.


upstream nodeapps{
  # Directs to the process with least number of connections.
  least_conn;
  server 127.0.0.1:8080 max_fails=0 fail_timeout=10s;
  server localhost:8080 max_fails=0 fail_timeout=10s;
  keepalive 512;
}

server {
  listen 80;
  server_name app.website.tld;
  client_max_body_size 16M;
  keepalive_timeout 10;

  # Make site accessible from http://localhost/
  root /var/www/[app-name]/app;
  location / {
    proxy_pass http://nodeapps;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-IP $remote_addr;
  }
}

Example: Typical nginx configuration at /etc/nginx/sites-available/app-name

Configure redis to run with a nodejs server

redis is a minimalistic yet feature complete in memory key-value data store engine. The need to have redis in addition to a database arises from the need to make realtime possible in clustered/multi-process nodejs deployment. It is possible to run as a standalone or clustered environment.

The location of redis configuration files depend on Operating System distribution, the application is hosted on. In our context, we assume that our operating system is Linux/Unix and redis is installed and configured at /etc/redis.conf.

Some other possible places are /usr/local/redis, /usr/local/etc/redis or any other location depending on how the operating system manages its filesystem.

There is little to no configuration required to run a redis instance, and same configuration data can be passed as arguments at start time.

port 6380
maxmemory 2mb

Example:

To launch redis via CLI, the following command can be typed on the interface: – $ redis-server --port 6380 --slaveof 127.0.0.1 6379 in this configuration, redis starts on localhost as a slave of another instance running on port 6379. – $ redis-server /usr/local/etc/redis.conf in this configuration, redis starts using configuration settings stated in /usr/local/etc/redis.conf.

We recommend reading “How to install redis and “How to install nodejs for more in-depth information that may not be found in current blog post. redis configuration manual

Configure mongodb as a database server for nodejs project

mongodb is a noSQL database engine that covers most use cases a nodejs application can do. It is possible to configure the database in a cluster, as well as standalone mode.

The location of mongodb configuration files depend on Operating System distribution, the database server is hosted on. In our context, we assume that our operating system is Linux/Unix and mongodb is installed and configured at /etc/mongod.conf.

Some other possible places are /usr/local/mongodb, /usr/local/etc/mongodb or any other location depending on how the operating system manages its filesystem. As always, init scripts can be found at //

There is not a lot of configurations to change to run mongodb server. It is possible to start using the service right after installation, with one exception: when running multiple mongodb instances on the same server, or needing replication and shard features.

processManagement:
   fork: true
net:
  - bindIp: localhost
  + bindIp: localhost,10.8.0.10,192.168.4.24,/tmp/mongod.sock
   port: 27017
storage:
  - dbPath: /srv/mongodb
  + dbPath: /custom/path/to/mongodb
systemLog:
   destination: file
  - path: "/var/log/mongodb/mongod.log"
  + path: "/custom/path/to/mongod.log"
  + logRotate: rename
    logAppend: true
storage:
   journal:
      enabled: true
+ security:
   keyFile: /srv/mongodb/keyfile

Example: typical mongodb configuration in /etc/mongod.conf

We recommend reading “How to install mongodb and “How to install nodejs for more in-depth information that may not be found in current blog post, mongodb administration configuration manual

Configure nginx to serve WebSockets with an expressjs and socket.io application

The configuration exposed above does make a nodejs server running on a private network public. However, since the protocol of communication is HTTP, any other protocol, for instance WebSocket, trying to communicated on the same channel will yield an error. To make WebSocket work and using the same port as HTTP(port:80), we will need nginx to upgrade HTTP so that WebSocket messages can pass as well.

The script below executes tasks in following order:

#1 Tells nginx which version to upgrade to #2 Tells nginx to upgrade HTTP to version 1.1 #3 Tells nginx to upgrade upon receiving socket flash request

server{
  #...
  location /{
      proxy_http_version 1.1; #1
      proxy_set_header Upgrade $http_upgrade; #2
      proxy_set_header Connection "upgrade"; #3
  }
}

Example: 3 lines that enable nginx to serve WebSockets

Proxying WebSockets in an nginx configuration is based on ideas of Chriss Lea's blog post Proxying WebSockets with Nginx

Configure upstart to start nodejs application

With configurations we have at this point, the system applications are going to be useable, if we so state from the command line interface. That is, every application will have to be specifically executed.

The issue in production environment is that the terminal has to be closed at some point, once all tasks regarding a command line interface are completed. There is already some services such as init or systemctl that are already shipped with the system. Using upstart for starting and stopping any application is due to its ease of configurations, asynchronous and reactive nature of the tool, that other tools mentioned above lack.

upstart is a free and open source task runner. It was designed with Ubuntu Linux distribution in mind, but can also work on other Linux/Unix distributions. It has expressive task declaration that even newbies can feel comfortable using.

The location of upstart configuration files depend on Operating System distribution the application is hosted on. In our context, we assume that our operating system is Linux/Unix and upstart is installed and configured at /etc/upstart.

Some other possible places are /usr/local/upstart, /usr/local/etc/upstart or any other location depending on how the operating system manages its filesystem.

At the end of a successful configuration of tasks, we should be able to start all of the system applications and services using the following script, either by running one by one, or by making an extra executable file to simplify our task

As a reminder, the command follows following statement sudo service <servicename> <control>, where the service-name is technically our application and will be located at /etc/init/<servicename>.conf, whereas control is one of start/restart or stop keywords

# testing validity of configurations
init-checkconf /etc/init/nginx.conf
init-checkconf /etc/init/redis.conf
init-checkconf /etc/init/mongod.conf
init-checkconf /etc/init/appname.conf

# restart to re-use same script post deployment
service nginx   restart  
service redis   restart  
service mongodb restart  
service appname restart  

Example: tasks to start/restart all deployment applications in appname/bin/start.sh or on a command line

Alternatively, we should be able to stop services either one by one, or all of the services, using the scripts as in the following example

service nginx   stop  
service redis   stop  
service mongodb stop  
service appname stop  

# In case mongod stops to halt 
sudo /usr/bin/mongod -f /etc/mongod.conf --shutdown

Example: tasks to stop applications in appname/bin/stop.sh or on a command line

Now that we know how to launch our services, the problem remains on how do we configure each one of those services we are just running. The following are typical example, aforementioned services can be brought online.

# nginx

description "nginx http daemon"
author "Author Name"

start on (filesystem and net-device-up IFACE!=lo)
stop on runlevel [!2345]

env DAEMON=/usr/sbin/nginx
env PID=/var/run/nginx.pid

expect fork
respawn
respawn limit 10 5
#oom never

pre-start script
        $DAEMON -t
        if [ $? -ne 0 ]
                then exit $?
        fi
end script

exec $DAEMON

Example: nginx job descriptor in /etc/init/nginx.conf source

The job that will be executed by the redis service is as in the following script.

description "redis server"

start on runlevel [23]
stop on shutdown

pre-stop script
    rm /var/run/redis.pid
end script

script
  echo $$ > /var/run/redis.pid
  exec sudo -u redis /usr/bin/redis-server /etc/redis/redis.conf
end script

respawn limit 15 5

Example: redis job descriptor in /etc/init/redis.conf

If planning to use external monitoring service, respawn limit 15 5 should be either removed, or restart with the monitoring tool the failing service after 15 * 5 seconds time.

The job that will be executed by the mongodb service is as in the following script.

This example is minimalistic, more details can be found on this resource: Github mongod.upstart. Some tuning may be required before use.

#!upstart
description "mongodb server"
author      "author name <author@email>"

start on runlevel [23]
stop on shutdown

pre-stop script
    rm /var/run/mongod.pid
end script

script
  echo $$ > /var/run/mongod.pid
  exec sudo -u mongod /usr/bin/mongod -f /etc/mongod.conf
end script

respawn limit 15 5

Example: mongod job descriptor in /etc/init/mongod.conf

If planning to use external monitoring service, respawn limit 15 5 should be either removed, or restart with the monitoring tool the failing service after 15 * 5 seconds time.

The next and last step in this section, is an example of the script used to start the nodejs server. At this point, any disruption or un-handled problem in the application, will bring down the system as a whole. Other services will be up and running, unfortunately the nodejs server wont! To make failure recovery automatic, we will need yet another tool described in the next section.

#!upstart
description "appname nodejs server"
author      "author name <author@email>"

start on startup
stop on shutdown

script
    export HOME="/var" # this is required by node to be set 
    echo $$ > /var/run/appname.pid
    exec sudo -u appname sh -c "/usr/bin/node /var/www/appname/server.js >> /var/log/appname.log 2>&1"
end script

pre-start script
    # Date format same as (new Date()).toISOString() for consistency
    echo "[`date -u +%Y-%m-%dT%T.%3NZ`] Starting" >> /var/log/appname.log
end script

pre-stop script
    rm /var/run/appname.pid
    echo "[`date -u +%Y-%m-%dT%T.%3NZ`] Stopping" >> /var/log/appname.log
end script

Example: appname job descriptor in /etc/init/mongod.conf

We recommend reading “How to install upstart and “How to install nodejs for more in-depth information that may not be found in current blog post. The upstart event system, what it is and how to use it. On nginx blog: Ubuntu upstart.

Configure monit to monitor nodejs application

The previous section discussed how to automate starting/stopping services. However, when something goes unpredictably wrong, we will not be able to know that something bad happened, nor to tell which system is the culprit. Moreover, we will not be able to recover the from the failure at least by restarting the failing service.

The monitoring tool discussed henceforth, addresses most of the statement made above.

monit is a free and open source monitoring alternative. With a little bit of ingenuity, it is possible to use it to trigger tasks execution, such as sending an alert when something goes off the rails or restarting a failing application.

The location of monit configuration files depend on Operating System distribution the application is hosted on. In our context, we assume that our operating system is Linux/Unix and monit is installed and configured at /etc/monit.

Some other possible places are /usr/local/monit, /usr/local/etc/monit or any other location depending on how the operating system manages its filesystem.

The order of monitoring will go as following:

  • When nginx goes down, we will have to notify the administrator.
  • When redis runs out of memory or goes down, we will force a restart
  • When mongodb may trigger daily backup scripts, may notify when database is down or failed restart attempts
  • When appname goes down|uses abnormally high CPU|run out of memory, we will have to restart and notify the administrator
# The application
check host appname with address 127.0.0.1
    start "/sbin/start appname"
    stop "/sbin/stop appname"
    restart program  = "/sbin/restart appname"
    if failed port 80 protocol http
        request /ok
        with timeout 5 seconds
        then restart
    if cpu > 95% for 2 cycles then alert          # Alert on excessive usage of CPU
    if total cpu > 99% for 10 cycles then restart # Restart if CPU reaches 99 after 10 checks

# Checking using PID 
check process nginx with pidfile /var/run/nginx.pid
    start program = "/etc/init/nginx start"   # service nginx start
    stop program = "/etc/init/nginx stop"     # service nginx stop
    restart program  = "/etc/init/nginx restart"
    if failed port 80 protocol http then restart  # restart when process up, but not answering
    if failed port 443 protocol https then restart

check process redis with pidfile /var/run/redis.pid
    start program = "/etc/init/redis start"   # service redis start
    stop program = "/etc/init/redis stop"     # service redis stop
    if memory > 50 MB then alert
    if total memory > 500 MB then restart

check process mongod with pidfile /var/run/mongod.pid
    start program = "/etc/init/mongod start"   # service mongod start
    stop program = "/etc/init/mongod stop"     # service mongod stop
    restart program  = "/etc/init/mongod restart"
    if failed port 27017 protocol mongo then restart  
    if disk read > 10 MB/s for 2 cycles then alert  # Alert on slow reads 

Example: in /etc/minit/monitrc

To debug validity of /etc/monit/monitrc the following command can be used: monit -t. In case everything looks good, to start services using monit can be done with the following command: monit start all.

There is one more aspect that was not discussed in scripts above, and that is “How does monit knows where to send messages in case of alerting?”. The answer is in the next script, as provided by the monit documentation, but that I feel sharing in this blog post:

# Where to send the email
set alert foo@bar
# What message format 
set mail-format {
      from: Monit Support <monit@foo.bar>
  reply-to: support@domain.com
   subject: $SERVICE $EVENT at $DATE
   message: Monit $ACTION $SERVICE at $DATE on $HOST: $DESCRIPTION.
            Yours sincerely,
            monit
 }
 # Setting the mailserver, in our case, mailgun 
 set mailserver smtp.mailgun.org port 587
  username mailgunusr@domain.com password <PASSWORD>
  using <SSL> with timeout 30 seconds
# <SSL> can be SSLV2 | SSLV3 | TLSV1 | TLSV11 | TLSV12 | TLSV13

Example: custom alert messages in /etc/minit/monitrc

This is an example of few things that can be achieved. There are more that monit can do to enhance the deployment experience, and free of charge. Those things can include, but not limited to, schedule reporting, database backups, purging sessions or accounts that looks not right, as well as sending triggering tasks to send emails.

We recommend reading “How to install monit and “How to install nodejs for more in-depth information that may not be found in current blog post. Quick tutorial on monit. How to install and configure monit, Creating issues when something goes wrong

Conclusion

The two tools that tie the whole system together, also need a system to start and stop them. Luckily, the Linux/Unix environment provides a way to make daemons start at start time.

References

#snippets #configurations #questions #discuss #y2020 #Jan2020