Tunneling nodejs applications
It is quite a challenge to open up the local environment for supervised access to the world, just as publicly accessible web servers do. Two scenarios that can make this statement a bit clearer are when wanting to demo to a remote customer, or want to allow a remote public server to send a WebHook payload to localhost. This communication that is otherwise seen as impossible, is made possible with a technique commonly referred to as tunneling. And that is going to be the subject of this blog post.
This blog will have additional information in near future.
In this article we will talk about:
- How to debug a remote server on a local development environment
- How to share local development work to the world using
ssh,vscode, and or online services - How to receive a WebHook payload from a remote server to a local instance for testing purposes
- How to expose
nodejsruntime to the browser for debugging purposes. - How to prevent local development environment from hitting remote services using proxy setup
Even though this blog post was designed to offer complementary materials to those who bought my Testing
nodejsApplications book, the content can help any software developer to tuneup working environment. You use this link to buy the book.
Tunneling
Tunneling does port forwarding from a client machine to a server. The client and server here are seen from the perspective of the machine that brokerages data. Tunneling has three faces: local port forwarding, remote port forwarding, and dynamic forwarding
Local port forwarding
A clear scenario where we need a local port forwarding is being on a private network that restricts access to service: restricted.service.com. A tunnel goes through another service(jump server) that isn't restricted: safe.com which forwards our local port 3000 to restricted.service.com:8080. restricted.service.com doesn't only need to be a service restricted on a private network, It can also be for instance a production database server(postgre sql/mongodb or mysql) provisioned via a reverse proxy-server or behind a firewall that we want to access to from our local instance, a monitoring tool such as uptime or monit that are not accessible to the public. And the list can go on.
# Local port forwarding
# now restricted.service.com is available on http://localhost:3000
$ ssh -nNT -L 3000:restricted.service.com:8080 usr@safe.com
# to access remote psql service using $ psql -h localhost -p 5431, do
$ ssh -nNT -L 5431:localhost:5432 usr@safe.com
# to access remote mongod service using $ mongod -h localhost -p 27016, do
$ ssh -nNT -L 27016:localhost:27017 usr@safe.com
How to read ssh -nNT -L 27016:localhost:27017 usr@safe.com commands in plain English: forward our local port 27016 to remote server's localhost:27017 which is totally fine once ssh'd into safe.com. In this case, the localhost:27017 is relative to safe.com's localhost. Our localhost is implicit in our command, as the extended version would look like ssh -nNT -L localhost:27016:localhost:27017 usr@safe.com. In this case, however, ssh -nNT -L 3000:restricted.service.com:8080 usr@safe.com we are using our safe server as a jump server to connect to a remote service.
Remote port forwarding
Another clear scenario where we need remote port forwarding is being able to make our local server, which is most definitely not available to the public, become somehow available to the public. The use case that may make this concept familiar, is when we want to demo something on a local private instance to a customer or a friend. Another scenario would be redirecting traffic from one (public) server to another (local) server — for instance, redirecting production read/writes to a local instance.
vi /etc/ssh/sshd_config
# edit: GatewayPorts yes
sudo service ssh restart
# now safe.com:8080 exposes our http://localhost:3000 to the public
$ ssh -nNT -R 8080:localhost:3000 usr@safe.com
Can it be possible to use ssh tunnel as a blue/green deployment switch.
Writing to remote server from a locally sourced data
This technique can be instrumental in two use cases:
The case when data have to be copied over to a remote server, for instance, deployments using FTP.
# using csp to download file to from remote server to local machine
$ scp username@url|ip:/path/to/file.tar /tmp
# using ssh keys
$ scp -i ~/.ssh/id_rsa.pub username@url|ip:/path/to/file.tar /tmp
The case where a long-living session on a remote server is necessary to perform series of tasks. For instance, monitoring a server for an extended period of time for troubleshooting reasons.
$ ssh [username]@v4.or.v6.ip
$ [passwd]
# curl download from github
$ curl -H "Authorization: token xxxx" -L -o semVer.tar.gz https://github.com/.../semVer.tar.gz
# untar + configure + install
# it may go on for X hours
scpcommand works both ways: to upload files to the remote server and to download files from a remote server
In both cases, to establish a remote session, an ssh tunnel is necessary as presented in two previous examples.
Reading remote server data to a local instance
This technique forwards a remote port to a local instance. The use cases it may be useful are, but not limited to, the following use cases
tail (or tee) production server logs to a local development environment for log analysis purposes
$ ssh -t server tail -f /var/log/remote.log >> /var/log/local.log
$ ssh -t server "tail -f /var/log/remote.log" | tee -a /var/log/local.log
Debug deployment environment code on a local instance
The techniques explained here can well be used in a container environment as well.
Forwarding WebHooks to a local instance
When integrating with asynchronous services such as a payment, or receiving notifications about shipping status, WebHooks play a big role to make this communication happen.
The problem is to be able to validate if the code that handles WebHook payload is working as advertised. One of the ways to test before the sh*t hits the fan is to receive an actual WebHook(or testing WebHook) to a local development instance. And that is not as easy as it sounds.
This section showcase how to achieve that in a few steps, using both hacks and third-party tools.
Conclusion
In this article, we revisited how to open ports so that remote servers and local servers can communicate in a secure way, using tunneling techniques. We also revisited a couple of applications where tunneling might be useful be in the development, demo, or support of a nodejs application in production. There are additional complimentary materials in the “Testing nodejs applications” book.