Orchestrating multiple dockers with Swarm

10 min readFeb 4, 2022

In case you are landing here directly, it’s recommended to read through this documentation first.

Following are the topics, which we shall be touching through this blog :-

Introduction to Docker-Swarm.
Need of Docker-Swarm.
Strong Competitor to Swarm.
Docker Virtual Private Network.
High Availability with Docker Swarm.
Launching Docker Swarm at local host.
Demonstrating Docker Swarm with Playground(https://labs.play-with-docker.com)
Launching multiple docker-instances, using swarm, inside docker playground.

Question:- What’s the need of Docker-Swarm ?

Answer:- In order to get these containers into production, we would need some orchestration tool. Docker-Swarm is one such tool available there, which can help us to take applications to production.

Question:- What can Docker-Swarm help us in achieving ?

Answer:- Swarm can help us in following aspects :-

It can enable us to run our containers on a cluster.
it can help us to perform redeployments with zero downtime.

Question :- What actually is Docker-Swarm ?

Answer:-

It’s an orchestration system and that means that all that can manage our containers in production.
Docker does actually contain its own built-In orchestration system, this thing called Docker-Swarm.

Question :- Apart from Docker-Swarm, Is there any other system also available in market for the same purpose ?

Answer:- Yes, of-course, we have Kubernetes, which is far more popular and widely used system for managing containers in production. It’s because Kubernetes has far more features than Swarm. OK, Swarm is less powerful, but it’s still a very feature full.

Describe :- Let’s go through the use-case of virtual private network that we have in-hand currently :-

Solution:- If you have zipped through the previous blogs (in this particular Docker Series, mentioned in references), we have a very simple system, with just two containers at the moment, but each of those containers is publishing a service.

We have a database service.
We also have a kind of Java container running a spring boot application.

We’ve seen that we can quite easily network those containers together, and we’ve seen that we end up with what is basically a virtual private network using DOCKER networking.

Question:- How does a practical use-case looks like in Industry ?

Answer:- In real life you might have hundreds or maybe thousands or possibly hundreds of thousands of these containers to manage. Certainly estates such as Netflix have that kind of complexity in their systems.

Question:- Is there a problem with, what we had been doing so far ? If yes, Can Horizontal Scalability address the problem ?

Answer:- Now, so far, we’ve been running all of our containers on a single computer, on a single hosted system.

Of course, two containers would be absolutely fine to run on a single physical computer, but once we start scaling up to dozens or hundreds of containers, then you’d probably struggle to find a server that can cope. You would certainly need a very, very expensive server to run so many containers.
So it’s very common to scale horizontally, which means to distribute the containers across multiple physical computers.

Question:- Is there some ready-made tool, in order to manage the containers across multiple machines ?

Answer:- Docker-Swarm is a tool that enables us to do manage this situation. In fact, swarm is more than that. It’s a full scale so-called container orchestration system.

Question:- Whats the best thing about Docker-Swarm from High-Availability point of view ?

Answer:- It’s going to allow us to make sure that our containers are always running And if any container fails, then the swarm will automatically restart those containers.

Question:- List down some important differences, with Swarm ?

Answer:- The basic concepts in the swarm looks like below :-

Rather than running on a single host computer, we’re going to distribute across multiple so-called nodes. For example, we might have four nodes open.
So, now we’ve been on a single hosted system and we’ve been running commands such as Docker Container Run. The big difference in a swarm is that we’ll log on to one of these nodes and we’re going to issue commands to docker swarm.

Question:- How does new containers are launched with Swarm ?

Answer:- Even if we’re logged into node one and we’re issuing our commands there, then when we tell the swarm that we want a new instance of a container, docker-swarm will be responsible for deciding on which node that new container shall be launched ?

Roughly what happens is a kind of a balancing system so that no one node has more containers than the other nodes. So that’s the very basic concepts of the docker swarm.

Question:- Can I use Swarm with a Single Node Server as well?

Answer:- Of course, you can use Swarm on a single node too, but it’s a nonsense, really. You would never in production have a single node swarm. It doesn’t make sense.

Question:- How doe we begin with Docker-Swarm on our local machine ?

Answer:- Docker-Swarm has a command something called as init and that will set up a swarm with a single node.

docker swarm init

This node is a manager now i.e. Docker-Swarm Manager.

Question:- How can we add another nodes to this Swarm Manager ?

Answer:- We may now want to add another nodes / workers, to this swarm, then we shall need to run the command being highlighted in the above snapshot.

docker swarm join --token <TOKEN_RECEIVED_FROM_SWARM-MANAGER>

Question:- What’s the fundamental thing required with Docker-Swarm?

Answer:- We need a special type of Bridge-Network, while working with Docker-Swarm. This is called as “Overlay-Network”.

Question:- Why do we need the different type of Network with Swarm ?

Answer:- The big difference, of course, is that our containers are now very likely to be on different physical nodes. So, the communications between them are not going to be within the same host machine. They’re going to be across different physical computers. And for that reason, in swarm, we have a different type of network. It’s called an overlay network. The difference is this network will work across multiple nodes.

Question:- Let’s demonstrate the process of creating an overlay Network with Swarm ?

Answer:- Here is how, we can create an overlay network :-

docker network create --driver overlay aditya-overlay-network

Let’s now list down our networks we have :-

docker network ls

Question:- How does process of launching a container into swarm is different from launching a container barebone ?

Answer:- Up until now, you will have been doing Docker Container Run, followed by that great, big, long list of command line arguments.

docker container run -v /Users/B0218162/Documents/LEARNINGS/MEDIUM-BLOG/Docker/DOCKER_LEARN/DATA_DIR_FOR_CONTAINER:/var/lib/mysql -d -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=fleetman --network aditya-network --name mysqldbpanga mysql:5

The difference is when you’re working in a swarm, instead, a swarm talks about services.

Question:- Whats the meaning of Service and how do we launch it ?

Answer:- A service is a container inside a swarm. Here is how, we can launch a service with Swarm :-

docker service create -d --network aditya-overlay-network -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=fleetman --name mysqldbpangavari mysql:5

Note here that, we have specified the name of our (to be launched) container as : “mysqldbpangavari”.
We are passing an intent to swarm that, container should be launched with a docker-image “mysql” with version as 5.
We also specified the network, while launching the container through the Swarm (Service). Here, we have used the overlay-network, the one that we launched above.
We also specified the TWO environment variables while launching the service with docker swarm i.e. password & database_name.

Question:- Can we verify whether swarm-service got launched successfully or not ?

Answer:- We can very well verify that, swarm-service has been launched or not :-

docker service ls

Question:- Can we verify that, whether container actually launched as well or not ?

Answer:- We can very well verify that, respected container have been launched by our docker swarm-service :-

docker container ls

But remember, if we were running in a real multiple node system, then when we started this service, that container would have been randomly created on one of the nodes of our system, not necessarily the one that we’re running on.

Question:- Let’s demonstrate Docker-Swarm in action ?

Step #1.) Let’s head to playground : https://labs.play-with-docker.com and launch 2 brand new instances there.

Next, At anyone of instance of our choice, we initialise 🚀 the docker-swarm :-

docker swarm init --advertise-addr <eth0_ip_address>

Step #2.) Next at the other instance, let’s ask that node to join the docker -swarm by issuing following command :-

docker swarm join --token <TOKEN_ID> <IP_WHERE_SWARM_WAS_INITALISED>

Step #3.) Now at anyone instance of our choice, let’s create the overlay network. Recall that, we specifically needed this kind of network because, there are multiple machines where our containers shall be distributed :-

docker network create --driver overlay <NAME_OF_NETWORK>

Step #4.) Note that, this network that we created @ Node1, would not exist @ the Node2 :-

docker network ls

Step #5.) Next, we would be creating our Service @ Node1 :-

docker service create -d --network <NAME_OF_NETWORK> -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=fleetman --name database mysql:5

Step #5.) Next, let’s watch the status of the service :-

docker service ls

Above first output shows us that, currently there are no instances of this container running. But we require one instance.The reason for that is the container will be starting up in the background.
Once that container started will that switch to a ONE/ONE. So, that means somewhere in our swarm, there is a container running for the database.

Step #6.) Next, let’s also verify that, whether actual container, has also been started :-

docker container ls

Note the following points :-

In our case, the container powering to mysql, has been launched @ the Node1 only, but it can be very much possible that, this particular instance could had been launched @ another node(i.e. other nodes which are part of Swarm) as well.
Now, say, if we launch another instance as well (i.e. as a swarm-service), then the same would be surely launched on the other instance, because swarm automatically balances the load and would make sure that, each node has 1 container running there.

Step #7.) Let’s observe the logs of the above launched container :-

docker container logs -f <NAME_OF_CONTAINER>

Step #8.) Recall that, docker-swarm manager was launched @ node1 and therefore, we shall be able to able to inquire for all the nodes available in this swarm-network :-

docker node ls

Step #9.) Note that, if we inquire for the nodes @ other node, then we shall not be able to get anything because, we can only execute this inquiry @ the manager node of swarm-network. Worker nodes can’t be used to view or modify cluster state :-

docker node ls

Now, we can very well launch the other service using swarm (it would internally launch the new container).

That’s all in this section. If you liked reading this blog, kindly do press on clap button multiple times, to indicate your appreciation. We would see you in next series.

References :-

Orchestrating multiple dockers with Swarm

Written by aditya goel