Thanks to Docker for AWS, setting up a Docker Swarm cluster on AWS can be done in a few clicks in minutes. The provided AWS CloudFormation template first asks for some values and then takes care of everything else automatically.
The requested values include the number of managers and workers, but also the SSH key to be used. Docker for AWS uses this to configure access to the individual EC2 instances. As external access is only allowed to the managers, you must first connect to one of them, before being able to connect to the workers from there.
Log in using SSH
This makes sense from a security perspective, and it works without any problems at first glance. When you log in to one of the managers using SSH and execute the following command, the manager does as expected and starts three instances of Nginx and distributes them in the cluster:
docker service create -p 80:80 -p 443:443 --replicas 3 nginx
Things become more difficult if you want to do the same with a private image, because then authentication is missing. Theoretically, you can solve this by logging in to Docker:
Anyway, this doesn't leave a good feeling, as personal access data, which are only needed every now and then, remain on the server.
Next, if you try to build an image based on a
Dockerfile, you will notice that the build context is no longer the local machine's one, and hence the build won't work. Of course, this shouldn't come as a surprise, as you have switched to a different system using SSH.
Remote access doesn't work
As a way out, it seems to be advisable to address the manager's Docker endpoint directly from the local machine. The assumption is that all you need to do is to set the local environment variable
DOCKER_HOST to the appropriate value. Then, authentication, should be possible by using certificates. But which certificate should be used?
As it turns out, Docker for AWS configures the managers in a way that their respective Docker endpoint can't be addressed over the network at all. Only local connections via
/var/run/docker.sock will be accepted. This prevents any remote access.
SSH to the rescue
The solution is to use SSH with local port forwarding to open a tunnel that connects the
/var/run/docker.sock path from one of the managers to a port on the local machine. To do this, use the
-L flag for SSH. You also have to tell SSH that it should only open the tunnel but not execute a command. This can be done by using the
-N flag. Finally, you need to specify the SSH key to use by providing the
ssh -i aws.pem -N -L 2375:/var/run/docker.sock docker@$IPADDRESS
This opens the connection and allows local access to Docker. To do this, set the environment variable
DOCKER_HOST to the value
tcp://localhost:2375. All other environment variables that may have been set by
docker-machine must be removed using
unset DOCKER_TLS_VERIFY unset DOCKER_CERT_PATH unset DOCKER_MACHINE_NAME export DOCKER_HOST=tcp://localhost:2375
For example, if you now execute the command
the local Docker client accesses one of the Docker managers running on AWS using the SSH tunnel. The tunnel remains open until the
ssh command is canceled by pressing <Ctrl>+<C>. Now the
docker build command uses the local build context and works as expected.
Authentication for Docker for AWS
However, access to private images is still problematic, for example when using
docker service create. Since the manager only assigns the task of downloading the appropriate image to one of the workers instead of doing it on its own, the worker fails because of authentication.
Fortunately, there is the
--with-registry-auth flag, which forwards the authentication of the local Docker client to the remote Docker daemon. This way the command
docker service create now finally works as expected.