Unix socket

.gitignore

@@ -52,3 +52,5 @@ docs/_build/
 
 # PyBuilder
 target/
+
+_trial_temp/

README.rst

@@ -67,12 +67,42 @@ Try it out
 
 Powerstrip ships as a Docker image, and adapters can be any HTTP endpoint, including other linked Docker containers.
 
+NOTE: **there are breaking changes in v0.0.2**
+
+Unix Socket
+-----------
+
+Powerstrip expects Docker to have been reconfigured to listen on ``/var/run/docker.real.sock``.  There are official instructions for doing that .. here: https://docs.docker.com/articles/basics/#bind-docker-to-another-hostport-or-a-unix-socket.
+
+For example, on Ubuntu, the default Docker options are found in ``/etc/default/docker`` which can be edited to say ``DOCKER_OPTS="-H unix:///var/run/docker.real.sock"`` and then run ``sudo service docker restart``.
+
+NOTE: ``boot2docker`` and ``docker-machine`` are not currently supported.
+
+socat
+-----
+
+Because powerstrip listens on a Unix Socket (since v0.0.2) - it means that the HTTP api is not exposed over the network.  If you need powerstrip to listen on a TCP socket - you can use something like socat.  However - it must be noted that it does not work if you run socat inside a container (which is the reason that powerstrip no longers offers an option to listen on a TCP port).
+
+Here is an example of a socat command that will forward traffic on port 2375 to the `/var/run/docker.sock` that powerstrip is listening to:
+
+```bash
+$ socat TCP-LISTEN:2375,reuseaddr,fork UNIX-CLIENT:/var/run/docker.sock
+```
+
+/var/run volume
+---------------
+
+Powerstrip also expects to have a volume for ``/var/run`` on the host bind-mounted to ``/host-var-run`` in the container.
+
+Powerstrip will then create ``/var/run/docker.sock`` from the host's perspective (``/host-var-run/docker.sock`` from inside its container) and normal Docker tools should carry on working as normal.
+
+Example Adapter
+---------------
+
 `Slowreq <https://github.com/clusterhq/powerstrip-slowreq>`_ is a trivial Powerstrip adapter (container) which adds a 1 second delay to all create commands.
 
 Try it out like this (assuming logged into an Ubuntu Docker host).
 
-If you are using ``boot2docker``, drop the ``sudo`` and also unset ``DOCKER_TLS_VERIFY``.
-
 .. code:: sh
 
     $ cd ~/
@@ -86,22 +116,21 @@ If you are using ``boot2docker``, drop the ``sudo`` and also unset ``DOCKER_TLS_
       slowreq: http://slowreq/slowreq-adapter
     EOF
 
-    $ sudo docker run -d --name powerstrip-slowreq \
+    $ sudo DOCKER_HOST="unix:///var/run/docker.real.sock" \
+           docker run -d --name powerstrip-slowreq \
                --expose 80 \
                clusterhq/powerstrip-slowreq:v0.0.1
-    $ sudo docker run -d --name powerstrip \
-               -v /var/run/docker.sock:/var/run/docker.sock \
-               -v $PWD/powerstrip-demo/adapters.yml:/etc/powerstrip/adapters.yml \
+    $ sudo DOCKER_HOST="unix:///var/run/docker.real.sock" \
+           docker run -d --name powerstrip \
+               -v /var/run:/host-var-run \
+               -v ${PWD}/powerstrip-demo/adapters.yml:/etc/powerstrip/adapters.yml \
                --link powerstrip-slowreq:slowreq \
-               -p 2375:2375 \
-               clusterhq/powerstrip:v0.0.1
+               clusterhq/powerstrip:unix-socket
 
     # Note how the second command takes a second longer than the first.
+    $ time sudo DOCKER_HOST="unix:///var/run/docker.real.sock" \
+                docker run ubuntu echo hello
     $ time sudo docker run ubuntu echo hello
-    $ time DOCKER_HOST=localhost:2375 docker run ubuntu echo hello
-
-**Warning:** Powerstrip exposes the Docker API unprotected on port 2375.
-Only use it in private, secure development environments.
 
 **Issues:** If you are using ``SELinux`` and having some issues, disable it or run the following commands:
 
@@ -327,6 +356,7 @@ v0.0.2:
 
 * Add integration tests against real Docker for ``run``, ``build`` and ``pull``, fix various bugs exposed therein.
 * In particular, fix docker ``attach``, streaming responses when there are no post-hooks, GET requests, skip pre-hooks with ``application/tar`` handling, stdin handling for ``attach``.
+* Powerstrip now listens on a Unix Socket and not TCP - this is to get around proxy problems when using a Docker container to foward the TCP traffic
 
 v0.0.1:
 
@@ -340,6 +370,14 @@ Additional Adapter Ideas
 * A pre hook for containers => create that will inject ENV variables loaded from `consul <https://github.com/hashicorp/consul>`_ or `etcd <https://github.com/coreos/etcd>`_
 * A post hook for containers => {start,stop} that will update `consul <https://github.com/hashicorp/consul>`_ or `etcd <https://github.com/coreos/etcd>`_ with the containers exposed endpoints
 
+Running Tests
+=============
+
+To run the test suite do the following commands:
+
+.. code::
+    sudo TEST_PASSTHRU=1 trial powerstrip.test
+
 License
 =======
 

powerstrip.tac

@@ -1,28 +1,34 @@
 import os
+import sys
 from twisted.application import service, internet
 #from twisted.protocols.policies import TrafficLoggingFactory
-from urlparse import urlparse
 
 from powerstrip.powerstrip import ServerProtocolFactory
+from powerstrip.resources import GetDockerHost,GetDockerAPICredentials
+
+TARGET_DOCKER_SOCKET = "/host-var-run/docker.sock"
 
 application = service.Application("Powerstrip")
 
-DOCKER_HOST = os.environ.get('DOCKER_HOST')
-if DOCKER_HOST is None:
-    # Default to assuming we've got a Docker socket bind-mounted into a
-    # container we're running in.
-    DOCKER_HOST = "unix:///var/run/docker.sock"
-if "://" not in DOCKER_HOST:
-    DOCKER_HOST = "tcp://" + DOCKER_HOST
-if DOCKER_HOST.startswith("tcp://"):
-    parsed = urlparse(DOCKER_HOST)
-    dockerAPI = ServerProtocolFactory(dockerAddr=parsed.hostname,
-        dockerPort=parsed.port)
-elif DOCKER_HOST.startswith("unix://"):
-    socketPath = DOCKER_HOST[len("unix://"):]
-    dockerAPI = ServerProtocolFactory(dockerSocket=socketPath)
-#logged = TrafficLoggingFactory(dockerAPI, "api-")
-dockerServer = internet.TCPServer(2375, dockerAPI, interface='0.0.0.0')
-dockerServer.setServiceParent(application)
+# we create a connection to the Docker server based on DOCKER_HOST (can be tcp or unix socket)
+DOCKER_HOST = GetDockerHost(os.environ.get('DOCKER_HOST'))
+dockerAPICredentials = GetDockerAPICredentials(DOCKER_HOST)
+
+# check that /var/run is mounted from the host (so we can write docker.sock to it)
+if not os.path.isdir("/host-var-run"):
+  sys.stderr.write("/var/run must be mounted as /host-var-run in the powerstrip container\n")
+  sys.exit(1)
 
-print r'export DOCKER_HOST=tcp://localhost:2375'
+# check that the docker unix socket that we are trying to connect to actually exists
+if dockerAPICredentials['dockerSocket'] and not os.path.exists(dockerAPICredentials["dockerSocket"]):
+  sys.stderr.write(dockerAPICredentials["dockerSocket"] + " does not exist as a docker unix socket to connect to\n")
+  sys.exit(1)
+
+# check that the unix socket we want to listen on does not already exist
+if os.path.exists(TARGET_DOCKER_SOCKET):
+  sys.stderr.write(TARGET_DOCKER_SOCKET + " already exists - we want to listen on this path\n")
+  sys.exit(1)
+
+dockerAPI = ServerProtocolFactory(**dockerAPICredentials)
+dockerServer = internet.UNIXServer(TARGET_DOCKER_SOCKET, dockerAPI, mode=0660)
+dockerServer.setServiceParent(application)

powerstrip/resources.py

@@ -5,6 +5,9 @@
 from twisted.internet import reactor
 from twisted.internet.task import deferLater
 from twisted.web.server import NOT_DONE_YET
+import os
+from urlparse import urlparse
+from powerstrip import ServerProtocolFactory
 
 class BaseProxyResource(proxy.ReverseProxyResource):
     def getChild(self, path, request):
@@ -21,3 +24,38 @@ def run():
 
 class DeleteContainerResource(BaseProxyResource):
     pass
+
+def GetDockerHost(DOCKER_HOST=None):
+    """
+    Logic for getting the default value of DOCKER_HOST if its either not given or
+    only partially given.
+    The DOCKER_HOST must either start with tcp:// or unix://
+    If no scheme is provided - we check for a leading slash to determine if its tcp or unix
+
+    it is normal to pass the ENV var DOCKER_HOST to this function:
+
+    dockerHost = GetDockerHost(os.environ.get('DOCKER_HOST'))
+    """
+
+    if DOCKER_HOST is None:
+        # Default to assuming we've got a Docker socket bind-mounted into a
+        # container we're running in.
+        DOCKER_HOST = "unix:///host-var-run/docker.real.sock"
+    if "://" not in DOCKER_HOST:
+        if DOCKER_HOST.startswith("/"):
+          DOCKER_HOST = "unix://" + DOCKER_HOST
+        else:
+          DOCKER_HOST = "tcp://" + DOCKER_HOST
+    return DOCKER_HOST
+
+def GetDockerAPICredentials(DOCKER_HOST="unix:///host-var-run/docker.real.sock"):
+    """
+    Logic for getting the arguments to be passed to a ServerProtocolFactory based on the DOCKER_HOST  
+    """
+    if DOCKER_HOST.startswith("tcp://"):
+        parsed = urlparse(DOCKER_HOST)
+        return dict(dockerAddr=parsed.hostname, dockerPort=parsed.port)
+        #ServerProtocolFactory(**mydictionary)
+    elif DOCKER_HOST.startswith("unix://"):
+        socketPath = DOCKER_HOST[len("unix://"):]
+        return dict(dockerSocket=socketPath)

powerstrip/test/test_utils.py

@@ -0,0 +1,80 @@
+# Copyright ClusterHQ Limited. See LICENSE file for details.
+# -*- test-case-name: powerstrip.test.test_utils -*-
+
+from twisted.trial.unittest import TestCase
+from ..resources import GetDockerHost,GetDockerAPICredentials
+
+"""
+Tests for the utils.
+"""
+
+class TestDockerHost(TestCase):
+
+    def test_get_default_docker_host(self):
+        """
+        Test that if nothing is supplied we get the default UNIX socket
+        """
+        dockerHost = GetDockerHost()
+        self.assertEqual(dockerHost, "unix:///host-var-run/docker.real.sock")
+
+    def test_get_path_based_docker_host(self):
+        """
+        Check that if a path with no scheme is supplied then we get a unix socket
+        """
+        dockerHost = GetDockerHost('/var/run/my.sock')
+        self.assertEqual(dockerHost, "unix:///var/run/my.sock")
+
+    def test_get_tcp_based_docker_host(self):
+        """
+        Check that if a path with no scheme is supplied then we get a unix socket
+        """
+        dockerHost = GetDockerHost('localhost:2375')
+        self.assertEqual(dockerHost, "tcp://localhost:2375")
+
+    def test_get_path_based_docker_host_unchanged(self):
+        """
+        Check that if a path with no scheme is supplied then we get a unix socket
+        """
+        dockerHost = GetDockerHost('unix:///var/run/yobedisfileyo')
+        self.assertEqual(dockerHost, "unix:///var/run/yobedisfileyo")
+
+    def test_get_tcp_based_docker_host_unchanged(self):
+        """
+        Check that if a path with no scheme is supplied then we get a unix socket
+        """
+        dockerHost = GetDockerHost('tcp://127.0.0.1:2375')
+        self.assertEqual(dockerHost, "tcp://127.0.0.1:2375")
+
+
+class TestDockerAPICredentials(TestCase):
+
+    def test_get_default_dockerapi_credentials(self):
+        """
+        Test that if nothing is supplied we get the default UNIX socket
+        """
+        dockerAPICredentials = GetDockerAPICredentials()
+
+        self.assertEqual(dockerAPICredentials['dockerSocket'], "/host-var-run/docker.real.sock")
+
+        self.assertNotIn("dockerAddr", dockerAPICredentials)
+        self.assertNotIn("dockerPort", dockerAPICredentials)
+
+    def test_get_tcp_dockerapi_credentials(self):
+        """
+        Test that if TCP is supplied we get the IP / port returned
+        """
+        dockerAPICredentials = GetDockerAPICredentials('tcp://127.0.0.1:2375')
+
+        self.assertEqual(dockerAPICredentials['dockerAddr'], "127.0.0.1")
+        self.assertEqual(dockerAPICredentials['dockerPort'], 2375)
+        self.assertNotIn("dockerSocket", dockerAPICredentials)
+
+    def test_get_unixsocket_dockerapi_credentials(self):
+        """
+        Test that if UNIX is supplied
+        """
+        dockerAPICredentials = GetDockerAPICredentials('unix:///var/run/yobedisfileyo')
+
+        self.assertEqual(dockerAPICredentials['dockerSocket'], "/var/run/yobedisfileyo")
+        self.assertNotIn("dockerAddr", dockerAPICredentials)
+        self.assertNotIn("dockerPort", dockerAPICredentials)