Migrating to docker compose

In the folder where we have our Dockerfile with the following content:

FROM ubuntu:22.04
ENV LC_ALL=C.UTF-8

RUN apt-get update && apt-get install -y \
  curl python2 \
  && update-alternatives --install /usr/bin/python python /usr/bin/python2 1

RUN curl -L https://github.com/ytdl-org/youtube-dl/releases/download/2021.12.17/youtube-dl -o /usr/local/bin/youtube-dl \
    && chmod a+rx /usr/local/bin/youtube-dl

WORKDIR /app
ENTRYPOINT ["/usr/local/bin/youtube-dl"]

we create a file called youtube-dl/docker-compose.yml

version: '3.8'

services:
  youtube-dl-ubuntu:
    image: youtube-dl-ubuntu
    build: .

The version setting is not very strict, it just needs to be above 2 because otherwise the syntax is significantly different. See https://docs.docker.com/compose/compose-file/ for more info. The key build: value can be set to a path (ubuntu), have an object with context and dockerfile keys or reference a url of a git repository.

Now we can build and push with just these commands:

docker compose build
docker compose push

To run the image as we did previously, we will need to add the volume bind mounts. Volumes in docker-compose are defined with the following syntax location-in-host:location-in-container. Compose can work without an absolute path:

version: '3.8'

services:
  youtube-dl-ubuntu:
    image: youtube-dl-ubuntu
    build: .
    volumes:
      - .:/app

Given youtube-dl/Dockerfile.alpine

FROM alpine:latest

ENV LC_ALL=C.UTF-8

RUN apk add curl \
  && curl -L https://github.com/ytdl-org/youtube-dl/releases/download/2021.12.17/youtube-dl -o /usr/local/bin/youtube-dl \
  && chmod +x /usr/local/bin/youtube-dl \
  && apk del curl

RUN apk add python3

WORKDIR /app

ENTRYPOINT ["/usr/local/bin/youtube-dl"]

Add youtube-dl-alpine

Answer:

youtube-dl/docker-compose.yml

version: '3.8'

services:
  youtube-dl-ubuntu:
    image: youtube-dl-ubuntu
    build:
      dockerfile: Dockerfile.ubuntu
    volumes:
      - .:/app

  youtube-dl-alpine:
    image: youtube-dl-alpine
    build:
      dockerfile: Dockerfile.alpine
    volumes:
      - .:/app

Compose is really meant for running web services, so let's move from simple binary wrappers to running a HTTP service.

https://github.com/jwilder/whoami is a simple service that prints the current container id (hostname).

docker container run -p 8000:8000 jwilder/whoami

Navigate with a browser or curl to localhost:8000, they both will answer with the id.

Take down the container so that it's not blocking port 8000.

Let's create a new folder and a docker-compose file whoami/docker-compose.yml from the command line options.

version: '3'

services:
  whoami:
    image: jwilder/whoami
    ports:
      - 8000:8000

Test it:

docker compose up
curl localhost:8000

Compose can also scale the service to run multiple instances:

$ docker compose up --scale whoami=3

  WARNING: The "whoami" service specifies a port on the host. If multiple containers for this service are created on a single host, the port will clash.

  Starting whoami_whoami_1 ... done
  Creating whoami_whoami_2 ... error
  Creating whoami_whoami_3 ... error

The command fails due to a port clash, as each instance will attempt to bind to the same host port (8000).

We can get around this by only specifying the container port. As mentioned in part 1, when leaving the host port unspecified, Docker will automatically choose a free port.

Update the ports definition in docker-compose.yml:

version: '3'

services:
  whoami:
    image: jwilder/whoami
    ports:
      - 8000

Then run the command again:

$ docker compose up --scale whoami=3
  Starting whoami_whoami_1 ... done
  Creating whoami_whoami_2 ... done
  Creating whoami_whoami_3 ... done

All three instances are now running and listening on random host ports. We can use docker compose port to find out which ports the instances are bound to.

$ docker compose port --index 1 whoami 8000
  0.0.0.0:32770

$ docker compose port --index 2 whoami 8000
  0.0.0.0:32769

$ docker compose port --index 3 whoami 8000
  0.0.0.0:32768

We can now curl from these ports:

$ curl 0.0.0.0:32769
  I'm 536e11304357

$ curl 0.0.0.0:32768
  I'm 1ae20cd990f7

In a server environment you'd often have a load balancer in-front of the service. For local environment (or a single server) one good solution is to use https://github.com/jwilder/nginx-proxy that configures nginx from docker daemon as containers are started and stopped.

Let's add the proxy to our compose file and remove the port bindings from the whoami service. We'll mount our docker.sock inside of the container in :ro read-only mode.

version: "3"

services:
  whoami:
    image: jwilder/whoami
  proxy:
    image: jwilder/nginx-proxy
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
    ports:
      - 80:80

When we start this and test

$ docker compose up -d --scale whoami=3
$ curl localhost:80
  <html>
  <head><title>503 Service Temporarily Unavailable</title></head>
  <body bgcolor="white">
  <center><h1>503 Service Temporarily Unavailable</h1></center>
  <hr><center>nginx/1.13.8</center>
  </body>
  </html>

It's "working", but the nginx just doesn't know which service we want. The nginx-proxy works with two environment variables: VIRTUAL_HOST and VIRTUAL_PORT. VIRTUAL_PORT is not needed if the service has EXPOSE in it's docker image. We can see that jwilder/whoami sets it: https://github.com/jwilder/whoami/blob/master/Dockerfile#L9

Note:

The domain colasloth.com is configured so that all subdomains point to 127.0.0.1. More information about how this works can be found at colasloth.github.io, but in brief it's a simple DNS "hack". Several other domains serving the same purpose exist, such as localtest.me, lvh.me, and vcap.me, to name a few. In any case, let's use colasloth.com here:

version: "3"

services:
  whoami:
    image: jwilder/whoami
    environment:
      - VIRTUAL_HOST=whoami.colasloth.com
  proxy:
    image: jwilder/nginx-proxy
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
    ports:
      - 80:80

Now the proxy works:

$ docker compose up -d --scale whoami=3
$ curl whoami.colasloth.com
  I'm f6f85f4848a8
$ curl whoami.colasloth.com
  I'm 740dc0de1954

Let's add couple of more containers behind the same proxy. We can use the official nginx image to serve a simple static web page. We don't have to even build the container images, we can just mount the content to the image. Let's prepare some content for two services called "hello" and "world".

echo "hello" > hello.html
echo "world" > world.html

Then add these services to the docker-compose.yml file where you mount just the content as index.html in the default nginx path:

hello:
  image: nginx:1.19-alpine
  volumes:
    - ./hello.html:/usr/share/nginx/html/index.html:ro
  environment:
    - VIRTUAL_HOST=hello.colasloth.com
world:
  image: nginx:1.19-alpine
  volumes:
    - ./world.html:/usr/share/nginx/html/index.html:ro
  environment:
    - VIRTUAL_HOST=world.colasloth.com

Now let's test:

$ docker compose up -d --scale whoami=3
$ curl hello.colasloth.com
  hello

$ curl world.colasloth.com
  world

$ curl whoami.colasloth.com
  I'm f6f85f4848a8

$ curl whoami.colasloth.com
  I'm 740dc0de1954

Now we have a basic single machine hosting setup up and running.

Test updating the hello.html without restarting the container, does it work?

You have reached the end of this section! Continue to the next section:

2. More Docker networking