How dokku works

2013-11-23 | docker

based on dokku v0.2.0-RC1-25-g698e5e5

What dokku is

Dokku markets itself as "Docker powered mini-Heroku. The smallest PaaS implementation you've ever seen" which pretty much describes what it is about.

The project uses Docker, Buildstep, ssh-command, pluginhook, ssh, git, nginx and some bash-glue to provide you with a simplified single-host version of Heroku.

Dokku is currently under quite active development and could probably change quite a bit after this is written.

The project's original author also wrote a small introduction on dokku on his website which also includes a nice screencast.

What you get

As it's tagline suggests, you don't get much. At least code-wise. The main script is about 100 lines long. The plugins that ship with dokku contain about 470 lines of shell-code.

There is a git plugin for handling the interaction with the git repositories, an nginx-vhosts plugin for making the apps available through nginx, a config plugin which manages the settings for a given app as well as a backup plugin which can backup and restore the dokku and application settings.

Dokku and the dokku-standard plugin let you view the url and logs of an application, run a command within the app's container, deploy and of course delete it again.

Besides that, the installation procedure includes docker for managing linux containers (which will host your applications) and a buildstep docker image which will be used as base image for all app deployments.

The other dependencies are used to tie this together in a useful and extendable way.

Also dokku can be easily tried out using the provided Vagrantfile.

What it does

SSH command

After installing dokku and configuring it as described within the project's README it is ready to use.

During configuration you have to add your public key to dokku (I am using vagrant in my example):

$ cat ~/.ssh/id_rsa.pub | vagrant ssh -- sudo sshcommand acl-add dokku rico

This adds a custom rule to the authorized_keys file for the dokku user on your dokku server:

command="FINGERPRINT=... NAME=rico `cat /home/dokku/.sshcommand` $SSH_ORIGINAL_COMMAND",no-agent-forwarding,no-user-rc,no-X11-forwarding,no-port-forwarding ssh-rsa ... ...@...

This line causes all commands received from a connection using the previously added public-key, to be redirected to the dokku command.

To split this up:

the command="..." part of the line specifies the command to be executed whenever the given given key (found at the end of the line) is used for authentication
FINGERPRINT=... and NAME=... specify some additional environment variables for the command to run while cat /home/dokku/.sshcommand reads the actual command to be executed from a separate file
the file /home/dokku/.sshcommand contains /usr/local/bin/dokku; this file is created during the installation by running sshcommand create dokku /usr/local/bin/dokku
several ssh feature exclusions are put into place; features which won't be of much use here anyway
finally the line ends with the public key starting with the key algorithm followed by the key itself and some (email-like) identifier

The FINGERPRINT and NAME variables are currently not used by dokku itself but probably could be used by plugin-writers for additional logging or integration purposes.

Normally using a "default" ssh setup you could run arbitrary commands on the remote host, passing them as argument to the ssh-client program. The uptime command for example could tell you how long the host has been up:

$ ssh someotherhost uptime
 07:51:53 up 18 days, 20:00,  0 users,  load average: 0.06, 0.12, 0.17

But when we try this on the dokku account, we don't get that information:

$ ssh -p2222 dokku@localhost uptime

It does not return anything.

The additional line inside the authorized_keys file causes the dokku command to be invoked instead of uptime.

Dokku's default behavior for actions not defined directly within the main script, is to call all available plugins and let them handle it. No default plugin handles uptime so not much is done.

But using a known command provides a neat way to remotely manage your dokku installation and deployed apps:

$ ssh -p2222 dokku@localhost help
    backup:export [file]                            Export dokku configuration files
    backup:import [file]                            Import dokku configuration files
    config <app>                                    display the config vars for an app
    config:get <app> KEY                            display a config value for an app
    config:set <app> KEY1=VALUE1 [KEY2=VALUE2 ...]  set one or more config vars
    config:unset <app> KEY1 [KEY2 ...]              unset one or more config vars
    delete <app>                                    Delete an application
    help            Print the list of commands
    logs <app> [-t]                                 Show the last logs for an application (-t follows)
    plugins-install Install active plugins
    plugins         Print active plugins
    run <app> <cmd>                                 Run a command in the environment of an application
    url <app>                                       Show the URL for an application
    version                                         Print dokku's version

Git push

What happens when you invoke git push dokku master is defined within the git plugin. It contains the following commands file:

$ cat /var/lib/dokku/plugins/git/commands
#!/usr/bin/env bash
set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x

case "$1" in
  git-hook)
    APP=$2

    while read oldrev newrev refname
    do
      # Only run this script for the master branch. You can remove this
      # if block if you wish to run it for others as well.
      if [[ $refname = "refs/heads/master" ]] ; then
        git archive $newrev | dokku receive $APP | sed -u "s/^/"$'\e[1G'"/"
      fi

    done
    ;;

  git-*)
    APP="$(echo $2 | perl -pe 's/(?<!\\)'\''//g' | sed 's/\\'\''/'\''/g')"
    APP_PATH=$DOKKU_ROOT/$APP

    if [[ $1 == "git-receive-pack" && ! -d $APP_PATH ]]; then
        git init --bare $APP_PATH > /dev/null
        PRERECEIVE_HOOK="$APP_PATH/hooks/pre-receive"
        cat > $PRERECEIVE_HOOK <<EOF
#!/usr/bin/env bash
set -e; set -o pipefail;
cat | DOKKU_ROOT="$DOKKU_ROOT" dokku git-hook $APP
EOF
        chmod +x $PRERECEIVE_HOOK
    fi

    args=$@
    git-shell -c "$args"
    ;;

esac
cat

When the git client program is used to push repository information to a remote repository using the ssh transport, it actually tries to run the git-receive-pack command on the remote machine (you can read more on this here or here).

Due to the ssh setup described earlier, this command will be fed into dokku. Internally dokku git-receive-pack 'repository-path' will be called, which in turn causes all plugin command files to be executed. The git plugin's commands file listed above is executed too and the git-*) case will match. The plugin then determines the application repository path, creates an empty repository if none exists and places a custom pre-receive hook file in there.

After that, it passes control to the git-shell command to let it handle the rest of the push.

The git-shell will then invoke the actual receive-pack command which calls the added pre-receive hook script which in turn will invoke dokku again like dokku git-hook $APP and feed it information about the push.

The call to dokku with git-hook will lead to yet another invocation of the git plugin commands file. This time the git-hook) case will match, which will trigger git archive to exports the pushed application as tar archive to stdout and feed this to dokku receive.

Deployment

At this point, the actual build and deployment happens within the main script:

case "$1" in
  receive)
    APP="$2"; IMAGE="app/$APP"
    echo "-----> Building $APP ..."
    cat | dokku build $APP $IMAGE
    echo "-----> Releasing $APP ..."
    dokku release $APP $IMAGE
    echo "-----> Deploying $APP ..."
    dokku deploy $APP $IMAGE
    echo "-----> Cleaning up ..."
    dokku cleanup
    echo "=====> Application deployed:"
    echo "       $(dokku url $APP)"
    echo
    ;;

First, the app is fed to the buildstep docker container by piping the received git archive output to dokku build. The buildstep container is responsible of handling heroku-style buildpacks. It will try to detect the type of application (think ruby, python, java, php, node ...) and run the proper buildpack.

After the build is finished, dokku release will take the application settings defined using the config plugin and add them to the new container.

dokku deploy first starts the application and invokes the post-deploy pluginhook. The nginx plugin contains a script for this hook which creates an appropriate nginx vhost for the application and reloads nginx.

Finally, after some cleanup, dokku will then happily report the url of the deployed application back to you!

Wrapping it up

Dokku provides a relatively simple way to deploy applications on your own servers in a heroku-like way.

Using the buildstep image as base for all app deployments shows one of docker's strengths. Due to docker's layers (union file system), disk usage is minimized. Only the code of the application code and the result of the buildpack run need to be saved on disk, the rest is shared.

Dokku can get your applications up and running. But you have to take care of the rest yourself. Your app will probably need a database, other services or monitoring. Fortunately, dokku supports plugins which can let you add this extra functionality to your dokku installation.

The scope of dokku is limited, but it is not intended to be much more than it currently is. If you want more, the documentation suggests that one could also take a look at flynn.io.