Containerization remains a hot topic, be it directly on Docker (for example aided by Docker Compose) or on big orchestration platforms such as Kubernetes. Containers, after all, come with the promise of making applications easier to deploy and maintain. But building containers that empower administrators to do so is a challenge of its own. The following blog will highlight some of the design decisions that have been made in the “kopano-docker” project to deliver a set of Docker containers that are both easy to use and easy to “make it your own”.
Simply interested in using Docker to easily spin up a Kopano environment? Head directly to our follow-up blog about this.
This year Docker celebrated its sixth birthday. During the past six years, several Kopano Docker containers were developed which you will find when you search the public Docker Hub or when you check for projects on Github. I am also quite sure more Kopano Docker containers exist at our partners and customers behind closed doors.
While there are plenty of people around crafting Docker containers and Docker files, a lot of these projects suffer either from lack of maintenance (combined with the missing ability for externals to easily take over) or inflexibility when it comes to configuration. And when I am talking about “inflexibility” I mean either falling back to having the user manually edit configuration files within the container and/or working just in narrow use cases with the user needing to manually create files and folders until the container in question finally starts up.
If you tried one of the above search queries, one of the projects you would have found is “kopano-docker”. We ourselves found this project in the beginning of 2018. It was already doing a lot of things in the right way (configurable exclusively through environment variables for example), but actually using it still meant a bit of trial and error (for example because of a missing docker-compose.yml directly in the repository). Below I want to showcase some of the practices that have been implemented in this project between 2018 and now to ease the use of it.
Don’t reinvent what you can reuse
This almost goes without saying, but there is already a huge library of ready to run “infrastructure” images on the Docker Hub. This starts with images for MySQL or MariaDB provided directly by the Docker Team. Even more complex topics such as a general purpose mail stack or an easy to extend OpenLDAP installation are already provided for.
And in case the container you want to run still needs some extra files (like for example the Kopano LDAP schema) one can simply write an own Dockerfile that uses an existing image as its base. An example of such a Dockerfile can be found in the LDAP image that is part of kopano-docker.
FROM osixia/openldap:1.2.4 COPY bootstrap /container/service/slapd/assets/config/bootstrap RUN rm /container/service/slapd/assets/config/bootstrap/schema/mmc/mail.schema RUN touch /etc/ldap/slapd.conf
Make compose file dynamic by using variables
Not only can you use the compose file to easily configure environment variables for your running containers, but you can just as easily adapt a single docker-compose.yml to multiple environments when working with variables within your compose file as well:
[..]
web:
image: ${docker_repo:-zokradonh}/kopano_web:${KWEB_VERSION:-latest}
restart: unless-stopped
ports:
- "${CADDY:-2015}:2015"
- "${HTTP:-80}:80"
- "${HTTPS:-443}:443"
environment:
- EMAIL=${EMAIL:-off}
- FQDN=${FQDN?err}
command: wrapper.sh
cap_drop:
- ALL
cap_add:
- NET_BIND_SERVICE
- CHOWN
- SETGID
- SETUID
volumes:
- web:/.kweb
networks:
web-net:
aliases:
- ${FQDNCLEANED?err}
[..]
As can be seen above the compose file format even allows for tricks such as variable substitution to define default values or force docker-compose to stop with an error if a variable has not been defined.
Experience has shown that the compose file should be part of the version controlled project to ease updating for existing users. This practice is made easier through the use of variables as well, since all installation specific values (including secrets like the database password) can be safely stored inside of a .env file (which in turn should not be version controlled).
Bundle a small script to ease first-time use
Because of the above dynamic we now have already two files that are very important to the administrator:
docker-compose.yml(taken care off through the git repository).env(which needs to be managed manually by the user, but is of vital importance to the compose file)
Both to prevent unsafe defaults for passwords, as well as spare users from the task to manually go through the compose file and create a .env with everything important, I would recommend bundling a small setup.sh script with your project.
Such a script can then, for example, generate passwords for use with the MySQL database or guide users when it comes to selecting the right timezone. As an added bonus the setup.sh script for kopano-docker even allows the user to select which plugins for Kopano WebApp will be available later.
$ ./setup.sh
Creating individual env files for containers (if they do not exist already)
Creating an .env file for you
Which tag do you want to use for Kopano Core components? [latest]:
Which tag do you want to use for Kopano WebApp? [latest]:
Which tag do you want to use for Z-Push? [latest]:
Which tag do you want to use for Kopano Konnect? [latest]:
Which tag do you want to use for Kopano Kwmserver? [latest]:
Which tag do you want to use for Kopano Meet? [latest]:
Which tag do you want to use for Kopano kDAV? [latest]:
Name of the Organisation for LDAP [Kopano Demo]:
FQDN to be used (for reverse proxy).
Tipp: use port 2015 in case port 443 is already in use on the system.
[kopano.demo]:
Email address to use for Lets Encrypt.
Use 'self_signed' as your email to create self-signed certificates.
Use 'off' if you want to run the service without tls encryption. Make sure to use an ssl-terminating reverse proxy in front in this case.
[self_signed]:
Name of the BASE DN for LDAP [dc=kopano,dc=demo]:
LDAP server to be used (defaults to the bundled OpenLDAP) [ldap://ldap:389]:
Use bundled LDAP with demo users? yes/no [yes]:
Timezone to be used [Europe/Berlin]:
E-Mail Address displayed for the 'postmaster' []:
Name/Address of Database server (defaults to the bundled one) [db]:
Available options:
1 ) de-at
2 ) de-ch
3 ) de-de
4 ) en
5 ) en-gb
6 ) es
7 ) fr
8 ) it
9 ) nl
10 ) pl-pl
Check language spell support (again to uncheck, ENTER when done):
Available options:
1 ) contactfax
2 ) desktopnotifications
3 ) filepreviewer
4 ) files
5 ) filesbackend-smb
6 ) filesbackend-owncloud
7 ) folderwidgets
8 ) gmaps
9 ) intranet
10 ) mattermost
11 ) mdm
12 ) pimfolder
13 ) quickitems
14 ) smime
15 ) titlecounter
16 ) webappmanual
17 ) zdeveloper
Check for additional plugins (again to uncheck, ENTER when done):
Integrate WhatsApp into DeskApp yes/no [no]:
Use override files for really environment specific details
Did you know that docker-compose can handle more than a single compose file at the same time? In fact when invoking docker-compose the command processes instructions both from docker-compose.yml and docker-compose.override.yml (and you can process even more files by passing the -f option to it). The override file can be used to easily extend your configuration from the actual compose file.
For example to pull in Watchtower to automatically update your containers when there are newer versions:
version: "3.5"
services:
watchtower:
image: v2tec/watchtower
volumes:
- /var/run/docker.sock:/var/run/docker.sock
Or to store attachments in Kopano in Minio instead of directly on disk:
version: "3.5"
# example file to store attachments in s3 (provided by minio)
# rename to docker-compose.override.yml and place it along the existing file to use it
# (and change accesskey an secretkey below)
services:
kopano_server:
depends_on:
- minio
environment:
- KCCONF_SERVER_ATTACHMENT_STORAGE=s3
- KCCONF_SERVER_LOG_LEVEL=6
- KCCONF_SERVER_ATTACHMENT_S3_HOSTNAME=minio:9000
- KCCONF_SERVER_ATTACHMENT_S3_PROTOCOL=http
- KCCONF_SERVER_ATTACHMENT_S3_URISTYLE=path
- KCCONF_SERVER_ATTACHMENT_S3_REGION=us-east-1
- KCCONF_SERVER_ATTACHMENT_S3_ACCESSKEYID=ACCESSKEY
- KCCONF_SERVER_ATTACHMENT_S3_SECRETACCESSKEY=SECRETKEY
- KCCONF_SERVER_ATTACHMENT_S3_BUCKETNAME=kopano
- KCCONF_SERVER_ATTACHMENT_PATH=attachments
minio:
image: minio/minio
ports:
- '9000:9000'
volumes:
- miniodata:/data
- minioconfig:/root/.minio
environment:
- "MINIO_ACCESS_KEY=ACCESSKEY"
- "MINIO_SECRET_KEY=SECRETKEY"
command: server /data
entrypoint: sh
command: -c 'mkdir -p /export/kopano && /usr/bin/minio server /export'
networks:
- kopano-net
volumes:
miniodata:
minioconfig:
Some more examples can be found in the examples directory of kopano-docker.
Take care of dependencies between containers
The good news is that the compose file format allows for general dependencies between containers like shown in the below example:
[..]
kopano_server:
image: ${docker_repo:-zokradonh}/kopano_core:${CORE_VERSION:-latest}
hostname: kopano_server
container_name: ${COMPOSE_PROJECT_NAME}_server
depends_on:
- db
- ldap
- kopano_ssl
- kopano_konnect
ports:
[..]
At the same time, the sad news is that this only allows for “when container A starts, container B needs to be started first” chains, without taking into account that the service within container B might take a few seconds to be fully operational. Essentially something like a healthcheck is missing for this relation. Luckily there are plenty of ways to resolve these situations from within the running container.
While looking for generally accepted solutions to this problem I finally ended up using dockerize for whenever a container needs to pause his startup until a certain condition is fulfilled. This way I can pause the startup of kopano-server until the MySQL database is responding and the ssl certificates for it have been created:
[..] dockerize \ -wait file://"$KCCONF_SERVER_SERVER_SSL_CA_FILE" \ -wait file://"$KCCONF_SERVER_SERVER_SSL_KEY_FILE" \ -wait "$DB_CONN" \ -timeout 360s [..]
Or I can pause the startup of kwmserver until the OpenID discovery document has been generated:
[..]
exec dockerize \
-wait http://kopano_konnect:8777/.well-known/openid-configuration \
-timeout 360s \
/usr/local/bin/docker-entrypoint.sh serve \
--registration-conf /kopano/ssl/konnectd-identifier-registration.yaml \
"$@"
[..]
And as you can see above this even works with a self defined timeout, meaning that in both cases the container will wait for a maximum of 360 seconds before giving up and stopping.
But waiting for dependencies is just a small part of the functionality of dockerize, to learn how you can use it to template configuration files have a look at its own README.
Handling configuration within Containers
I personally think that a good container should not create the need for the user to configure something inside of the container manually. Something like a “first start wizard” may be acceptable for services that bring their own (web) ui, but ideally one should be able to configure the application inside of the container directly when launching it.
At Kopano we have three different styles of how components are configured:
- “old style” services in C++ or Python, like kopano-server or kopano-search
- have a configuration file named after the service in question (e.g. server.cfg for kopano-server)
- php based components like WebApp its plugins or Z-Push
- have configuration options stored inside of php files
- “new style” services in Golang (e.g. Konnect and Kwmserver)
- get configuration from environment variables (for packaged installation the configuration file is loaded into the environment through systemd)
The last case is quite easy to implement in a container-centric world, but the first two require additional tooling.
For the “old style” services the creator of the project already implemented a python module to retrieve configuration from an environment variable and write it back into the desired configuration file as can be seen in:
import os
import kcconf
# Component specific configurations
kcconf.configkopano({
r"/etc/kopano/server.cfg":
{
'log_file': "-",
'log_level': "3",
'attachment_path': "/kopano/data/attachments/",
'user_plugin': "ldap",
'server_listen': "*:236",
'server_listen_tls': "*:237",
'sync_gab_realtime': "no",
'kcoidc_initialize_timeout': "360"
}
})
# Override configs from environment variables
kcconf.configkopano(kcconf.parseenvironmentvariables(r"/etc/kopano/"))
Since Python is not really one of my own strengths I decided to implement the logic for updating the php based configuration with good ol’ bash & sed:
php_cfg_gen() {
local cfg_file="$1"
local cfg_setting="$2"
local cfg_value="$3"
if [ -e "$cfg_file" ]; then
if grep -q "$cfg_setting" "$cfg_file"; then
echo "Setting $cfg_setting = $cfg_value in $cfg_file"
case $cfg_value in
true|TRUE|false|FALSE)
echo boolean value
sed -ri "s#(\s*define).+${cfg_setting}.+#\tdefine(\x27${cfg_setting}\x27, ${cfg_value}\);#g" "$cfg_file"
;;
*)
sed -ri "s#(\s*define).+${cfg_setting}.+#\tdefine(\x27${cfg_setting}\x27, \x27${cfg_value}\x27\);#g" "$cfg_file"
;;
esac
else
echo "Error: Config option $cfg_setting not found in $cfg_file"
cat "$cfg_file"
exit 1
fi
else
echo "Error: Config file $cfg_file not found. Plugin not installed?"
local dir
dir=$(dirname "$cfg_file")
ls -la "$dir"
exit 1
fi
}
Add a Makefile to ease rebuilding
With the above pieces in place, users can already use your containers in an easy way, but what if a user wants to locally rebuild the container (for example since he requires signed images or because he wants to use a newer version than the one available from the Docker Hub)? Building and tagging a single container is fine to do, but what if you want something more modular? Where you can just run subsets of your whole script? This is where a Makefile proves its worth.
In kopano-docker the bundled Makefile has the ability to build, tag and publish either single images or all of them. And just like in the compose file the user has the ability to use environment variables to define the name of his how container registry. On top of that the Makefile can be used to clean up the local development state (remove data volumes), automate starting and stopping containers as well as linting scripts and Dockerfiles and checking if all containers still start up after local changes.
Use build-time variables to allow for customization
But a Makefile is only half of the puzzle. To allow the user to influence the location where the final image will be pushed to and which version of our software is included in the container build-time variables can be utilised:
[..]
build:
[..]
docker build \
--pull \
--build-arg docker_repo=${docker_repo} \
--build-arg KOPANO_CORE_VERSION=${core_download_version} \
--build-arg KOPANO_$(COMPONENT)_VERSION=${$(component)_download_version} \
--build-arg KOPANO_CORE_REPOSITORY_URL=$(KOPANO_CORE_REPOSITORY_URL) \
--build-arg KOPANO_MEET_REPOSITORY_URL=$(KOPANO_MEET_REPOSITORY_URL) \
--build-arg KOPANO_WEBAPP_REPOSITORY_URL=$(KOPANO_WEBAPP_REPOSITORY_URL) \
--build-arg KOPANO_WEBAPP_FILES_REPOSITORY_URL=$(KOPANO_WEBAPP_FILES_REPOSITORY_URL) \
--build-arg KOPANO_WEBAPP_MDM_REPOSITORY_URL=$(KOPANO_WEBAPP_MDM_REPOSITORY_URL) \
--build-arg KOPANO_WEBAPP_SMIME_REPOSITORY_URL=$(KOPANO_WEBAPP_SMIME_REPOSITORY_URL) \
--build-arg KOPANO_ZPUSH_REPOSITORY_URL=$(KOPANO_ZPUSH_REPOSITORY_URL) \
--build-arg RELEASE_KEY_DOWNLOAD=$(RELEASE_KEY_DOWNLOAD) \
--build-arg DOWNLOAD_COMMUNITY_PACKAGES=$(DOWNLOAD_COMMUNITY_PACKAGES) \
--build-arg ADDITIONAL_KOPANO_PACKAGES="$(ADDITIONAL_KOPANO_PACKAGES)" \
--build-arg ADDITIONAL_KOPANO_WEBAPP_PLUGINS="$(ADDITIONAL_KOPANO_WEBAPP_PLUGINS)" \
--cache-from $(docker_repo)/kopano_$(component) \
--cache-from $(docker_repo)/kopano_$(component):builder \
-t $(docker_repo)/kopano_$(component) $(component)/
[..]
These can then be picked up within the Dockerfile and optionally be preserved as environment variables within the resulting container:
ARG docker_repo=zokradonh
FROM ${docker_repo}/kopano_base
ARG DEBIAN_FRONTEND=noninteractive
ARG ADDITIONAL_KOPANO_PACKAGES=""
ENV ADDITIONAL_KOPANO_PACKAGES=$ADDITIONAL_KOPANO_PACKAGES
ARG DOWNLOAD_COMMUNITY_PACKAGES=1
ENV DOWNLOAD_COMMUNITY_PACKAGES=$DOWNLOAD_COMMUNITY_PACKAGES
ARG KOPANO_CORE_REPOSITORY_URL="file:/kopano/repo/core"
ENV KOPANO_CORE_REPOSITORY_URL=$KOPANO_CORE_REPOSITORY_URL
ARG KOPANO_CORE_VERSION=newest
ENV KOPANO_CORE_VERSION=$KOPANO_CORE_VERSION
ARG KOPANO_REPOSITORY_FLAGS="trusted=yes"
ENV KOPANO_REPOSITORY_FLAGS=$KOPANO_REPOSITORY_FLAGS
ARG RELEASE_KEY_DOWNLOAD=0
ENV RELEASE_KEY_DOWNLOAD=$RELEASE_KEY_DOWNLOAD
[..]
And don’t worry about build-time variables that are passed to docker build and then not used afterward, as docker will only print a warning about them.
Closing words
I hope you have enjoyed this hopefully a bit different look at practices for containerization. If you want to see the above-explained patterns in practice make sure to take a look at kopano-docker. I’m looking forward to your feedback (for example in form of issues and pull requests).
