Minio – The Secure Alternative To Amazon S3 For Object Storage

July 24, 2018

Von
Kategorie: blog

Earlier this year we  published an app to the Univention Appcenter providing the Minio object storage server. Even though Minio has been around for quite a while and offers benefits for people who want to keep their data under their own control, it is not as widely known is it could be.

So, What Is Minio Object Storage?

The short and simplified answer is “It’s like Amazon S3, but hosted locally.”

Minio is an object storage server that implements the same public API as Amazon S3. This means that applications that can be configured to talk to Amazon S3 can also be configured to talk to Minio. An object store, such as Minio, can then be used to store unstructured data such as photos, videos, log files, backups and container/VM images. The size of a single object can range from only a few KB to a maximum of 5TB. Files are organised in so -called “buckets”, which are logical separators for your stored data and need to be passed to your application along with your access key, secret key and the HTTP address of the Minio instance.

Great for Handling Big Amounts of Data

At Kopano we are used to juggling with large sets of data. So we know that when you’re running large data sets in distributed setups, you do not always want to have your attachment data (which can account for as much as 90% of your total mailbox size) stored on the same system you are running Kopano on. Also, ever-growing mailboxes tend to cause headaches, since disk storage needs to be extended and replicated to protect against disk failure.

Minio offers an easy way out of this dilemma. On the one hand by decoupling disk storage from your local machines, by making them accessible through a simple HTTP interface. And on the other hand by creating redundant data storage without the need to deal with cluster filesystems.

Easy setup

A single Minio server is really easy to setup, since its just a single Go binary, which will create all neccesary configuration on first execution. If you are not using our Minio app in the Univention Appcenter, then the Minio website offers plenty of other ways to get started with your own Minio setup.

For the rest of the blog we are going to assume that you have started the following way:

$ docker run -p 9000:9000 minio/minio server /data
Unable to find image 'minio/minio:latest' locally
latest: Pulling from minio/minio

911c6d0c7995: Pull complete
3d8b713e0b9a: Pull complete
721ac36e4eb4: Pull complete
Digest: sha256:7ea6405694d6f9265287e961adaaebcde10c667ebbf9a8522af2a2c693b460d0
Status: Downloaded newer image for minio/minio:latest

Created minio configuration file successfully at /root/.minio

Endpoint:  http://172.17.0.6:9000  http://127.0.0.1:9000
AccessKey: QQVJ37HFJSL3H9DTPBIF
SecretKey: qunsTA432ZFARMVyFdWvRGTJBx+xK99EBhaAtjFl

Browser Access:
   http://172.17.0.6:9000  http://127.0.0.1:9000

Command-line Access: https://docs.minio.io/docs/minio-client-quickstart-guide
   $ mc config host add myminio http://172.17.0.6:9000 QQVJ37HFJSL3H9DTPBIF qunsTA432ZFARMVyFdWvRGTJBx+xK99EBhaAtjFl

Object API (Amazon S3 compatible):
   Go:         https://docs.minio.io/docs/golang-client-quickstart-guide
   Java:       https://docs.minio.io/docs/java-client-quickstart-guide
   Python:     https://docs.minio.io/docs/python-client-quickstart-guide
   JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
   .NET:       https://docs.minio.io/docs/dotnet-client-quickstart-guide

As you can see above this command will pull the latest Minio release from Docker and then start the Minio process in the forground. In doing so it will print the AccessKey and SecretKey which will be used in client to authenticate.

How Can I Use Minio For My Kopano Data?

General support for storing attachments in Amazon S3 has been introduced in Kopano Groupware Core back in the 8.2 release. For new installation the following lines simply need to be added to the server.cfg:

attachment_storage=s3
attachment_s3_hostname=localhost:9000
attachment_s3_region=us-east-1
attachment_s3_protocol=http
attachment_s3_uristyle=path
attachment_s3_accesskeyid=QQVJ37HFJSL3H9DTPBIF
attachment_s3_secretaccesskey=qunsTA432ZFARMVyFdWvRGTJBx+xK99EBhaAtjFl
attachment_s3_bucketname=bucket
attachment_path=attachments

With these set Kopano will automatically use Minio for attachments. Converting an existing installation is possible as well, but slightly more complicated. Please contact the Kopano support for an offer on how to convert your current installation from disk-based attachments to S3 attachments.

Should I now migrate all my apps to use Minio/S3 for file storage?

Amazon S3 is the gold standard when it comes to backend file storage for cloud applications. Tools like Minio provide the same easy application access (simple HTTP requests) to your apps, without giving up control over your data. But does that mean you should now migrate all your applications to use S3? Probably not, but this also greatly depends on the overall architecture of your IT infrastructure. For example, when you are already using Amazon S3 for storage within your applications, then it’s quite easy to replace the external dependency on Amazon with a self-hosted Minio instance and move existing buckets to your Minio installation through the command line tools provided by Minio.

Still not sure? Just try Minio out yourself!

This is a slightly updated version of a blog post we published on the Univention blog earlier this year. Are you already running Kopano on Univention? Then you may be interested in our Minio app for Univention as well.