Julien Neuhart IT Architect

Or how to wrap useful libraries into a scalable API.

TL;DR

We're providing a simple Docker image which is able to convert files into PDF. It accepts:

  • Office documents
  • HTML files
  • Markdown files

You may also send many files which will all be merged into a single PDF.

Why?

At TheCodingMachine, we build a lot of web applications (intranets, extranets and so on) which require to generate PDF from various sources. Each time, we ended up using some well known libraries like wkhtmltopdf or unoconv and kind of lost time by reimplementing a solution from a project to another project. Meh.

How?

To answer this regular need, I've decided to develop an API. Its contract is quite simple:

  • Simplicity: one entry point, which accepts one or more files to convert and always returns a resulting PDF file
  • Stateless: it should not rely on any kind of database and should be able to restart without any knowledge from previous runs

I ended up writing it using Golang, a simple yet powerful language which provides everything I needed to make the API robust and efficient.

Also, since a year or two, the Docker revolution has enlightening us and we love using Docker Compose for setting up our environments. And what we love most is to simply drop an image and use it straight away.

That's why I've decided to distribute the API through a Docker image. The actual image wraps two things:

  • The libraries which do the conversions: markdown-pdf (Markdown to PDF), wkhtmltopdf (HTML to PDF), unoconv (Office documents to PDF) and pdftk (merging PDF)
  • The API

Usage

Let's say you're starting the API using this simple command:

$ docker run --rm -p 3000:3000 thecodingmachine/gotenberg:1.0.0

The API is now available on your host under http://127.0.0.1:3000.

It accepts POST requests with a multipart/form-data Content-Type. Your form data should provide one or more files to convert. It currently accepts the following:

  • Markdown files
  • HTML files
  • Office documents (.docx, .doc, .odt, .pptx, .ppt, .odp and so on)
  • PDF files (if more than one file to convert)
Heads up!

The API relies on the file extension to determine which library to use for conversion.

There are two use cases:

  • If you send one file, it will convert it and return the resulting PDF.
  • If many files, it will convert them to PDF, merge the resulting PDFs into a single PDF and return it.

Examples:

  • One file
$ curl --request POST \
    --url http://127.0.0.1:3000 \
    --header 'Content-Type: multipart/form-data' \
    --header 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
    --form files=@file.docx \
    > result.pdf
  • Many files
$ curl --request POST \
    --url http://127.0.0.1:3000 \
    --header 'Content-Type: multipart/form-data' \
    --header 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
    --form files=@file.md \
    --form files=@file.html \
    --form files=@file.pdf \
    --form files=@file.docx \
    > result.pdf

Security

The API does not provide any authentication mechanism. Make sure to not put it on a public facing port and your client(s) should always control what is sent to the API.

Scalability

Some libraries like unoconv cannot perform concurrent conversions. That's why the API does only one conversion at a time. If your API is under heavy load, a request will take time to be processed.

Fortunately, you may pass through this limitation by scaling the API.

In the following example, I'll demonstrate how to do some vertical scaling (= on the same machine) with Docker Compose, but of course horizontal scaling works too!

version: '3'

services:

  gotenberg:
    image: gotenberg:1.0.0

You may now launch your services using:

docker-compose up --scale gotenberg=your_number_of_instances

When requesting the Gotenberg service with your client(s), Docker will automatically redirect a request to a Gotenberg container according to the round-robin strategy.

Custom implementation

The API relies on a simple YAML configuration file called gotenberg.yml. It allows you to tweak some values and even provides you a way to change the commands called for each kind of conversion.

Here the default configuration:

# The port the application will listen to.
port: 3000

logs:
  # Accepted values, in order of severity: DEBUG, INFO, WARN, ERROR, FATAL, PANIC.
  # Messages at and above the selected level will be logged.
  level: "INFO"

  # Accepted values: text, json.
  # When a TTY is not attached, the output will be in the defined format.
  format: "text"

# You don't like a library which is used for a conversion? You may provide here your own implementation.
commands:

  markdown:
    # Duration in seconds after which the command will be killed if it has not finished.
    timeout: 30
    # The command template: you have access to FilePath and ResultFilePath variables.
    template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}"

  html:
    timeout: 30
    template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}"

  office:
    timeout: 30
    template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\""

  merge:
    timeout: 30
     # Unlike others commands' templates, you have access to FilesPaths instead of FilePath: it gathers all PDF files which should be merged.
    template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}"

VoilĂ !

Happy converting! :-)