Julien Neuhart IT Architect

Or how to wrap useful libraries into a scalable API.

Updates:
Jun. 12th, 2018:
  • Updated Gotenberg version: from 1.0.0 to 2.0.0
  • Added PHP section

TL;DR

We're providing a simple Docker image which is able to convert files into PDF. It accepts:

  • Office documents
  • HTML files
  • Markdown files

You may also send many files which will all be merged into a single PDF.

Why?

At TheCodingMachine, we build a lot of web applications (intranets, extranets and so on) which require to generate PDF from various sources. Each time, we ended up using some well known libraries like wkhtmltopdf or unoconv and kind of lost time by reimplementing a solution from a project to another project. Meh.

How?

To answer this regular need, I've decided to develop an API. Its contract is quite simple:

  • Simplicity: one entry point, which accepts one or more files to convert and always returns a resulting PDF file
  • Stateless: it should not rely on any kind of database and should be able to restart without any knowledge from previous runs

I ended up writing it using Golang, a simple yet powerful language which provides everything I needed to make the API robust and efficient.

Also, since a year or two, the Docker revolution has enlightening us and we love using Docker Compose for setting up our environments. And what we love most is to simply drop an image and use it straight away.

That's why I've decided to distribute the API through a Docker image. The actual image wraps two things:

  • The libraries which do the conversions: markdown-pdf (Markdown to PDF), wkhtmltopdf (HTML to PDF), unoconv (Office documents to PDF) and pdftk (merging PDF)
  • The API

Usage

Let's say you're starting the API using this simple command:

$ docker run --rm -p 3000:3000 thecodingmachine/gotenberg:2.0.0

The API is now available on your host under http://127.0.0.1:3000.

It accepts POST requests with a multipart/form-data Content-Type. Your form data should provide one or more files to convert. It currently accepts the following:

  • Markdown files
  • HTML files
  • Office documents (.docx, .doc, .odt, .pptx, .ppt, .odp and so on)
  • PDF files
Heads up!

The API relies on the file extension to determine which library to use for conversion.

There are two use cases:

  • If you send one file, it will convert it and return the resulting PDF.
  • If many files, it will convert them to PDF, merge the resulting PDFs into a single PDF and return it.

Examples:

  • One file
$ curl --request POST \
    --url http://127.0.0.1:3000 \
    --header 'Content-Type: multipart/form-data' \
    --form files=@file.docx \
    > result.pdf
  • Many files
$ curl --request POST \
    --url http://127.0.0.1:3000 \
    --header 'Content-Type: multipart/form-data' \
    --form files=@file.md \
    --form files=@file.html \
    --form files=@file.pdf \
    --form files=@file.docx \
    > result.pdf

Security

The API does not provide any authentication mechanism. Make sure to not put it on a public facing port and your client(s) should always control what is sent to the API.

Scalability

Some libraries like unoconv cannot perform concurrent conversions. That's why the API does only one conversion at a time. If your API is under heavy load, a request will take time to be processed.

Fortunately, you may pass through this limitation by scaling the API.

In the following example, I'll demonstrate how to do some vertical scaling (= on the same machine) with Docker Compose, but of course horizontal scaling works too!

version: '3'

services:

  gotenberg:
    image: gotenberg:2.0.0

You may now launch your services using:

docker-compose up --scale gotenberg=your_number_of_instances

When requesting the Gotenberg service with your client(s), Docker will automatically redirect a request to a Gotenberg container according to the round-robin strategy.

Custom implementation

The API relies on a simple YAML configuration file called gotenberg.yml`. It allows you to tweak some values and even provides you a way to change the commands called for each kind of conversion.

Below the default configuration file:

# The port the application will listen to.
port: 3000

logs:
  # Accepted values, in order of severity: DEBUG, INFO, WARN, ERROR, FATAL, PANIC.
  # Messages at and above the selected level will be logged.
  level: "INFO"

  # Accepted values: text, json.
  # When a TTY is not attached, the output will be in the defined format.
  formatter: "text"

# You don't like a library which is used for a conversion? You want to handle a new file type?
# You may provide here your own implementation!
commands:

  # Some libraries like unoconv cannot perform concurrent conversions. That's why the API does only one conversion at a time.
  # If your current implementation uses libraries which are able to perform concurrent conversions, you may
  # change this value to false.
  lock: true

  # Unlike others commands' templates, you have access to FilesPaths instead of FilePath: it gathers all PDF files which should be merged.
  merge:
    template: "pdftk {{ range $filePath := .FilesPaths }} {{ $filePath }} {{ end }} cat output {{ .ResultFilePath }}"
    interpreter: "/bin/sh -c"
    timeout: 30

  # You may add more commands (or less, or even none).
  conversions:

      # The command template: you have access to FilePath and ResultFilePath variables.
    - template: "markdown-pdf {{ .FilePath }} -o {{ .ResultFilePath }}"
      # The binary which will call the command.
      interpreter: "/bin/sh -c"
      # Duration in seconds after which the command will be killed if it has not finished.
      timeout: 30
      # Files with the following extensions will be converted by the current command.
      extensions:
        - ".md"

    - template: "xvfb-run -e /dev/stdout wkhtmltopdf {{ .FilePath }} {{ .ResultFilePath }}"
      interpreter: "/bin/sh -c"
      timeout: 30
      extensions:
        - ".html"
        - ".htm"

    - template: "unoconv --format pdf --output \"{{ .ResultFilePath }}\" \"{{ .FilePath }}\""
      interpreter: "/bin/sh -c"
      timeout: 30
      extensions:
        - ".doc"
        - ".docx"
        - ".odt"
        - ".xls"
        - ".xlsx"
        - ".ods"
        - ".ppt"
        - ".pptx"
        - ".odp"

We provide binaries for a wide range of OS and architecture in the releases page, so feel free to create your own Docker image for your implementation of the Gotenberg API 🤘

PHP Client

We also provide a simple PHP client for interacting with your Gotenberg API.

The installation is straightforward:

$ composer require php-http/guzzle6-adapter # Unless your project already has a PSR7 HttpClient 
$ composer require thecodingmachine/gotenberg-php-client

Same for the usage :-)

<?php

namespace YourAwesomeNamespace;

use TheCodingMachine\Gotenberg\Client;
use TheCodingMachine\Gotenberg\ClientException;
use TheCodingMachine\Gotenberg\DocumentFactory;

use GuzzleHttp\Psr7\LazyOpenStream;

class YourAwesomeClass {

    public function yourAwesomeMethod()
    {
        $client = new Client('gotenberg:3000', new \Http\Adapter\Guzzle6\Client());
        # or the following if you want the client to discover automatically an installed implementation of the PSR7 `HttpClient`.
        $client = new Client('gotenberg:3000');

        # let's instantiate some documents you wish to convert.
        $yourOfficeDocument = DocumentFactory::makeFromPath('file.docx', '/path/to/file');
        $yourHTMLDocument = DocumentFactory::makeFromStream('file.html', new LazyOpenStream('path/to/file', 'r'));

        # now let's send those documents!
        try {
            # store method allows you to... store the resulting PDF in a particular folder.
            # this method also returns the resulting PDF path.
            $filePath = $client->store([
                $yourOfficeDocument,
                $yourHTMLDocument
            ], 'path/to/folder/you/want/the/pdf/to/be/store');

            # if you wish to redirect the response directly to the browser, you may also use:
            $response = $client->forward([
                $yourOfficeDocument,
                $yourHTMLDocument
            ]);

            # you may also retrieve the associated resource for storing the resulting PDF in database.
            $resource = $response->getBody()->detach();

        } catch (ClientException $e) {
            # this exception is thrown by the client if the API has returned a code != 200.
        } catch (\Http\Client\Exception $e) {
            # some PSR7 exception.
        } catch (\Exception $e) {
            # some (random?) exception.
        }
    }

}

Voilà!

Happy converting! :-)