[PSR-11+] An overview of interoperable PHP modules

A lot has been going on in the past months regarding the creation and standardization of interoperable PHP packages. We (the team behind container-interop) have been testing a lot of different solutions, and many of them have the potential to become a good standard. In this article, I'm taking a step back at what has been accomplished so far and presenting our findings.

TL/DR?

Jump directly to the conclusion section at the bottom of this document!

Why is it deeply needed?

Let’s assume you are a PHP package author. If you want your classes to be easily usable in framework A or framework B, you must write a « meta-package » for each framework. This meta package usually contains « services definitions »: it explains to the framework how you build your objects. So the job of a PHP package developer is to write his/her package, and then to write X meta-packages (one for each framework it wants to target). This is a problem, as X is quite large in the PHP world. Have a look at the number of meta packages needed for the Glide image manipulation library:

We are looking for a solution to this problem. Hence, we are looking for a standard that would allow a package to put/modify entries in the application's container.

Important note: Entries can be put in the container by the application's developer and by packages. Ultimately, the responsibility of configuring the container belongs to the application's developer. If an application's developer decides he does not want to use entries provided by a package, he should always be able to define its own entries. Entries provided by packages are only here to help a developer getting started with sensible defaults. We are not looking to standardize how we configure all containers (this is what makes containers special). We are only looking for a common way for packages to declare entries in a container.

What features do we need?

A quick glance at existing bundles / modules / meta package systems out there tells us that a package should be able to:

• Declare new container entries

  • Using the new keyword
    Using a Pimple like syntax, something that would be similar to:

    function() {
        return new MyService();
    }

  • Using a factory
    Something similar to:

    function() {
        return Factory::getMyService();
    }

  • Pass parameters to the constructor, the factory, or call setters or set public properties.
    That would translate in this code:

    function(ContainerInterface $container) {
        $service = new MyService('foo', $container->get('baz'));
        $service->setBar(42);
        $service->hello = 'world';
        return $service;
    }

• Extend a container entry provided by another package.
  For instance, if a package provides a Twig_Environment service, another package should be able to extend that service and call the addExtension method of the service to register an extension.

Note: there are countless interesting features we could add to those features listed here, but let's only focus on these "must have" features for now.

What solutions do we have so far?

Good news! We have a number of existing solutions that solve this problem. We have investigated some of them that are presented in this blog post.

1. Having each module provide its own PSR-11 compatible container
2. or defining standard container entry file format
3. or defining standard PHP objects/interfaces describing container definitions
4. or defining standard PHP objects/interfaces representing “dumpable” container definitions
5. or defining standard service providers

Solution 1: PSR-11 and composite containers

Idea: each module can provide its own container, all containers can be chained (using PSR-11) to build a main "composite" container.

This approach has been considered but it has one main shortcoming: it does not allow to extend a container entry. So it does not really cover enough to be considered a viable solution. We need something stronger than this.

Solution 2: A standard configuration format

This solution is guaranteed to work. This is what is done in Java with CDI or formerly with the Spring framework. This is also to some extend what Symfony does with its services.yml file.

We haven't experimented this solution directly because of the amount of work required to propose a correct draft.

Such a configuration file could be in XML or JSON. A sample file could look like this:

{
  "schema":   "https:/some-url-here.org/schema#",
  "services": {
    "logger":     {
      "factory":   "AppProvidersLoggerProvider::createLogger",
      "arguments": [],
      "type":      "PsrLogLoggerInterface",
      "meta":      "JSON has no comments, so this could be a useful"
    },
    "middleware": {
      "factory":   "AppProvidersMiddleware::createMiddleware",
      "arguments": [
        "@logger",
        "string"
      ],
      "type":      "callable"
    }
  }
}

Note: this sample credited to Giuseppe Mazzapica here. A similar proposal was made by Larry Garfield when discussing PSR-11 entrance vote.

Here is the list of strong and weak points of this solution:

Strengths:

• It can cover all the features we need for a module system (creation of new entries and extension of existing one)
• It is the only solution that allows static analysis. External tools could be used to analyze / scan / edit such configuration files. Think about: Packagist allowing you to search a particular instance, auto-completion in your IDE, etc...
• Implementation is relatively easy for compiled containers.

Weaknesses:

• For each way to create a service, we need a particular syntax:
  • A syntax to create a service with the new keyword
  • A syntax to create a service using a static factory (Factory::getMyService())
  • A syntax to create a service using a factory instance that is itself a service ($container->get('myFactory')::getMyService())
  • A syntax to create arrays of services, string parameters, etc...
  • A syntax to call setters / methods
  • ...
• This is necessarily complex and a cognitive burden for the developers (yet another file format they need to master)
• Furthermore, even with a very feature-rich format, we will always reach a point where we cannot do something. For instance, what if I want to concatenate 2 configuration strings and pass those as a parameter to a constructor? (Symfony solves this with the Expression Language, but this becomes clearly out of reach of any standard).
• The more features we add, the harder the implementation for containers. We have to find a correct balance between available features and easiness of implementation. This might be a difficult balance to strike.
• Performance-wise, this is dreadful for "runtime containers". A container cannot parse all configuration files on every request. The only possible solutions are doing heavy caching or preprocessing the configuration files and generating a PHP class (i.e. "compiling" the container). De-facto, this means adding the notion of a "build stage" to all applications. This is a big step for small/light frameworks. Actually, I know it is a show-stopper for some framework developers out there.
• Also, with static configuration files, we need to agree on a common file format! This is a huge challenge. There are many formats out there: JSON / XML / YML / Neon, etc... Getting the PHP-FIG members to agree on one format might be tough. At container-interop, we had some talks about what the best format would be. We currently lean towards XML because unlike JSON, it has comments and strong support for schema validation / autocompletion (in IDE). I know other people would really prefer JSON that is less verbose... so this will be a tough debate.

Because there is going to be strong issues regarding the format of the configuration file, we thought of another solution. Rather than defining a file format, we could standardize a set of interfaces that represent the definition of container entries.

Solution 3: standard definition interfaces

With this solution, we are trying to standardize interfaces that describe container entries. We have done extensive research and tests on this solution with the container-interop/definition-interop project. I've also blogged a bit about it.

The whole idea is to have one interface per technique to construct container entries. You want to declare an entry with the new keyword? Use the ObjectDefinitionInterface. You want to declare an entry created by a factory? Use the FactoryCallDefinitionInterface... There is one interface per supported way of creating a container entry.

As I said, we have tested this technique for several months. There is a direct implementation of the interface named Assembly.

We have several containers consuming this interface:

• Assembly provides a PSR-11 compliant container consuming container definitions at runtime.
• Yaco is a compiler that generates a PSR-11 compliant container from container definitions (i.e. a container builder or container compiler).
• Although not completely ready, we started working on a Symfony compiler pass that can consume container definitions. It is definitely possible possible and easy to implement.

Finally, we wrote a proof-of-concept that takes YML service files (in the Symfony format) and converts them into definitions compatible with definition-interop. What is really great with this approach is that we don't have to choose a file format any more. Anyone can define its own file format (be it JSON / XML / YML / annotations / whatever...) as long as it can be converted to a set of definition objects.

Strengths:

• It can cover all the features we need for a module system (creation of new entries and extension of existing one)
• Implementation is relatively easy
• Offers some freedom regarding the best file format

Weaknesses:

Most of the weaknesses are shared with the previous solution (a common file format):

• For each way to create an service, we need a particular interface:
  • An interface to create a service with the new keyword (ObjectDefinitionInterface)
  • An interface to create a service using a static factory (FactoryCallDefinitionInterface)
  • A syntax to create a service using a factory instance that is itself a service (FactoryCallDefinitionInterface)
  • A syntax to create arrays of services, string parameters, etc... ([ParameterDefinitionInterface](https://github.com/container-interop/definition-interop/blob/master/src/ParameterDefinitionInterface.php) ...)
  • A syntax to call setters / methods
  • ...
• This is necessarily complex and a cognitive burden for the developers. The global architecture (definitions, definition loaders...) is somehow complex to understand for newcomers.
• Furthermore, even with numerous interfaces, we will always reach a point where we cannot do something. For instance, what if I want to concatenate 2 configuration strings and pass those as a parameter to a constructor?
• The more features we add, the harder the implementation for containers. We have to find a correct balance between available features and easiness of implementation. This might be a difficult balance to strike.
• Performance-wise, this is bad for "runtime containers". For each entry we want to create, we need to instantiate several definition objects. If the objects are generated from "loaders", it gets even worse. The only possible solutions are doing heavy caching or preprocessing the configuration files and generating a PHP class (i.e. "compiling" the container). De-facto, this means adding the notion of a "build stage" to all applications.

On a personal note, I must say I've spent quite some time testing and playing with this solution. I used to like it a lot and think it was the way to go. This was before I played with service providers.

Solution 4: dumpable definition interfaces

This solution tries to tackle one of the big issues of the two previous solutions: that fact that we need to have one syntax/interface per way to build an object.

This solution assumes that we will build/compile a container. Rather than having interfaces that describe definitions, we are standardizing here how a definition is transformed into an actual entry.

This has been tested with container-interop/compiler-interop. There is a single important interface here: DumpableDefinitionInterface. This interface contains one important method: toPhpCode.

Any object implementing the DumpableDefinitionInterface is a container definition that can cast itself into a PHP code string (and therefore be pasted into a compiled container).

Note that there is a shift of responsibility. Usually, the container builder is in charge of compiling the definitions. With this idea, the definitions are in charge of compiling themselves. The container builder is "dumb" and is just appending definitions code together. In my opinion, this is a good thing as it offers far greater extensibility.

This has been implemented in early version of Yaco.

You can find more about this solution in an earlier blog post.

Strengths:

• It can cover all the features we need for a module system (creation of new entries and extension of existing one)
• Implementation is relatively easy
• Offers maximum freedom. There is no limit on the way one can create an instance. You could easily write a new dumpable definition that creates proxies or lazy loaded objects, etc... the sky is the limit :)

Weaknesses:

• The most obvious limit is that it requires compiling. There is no way you can use a runtime container with this solution.
• Also, the whole architecture is complex to understand for newcomers (you need to understand the concept of container, compilers, definitions, definition loaders...)

On a personal level, I must admit I love this solution... and it seems I'm the only one out there. I guess returning PHP code as a string feels weird to many people. So let's go directly to the next solution.

Solution 5: service providers

Last but not least, we have tested the notion of service providers.

At boot time, the container is calling each service provider he knows of. Each service provider can declare a set of services that need to be created / extended. Each service is wrapped in a callback that can be executed only when the service is fetched (Pimple-style).

We have been extensively testing this solution with container-interop/service-provider. Especially, we managed quite easily to write adapters for both Laravel (a runtime container) and Symfony (a compiled container).

Note: do not let yourself be fooled by the current design of the interface. It features "static" methods but this is really an implementation detail that is currently discussed.

Strengths:

• It can cover all the features we need for a module system (creation of new entries and extension of existing one)
• The standard is much simpler, which means it is easier to explain, understand and agree upon
  
• It is easier to use as it relies on plain old PHP code
• It is easy to implement in containers
• Offers maximum freedom. There is no limit on the way one can create an instance since it is pure PHP.

Weaknesses:

Weaknesses are not easy to spot. Here is a small one:

• Service providers can be "compiled" into a compiled containers. However, the call to a service provider might incur an additional function call for every service fetched from the container (the extra call to the factory). In practice however, I'm fairly confident the impact on performance is close to 0.

On a personal level, when I started working on container interoperability, service providers were really the last solution I would have tried. I had a very strong feeling against this solution because I was under the impression that the more you add services, the slower the booting of the container becomes. It turns out I was completely wrong, since service providers can even be compiled into a container (as shown in the Symfony integration).

Conclusion

Below is a comparison table summarizing the pros and cons of each solution:

	PSR-11 + composite container	Standard file format	Container definition	Dumpable definition	Service provider
Can define new entries
Can extend existing entries
Can perform static analysis
Can create entries in any way (vs entries creation is tied to the standard)
Easy to understand / learn
Performance on runtime containers	?			N/A
Performance on compiled containers	?
Easy to specify / agree upon as a standard		(difficult to agree on file format and hard to agree on supported features)	~ (hard to agree on supported features)
Easily debuggable	Depends on the containers

It's been almost 10 months since we started working on this topic. We tried to tackle the problem in a very open-minded way. We tested several solutions and the results were surprising to me. 10 months ago, I would never have favoured the service-provider solution. Today, I'm glad to present it. I'm fairly confident that the service provider route is the best solution we have to build cross-framework modules, and I'd like to propose that to the PHP community at large.

What's next?

The discussion is ongoing on the PHP-FIG mailing list. This blog article is my contribution to the building.

I hope we will have some feedback. Maybe we will gather other solutions we did not think about, and finally, I hope we can validate which solution we should push forward.

Whatever solution we choose, we will be working on a PSR to formalize it. Do not hesitate to give us some feedback (ideally in the dedicated PHP-FIG mailing list thread). We are eager to hear from the PHP community (yes, this is you!)

About the author

David is CTO and co-founder of TheCodingMachine and WorkAdventure. He is the co-editor of PSR-11, the standard that provides interoperability between dependency injection containers. He is also the lead developper of GraphQLite, a framework-agnostic PHP library to implement a GraphQL API easily.