[PSR-11] Performance impact of the delegate lookup feature

TL;DR

In short, I measured the load time of a demo Symfony application. Then, I modified Symfony to make it PSR-11 compliant and I added 20 additional containers that are running side-by-side with Symfony container. I was not able to detect any meaningful performance impact.

My test setup

For this test, I decided to work on a Symfony 2 application. My idea is to test the default "demo" page that comes with Symfony when you install the framework (the one described in the Quick tour document). The first step was to modify Symfony to make it compatible with PSR-11 (actually, I made it compatible with container-interop, which is the precursor of PSR-11). This was remarkably simple to do. Most of the work was about adding the "delegate lookup" feature. Here is the changelog that makes Symfony compatible with container-interop. The modified Symfony repository is here. Then, I added a "composite container" (the one that comes with Acclimate) and I plugged into it the Symfony container and many additional containers. For the additional containers, I chose Picotainer. This is a minimalist container (a bit like Pimple but compatible with container-interop / PSR-11). Picotainer has a few interesting characteristic, one of them being it is quite fast to setup. This is because I want to measure the time spent in the composite container, not the time spent initializing containers (which is container dependent). Overall, containers are declared in Symfony's AppKernel this way:

class AppKernel extends Kernel
{
    //...

    /**
     * Initializes the service container.
     *
     * Use this method to initialize your own DI container and register it
     * in Symfony DI container.
     */
    protected function initializeContainer()
    {
        parent::initializeContainer();
        $compositeContainer = new AcclimateContainerCompositeContainer();
        // Create a Picotainer container
        $picotainer = new Picotainer([
            'my_service', function() { return new stdClass(); }
        ], $compositeContainer);
        $sfContainer = $this->container;
        $sfContainer->setDelegateContainer($compositeContainer);

        $compositeContainer->addContainer($picotainer);
        $compositeContainer->addContainer($sfContainer);
    }
}

The "prod" environment is used in Symfony and Composer autoloader is optimized using the "-o" option. Then, I used Blackfire.io (another Sensiolabs product!) to test the performance impact of my many containers on the load time of the page. Finally, the environment is initialized in a Docker container with a stock version of PHP (no funny extensions, no Xdebug, etc...)

Test runs

I ran the following tests (each test is stored in a separate branch of this github repository):

• standard: the standard, unpatched Symfony distribution
• psr11: Symfony with the patched container to add delegate lookup support. The feature is added to the container but is not used in that test. The goal is to check that performance is not altered by the modifications performed on the Symfony container.
• 2-containers: Symfony with a patched container. A second container is added (Picotainer). Symfony is first and Picotainer second container
• 2-containers-sf-last: Symfony with a patched container. A second container is added (Picotainer). Picotainer is first and Symfony second container. Since all container entries are stored in Symfony, this test should have lesser performances than the preceding one (for each dependency, we need to test Picotainer before finding the instance in Symfony)
• 20-containers-sf-last: Symfony with a patched container. 20 Picotainer containers are added (!). Symfony is last. This is clearly the worst case scenario, as for each container entry, we must check 20 picotainer containers before finding the entries in Symfony's container.

Results

Impact of the PSR-11 version of Symfony VS "standard" Symfony: Blackfire results

• Symfony standard: 74.3 ms
• PSR-11 Symfony: 70.3 ms (about 5% faster)
• Can the PSR-11 version of Symfony be faster? No of course. The code is almost identical (see the changelog). The expected result is that we would have roughly the same response time. What we are measuring here is the margin of error of the tests: 5%
• Conclusion: no impact

Impact of the Symfony with 2 containers (Symfony first, Picotainer last) VS "standard" Symfony: Blackfire results

Code on Github

• Symfony standard: 74.3 ms
• Symfony with 2 containers (Symfony first, Picotainer last): 75.4 ms (about 1% slower, within error margin)
• Conclusion: impact too small to be measured
• Interesting facts: the composite container is called 33 times.

Impact of the Symfony with 2 containers (Picotainer first, Symfony last) VS "standard" Symfony: Blackfire results

Code on Github

• Symfony standard: 74.3 ms
• Symfony with 2 containers (Picotainer first, Symfony last): 72.0 ms (about 2% faster, within error margin)
• Conclusion: impact too small to be measured

Impact of the Symfony with 21 containers (20 Picotainers first, Symfony last) VS "standard" Symfony: Blackfire results

Code on Github

• This is the worst possible case we can imagine.
• Symfony standard: 74.3 ms
• Symfony with 21 containers (20 Picotainers first, Symfony last): 74.3 ms (same time)
• Conclusion: impact too small to be measured

Conclusion

Surprisingly enough, even in the worst case scenario (21 containers running side-by-side), I was unable to measure any meaningful impact of the response time.

And this is with a "naive" implementation of the CompositeContainer that simply loops within each container for each instance to see if the instance is available. There are many ways to improve the performance of the CompositeContainer. We could for instance build a map of instances and their related container and put this map into cache. The fact is that time is spent elsewhere in the application and that the CompositeContainer and the additional containers are not adding any meaningful load to the demo application.

I open-sourced all tests that I ran and I would be interested into getting some feedback. I plan to discuss this further on the PHP-FIG mailing list.

About the author

David is CTO and co-founder of TheCodingMachine and WorkAdventure. He is the co-editor of PSR-11, the standard that provides interoperability between dependency injection containers. He is also the lead developper of GraphQLite, a framework-agnostic PHP library to implement a GraphQL API easily.