Running single site and multisite WordPress integration tests

Running WordPress integration tests in single site and multisite installation mode.

The context

WordPress plugins and themes might run in the context of a single site installation as well as in that of a multisite one.
If the plugin or theme does not need to tap into any additional feature offered by a multisite installation functions and methods interacting with the WordPress API will all fall back on reading and writing from the database tables of the current blog (where “current blog” is the blog the user is currently visiting).
Sometimes, though, plugins might need or want to “take notice of the surroundings” and perform different actions depending on the installation type; a good example is a plugin that will add a settings page on single site installations and a network settings page in multisite installations.
Running tests in the two contexts is not difficult bur requires some care.
I will use Codeception and wp-browser in the following examples.

Setting up

When it comes to integration tests, those using the PHPUnit based automated tests Core suite or wp-browser WPLoader module, the WordPress local installation will simply provide the code base; as such whether the local WordPress installation is a single site or a multisite one does not make any difference.
In the case of the examples below I’m running a single site installation.
I will scaffold a new plugin using wp-cli:

wp scaffold pluging example

and will then scaffold wp-browser based tests using wp-browser wp-cli extension:

wp wpb-scaffold plugin-tests example

and have a ready to run plugin and tests after a step by step guide:

[wpb-scaffold]

To make sure everything is working I will run the tests:

codecept run

[tests]

Looking at the integration.suite.yml configuration file I can see the tests will run in single site installation mode;

# Codeception Test Suite Configuration

# Suite for integration tests.
# Load WordPress and test classes that rely on its functions and classes.


class_name: IntegrationTester
modules:
    enabled:
        - \Helper\Integration
        - WPLoader
    config:
        WPLoader:
            wpRootFolder: /Users/Luca/Sites/wp
            dbName: wp-tests
            dbHost: 127.0.0.1
            dbUser: root
            dbPassword: root
            tablePrefix: int_wp_
            domain: wp.dev
            adminEmail: [email protected]
            title: WP Tests
            plugins: [example/example.php]
            activatePlugins: [example/example.php]

A quick update to the plugin code is in order to have some code to run the tests against.

A basic plugin

Here is the code of the plugin I will test:

<?php
/**
 * Plugin Name:     Example
 * Plugin URI:      PLUGIN SITE HERE
 * Description:     PLUGIN DESCRIPTION HERE
 * Author:          YOUR NAME HERE
 * Author URI:      YOUR SITE HERE
 * Text Domain:     example
 * Domain Path:     /languages
 * Version:         0.1.0
 *
 * @package         Example
 */

if(is_multisite()){
    update_network_option( null, 'example', 'multisite' );
} else {
    update_option('example','single');
}

The plugin will take notice of the WordPress installation type and conditionally set an option in the current blog or the network table.

Single site tests

I will add a test case to the integration suite to make sure the plugin is working as intended in a single site installation:

wpcept generate:wpunit integration Single

and edit the test case to add a test method:

<?php

class SingleTest extends \Codeception\TestCase\WPTestCase
{

    public function setUp()
    {
        // before
        parent::setUp();

        // your set up methods here
    }

    public function tearDown()
    {
        // your tear down methods here

        // then
        parent::tearDown();
    }

    public function testSingle()
    {
        $this->assertTrue(is_plugin_active('example/example.php' )); 

        $this->assertEquals('single', get_option('example'));
    }
}

Running the test will yield the expected result:

codecept run integration

[test-1]

A multisite test

To run tests in multisite mode I will create another suite entirely:

codecept generate:suite muintegration

and configure it to run WPLoader in multisite mode:

class_name: MuintegrationTester
modules:
    enabled:
        - \Helper\Integration
        - WPLoader
    config:
        WPLoader:
            multisite: true
            wpRootFolder: /Users/Luca/Sites/wp
            dbName: wp-tests
            dbHost: 127.0.0.1
            dbUser: root
            dbPassword: root
            tablePrefix: int_wp_
            domain: wp.dev
            adminEmail: [email protected]
            title: WP Tests
            plugins: [example/example.php]
            activatePlugins: [example/example.php]
            bootstrapActions: []

After an update to the suite configuration file Codeception will need to rebuild the modules:

codecept build

I will add a test case to the suite to test that the plugin is aware of its surroundings and working as intended:

wpcept generate:wpunit muintegration Multisite

and edit the test case to this:

<?php

class MultisiteTest extends \Codeception\TestCase\WPTestCase
{

    public function setUp()
    {
        // before
        parent::setUp();

        // your set up methods here
    }

    public function tearDown()
    {
        // your tear down methods here

        // then
        parent::tearDown();
    }

    public function testMultisite()
    {
        $this->assertTrue(is_plugin_active_for_network('example/example.php'));

        $this->assertFalse(get_option('example'));

        $this->assertEquals('multisite', get_network_option(null, 'example'));
    }
}

Running the suite will see the tests pass:

codecept run muintegration

[test-2]

All together now

Running both suites at the same time will, but, see something fail:

codecept run

[test-3]

The reason is quickly found updating the failing test case:

public function testMultisite()
{
    // test fails here!
    $this->assertTrue(defined('MULTISITE') && MULTISITE === true);

    $this->assertTrue(is_plugin_active_for_network('example/example.php'));

    $this->assertFalse(get_option('example'));

    $this->assertEquals('multisite', get_network_option(null, 'example'));
}

Reason and solution

The tests, whether running using the Core suite or wp-browser, rely on the same codebase and will both bootstrap a WordPress instance in the same variable scope as the tests; was this not the case then writing any assertion relying on WordPress functions like get_option() or is_plugin_active_for_network() would not be possible.
With this clear it’s easy to understand that the first suite relying on the WPLoader module will define the MULTISITE constant in a first-come, first-served principle; in this case the integration suite will run first setting MULTISITE to false.
This is a direct consequence of WordPress reliance on globals and side-effects.
The first very easy solution is not to run multiple suites relying on WPLoader configured to run WordPress tests in different installation modes together:

codecept run integration && \
codecept run muintegration

[test-4]

While not ideal and violating the principle that would see test code never depend or influence itself this is the only viable solution to avoid the pitfalls of the “global pollution”.

I appreciate your input