Better mocking or system commands in Behat tests

Mocking system commands in Behat tests using PHP process shared memory feature.

The problem

When I say “command mocking” I mean replacing a system command with a mock that allows me to set expectations on it and verify calls, post conditions and usage.
While this seems removed from any practical usage I needed that many times during the development of a WP-Cli wp-browser specific package to scaffold wp-browser based tests.
I will not dive into that particular code though and come up with a more general and comprehensive example.
Let’s start with a Behat feature:

Feature: users of the ccc command should be able to decide to git push or not using a flag.

  Scenario: avoiding the final git push
    When I run 'ccc --no-push'
    Then 'composer' should have been called
    And 'git' should not have been called

The command itself is this:

#!/usr/bin/env php

<?php

// run composer update without dev dependencies
exec('composer update --no-dev', $output, $status);

if ($status !== 0) {
    echo (implode("\n", $output));
    return -1;
}

// maybe not push
$options = getopt('n', array('no-push'));

if (isset($options['no-push']) || isset($options['n'])) {
    return 0;
}

// ok, push
exec('git push', $output, $status);

if ($status !== 0) {
    echo (implode("\n", $output));
    return -1;
}

return 0;

The script will use two system commands: composer and git; I want to be sure those will be called by the script but I surely do not want a composer update or a git push to run on each test run.
The problem lies in replacing those two system commands with something less active (ideally doing nothing) and be able to assert one or both commands were called by the script.

The solution

After setting up Behat to run my single feature I’ve set up the FeatureContext class, the context in which each Behat scenario will run, to support my two step command mocking solution.
First step is to prepend the path to a feeatures/data folder to the system PATH; the script is not specifying where the composer and git commands are located and is relying on the system cascading path resolution.
This means I can point the system PATH where I want and have my composer and git commands called.
The second step is to use PHP shared memory feature feature to create a shared memory portion where different PHP processes will be able to read and write data.

<?php

use Behat\Behat\Context\Context;

/**
 * Defines application features from the specific context.
 */
class FeatureContext implements Context {

    protected $semaphore = 100;
    protected $segment   = 200;
    protected $processes = 23;
    protected $path;

    /**
     * @BeforeScenario
     */
    public function addDataToPathAndSetSemaphore() {
        // prepend the `features/data` folder to PATH
        $this->path = getenv( 'PATH' );
        putenv( 'PATH=' . dirname( __DIR__ ) . '/data:' . $this->path );

        // set up the shared memory portion
        $this->setupSemaphore();
    }

    protected function setupSemaphore() {
        // get a handle to the semaphore associated with the shared memory
        // segment we want
        $sem = sem_get( $this->semaphore, 1, 0600 );

        // ensure exclusive access to the semaphore
        $acquired = sem_acquire( $sem );
        if ( ! $acquired ) {
            throw new RuntimeException( 'Could not acquire semaphore' );
        }

        // get a handle to our shared memory segment
        $shm = shm_attach( $this->segment, 16384, 0600 );

        // store the value back in the shared memory segment
        shm_put_var( $shm, $this->processes, array() );

        // release the handle to the shared memory segment
        shm_detach( $shm );

        // release the semaphore so other processes can acquire it
        sem_release( $sem );
    }

    /**
     * @AfterScenario
     */
    public function restorePath() {
        putenv( 'PATH=' . $this->path );
    }

    /**
     * @When I run :command
     */
    public function iRun( $command ) {
        exec( dirname( dirname( __DIR__ ) ) . '/' . $command );
    }

    /**
     * @Then :command should have been called
     */
    public function shouldHaveBeenCalled( $command ) {
        PHPUnit_Framework_Assert::assertContains(
            $command, $this->getRanCommands(), 'Calls where: ' . print_r( $this->getRanCommands(), true )
        );
    }

    /**
     * @Then :command should not have been called
     */
    public function shouldNotHaveBeenCalled( $command ) {
        PHPUnit_Framework_Assert::assertNotContains(
            $command, $this->getRanCommands(), 'Calls where: ' . print_r( $this->getRanCommands(), true )
        );
    }

    protected function getRanCommands() {
        $sem      = sem_get( $this->semaphore, 1, 0600 );
        $acquired = sem_acquire( $sem );

        if ( ! $acquired ) {
            throw new RuntimeException( 'Could not acquire semaphore' );
        }

        $shm = shm_attach( $this->segment, 16384, 0600 );

        $processes = shm_get_var( $shm, $this->processes );
        $processes = is_array( $processes ) ? $processes : array();

        shm_detach( $shm );
        sem_release( $sem );

        return $processes;
    }
}

I say “processes” to indicate separate, parallel, asynchronous PHP process not sharing environment, data, memory or variable scope.
The “processes” will be, in the case of the feature above:

  1. the behat one
  2. the one running ccc in the FeatureContext::iRun method call
  3. the composer one called in the ccc one using exec
  4. the git one called in the ccc one using exec

The shared memory solution is needed here as it’s the only way processes will be able to communicate one with the other; think of a box where people unable to see and talk to each other will leave information useful to everyone.
If the FeatureContext class is in charge of setting up the shared memory portion and making sure the shell will look up the features/data folder first the other side of the “command mocking” are the two mock command themselves.
They differ but for a string and here is the “mock” composer executable:

#!/usr/bin/env php

<?php
$semaphoreId = 100; // same from FeatureContext!
$segmentId   = 200; // same from FeatureContext!
$processesId = 23; // same from FeatureContext!

// get a handle to the semaphore associated with the shared memory
// segment we want
$sem = sem_get($semaphoreId,1,0600);

// ensure exclusive access to the semaphore
$acquired = sem_acquire( $sem );
if ( ! $acquired ) {
    throw new RuntimeException( 'Could not acquire semaphore' );
}

// get a handle to our shared memory segment
$shm = shm_attach($segmentId,16384,0600);

// retrieve a value from the shared memory segment
$processes = shm_get_var($shm,$processesId);

// store this process call
array_shift($argv);
$processes[] = 'composer ' . implode(' ',$argv);

// store the value back in the shared memory segment
shm_put_var($shm,$processesId,$processes);

// release the handle to the shared memory segment
shm_detach($shm);

// release the semaphore so other processes can acquire it
sem_release($sem);

Semaphore related ids have to be hardcoded (that could be avoided making it include a php configuration file but that’s overkill) and what the mock command will do is just updating the record of called commands and add its own call to the shared memory portion.
After that it’s merely a question of writing the features and scenarios and running them:
Successful run of Behat tests

On GitHub

I’ve pushed the example code for my future reference on GitHub.

I appreciate your input