End-to-End testing in WordPress with Nightwatch.js

Nightwatch.js logo

End-to-End testing (also called e2e testing) is a really great way to make sure that your plugin or even your entire site is working the way it should.

I’ve written previously about unit testing WordPress plugins which is awesome for checking that the logic of your plugin is working as expected, whereas e2e tests simulate actual user scenarios. You can test how a real user uses your site and that everything is working as it should.

We’re using Nighwatch.js to setup e2e tests for your plugin or application. It’s not the simplest of setups, and the documentation left me scratching my head more than once. This guide will hopefully help you get started without too much pain.

Ok, ok, what do I do already?!

Nightwatch.js uses Selenium which runs a browser that will automatically run your tests. (Don’t worry if it seems a little daunting. Just keep going and it’ll all make sense at the end!).

Selenium runs on Java (yes, yes). If you’re running MacOS you can install Java using brew. If you haven’t got brew installed (why not, it’s amaze?!?!) just yet pop along to https://brew.sh/ to get started and them come back here.

Now run

brew update

followed by

brew cask install java

If you’re running Windows just head on over to https://www.java.com/en/download/help/download_options.xml and come back when you’re done.

We can install Nightwatch.js and the other stuff we need using npm.

Run this in your plugin or theme that needs testing.

npm install nightwatch selenium-server chromedriver --save-dev

or

yarn add nightwatch selenium-server chromedriver --dev

if that’s how you roll. Cool, that’s all our dependencies sorted.

Configuring Nightwatch

Now in the same directory create a file called nightwatch.conf.js and add the following code:

const seleniumServer = require("selenium-server");
const chromedriver = require("chromedriver");

module.exports = {
	"src_folders": [
		"tests/e2e" // Change this to the directory where your test are
	],
	"output_folder": "./reports", // the directory for your rest reports
	"selenium": {
		"start_process": true,
		"server_path": seleniumServer.path,
		"host": "127.0.0.1",
		"port": 4444, // selenium usually runs on 4444
		"cli_args": {
			"webdriver.chrome.driver" : chromedriver.path // There are multiple drivers you can use, eg gecko or IE
		}
	},
	"test_settings": {
		"default": {
			"globals": {
				"waitForConditionTimeout": 5000
			},
			"desiredCapabilities": {
				"browserName": "chrome"
			}
		},
		"chrome": {
			"desiredCapabilities": {
				"browserName": "chrome",
				"javascriptEnabled": true
			}
		}
	}
};

Now in the directory you specified you can write your first test!!

module.exports = {
	'Test Something' (browser) {
		browser
			.url( 'https://wordpress.org/' )
			.waitForElementVisible( '#download' )
			.click( '#download' )
			.waitForElementVisible( '.download-meta' )
			.assert.urlContains( 'download' )
			.end();
	}
};

Run the tests

./node_modules/nightwatch/bin/nightwatch

and be amazed!

Bonus!

Impress your hipster friends by adding the tests to the scripts section of your package.json.

"scripts": {
   "test": "node_modules/nightwatch/bin/nightwatch",
}

Now you can just run

npm test

every time you wish to run your tests.

For documentation on all the assertions you can use just head over to http://nightwatchjs.org/api

Caveats

Nightwatch is generally great, but I ran into occasional issues with clicking on buttons or links outside the browser viewport. It’s unclear to me whether it’s Selenium or Nightwatch that’s to blame here but whatever, it’s annoying!

I got around this by using the following (pretty nasty) workaround.

'Test Complete Order' (browser) {
	browser
		.url( 'http://lavendla.se.dev.synot.io/test/' )
		.waitForElementVisible( '.cart', 5000 )
    		.click( '.go-to-checkout' ) // This didn't work
		.execute(function() { // This DID work!
			document.querySelector('.go-to-checkout').click()
		})
    // ...test something
}

Unit Testing in WordPress – Testing integrations

Unit Testing in Wordpress

I recently wrote a post about setting up PHPUnit tests for WordPress plugins. A colleague where I work at (the most excellent Angry Creative) suggested I provide a more concrete example of how to implement Unit Tests with WordPress itself and even other plugins. Here you go!

If you haven’t read that post yet, you should probably start there.

Integrating PHPUnit Testing with WordPress

When testing integration with WordPress, a good rule of thumb is that you don’t need to test WordPress’s own APIs. WordPress is responsible for this via their own test suite. In other words, in your test class you don’t need to do this:

<?php

class MyPluginTests extends WP_UnitTestCase {
    
    /** You don't need to do this! */
    function test_something_for_no_good_reason() {
        $post_title = 'Encounter at Farpoint';
        $post_id    = wp_insert_post([
            'post_title'  => $post_title,
            'post_status' => 'publish'
        ]);

        $this->assertInternalType( 'int', $post_id );

        $post = get_post( $post_id );
        $this->assertEquals( $post_title, $post->post_title );
    }

}

Having said that, I’ve recently written tests for a plugin that imports data via a CSV file using WordPress’s APIs. In this case, I think tests are very useful to make sure that all of the data I want to import is actually imported.

Here’s an example of what I mean.

<?php

class MyPluginTests extends WP_UnitTestCase {

	/**
	 * @return array
	 */
	function data_provider() {
		return [
			'title'   => 'Babylon 5',
			'country' => 'Sweden',
			'city'    => 'Malmö',
			'address' => 'Studentgatan 4, 211 38 Malmö, Sweden',
		];
	}

	/**
	 * Test the entire data import flow.
	 */
	function test_insert_post() {
		$data     = $this->data_provider();
		$importer = new Bulk_Importer( $data );
		$post_id  = $importer->init();

		// Check the taxonomies have successfully added and attached to the post
		$countries = wp_get_post_terms( $post_id, 'country' );
		$this->assertEquals( count( $countries ), 1 );
		$this->assertEquals( $countries[0]->name, $data['country'] );

		$city = wp_get_post_terms( $post_id, 'city' );
		$this->assertEquals( count( $city ), 1 );
		$this->assertEquals( $city[0]->name, $data['city'] );

		// Check the post meta has been successfully saved
		$location = get_post_meta( $post_id, 'retailer_position', true );
		$this->assertArrayHasKey( 'address', $location );
		$this->assertArrayHasKey( 'lat', $location );
		$this->assertArrayHasKey( 'lng', $location );
		$this->assertNotEmpty( $location['lat'] );
		$this->assertNotEmpty( $location['lng'] );
		$this->assertEquals( $location['address'], $data['address'] );
	}

}

It’s not really necessary to show the ‘Bulk_Importer’ class here, but suffice to say it adds the data in the ‘data_provider’ method to the database.

The city and country fields are added as terms in 2 separate taxonomies, respectively ‘country’ and ‘city’. The address is geo-encoded via the Google Maps API and that’s handy to have tested, just to make sure my plugin logic is working the way I thought it should. The post_meta ‘retailer_position’ should look like this after it’s been imported:

[
    'address' => 'Studentgatan 4, 211 38 Malmö, Sweden',
    'lat'     => '55.6033432',
    'lng'     => '13.0050809',
];

In my tests I check that the metadata has been saved in the correct format and that the respective taxonomy terms have been appended. I can check the address is the same, but I don’t need to check the result of the Google Maps API. They are responsible for their own APIs!

Integrating PHPUnit Testing with other plugins

When you scaffold your plugin tests via WP-CLI you’ll notice a bunch of files get added to your plugin. If you look at the file at tests/bootstrap.php you’ll see something like this:

<?php

$_tests_dir = getenv( 'WP_TESTS_DIR' );
if ( ! $_tests_dir ) {
    $_tests_dir = '/tmp/wordpress-tests-lib';
}

// Give access to tests_add_filter() function.
require_once $_tests_dir . '/includes/functions.php';

/**
 * Manually load the plugin being tested.
 */
function _manually_load_plugin() {
	require dirname( dirname( __FILE__ ) ) . '/your-plugin.php';
}
tests_add_filter( 'muplugins_loaded', '_manually_load_plugin' );

// Start up the WP testing environment.
require $_tests_dir . '/includes/bootstrap.php';

Note the ‘_manually_load_plugin()’ function. If you need to load additional plugins when you run your tests just add them to the list and they be autoloaded when you run your tests.

<?php

function _manually_load_plugin() {
    require dirname( dirname( __FILE__ ) ) . '/your-plugin.php';
    require dirname( dirname( dirname( __FILE__ ) ) ) . '/woocommerce/woocommerce.php';
}
tests_add_filter( 'muplugins_loaded', '_manually_load_plugin' );

Now in my tests i can use WooCommerce’s APIs without issue.

<?php

class MyPluginTests extends WP_UnitTestCase {

    /**
     * Test that relies on WooCommerce
     */
    function test_woocommerce() {
        // Assuming the product actually exists 🙂
        $product_id = 666;
        WC()->cart->add_to_cart( $product_id );

        $this->assertNotEmpty( WC()->cart->cart_contents );
    }

}

Again remember that we don’t actually need to test WooCommerce’s APIs. They are responsible for that, this is just to demonstrate that the APIs are in fact available to us.

Setting up data before we run our tests

Often when we want to test our plugin code we will need to create actual data to work with (see the above example). You’ll probably have noticed that our plugin tests class extends WP_UnitTestCase and not PHPUnit\Framework\TestCase.

WP_UnitTestCase class has a bunch of factory objects that you can use for creating posts, attachments, terms, users, comments, blogs, categories, terms and even networks. There doesn’t seem to be any official documentation about the factory classes which is a bit rubbish if you ask me, but google turned up this gist that has some data at least.

Using the setUp and tearDown methods

In your tests class there are 2 methods that will be called before and after each test method in the class is run, namely setUp and tearDown. The setUp method is a good place to use the respective factory object methods to create any data you’ll need in your tests. You can use the factory objects to go nuts and add a bunch of data to your test database here.

<?php

class MyPluginTests extends WP_UnitTestCase {

    function setUp() {
         // Call the setup method on the parent or the factory objects won't be loaded!
        parent::setUp();

        // Accepts the same arguments as wp_insert_post
        $post_id = $this->factory->post->create([
            'post_type' => 'page',
            'post_title => 'The name of the place is Babylon 5.',
        ]);

        // Create a bunch of posts, just to have some data to play around with.
        $post_ids = $this->factory->post->create_many( 50 );

        // Create an admin user
        $user_id = $this->factory->user->create([
            'user_login' => 'test',
            'role'       => 'administrator'
        ]);

        // Create a bunch of users
        $user_ids = $this->factory->user->create_many( 4 );

        // Create a single term
        $term_id = $this->factory->term->create([
            'name'     => 'Babylon 5',
            'taxonomy' => 'category',
            'slug'     => 'babylon-5'
        ]);

        // Create a bunch of terms
        $term_ids = $this->factory->term->create_many( 10 );
    }


}

Of course, you can use the factory methods anywhere in your class.

<?php

class MyPluginTests extends WP_UnitTestCase {

    // Remember to always call the parent::setUp method to make the factory objects available.
    function setUp() {
        parent::setUp();
    }


    function test_something() {
        $post_id = $this->factory->post->create([ 'post_title' => 'Trees and people used to be good friends' ]);

        $post = get_post( $post_id );
        // Go ahead! Test something 🙂
    }

}

#protip – if you’re using custom custom taxonomies that are not created the plugin you’re testing you can create them in the setUp method so that they’re available in your tests.

<?php

class MyPluginTests extends WP_UnitTestCase {

    function setUp() {
        parent::setUp();

        register_post_type( 'book' );
        register_taxonomy( 'genre', 'book' );

        ...
    }

}

Real world example

Ok, here’s a part of some actual tests i wrote to test a discount plugin I wrote for WooCommerce. The setUp method creates the products and adds them to the cart. The tearDown method removes the products from the cart and deletes them from the database.

<?php

class MyPluginTests extends WP_UnitTestCase {

    /** @var array */
    public $products = [];

    /**
     * SetUp
     */
    public function setUp() {
        parent::setUp();

        $this->createProducts();
        foreach ( $this->products as $product_id ) {
            WC()->cart->add_to_cart( $product_id );
        }

    }

    /**
     * TearDown
     */
    public function tearDown() {
        WC()->cart->empty_cart( true );

        foreach ( $this->products as $product_id ) {
            wp_delete_post( $product_id, true );
        }
    }


    /**
     * Create the products.
     */
    public function createProducts() {
        $products = [
            [
                'title' => 'Product 500',
            ],
            [
                'title' => 'Product 400',
            ],
            [
                'title' => 'Product 300',
            ],
        ];

        foreach ( $products as $product ) {
            $this->products[] = $this->factory->post->create([
               'post_title'  => $product['title'],
               'post_type'   => 'product',
               'post_status' => 'publish',
            ]);

            // Plus a bunch more stuff to actually create a valid WooCommerce product.
            ...
         }

    }

    /**
     * WC()->cart now contains the 3 products I created, so I can do some tests on
     * manipulating the cart with 'real' data using the WC_Cart APIs.
     */
    function test_the_cart() {
        ...
    }

}

Awesome, now we can test ALL OF THE THINGS!!

Ping me at @richardsweeney if you have any questions about PHPUnit testing in WordPress. I’d be happy to help out if I can 🙂

Unit Testing in WordPress

Unit Testing in Wordpress

Unit testing in PHP is one of those things that people talk about but can be difficult to get your head around until you take the leap and actually write your first test.

It can also be difficult to see the value in taking the time to write tests if you’ve never done it before, but take my word for it – it’s one of those things that when you start writing tests you’ll wonder why the hell it took you so long to start writing tests and how you ever managed before!

I can pretty much guarantee you it will make you change the way you structure your code too, in my case very much for the better.

Getting started

First you’ll need to install PHPUnit.

composer global require phpunit/phpunit

The easiest way to add tests to a plugin is to use WP-CLI.

wp scaffold plugin-tests {plugin-name}

Check out the documentation for more details about this command.

When you’ve run the command, cd to your plugin directory and run.

./bin/install-wp-tests.sh {db-name} {db-user} {db-pass} [db-host] [wp-version] [skip-database-creation]

Just run something like the following:

./bin/install-wp-tests.sh wordrpress_tests mysql_username mysql_password

To install the test library and the tests database. Do not use a real database for this as it will be deleted when you run your tests!!

Writing your tests

Alright, you’ve made it this far! You’ll notice a couple of files added to your plugin now. Check out the default tests class at: tests/tests-sample.php. It should look like this:

/**
 * Sample test case.
 */
class SampleTest extends WP_UnitTestCase {

    /**
     * A single example test.
     */
    function test_sample() {
        $this->assertTrue( true );
    }
}

If you run

phpunit

from your plugin directory you should see something like the following:

Unit tests result

Awesome!! Now you just need to write some actual tests. Sitepoint has a pretty good introduction article about this. I suggest just playing around with the API it feels natural.

Try adding the following to your tests class and see what happens 🙂

function test_something() {
    $string = 'Nice :)';
    $this->assertEquals( 'Nice :)', $string );
}

function test_something_that_fails() {
    $string = 'Nice :)';
    $this->assertEquals( 'BAD :(', $string );
}

There’s a bunch of assertions you can use depending on your use case. They’re all detailed on the PHPUnit site.

Happy testing!

Update! I’ve since written a sort of part 2 to this post that goes a bit deeper into how to integrate your tests with WordPress and even other plugins. Check it out, if you fancy diving a wee bit deeper into the subject.