How to write tests for MediaWiki with PHPUnit?

Ashar 'hashar' Voultoiz

¿MediaWiki hacker?

Getting PHPUnit

Get PHPUnit from pear

See http://pear.phpunit.de/ for installation manual

Most linux distributions are lagging out.

To update later on:

# pear update-channels
# pear list-upgrades
# pear upgrade

MediaWiki PHPUnit test suite

• located in phase3/tests/phpunit
• tests written using PHP
• start tests with 'make'

Caution: tests are run against your live database.

MakeFile comes with several self documented helpers

Example next slide -->

makefile target example

$ make databaseless
php phpunit.php -c tests/phpunit/suite.xml \
 --exclude-group Broken, Destructive, Database, Stub

PHPUnit 3.5.13 by Sebastian Bergmann.

ISSSSSS............I...IIIIIIIIIIIIIII.......................   61 / 1629 (  3%)
.............................................................  122 / 1629 (  7%)
.............................................................  183 / 1629 ( 11%)

<snip>

.............................................................  671 / 1629 ( 41%)
..EEEEEEEE...................................................  732 / 1629 ( 44%)
............S................................................  793 / 1629 ( 48%)
..............................F...

I: incomplete test
S: skipped test
E: error test
.: successful 

Metrics!

As a footer, PHPUnit provides useful metrics:

Time: 22 seconds, Memory: 60.00Mb
FAILURES!
Tests: 827, Assertions: 74034, Failures: 1,
Errors: 8, Incomplete: 34, Skipped: 7.

More output options aimed at different publics:

--tap for perl hackers

--testdox for the product manager

Where do we need tests?

• Code coverage has the answer
• phpunit --coverage-html <output_path>

Coverage for phase3/includes

• Only 25% of methods in phase3/includes are tested!
• orange: NOT executed
• green: executed

Coverage for includes/xml.php

Writing tests

• Files in phase3/tests/phpunit/includes
• Filename must end with 'Test.php'
• Must have exactly one class (second ignored)]
• Should extends our class MediaWikiTestCase

Example:

<?php
class DumbClassTest extends MediaWikiTestCase {
	function testDumbest() {
		// your code there
	}
}
To add a test, just add a method whose name begins by 'test'

A dummy test

Lets test MediaWiki XmlSelect:getAttribute()

Edit phase3/tests/phpunit/includes/XmlSelectTest.php

Add a new method:

public function testGetAttributes() {
 $this->select->setAttribute( 1911, 'razor' );
 $this->assertEquals(
    $this->select->getAttribute( '1911' ), # code
    null,  # expected result
 }

Testing test result

Lets test testGetAttributes() from the previous slide

$ php phpunit.php --filter testGetAttributes
PHPUnit 3.5.13 by Sebastian Bergmann.

F   <-- HOUSTON WE HAVE AN ISSUE!

Time: 1 second, Memory: 26.75Mb
There was 1 failure:

1) XmlSelectTest::testGetAttributes
Failed asserting that two strings are equal.
// the usual diff:
--- Expected
+++ Actual
@@ @@
-razor
+

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

Fixing code or test

Fix the code or the test!

In this case the test is wrong, here is the patch:

$this->select->setAttribute( 1911, 'razor' );
   $this->assertEquals(
     $this->select->getAttribute( '1911' ), # code
-    null,  # expected result
+    'razor',  # expected result
 }

Fixing a test but still having wrong code is not a good idea.

And you will end up in CR wall of shame.

Re run fixed test

$ php phpunit.php --filter testGetAttributes
PHPUnit 3.5.13 by Sebastian Bergmann.
.    <--- LOOKS GREAT!

Time: 1 second, Memory: 26.50Mb

OK (1 test, 1 assertions)
$

• The test is right...
• ...and prove code is wrong.

Verify coverage

For those not sleeping yet. We rerun code coverage

$ php phpunit.php --filter testGetAttributes --coverage-html /tmp/XmlSel
ectGetAttr
PHPUnit 3.5.13 by Sebastian Bergmann.

.

Time: 3 seconds, Memory: 29.00Mb

OK (1 test, 1 assertion)

Generating code coverage report, this may take a moment.

getAttribute() covered!

Verify line coverage

Looking at the code: everything is green:

Does anyone read titles?

Für questions: come to me after this session.