Unit Testing

From Habari Project

Jump to: navigation, search

Contents

Introduction

Habari has an existing set of unit tests that use the PHPUnit harness. As with all software projects, the coverage of the tests could be improved.

A specification for the Posts class exists and while there is documentation for other core Habari classes, it's not clear this list is comprehensive or 100% complete.

This needn't be a problem though as it's relatively easy to construct unit test scenarios:

  • Examine the source code to see what the inputs and outputs and the pre and post conditions are for each method.
  • Use your knowledge of the full Habari system to construct test cases (e.g. when a post is published with tomorrow's date, the status should be 'Scheduled').
  • Use the coverage reports generated by PHPUnit to identify code branches which are not yet exercised.
  • Review bug reports (open, closed) to construct regression tests.
  • Track new code and add unit tests simultaneously. Obviously, developers are best placed to do this. However, paradoxically, developers are not always the best resource to do this as they are often too close to the code.
  • Ask the developers themselves what needs testing or is known to need fixing or improving.


Current status

The current status of the Unit Tests (11 October 2010) for 95 system classes and 34,212 lines of PHP code is:

Tests: 244, Assertions: 397, Failures: 16, Errors: 5, Incomplete: 94, Skipped: 14.

PHPUnit can also produce a summary HTML report for execution of the complete test suite or individual unit tests.

Habari-PHPUnit.png

PHPUnit can also report on source code coverage, colour coding lines that have/have not be exercised. This can help guide testing and increase the level of code coverage.

PHPUnit-ClassCoverage.png

Open questions

  • When testing, can you assume that other methods (e.g. Posts::Delete) work correctly when restoring the environment back to the original state (TearDown) ?
  • What is the initial environment - a newly installed Habari blog ? Does it contain any posts, users, comments etc ?
  • None of the above methods would detect the absence of a required method from a class.
  • Review all the 'Incomplete', 'Skipped' and 'Failed' tests.
  • How to run the API Cache enabled tests ?
  • Core (and third party) plugins should also have unit tests.
  • What about more complex scenarios ? For example, does deleting a post which is the sole reference to the 'fruit' tag remove that tag ? Is this better impemented as a functional test ?
  • It is likely that Unit Testing will result in questions, bug reports and changes to documentation. However, it is potentially dangerous for a non-technical 'tester' (like me) to infer changes to the functional specification from the result of a potentially flawed test. This means that any changes to project documentation should always be reviewed by a developer.
  • Look how other Open Source projects approach the problem. I believe Drupal 7 also uses PHPUnit and greatly increased test coverage and helped to improve the code quality.
  • Drupal 7 includes the ability for the user to execute the suite of tests from the Administration dashboard which is an excellent feature. In the longer term, Habari should also include this functionality.
  • Read the PHPUnit manual.
  • Functional testing simulating Web browser sessions using the SimpleTest framework also looks useful. In fact, at a brief glance, it looks like SimpleTest also be used for the Unit Tests. Using a single test framework for all Habari tests would be preferable. How does it differ from PHPUnit ? Selenium appears to be a Web testing framework that is supported by PHPUnit. Need to research this area more without letting the obsession with the tools overwhelm the production of actual tests.
  • Some code sections may intentionally never be tested so the @codeCoverageIgnore tags may be used here.


What, who, when, where ?

Lines of code is a notoriously poor indicator of code complexity and PHPUnit reports include a scatter graph of code complexity but here is a snapshot list of the 'system/classes' directory.


Class
Lines
Test Class
acl.php
849
Y
actionhandler.php
123

adminhandler.php
3750

ajaxhandler.php
57

ajaxresponse.php
70

apccache.php
235
Y
atomhandler.php
881

bitmask.php
173
Y
block.php
288
Y
cache.php
237

colorutils.php
177
Y
commentinfo.php
20

comments.php
881

config.php
86

controller.php
179

cronjob.php
218

curlrequestprocessor.php
148

databaseconnection.php
841

db.php
470

error.php
261

eventlog.php
328

feedbackhandler.php
218

filecache.php
326
Y
format.php
545
Y
formstorage.php
28

formui.php
1991

habaridatetime.php
353

habarilocale.php
437
Y
hiengine.php
428

htmltokenzier.php
352

htmltokenset.php
264

importer.php
31

infoobject.php
22

inforecords.php
258

inputfilter.php
465

installhandler.php
1871

iscontent.php
25

logentry.php
363

mediaasset.php
193

media.php
232

mediasilo.php
68

modules.php
243

multibyte.php
413
Y
options.php
275
Y
pluggable.php
289

pluginhandler.php
58

plugin.php
87

plugins.php
553

postinfo.php
17

post.php
1293
Y
posts.php
1089
Y
queryprofile.php
65

queryrecord.php
195
Y
rawphpengine.php
187

remoterequest.php
347
Y
rewriterule.php
250

rewriterules.php
163

rpcclient.php
60

session.php
454

singleton.php
66

site.php
387

smartyengine.php
201

socketrequestprocessor.php
201

stack.php
276

superglobal.php
265

tag.php
285
Y
tags.php
232

templateengine.php
122

term.php
405

theme.php
1187

themes.php
253

update.php
131

url.php
290

urlproperties.php
18

usergroup.php
420

usergroups.php
156

userhandler.php
208

userinfo.php
21

user.php
672

userthemehandler.php
82

utils.php
1143
Y
uuid.php
94
Y
version.php
113
Y
vocabulary.php
761

xmlrpcbinary.php
29

xmlrpcclient.php
149

xmlrpcdate.php
45

xmlrpcexception.php
102

xmlrpcserver.php
68

xmlrpcstruct.php 76
xmlrpcutils.php 121


Let's use a key class Posts that contains a significant amount of code, has a specification, some existing tests and start to write what tests might be useful.

Notes from a brief chat with Michael.

The Posts class is an important one and already has some existing unit tests which can be extended.

Useful links for the specification of the Posts class:


Resources

Possibly useful Wiki pages documenting other Habari core classes


To Do

Have a look at this unit testing branch - trac.habariproject.org/habari/browser/branches/phpunit

Personal tools