diff options
author | cinap_lenrek <cinap_lenrek@localhost> | 2011-05-03 11:25:13 +0000 |
---|---|---|
committer | cinap_lenrek <cinap_lenrek@localhost> | 2011-05-03 11:25:13 +0000 |
commit | 458120dd40db6b4df55a4e96b650e16798ef06a0 (patch) | |
tree | 8f82685be24fef97e715c6f5ca4c68d34d5074ee /sys/src/cmd/python/Doc/lib/libunittest.tex | |
parent | 3a742c699f6806c1145aea5149bf15de15a0afd7 (diff) |
add hg and python
Diffstat (limited to 'sys/src/cmd/python/Doc/lib/libunittest.tex')
-rw-r--r-- | sys/src/cmd/python/Doc/lib/libunittest.tex | 974 |
1 files changed, 974 insertions, 0 deletions
diff --git a/sys/src/cmd/python/Doc/lib/libunittest.tex b/sys/src/cmd/python/Doc/lib/libunittest.tex new file mode 100644 index 000000000..f1e1033c1 --- /dev/null +++ b/sys/src/cmd/python/Doc/lib/libunittest.tex @@ -0,0 +1,974 @@ +\section{\module{unittest} --- + Unit testing framework} + +\declaremodule{standard}{unittest} +\modulesynopsis{Unit testing framework for Python.} +\moduleauthor{Steve Purcell}{stephen\textunderscore{}purcell@yahoo.com} +\sectionauthor{Steve Purcell}{stephen\textunderscore{}purcell@yahoo.com} +\sectionauthor{Fred L. Drake, Jr.}{fdrake@acm.org} +\sectionauthor{Raymond Hettinger}{python@rcn.com} + +\versionadded{2.1} + +The Python unit testing framework, sometimes referred to as ``PyUnit,'' is +a Python language version of JUnit, by Kent Beck and Erich Gamma. +JUnit is, in turn, a Java version of Kent's Smalltalk testing +framework. Each is the de facto standard unit testing framework for +its respective language. + +\module{unittest} supports test automation, sharing of setup and shutdown +code for tests, aggregation of tests into collections, and independence of +the tests from the reporting framework. The \module{unittest} module +provides classes that make it easy to support these qualities for a +set of tests. + +To achieve this, \module{unittest} supports some important concepts: + +\begin{definitions} +\term{test fixture} +A \dfn{test fixture} represents the preparation needed to perform one +or more tests, and any associate cleanup actions. This may involve, +for example, creating temporary or proxy databases, directories, or +starting a server process. + +\term{test case} +A \dfn{test case} is the smallest unit of testing. It checks for a +specific response to a particular set of inputs. \module{unittest} +provides a base class, \class{TestCase}, which may be used to create +new test cases. + +\term{test suite} +A \dfn{test suite} is a collection of test cases, test suites, or +both. It is used to aggregate tests that should be executed +together. + +\term{test runner} +A \dfn{test runner} is a component which orchestrates the execution of +tests and provides the outcome to the user. The runner may use a +graphical interface, a textual interface, or return a special value to +indicate the results of executing the tests. +\end{definitions} + + +The test case and test fixture concepts are supported through the +\class{TestCase} and \class{FunctionTestCase} classes; the former +should be used when creating new tests, and the latter can be used when +integrating existing test code with a \module{unittest}-driven framework. +When building test fixtures using \class{TestCase}, the \method{setUp()} +and \method{tearDown()} methods can be overridden to provide +initialization and cleanup for the fixture. With +\class{FunctionTestCase}, existing functions can be passed to the +constructor for these purposes. When the test is run, the +fixture initialization is run first; if it succeeds, the cleanup +method is run after the test has been executed, regardless of the +outcome of the test. Each instance of the \class{TestCase} will only +be used to run a single test method, so a new fixture is created for +each test. + +Test suites are implemented by the \class{TestSuite} class. This +class allows individual tests and test suites to be aggregated; when +the suite is executed, all tests added directly to the suite and in +``child'' test suites are run. + +A test runner is an object that provides a single method, +\method{run()}, which accepts a \class{TestCase} or \class{TestSuite} +object as a parameter, and returns a result object. The class +\class{TestResult} is provided for use as the result object. +\module{unittest} provides the \class{TextTestRunner} as an example +test runner which reports test results on the standard error stream by +default. Alternate runners can be implemented for other environments +(such as graphical environments) without any need to derive from a +specific class. + + +\begin{seealso} + \seemodule{doctest}{Another test-support module with a very + different flavor.} + \seetitle[http://www.XProgramming.com/testfram.htm]{Simple Smalltalk + Testing: With Patterns}{Kent Beck's original paper on + testing frameworks using the pattern shared by + \module{unittest}.} +\end{seealso} + + +\subsection{Basic example \label{minimal-example}} + +The \module{unittest} module provides a rich set of tools for +constructing and running tests. This section demonstrates that a +small subset of the tools suffice to meet the needs of most users. + +Here is a short script to test three functions from the +\refmodule{random} module: + +\begin{verbatim} +import random +import unittest + +class TestSequenceFunctions(unittest.TestCase): + + def setUp(self): + self.seq = range(10) + + def testshuffle(self): + # make sure the shuffled sequence does not lose any elements + random.shuffle(self.seq) + self.seq.sort() + self.assertEqual(self.seq, range(10)) + + def testchoice(self): + element = random.choice(self.seq) + self.assert_(element in self.seq) + + def testsample(self): + self.assertRaises(ValueError, random.sample, self.seq, 20) + for element in random.sample(self.seq, 5): + self.assert_(element in self.seq) + +if __name__ == '__main__': + unittest.main() +\end{verbatim} + +A testcase is created by subclassing \class{unittest.TestCase}. +The three individual tests are defined with methods whose names start with +the letters \samp{test}. This naming convention informs the test runner +about which methods represent tests. + +The crux of each test is a call to \method{assertEqual()} to +check for an expected result; \method{assert_()} to verify a condition; +or \method{assertRaises()} to verify that an expected exception gets +raised. These methods are used instead of the \keyword{assert} statement +so the test runner can accumulate all test results and produce a report. + +When a \method{setUp()} method is defined, the test runner will run that +method prior to each test. Likewise, if a \method{tearDown()} method is +defined, the test runner will invoke that method after each test. In the +example, \method{setUp()} was used to create a fresh sequence for each test. + +The final block shows a simple way to run the tests. +\function{unittest.main()} provides a command line interface to the +test script. When run from the command line, the above script +produces an output that looks like this: + +\begin{verbatim} +... +---------------------------------------------------------------------- +Ran 3 tests in 0.000s + +OK +\end{verbatim} + +Instead of \function{unittest.main()}, there are other ways to run the tests +with a finer level of control, less terse output, and no requirement to be +run from the command line. For example, the last two lines may be replaced +with: + +\begin{verbatim} +suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions) +unittest.TextTestRunner(verbosity=2).run(suite) +\end{verbatim} + +Running the revised script from the interpreter or another script +produces the following output: + +\begin{verbatim} +testchoice (__main__.TestSequenceFunctions) ... ok +testsample (__main__.TestSequenceFunctions) ... ok +testshuffle (__main__.TestSequenceFunctions) ... ok + +---------------------------------------------------------------------- +Ran 3 tests in 0.110s + +OK +\end{verbatim} + +The above examples show the most commonly used \module{unittest} features +which are sufficient to meet many everyday testing needs. The remainder +of the documentation explores the full feature set from first principles. + + +\subsection{Organizing test code + \label{organizing-tests}} + +The basic building blocks of unit testing are \dfn{test cases} --- +single scenarios that must be set up and checked for correctness. In +\module{unittest}, test cases are represented by instances of +\module{unittest}'s \class{TestCase} class. To make +your own test cases you must write subclasses of \class{TestCase}, or +use \class{FunctionTestCase}. + +An instance of a \class{TestCase}-derived class is an object that can +completely run a single test method, together with optional set-up +and tidy-up code. + +The testing code of a \class{TestCase} instance should be entirely +self contained, such that it can be run either in isolation or in +arbitrary combination with any number of other test cases. + +The simplest \class{TestCase} subclass will simply override the +\method{runTest()} method in order to perform specific testing code: + +\begin{verbatim} +import unittest + +class DefaultWidgetSizeTestCase(unittest.TestCase): + def runTest(self): + widget = Widget('The widget') + self.assertEqual(widget.size(), (50, 50), 'incorrect default size') +\end{verbatim} + +Note that in order to test something, we use the one of the +\method{assert*()} or \method{fail*()} methods provided by the +\class{TestCase} base class. If the test fails, an exception will be +raised, and \module{unittest} will identify the test case as a +\dfn{failure}. Any other exceptions will be treated as \dfn{errors}. +This helps you identify where the problem is: \dfn{failures} are caused by +incorrect results - a 5 where you expected a 6. \dfn{Errors} are caused by +incorrect code - e.g., a \exception{TypeError} caused by an incorrect +function call. + +The way to run a test case will be described later. For now, note +that to construct an instance of such a test case, we call its +constructor without arguments: + +\begin{verbatim} +testCase = DefaultWidgetSizeTestCase() +\end{verbatim} + +Now, such test cases can be numerous, and their set-up can be +repetitive. In the above case, constructing a \class{Widget} in each of +100 Widget test case subclasses would mean unsightly duplication. + +Luckily, we can factor out such set-up code by implementing a method +called \method{setUp()}, which the testing framework will +automatically call for us when we run the test: + +\begin{verbatim} +import unittest + +class SimpleWidgetTestCase(unittest.TestCase): + def setUp(self): + self.widget = Widget('The widget') + +class DefaultWidgetSizeTestCase(SimpleWidgetTestCase): + def runTest(self): + self.failUnless(self.widget.size() == (50,50), + 'incorrect default size') + +class WidgetResizeTestCase(SimpleWidgetTestCase): + def runTest(self): + self.widget.resize(100,150) + self.failUnless(self.widget.size() == (100,150), + 'wrong size after resize') +\end{verbatim} + +If the \method{setUp()} method raises an exception while the test is +running, the framework will consider the test to have suffered an +error, and the \method{runTest()} method will not be executed. + +Similarly, we can provide a \method{tearDown()} method that tidies up +after the \method{runTest()} method has been run: + +\begin{verbatim} +import unittest + +class SimpleWidgetTestCase(unittest.TestCase): + def setUp(self): + self.widget = Widget('The widget') + + def tearDown(self): + self.widget.dispose() + self.widget = None +\end{verbatim} + +If \method{setUp()} succeeded, the \method{tearDown()} method will be +run whether \method{runTest()} succeeded or not. + +Such a working environment for the testing code is called a +\dfn{fixture}. + +Often, many small test cases will use the same fixture. In this case, +we would end up subclassing \class{SimpleWidgetTestCase} into many +small one-method classes such as +\class{DefaultWidgetSizeTestCase}. This is time-consuming and +discouraging, so in the same vein as JUnit, \module{unittest} provides +a simpler mechanism: + +\begin{verbatim} +import unittest + +class WidgetTestCase(unittest.TestCase): + def setUp(self): + self.widget = Widget('The widget') + + def tearDown(self): + self.widget.dispose() + self.widget = None + + def testDefaultSize(self): + self.failUnless(self.widget.size() == (50,50), + 'incorrect default size') + + def testResize(self): + self.widget.resize(100,150) + self.failUnless(self.widget.size() == (100,150), + 'wrong size after resize') +\end{verbatim} + +Here we have not provided a \method{runTest()} method, but have +instead provided two different test methods. Class instances will now +each run one of the \method{test*()} methods, with \code{self.widget} +created and destroyed separately for each instance. When creating an +instance we must specify the test method it is to run. We do this by +passing the method name in the constructor: + +\begin{verbatim} +defaultSizeTestCase = WidgetTestCase('testDefaultSize') +resizeTestCase = WidgetTestCase('testResize') +\end{verbatim} + +Test case instances are grouped together according to the features +they test. \module{unittest} provides a mechanism for this: the +\dfn{test suite}, represented by \module{unittest}'s \class{TestSuite} +class: + +\begin{verbatim} +widgetTestSuite = unittest.TestSuite() +widgetTestSuite.addTest(WidgetTestCase('testDefaultSize')) +widgetTestSuite.addTest(WidgetTestCase('testResize')) +\end{verbatim} + +For the ease of running tests, as we will see later, it is a good +idea to provide in each test module a callable object that returns a +pre-built test suite: + +\begin{verbatim} +def suite(): + suite = unittest.TestSuite() + suite.addTest(WidgetTestCase('testDefaultSize')) + suite.addTest(WidgetTestCase('testResize')) + return suite +\end{verbatim} + +or even: + +\begin{verbatim} +def suite(): + tests = ['testDefaultSize', 'testResize'] + + return unittest.TestSuite(map(WidgetTestCase, tests)) +\end{verbatim} + +Since it is a common pattern to create a \class{TestCase} subclass +with many similarly named test functions, \module{unittest} provides a +\class{TestLoader} class that can be used to automate the process of +creating a test suite and populating it with individual tests. +For example, + +\begin{verbatim} +suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase) +\end{verbatim} + +will create a test suite that will run +\code{WidgetTestCase.testDefaultSize()} and \code{WidgetTestCase.testResize}. +\class{TestLoader} uses the \code{'test'} method name prefix to identify +test methods automatically. + +Note that the order in which the various test cases will be run is +determined by sorting the test function names with the built-in +\function{cmp()} function. + +Often it is desirable to group suites of test cases together, so as to +run tests for the whole system at once. This is easy, since +\class{TestSuite} instances can be added to a \class{TestSuite} just +as \class{TestCase} instances can be added to a \class{TestSuite}: + +\begin{verbatim} +suite1 = module1.TheTestSuite() +suite2 = module2.TheTestSuite() +alltests = unittest.TestSuite([suite1, suite2]) +\end{verbatim} + +You can place the definitions of test cases and test suites in the +same modules as the code they are to test (such as \file{widget.py}), +but there are several advantages to placing the test code in a +separate module, such as \file{test_widget.py}: + +\begin{itemize} + \item The test module can be run standalone from the command line. + \item The test code can more easily be separated from shipped code. + \item There is less temptation to change test code to fit the code + it tests without a good reason. + \item Test code should be modified much less frequently than the + code it tests. + \item Tested code can be refactored more easily. + \item Tests for modules written in C must be in separate modules + anyway, so why not be consistent? + \item If the testing strategy changes, there is no need to change + the source code. +\end{itemize} + + +\subsection{Re-using old test code + \label{legacy-unit-tests}} + +Some users will find that they have existing test code that they would +like to run from \module{unittest}, without converting every old test +function to a \class{TestCase} subclass. + +For this reason, \module{unittest} provides a \class{FunctionTestCase} +class. This subclass of \class{TestCase} can be used to wrap an existing +test function. Set-up and tear-down functions can also be provided. + +Given the following test function: + +\begin{verbatim} +def testSomething(): + something = makeSomething() + assert something.name is not None + # ... +\end{verbatim} + +one can create an equivalent test case instance as follows: + +\begin{verbatim} +testcase = unittest.FunctionTestCase(testSomething) +\end{verbatim} + +If there are additional set-up and tear-down methods that should be +called as part of the test case's operation, they can also be provided +like so: + +\begin{verbatim} +testcase = unittest.FunctionTestCase(testSomething, + setUp=makeSomethingDB, + tearDown=deleteSomethingDB) +\end{verbatim} + +To make migrating existing test suites easier, \module{unittest} +supports tests raising \exception{AssertionError} to indicate test failure. +However, it is recommended that you use the explicit +\method{TestCase.fail*()} and \method{TestCase.assert*()} methods instead, +as future versions of \module{unittest} may treat \exception{AssertionError} +differently. + +\note{Even though \class{FunctionTestCase} can be used to quickly convert +an existing test base over to a \module{unittest}-based system, this +approach is not recommended. Taking the time to set up proper +\class{TestCase} subclasses will make future test refactorings infinitely +easier.} + + + +\subsection{Classes and functions + \label{unittest-contents}} + +\begin{classdesc}{TestCase}{\optional{methodName}} + Instances of the \class{TestCase} class represent the smallest + testable units in the \module{unittest} universe. This class is + intended to be used as a base class, with specific tests being + implemented by concrete subclasses. This class implements the + interface needed by the test runner to allow it to drive the + test, and methods that the test code can use to check for and + report various kinds of failure. + + Each instance of \class{TestCase} will run a single test method: + the method named \var{methodName}. If you remember, we had an + earlier example that went something like this: + + \begin{verbatim} + def suite(): + suite = unittest.TestSuite() + suite.addTest(WidgetTestCase('testDefaultSize')) + suite.addTest(WidgetTestCase('testResize')) + return suite + \end{verbatim} + + Here, we create two instances of \class{WidgetTestCase}, each of + which runs a single test. + + \var{methodName} defaults to \code{'runTest'}. +\end{classdesc} + +\begin{classdesc}{FunctionTestCase}{testFunc\optional{, + setUp\optional{, tearDown\optional{, description}}}} + This class implements the portion of the \class{TestCase} interface + which allows the test runner to drive the test, but does not provide + the methods which test code can use to check and report errors. + This is used to create test cases using legacy test code, allowing + it to be integrated into a \refmodule{unittest}-based test + framework. +\end{classdesc} + +\begin{classdesc}{TestSuite}{\optional{tests}} + This class represents an aggregation of individual tests cases and + test suites. The class presents the interface needed by the test + runner to allow it to be run as any other test case. Running a + \class{TestSuite} instance is the same as iterating over the suite, + running each test individually. + + If \var{tests} is given, it must be an iterable of individual test cases or + other test suites that will be used to build the suite initially. + Additional methods are provided to add test cases and suites to the + collection later on. +\end{classdesc} + +\begin{classdesc}{TestLoader}{} + This class is responsible for loading tests according to various + criteria and returning them wrapped in a \class{TestSuite}. + It can load all tests within a given module or \class{TestCase} + subclass. +\end{classdesc} + +\begin{classdesc}{TestResult}{} + This class is used to compile information about which tests have succeeded + and which have failed. +\end{classdesc} + +\begin{datadesc}{defaultTestLoader} + Instance of the \class{TestLoader} class intended to be shared. If no + customization of the \class{TestLoader} is needed, this instance can + be used instead of repeatedly creating new instances. +\end{datadesc} + +\begin{classdesc}{TextTestRunner}{\optional{stream\optional{, + descriptions\optional{, verbosity}}}} + A basic test runner implementation which prints results on standard + error. It has a few configurable parameters, but is essentially + very simple. Graphical applications which run test suites should + provide alternate implementations. +\end{classdesc} + +\begin{funcdesc}{main}{\optional{module\optional{, + defaultTest\optional{, argv\optional{, + testRunner\optional{, testLoader}}}}}} + A command-line program that runs a set of tests; this is primarily + for making test modules conveniently executable. The simplest use + for this function is to include the following line at the end of a + test script: + +\begin{verbatim} +if __name__ == '__main__': + unittest.main() +\end{verbatim} +\end{funcdesc} + +In some cases, the existing tests may have been written using the +\refmodule{doctest} module. If so, that module provides a +\class{DocTestSuite} class that can automatically build +\class{unittest.TestSuite} instances from the existing +\module{doctest}-based tests. +\versionadded{2.3} + + +\subsection{TestCase Objects + \label{testcase-objects}} + +Each \class{TestCase} instance represents a single test, but each +concrete subclass may be used to define multiple tests --- the +concrete class represents a single test fixture. The fixture is +created and cleaned up for each test case. + +\class{TestCase} instances provide three groups of methods: one group +used to run the test, another used by the test implementation to +check conditions and report failures, and some inquiry methods +allowing information about the test itself to be gathered. + +Methods in the first group (running the test) are: + +\begin{methoddesc}[TestCase]{setUp}{} + Method called to prepare the test fixture. This is called + immediately before calling the test method; any exception raised by + this method will be considered an error rather than a test failure. + The default implementation does nothing. +\end{methoddesc} + +\begin{methoddesc}[TestCase]{tearDown}{} + Method called immediately after the test method has been called and + the result recorded. This is called even if the test method raised + an exception, so the implementation in subclasses may need to be + particularly careful about checking internal state. Any exception + raised by this method will be considered an error rather than a test + failure. This method will only be called if the \method{setUp()} + succeeds, regardless of the outcome of the test method. + The default implementation does nothing. +\end{methoddesc} + +\begin{methoddesc}[TestCase]{run}{\optional{result}} + Run the test, collecting the result into the test result object + passed as \var{result}. If \var{result} is omitted or \constant{None}, + a temporary result object is created (by calling the + \method{defaultTestCase()} method) and used; this result object is not + returned to \method{run()}'s caller. + + The same effect may be had by simply calling the \class{TestCase} + instance. +\end{methoddesc} + +\begin{methoddesc}[TestCase]{debug}{} + Run the test without collecting the result. This allows exceptions + raised by the test to be propagated to the caller, and can be used + to support running tests under a debugger. +\end{methoddesc} + + +The test code can use any of the following methods to check for and +report failures. + +\begin{methoddesc}[TestCase]{assert_}{expr\optional{, msg}} +\methodline{failUnless}{expr\optional{, msg}} + Signal a test failure if \var{expr} is false; the explanation for + the error will be \var{msg} if given, otherwise it will be + \constant{None}. +\end{methoddesc} + +\begin{methoddesc}[TestCase]{assertEqual}{first, second\optional{, msg}} +\methodline{failUnlessEqual}{first, second\optional{, msg}} + Test that \var{first} and \var{second} are equal. If the values do + not compare equal, the test will fail with the explanation given by + \var{msg}, or \constant{None}. Note that using \method{failUnlessEqual()} + improves upon doing the comparison as the first parameter to + \method{failUnless()}: the default value for \var{msg} can be + computed to include representations of both \var{first} and + \var{second}. +\end{methoddesc} + +\begin{methoddesc}[TestCase]{assertNotEqual}{first, second\optional{, msg}} +\methodline{failIfEqual}{first, second\optional{, msg}} + Test that \var{first} and \var{second} are not equal. If the values + do compare equal, the test will fail with the explanation given by + \var{msg}, or \constant{None}. Note that using \method{failIfEqual()} + improves upon doing the comparison as the first parameter to + \method{failUnless()} is that the default value for \var{msg} can be + computed to include representations of both \var{first} and + \var{second}. +\end{methoddesc} + +\begin{methoddesc}[TestCase]{assertAlmostEqual}{first, second\optional{, + places\optional{, msg}}} +\methodline{failUnlessAlmostEqual}{first, second\optional{, + places\optional{, msg}}} + Test that \var{first} and \var{second} are approximately equal + by computing the difference, rounding to the given number of \var{places}, + and comparing to zero. Note that comparing a given number of decimal places + is not the same as comparing a given number of significant digits. + If the values do not compare equal, the test will fail with the explanation + given by \var{msg}, or \constant{None}. +\end{methoddesc} + +\begin{methoddesc}[TestCase]{assertNotAlmostEqual}{first, second\optional{, + places\optional{, msg}}} +\methodline{failIfAlmostEqual}{first, second\optional{, + places\optional{, msg}}} + Test that \var{first} and \var{second} are not approximately equal + by computing the difference, rounding to the given number of \var{places}, + and comparing to zero. Note that comparing a given number of decimal places + is not the same as comparing a given number of significant digits. + If the values do not compare equal, the test will fail with the explanation + given by \var{msg}, or \constant{None}. +\end{methoddesc} + +\begin{methoddesc}[TestCase]{assertRaises}{exception, callable, \moreargs} +\methodline{failUnlessRaises}{exception, callable, \moreargs} + Test that an exception is raised when \var{callable} is called with + any positional or keyword arguments that are also passed to + \method{assertRaises()}. The test passes if \var{exception} is + raised, is an error if another exception is raised, or fails if no + exception is raised. To catch any of a group of exceptions, a tuple + containing the exception classes may be passed as \var{exception}. +\end{methoddesc} + +\begin{methoddesc}[TestCase]{failIf}{expr\optional{, msg}} + The inverse of the \method{failUnless()} method is the + \method{failIf()} method. This signals a test failure if \var{expr} + is true, with \var{msg} or \constant{None} for the error message. +\end{methoddesc} + +\begin{methoddesc}[TestCase]{fail}{\optional{msg}} + Signals a test failure unconditionally, with \var{msg} or + \constant{None} for the error message. +\end{methoddesc} + +\begin{memberdesc}[TestCase]{failureException} + This class attribute gives the exception raised by the + \method{test()} method. If a test framework needs to use a + specialized exception, possibly to carry additional information, it + must subclass this exception in order to ``play fair'' with the + framework. The initial value of this attribute is + \exception{AssertionError}. +\end{memberdesc} + + +Testing frameworks can use the following methods to collect +information on the test: + +\begin{methoddesc}[TestCase]{countTestCases}{} + Return the number of tests represented by this test object. For + \class{TestCase} instances, this will always be \code{1}. +\end{methoddesc} + +\begin{methoddesc}[TestCase]{defaultTestResult}{} + Return an instance of the test result class that should be used + for this test case class (if no other result instance is provided + to the \method{run()} method). + + For \class{TestCase} instances, this will always be an instance of + \class{TestResult}; subclasses of \class{TestCase} should + override this as necessary. +\end{methoddesc} + +\begin{methoddesc}[TestCase]{id}{} + Return a string identifying the specific test case. This is usually + the full name of the test method, including the module and class + name. +\end{methoddesc} + +\begin{methoddesc}[TestCase]{shortDescription}{} + Returns a one-line description of the test, or \constant{None} if no + description has been provided. The default implementation of this + method returns the first line of the test method's docstring, if + available, or \constant{None}. +\end{methoddesc} + + +\subsection{TestSuite Objects + \label{testsuite-objects}} + +\class{TestSuite} objects behave much like \class{TestCase} objects, +except they do not actually implement a test. Instead, they are used +to aggregate tests into groups of tests that should be run together. +Some additional methods are available to add tests to \class{TestSuite} +instances: + +\begin{methoddesc}[TestSuite]{addTest}{test} + Add a \class{TestCase} or \class{TestSuite} to the suite. +\end{methoddesc} + +\begin{methoddesc}[TestSuite]{addTests}{tests} + Add all the tests from an iterable of \class{TestCase} and + \class{TestSuite} instances to this test suite. + + This is equivalent to iterating over \var{tests}, calling + \method{addTest()} for each element. +\end{methoddesc} + +\class{TestSuite} shares the following methods with \class{TestCase}: + +\begin{methoddesc}[TestSuite]{run}{result} + Run the tests associated with this suite, collecting the result into + the test result object passed as \var{result}. Note that unlike + \method{TestCase.run()}, \method{TestSuite.run()} requires the + result object to be passed in. +\end{methoddesc} + +\begin{methoddesc}[TestSuite]{debug}{} + Run the tests associated with this suite without collecting the result. + This allows exceptions raised by the test to be propagated to the caller + and can be used to support running tests under a debugger. +\end{methoddesc} + +\begin{methoddesc}[TestSuite]{countTestCases}{} + Return the number of tests represented by this test object, including + all individual tests and sub-suites. +\end{methoddesc} + +In the typical usage of a \class{TestSuite} object, the \method{run()} +method is invoked by a \class{TestRunner} rather than by the end-user +test harness. + + +\subsection{TestResult Objects + \label{testresult-objects}} + +A \class{TestResult} object stores the results of a set of tests. The +\class{TestCase} and \class{TestSuite} classes ensure that results are +properly recorded; test authors do not need to worry about recording the +outcome of tests. + +Testing frameworks built on top of \refmodule{unittest} may want +access to the \class{TestResult} object generated by running a set of +tests for reporting purposes; a \class{TestResult} instance is +returned by the \method{TestRunner.run()} method for this purpose. + +\class{TestResult} instances have the following attributes that will +be of interest when inspecting the results of running a set of tests: + +\begin{memberdesc}[TestResult]{errors} + A list containing 2-tuples of \class{TestCase} instances and + strings holding formatted tracebacks. Each tuple represents a test which + raised an unexpected exception. + \versionchanged[Contains formatted tracebacks instead of + \function{sys.exc_info()} results]{2.2} +\end{memberdesc} + +\begin{memberdesc}[TestResult]{failures} + A list containing 2-tuples of \class{TestCase} instances and strings + holding formatted tracebacks. Each tuple represents a test where a failure + was explicitly signalled using the \method{TestCase.fail*()} or + \method{TestCase.assert*()} methods. + \versionchanged[Contains formatted tracebacks instead of + \function{sys.exc_info()} results]{2.2} +\end{memberdesc} + +\begin{memberdesc}[TestResult]{testsRun} + The total number of tests run so far. +\end{memberdesc} + +\begin{methoddesc}[TestResult]{wasSuccessful}{} + Returns \constant{True} if all tests run so far have passed, + otherwise returns \constant{False}. +\end{methoddesc} + +\begin{methoddesc}[TestResult]{stop}{} + This method can be called to signal that the set of tests being run + should be aborted by setting the \class{TestResult}'s \code{shouldStop} + attribute to \constant{True}. \class{TestRunner} objects should respect + this flag and return without running any additional tests. + + For example, this feature is used by the \class{TextTestRunner} class + to stop the test framework when the user signals an interrupt from + the keyboard. Interactive tools which provide \class{TestRunner} + implementations can use this in a similar manner. +\end{methoddesc} + + +The following methods of the \class{TestResult} class are used to +maintain the internal data structures, and may be extended in +subclasses to support additional reporting requirements. This is +particularly useful in building tools which support interactive +reporting while tests are being run. + +\begin{methoddesc}[TestResult]{startTest}{test} + Called when the test case \var{test} is about to be run. + + The default implementation simply increments the instance's + \code{testsRun} counter. +\end{methoddesc} + +\begin{methoddesc}[TestResult]{stopTest}{test} + Called after the test case \var{test} has been executed, regardless + of the outcome. + + The default implementation does nothing. +\end{methoddesc} + +\begin{methoddesc}[TestResult]{addError}{test, err} + Called when the test case \var{test} raises an unexpected exception + \var{err} is a tuple of the form returned by \function{sys.exc_info()}: + \code{(\var{type}, \var{value}, \var{traceback})}. + + The default implementation appends \code{(\var{test}, \var{err})} to + the instance's \code{errors} attribute. +\end{methoddesc} + +\begin{methoddesc}[TestResult]{addFailure}{test, err} + Called when the test case \var{test} signals a failure. + \var{err} is a tuple of the form returned by + \function{sys.exc_info()}: \code{(\var{type}, \var{value}, + \var{traceback})}. + + The default implementation appends \code{(\var{test}, \var{err})} to + the instance's \code{failures} attribute. +\end{methoddesc} + +\begin{methoddesc}[TestResult]{addSuccess}{test} + Called when the test case \var{test} succeeds. + + The default implementation does nothing. +\end{methoddesc} + + + +\subsection{TestLoader Objects + \label{testloader-objects}} + +The \class{TestLoader} class is used to create test suites from +classes and modules. Normally, there is no need to create an instance +of this class; the \refmodule{unittest} module provides an instance +that can be shared as \code{unittest.defaultTestLoader}. +Using a subclass or instance, however, allows customization of some +configurable properties. + +\class{TestLoader} objects have the following methods: + +\begin{methoddesc}[TestLoader]{loadTestsFromTestCase}{testCaseClass} + Return a suite of all tests cases contained in the + \class{TestCase}-derived \class{testCaseClass}. +\end{methoddesc} + +\begin{methoddesc}[TestLoader]{loadTestsFromModule}{module} + Return a suite of all tests cases contained in the given module. + This method searches \var{module} for classes derived from + \class{TestCase} and creates an instance of the class for each test + method defined for the class. + + \warning{While using a hierarchy of + \class{TestCase}-derived classes can be convenient in sharing + fixtures and helper functions, defining test methods on base classes + that are not intended to be instantiated directly does not play well + with this method. Doing so, however, can be useful when the + fixtures are different and defined in subclasses.} +\end{methoddesc} + +\begin{methoddesc}[TestLoader]{loadTestsFromName}{name\optional{, module}} + Return a suite of all tests cases given a string specifier. + + The specifier \var{name} is a ``dotted name'' that may resolve + either to a module, a test case class, a test method within a test + case class, a \class{TestSuite} instance, or a callable object which + returns a \class{TestCase} or \class{TestSuite} instance. These checks + are applied in the order listed here; that is, a method on a possible + test case class will be picked up as ``a test method within a test + case class'', rather than ``a callable object''. + + For example, if you have a module \module{SampleTests} containing a + \class{TestCase}-derived class \class{SampleTestCase} with three test + methods (\method{test_one()}, \method{test_two()}, and + \method{test_three()}), the specifier \code{'SampleTests.SampleTestCase'} + would cause this method to return a suite which will run all three test + methods. Using the specifier \code{'SampleTests.SampleTestCase.test_two'} + would cause it to return a test suite which will run only the + \method{test_two()} test method. The specifier can refer to modules + and packages which have not been imported; they will be imported as + a side-effect. + + The method optionally resolves \var{name} relative to the given + \var{module}. +\end{methoddesc} + +\begin{methoddesc}[TestLoader]{loadTestsFromNames}{names\optional{, module}} + Similar to \method{loadTestsFromName()}, but takes a sequence of + names rather than a single name. The return value is a test suite + which supports all the tests defined for each name. +\end{methoddesc} + +\begin{methoddesc}[TestLoader]{getTestCaseNames}{testCaseClass} + Return a sorted sequence of method names found within + \var{testCaseClass}; this should be a subclass of \class{TestCase}. +\end{methoddesc} + + +The following attributes of a \class{TestLoader} can be configured +either by subclassing or assignment on an instance: + +\begin{memberdesc}[TestLoader]{testMethodPrefix} + String giving the prefix of method names which will be interpreted + as test methods. The default value is \code{'test'}. + + This affects \method{getTestCaseNames()} and all the + \method{loadTestsFrom*()} methods. +\end{memberdesc} + +\begin{memberdesc}[TestLoader]{sortTestMethodsUsing} + Function to be used to compare method names when sorting them in + \method{getTestCaseNames()} and all the \method{loadTestsFrom*()} methods. + The default value is the built-in \function{cmp()} function; the attribute + can also be set to \constant{None} to disable the sort. +\end{memberdesc} + +\begin{memberdesc}[TestLoader]{suiteClass} + Callable object that constructs a test suite from a list of tests. + No methods on the resulting object are needed. The default value is + the \class{TestSuite} class. + + This affects all the \method{loadTestsFrom*()} methods. +\end{memberdesc} |