summaryrefslogtreecommitdiff
path: root/sys/src/cmd/python/Doc/lib/libunittest.tex
diff options
context:
space:
mode:
authorcinap_lenrek <cinap_lenrek@localhost>2011-05-03 11:25:13 +0000
committercinap_lenrek <cinap_lenrek@localhost>2011-05-03 11:25:13 +0000
commit458120dd40db6b4df55a4e96b650e16798ef06a0 (patch)
tree8f82685be24fef97e715c6f5ca4c68d34d5074ee /sys/src/cmd/python/Doc/lib/libunittest.tex
parent3a742c699f6806c1145aea5149bf15de15a0afd7 (diff)
add hg and python
Diffstat (limited to 'sys/src/cmd/python/Doc/lib/libunittest.tex')
-rw-r--r--sys/src/cmd/python/Doc/lib/libunittest.tex974
1 files changed, 974 insertions, 0 deletions
diff --git a/sys/src/cmd/python/Doc/lib/libunittest.tex b/sys/src/cmd/python/Doc/lib/libunittest.tex
new file mode 100644
index 000000000..f1e1033c1
--- /dev/null
+++ b/sys/src/cmd/python/Doc/lib/libunittest.tex
@@ -0,0 +1,974 @@
+\section{\module{unittest} ---
+ Unit testing framework}
+
+\declaremodule{standard}{unittest}
+\modulesynopsis{Unit testing framework for Python.}
+\moduleauthor{Steve Purcell}{stephen\textunderscore{}purcell@yahoo.com}
+\sectionauthor{Steve Purcell}{stephen\textunderscore{}purcell@yahoo.com}
+\sectionauthor{Fred L. Drake, Jr.}{fdrake@acm.org}
+\sectionauthor{Raymond Hettinger}{python@rcn.com}
+
+\versionadded{2.1}
+
+The Python unit testing framework, sometimes referred to as ``PyUnit,'' is
+a Python language version of JUnit, by Kent Beck and Erich Gamma.
+JUnit is, in turn, a Java version of Kent's Smalltalk testing
+framework. Each is the de facto standard unit testing framework for
+its respective language.
+
+\module{unittest} supports test automation, sharing of setup and shutdown
+code for tests, aggregation of tests into collections, and independence of
+the tests from the reporting framework. The \module{unittest} module
+provides classes that make it easy to support these qualities for a
+set of tests.
+
+To achieve this, \module{unittest} supports some important concepts:
+
+\begin{definitions}
+\term{test fixture}
+A \dfn{test fixture} represents the preparation needed to perform one
+or more tests, and any associate cleanup actions. This may involve,
+for example, creating temporary or proxy databases, directories, or
+starting a server process.
+
+\term{test case}
+A \dfn{test case} is the smallest unit of testing. It checks for a
+specific response to a particular set of inputs. \module{unittest}
+provides a base class, \class{TestCase}, which may be used to create
+new test cases.
+
+\term{test suite}
+A \dfn{test suite} is a collection of test cases, test suites, or
+both. It is used to aggregate tests that should be executed
+together.
+
+\term{test runner}
+A \dfn{test runner} is a component which orchestrates the execution of
+tests and provides the outcome to the user. The runner may use a
+graphical interface, a textual interface, or return a special value to
+indicate the results of executing the tests.
+\end{definitions}
+
+
+The test case and test fixture concepts are supported through the
+\class{TestCase} and \class{FunctionTestCase} classes; the former
+should be used when creating new tests, and the latter can be used when
+integrating existing test code with a \module{unittest}-driven framework.
+When building test fixtures using \class{TestCase}, the \method{setUp()}
+and \method{tearDown()} methods can be overridden to provide
+initialization and cleanup for the fixture. With
+\class{FunctionTestCase}, existing functions can be passed to the
+constructor for these purposes. When the test is run, the
+fixture initialization is run first; if it succeeds, the cleanup
+method is run after the test has been executed, regardless of the
+outcome of the test. Each instance of the \class{TestCase} will only
+be used to run a single test method, so a new fixture is created for
+each test.
+
+Test suites are implemented by the \class{TestSuite} class. This
+class allows individual tests and test suites to be aggregated; when
+the suite is executed, all tests added directly to the suite and in
+``child'' test suites are run.
+
+A test runner is an object that provides a single method,
+\method{run()}, which accepts a \class{TestCase} or \class{TestSuite}
+object as a parameter, and returns a result object. The class
+\class{TestResult} is provided for use as the result object.
+\module{unittest} provides the \class{TextTestRunner} as an example
+test runner which reports test results on the standard error stream by
+default. Alternate runners can be implemented for other environments
+(such as graphical environments) without any need to derive from a
+specific class.
+
+
+\begin{seealso}
+ \seemodule{doctest}{Another test-support module with a very
+ different flavor.}
+ \seetitle[http://www.XProgramming.com/testfram.htm]{Simple Smalltalk
+ Testing: With Patterns}{Kent Beck's original paper on
+ testing frameworks using the pattern shared by
+ \module{unittest}.}
+\end{seealso}
+
+
+\subsection{Basic example \label{minimal-example}}
+
+The \module{unittest} module provides a rich set of tools for
+constructing and running tests. This section demonstrates that a
+small subset of the tools suffice to meet the needs of most users.
+
+Here is a short script to test three functions from the
+\refmodule{random} module:
+
+\begin{verbatim}
+import random
+import unittest
+
+class TestSequenceFunctions(unittest.TestCase):
+
+ def setUp(self):
+ self.seq = range(10)
+
+ def testshuffle(self):
+ # make sure the shuffled sequence does not lose any elements
+ random.shuffle(self.seq)
+ self.seq.sort()
+ self.assertEqual(self.seq, range(10))
+
+ def testchoice(self):
+ element = random.choice(self.seq)
+ self.assert_(element in self.seq)
+
+ def testsample(self):
+ self.assertRaises(ValueError, random.sample, self.seq, 20)
+ for element in random.sample(self.seq, 5):
+ self.assert_(element in self.seq)
+
+if __name__ == '__main__':
+ unittest.main()
+\end{verbatim}
+
+A testcase is created by subclassing \class{unittest.TestCase}.
+The three individual tests are defined with methods whose names start with
+the letters \samp{test}. This naming convention informs the test runner
+about which methods represent tests.
+
+The crux of each test is a call to \method{assertEqual()} to
+check for an expected result; \method{assert_()} to verify a condition;
+or \method{assertRaises()} to verify that an expected exception gets
+raised. These methods are used instead of the \keyword{assert} statement
+so the test runner can accumulate all test results and produce a report.
+
+When a \method{setUp()} method is defined, the test runner will run that
+method prior to each test. Likewise, if a \method{tearDown()} method is
+defined, the test runner will invoke that method after each test. In the
+example, \method{setUp()} was used to create a fresh sequence for each test.
+
+The final block shows a simple way to run the tests.
+\function{unittest.main()} provides a command line interface to the
+test script. When run from the command line, the above script
+produces an output that looks like this:
+
+\begin{verbatim}
+...
+----------------------------------------------------------------------
+Ran 3 tests in 0.000s
+
+OK
+\end{verbatim}
+
+Instead of \function{unittest.main()}, there are other ways to run the tests
+with a finer level of control, less terse output, and no requirement to be
+run from the command line. For example, the last two lines may be replaced
+with:
+
+\begin{verbatim}
+suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions)
+unittest.TextTestRunner(verbosity=2).run(suite)
+\end{verbatim}
+
+Running the revised script from the interpreter or another script
+produces the following output:
+
+\begin{verbatim}
+testchoice (__main__.TestSequenceFunctions) ... ok
+testsample (__main__.TestSequenceFunctions) ... ok
+testshuffle (__main__.TestSequenceFunctions) ... ok
+
+----------------------------------------------------------------------
+Ran 3 tests in 0.110s
+
+OK
+\end{verbatim}
+
+The above examples show the most commonly used \module{unittest} features
+which are sufficient to meet many everyday testing needs. The remainder
+of the documentation explores the full feature set from first principles.
+
+
+\subsection{Organizing test code
+ \label{organizing-tests}}
+
+The basic building blocks of unit testing are \dfn{test cases} ---
+single scenarios that must be set up and checked for correctness. In
+\module{unittest}, test cases are represented by instances of
+\module{unittest}'s \class{TestCase} class. To make
+your own test cases you must write subclasses of \class{TestCase}, or
+use \class{FunctionTestCase}.
+
+An instance of a \class{TestCase}-derived class is an object that can
+completely run a single test method, together with optional set-up
+and tidy-up code.
+
+The testing code of a \class{TestCase} instance should be entirely
+self contained, such that it can be run either in isolation or in
+arbitrary combination with any number of other test cases.
+
+The simplest \class{TestCase} subclass will simply override the
+\method{runTest()} method in order to perform specific testing code:
+
+\begin{verbatim}
+import unittest
+
+class DefaultWidgetSizeTestCase(unittest.TestCase):
+ def runTest(self):
+ widget = Widget('The widget')
+ self.assertEqual(widget.size(), (50, 50), 'incorrect default size')
+\end{verbatim}
+
+Note that in order to test something, we use the one of the
+\method{assert*()} or \method{fail*()} methods provided by the
+\class{TestCase} base class. If the test fails, an exception will be
+raised, and \module{unittest} will identify the test case as a
+\dfn{failure}. Any other exceptions will be treated as \dfn{errors}.
+This helps you identify where the problem is: \dfn{failures} are caused by
+incorrect results - a 5 where you expected a 6. \dfn{Errors} are caused by
+incorrect code - e.g., a \exception{TypeError} caused by an incorrect
+function call.
+
+The way to run a test case will be described later. For now, note
+that to construct an instance of such a test case, we call its
+constructor without arguments:
+
+\begin{verbatim}
+testCase = DefaultWidgetSizeTestCase()
+\end{verbatim}
+
+Now, such test cases can be numerous, and their set-up can be
+repetitive. In the above case, constructing a \class{Widget} in each of
+100 Widget test case subclasses would mean unsightly duplication.
+
+Luckily, we can factor out such set-up code by implementing a method
+called \method{setUp()}, which the testing framework will
+automatically call for us when we run the test:
+
+\begin{verbatim}
+import unittest
+
+class SimpleWidgetTestCase(unittest.TestCase):
+ def setUp(self):
+ self.widget = Widget('The widget')
+
+class DefaultWidgetSizeTestCase(SimpleWidgetTestCase):
+ def runTest(self):
+ self.failUnless(self.widget.size() == (50,50),
+ 'incorrect default size')
+
+class WidgetResizeTestCase(SimpleWidgetTestCase):
+ def runTest(self):
+ self.widget.resize(100,150)
+ self.failUnless(self.widget.size() == (100,150),
+ 'wrong size after resize')
+\end{verbatim}
+
+If the \method{setUp()} method raises an exception while the test is
+running, the framework will consider the test to have suffered an
+error, and the \method{runTest()} method will not be executed.
+
+Similarly, we can provide a \method{tearDown()} method that tidies up
+after the \method{runTest()} method has been run:
+
+\begin{verbatim}
+import unittest
+
+class SimpleWidgetTestCase(unittest.TestCase):
+ def setUp(self):
+ self.widget = Widget('The widget')
+
+ def tearDown(self):
+ self.widget.dispose()
+ self.widget = None
+\end{verbatim}
+
+If \method{setUp()} succeeded, the \method{tearDown()} method will be
+run whether \method{runTest()} succeeded or not.
+
+Such a working environment for the testing code is called a
+\dfn{fixture}.
+
+Often, many small test cases will use the same fixture. In this case,
+we would end up subclassing \class{SimpleWidgetTestCase} into many
+small one-method classes such as
+\class{DefaultWidgetSizeTestCase}. This is time-consuming and
+discouraging, so in the same vein as JUnit, \module{unittest} provides
+a simpler mechanism:
+
+\begin{verbatim}
+import unittest
+
+class WidgetTestCase(unittest.TestCase):
+ def setUp(self):
+ self.widget = Widget('The widget')
+
+ def tearDown(self):
+ self.widget.dispose()
+ self.widget = None
+
+ def testDefaultSize(self):
+ self.failUnless(self.widget.size() == (50,50),
+ 'incorrect default size')
+
+ def testResize(self):
+ self.widget.resize(100,150)
+ self.failUnless(self.widget.size() == (100,150),
+ 'wrong size after resize')
+\end{verbatim}
+
+Here we have not provided a \method{runTest()} method, but have
+instead provided two different test methods. Class instances will now
+each run one of the \method{test*()} methods, with \code{self.widget}
+created and destroyed separately for each instance. When creating an
+instance we must specify the test method it is to run. We do this by
+passing the method name in the constructor:
+
+\begin{verbatim}
+defaultSizeTestCase = WidgetTestCase('testDefaultSize')
+resizeTestCase = WidgetTestCase('testResize')
+\end{verbatim}
+
+Test case instances are grouped together according to the features
+they test. \module{unittest} provides a mechanism for this: the
+\dfn{test suite}, represented by \module{unittest}'s \class{TestSuite}
+class:
+
+\begin{verbatim}
+widgetTestSuite = unittest.TestSuite()
+widgetTestSuite.addTest(WidgetTestCase('testDefaultSize'))
+widgetTestSuite.addTest(WidgetTestCase('testResize'))
+\end{verbatim}
+
+For the ease of running tests, as we will see later, it is a good
+idea to provide in each test module a callable object that returns a
+pre-built test suite:
+
+\begin{verbatim}
+def suite():
+ suite = unittest.TestSuite()
+ suite.addTest(WidgetTestCase('testDefaultSize'))
+ suite.addTest(WidgetTestCase('testResize'))
+ return suite
+\end{verbatim}
+
+or even:
+
+\begin{verbatim}
+def suite():
+ tests = ['testDefaultSize', 'testResize']
+
+ return unittest.TestSuite(map(WidgetTestCase, tests))
+\end{verbatim}
+
+Since it is a common pattern to create a \class{TestCase} subclass
+with many similarly named test functions, \module{unittest} provides a
+\class{TestLoader} class that can be used to automate the process of
+creating a test suite and populating it with individual tests.
+For example,
+
+\begin{verbatim}
+suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
+\end{verbatim}
+
+will create a test suite that will run
+\code{WidgetTestCase.testDefaultSize()} and \code{WidgetTestCase.testResize}.
+\class{TestLoader} uses the \code{'test'} method name prefix to identify
+test methods automatically.
+
+Note that the order in which the various test cases will be run is
+determined by sorting the test function names with the built-in
+\function{cmp()} function.
+
+Often it is desirable to group suites of test cases together, so as to
+run tests for the whole system at once. This is easy, since
+\class{TestSuite} instances can be added to a \class{TestSuite} just
+as \class{TestCase} instances can be added to a \class{TestSuite}:
+
+\begin{verbatim}
+suite1 = module1.TheTestSuite()
+suite2 = module2.TheTestSuite()
+alltests = unittest.TestSuite([suite1, suite2])
+\end{verbatim}
+
+You can place the definitions of test cases and test suites in the
+same modules as the code they are to test (such as \file{widget.py}),
+but there are several advantages to placing the test code in a
+separate module, such as \file{test_widget.py}:
+
+\begin{itemize}
+ \item The test module can be run standalone from the command line.
+ \item The test code can more easily be separated from shipped code.
+ \item There is less temptation to change test code to fit the code
+ it tests without a good reason.
+ \item Test code should be modified much less frequently than the
+ code it tests.
+ \item Tested code can be refactored more easily.
+ \item Tests for modules written in C must be in separate modules
+ anyway, so why not be consistent?
+ \item If the testing strategy changes, there is no need to change
+ the source code.
+\end{itemize}
+
+
+\subsection{Re-using old test code
+ \label{legacy-unit-tests}}
+
+Some users will find that they have existing test code that they would
+like to run from \module{unittest}, without converting every old test
+function to a \class{TestCase} subclass.
+
+For this reason, \module{unittest} provides a \class{FunctionTestCase}
+class. This subclass of \class{TestCase} can be used to wrap an existing
+test function. Set-up and tear-down functions can also be provided.
+
+Given the following test function:
+
+\begin{verbatim}
+def testSomething():
+ something = makeSomething()
+ assert something.name is not None
+ # ...
+\end{verbatim}
+
+one can create an equivalent test case instance as follows:
+
+\begin{verbatim}
+testcase = unittest.FunctionTestCase(testSomething)
+\end{verbatim}
+
+If there are additional set-up and tear-down methods that should be
+called as part of the test case's operation, they can also be provided
+like so:
+
+\begin{verbatim}
+testcase = unittest.FunctionTestCase(testSomething,
+ setUp=makeSomethingDB,
+ tearDown=deleteSomethingDB)
+\end{verbatim}
+
+To make migrating existing test suites easier, \module{unittest}
+supports tests raising \exception{AssertionError} to indicate test failure.
+However, it is recommended that you use the explicit
+\method{TestCase.fail*()} and \method{TestCase.assert*()} methods instead,
+as future versions of \module{unittest} may treat \exception{AssertionError}
+differently.
+
+\note{Even though \class{FunctionTestCase} can be used to quickly convert
+an existing test base over to a \module{unittest}-based system, this
+approach is not recommended. Taking the time to set up proper
+\class{TestCase} subclasses will make future test refactorings infinitely
+easier.}
+
+
+
+\subsection{Classes and functions
+ \label{unittest-contents}}
+
+\begin{classdesc}{TestCase}{\optional{methodName}}
+ Instances of the \class{TestCase} class represent the smallest
+ testable units in the \module{unittest} universe. This class is
+ intended to be used as a base class, with specific tests being
+ implemented by concrete subclasses. This class implements the
+ interface needed by the test runner to allow it to drive the
+ test, and methods that the test code can use to check for and
+ report various kinds of failure.
+
+ Each instance of \class{TestCase} will run a single test method:
+ the method named \var{methodName}. If you remember, we had an
+ earlier example that went something like this:
+
+ \begin{verbatim}
+ def suite():
+ suite = unittest.TestSuite()
+ suite.addTest(WidgetTestCase('testDefaultSize'))
+ suite.addTest(WidgetTestCase('testResize'))
+ return suite
+ \end{verbatim}
+
+ Here, we create two instances of \class{WidgetTestCase}, each of
+ which runs a single test.
+
+ \var{methodName} defaults to \code{'runTest'}.
+\end{classdesc}
+
+\begin{classdesc}{FunctionTestCase}{testFunc\optional{,
+ setUp\optional{, tearDown\optional{, description}}}}
+ This class implements the portion of the \class{TestCase} interface
+ which allows the test runner to drive the test, but does not provide
+ the methods which test code can use to check and report errors.
+ This is used to create test cases using legacy test code, allowing
+ it to be integrated into a \refmodule{unittest}-based test
+ framework.
+\end{classdesc}
+
+\begin{classdesc}{TestSuite}{\optional{tests}}
+ This class represents an aggregation of individual tests cases and
+ test suites. The class presents the interface needed by the test
+ runner to allow it to be run as any other test case. Running a
+ \class{TestSuite} instance is the same as iterating over the suite,
+ running each test individually.
+
+ If \var{tests} is given, it must be an iterable of individual test cases or
+ other test suites that will be used to build the suite initially.
+ Additional methods are provided to add test cases and suites to the
+ collection later on.
+\end{classdesc}
+
+\begin{classdesc}{TestLoader}{}
+ This class is responsible for loading tests according to various
+ criteria and returning them wrapped in a \class{TestSuite}.
+ It can load all tests within a given module or \class{TestCase}
+ subclass.
+\end{classdesc}
+
+\begin{classdesc}{TestResult}{}
+ This class is used to compile information about which tests have succeeded
+ and which have failed.
+\end{classdesc}
+
+\begin{datadesc}{defaultTestLoader}
+ Instance of the \class{TestLoader} class intended to be shared. If no
+ customization of the \class{TestLoader} is needed, this instance can
+ be used instead of repeatedly creating new instances.
+\end{datadesc}
+
+\begin{classdesc}{TextTestRunner}{\optional{stream\optional{,
+ descriptions\optional{, verbosity}}}}
+ A basic test runner implementation which prints results on standard
+ error. It has a few configurable parameters, but is essentially
+ very simple. Graphical applications which run test suites should
+ provide alternate implementations.
+\end{classdesc}
+
+\begin{funcdesc}{main}{\optional{module\optional{,
+ defaultTest\optional{, argv\optional{,
+ testRunner\optional{, testLoader}}}}}}
+ A command-line program that runs a set of tests; this is primarily
+ for making test modules conveniently executable. The simplest use
+ for this function is to include the following line at the end of a
+ test script:
+
+\begin{verbatim}
+if __name__ == '__main__':
+ unittest.main()
+\end{verbatim}
+\end{funcdesc}
+
+In some cases, the existing tests may have been written using the
+\refmodule{doctest} module. If so, that module provides a
+\class{DocTestSuite} class that can automatically build
+\class{unittest.TestSuite} instances from the existing
+\module{doctest}-based tests.
+\versionadded{2.3}
+
+
+\subsection{TestCase Objects
+ \label{testcase-objects}}
+
+Each \class{TestCase} instance represents a single test, but each
+concrete subclass may be used to define multiple tests --- the
+concrete class represents a single test fixture. The fixture is
+created and cleaned up for each test case.
+
+\class{TestCase} instances provide three groups of methods: one group
+used to run the test, another used by the test implementation to
+check conditions and report failures, and some inquiry methods
+allowing information about the test itself to be gathered.
+
+Methods in the first group (running the test) are:
+
+\begin{methoddesc}[TestCase]{setUp}{}
+ Method called to prepare the test fixture. This is called
+ immediately before calling the test method; any exception raised by
+ this method will be considered an error rather than a test failure.
+ The default implementation does nothing.
+\end{methoddesc}
+
+\begin{methoddesc}[TestCase]{tearDown}{}
+ Method called immediately after the test method has been called and
+ the result recorded. This is called even if the test method raised
+ an exception, so the implementation in subclasses may need to be
+ particularly careful about checking internal state. Any exception
+ raised by this method will be considered an error rather than a test
+ failure. This method will only be called if the \method{setUp()}
+ succeeds, regardless of the outcome of the test method.
+ The default implementation does nothing.
+\end{methoddesc}
+
+\begin{methoddesc}[TestCase]{run}{\optional{result}}
+ Run the test, collecting the result into the test result object
+ passed as \var{result}. If \var{result} is omitted or \constant{None},
+ a temporary result object is created (by calling the
+ \method{defaultTestCase()} method) and used; this result object is not
+ returned to \method{run()}'s caller.
+
+ The same effect may be had by simply calling the \class{TestCase}
+ instance.
+\end{methoddesc}
+
+\begin{methoddesc}[TestCase]{debug}{}
+ Run the test without collecting the result. This allows exceptions
+ raised by the test to be propagated to the caller, and can be used
+ to support running tests under a debugger.
+\end{methoddesc}
+
+
+The test code can use any of the following methods to check for and
+report failures.
+
+\begin{methoddesc}[TestCase]{assert_}{expr\optional{, msg}}
+\methodline{failUnless}{expr\optional{, msg}}
+ Signal a test failure if \var{expr} is false; the explanation for
+ the error will be \var{msg} if given, otherwise it will be
+ \constant{None}.
+\end{methoddesc}
+
+\begin{methoddesc}[TestCase]{assertEqual}{first, second\optional{, msg}}
+\methodline{failUnlessEqual}{first, second\optional{, msg}}
+ Test that \var{first} and \var{second} are equal. If the values do
+ not compare equal, the test will fail with the explanation given by
+ \var{msg}, or \constant{None}. Note that using \method{failUnlessEqual()}
+ improves upon doing the comparison as the first parameter to
+ \method{failUnless()}: the default value for \var{msg} can be
+ computed to include representations of both \var{first} and
+ \var{second}.
+\end{methoddesc}
+
+\begin{methoddesc}[TestCase]{assertNotEqual}{first, second\optional{, msg}}
+\methodline{failIfEqual}{first, second\optional{, msg}}
+ Test that \var{first} and \var{second} are not equal. If the values
+ do compare equal, the test will fail with the explanation given by
+ \var{msg}, or \constant{None}. Note that using \method{failIfEqual()}
+ improves upon doing the comparison as the first parameter to
+ \method{failUnless()} is that the default value for \var{msg} can be
+ computed to include representations of both \var{first} and
+ \var{second}.
+\end{methoddesc}
+
+\begin{methoddesc}[TestCase]{assertAlmostEqual}{first, second\optional{,
+ places\optional{, msg}}}
+\methodline{failUnlessAlmostEqual}{first, second\optional{,
+ places\optional{, msg}}}
+ Test that \var{first} and \var{second} are approximately equal
+ by computing the difference, rounding to the given number of \var{places},
+ and comparing to zero. Note that comparing a given number of decimal places
+ is not the same as comparing a given number of significant digits.
+ If the values do not compare equal, the test will fail with the explanation
+ given by \var{msg}, or \constant{None}.
+\end{methoddesc}
+
+\begin{methoddesc}[TestCase]{assertNotAlmostEqual}{first, second\optional{,
+ places\optional{, msg}}}
+\methodline{failIfAlmostEqual}{first, second\optional{,
+ places\optional{, msg}}}
+ Test that \var{first} and \var{second} are not approximately equal
+ by computing the difference, rounding to the given number of \var{places},
+ and comparing to zero. Note that comparing a given number of decimal places
+ is not the same as comparing a given number of significant digits.
+ If the values do not compare equal, the test will fail with the explanation
+ given by \var{msg}, or \constant{None}.
+\end{methoddesc}
+
+\begin{methoddesc}[TestCase]{assertRaises}{exception, callable, \moreargs}
+\methodline{failUnlessRaises}{exception, callable, \moreargs}
+ Test that an exception is raised when \var{callable} is called with
+ any positional or keyword arguments that are also passed to
+ \method{assertRaises()}. The test passes if \var{exception} is
+ raised, is an error if another exception is raised, or fails if no
+ exception is raised. To catch any of a group of exceptions, a tuple
+ containing the exception classes may be passed as \var{exception}.
+\end{methoddesc}
+
+\begin{methoddesc}[TestCase]{failIf}{expr\optional{, msg}}
+ The inverse of the \method{failUnless()} method is the
+ \method{failIf()} method. This signals a test failure if \var{expr}
+ is true, with \var{msg} or \constant{None} for the error message.
+\end{methoddesc}
+
+\begin{methoddesc}[TestCase]{fail}{\optional{msg}}
+ Signals a test failure unconditionally, with \var{msg} or
+ \constant{None} for the error message.
+\end{methoddesc}
+
+\begin{memberdesc}[TestCase]{failureException}
+ This class attribute gives the exception raised by the
+ \method{test()} method. If a test framework needs to use a
+ specialized exception, possibly to carry additional information, it
+ must subclass this exception in order to ``play fair'' with the
+ framework. The initial value of this attribute is
+ \exception{AssertionError}.
+\end{memberdesc}
+
+
+Testing frameworks can use the following methods to collect
+information on the test:
+
+\begin{methoddesc}[TestCase]{countTestCases}{}
+ Return the number of tests represented by this test object. For
+ \class{TestCase} instances, this will always be \code{1}.
+\end{methoddesc}
+
+\begin{methoddesc}[TestCase]{defaultTestResult}{}
+ Return an instance of the test result class that should be used
+ for this test case class (if no other result instance is provided
+ to the \method{run()} method).
+
+ For \class{TestCase} instances, this will always be an instance of
+ \class{TestResult}; subclasses of \class{TestCase} should
+ override this as necessary.
+\end{methoddesc}
+
+\begin{methoddesc}[TestCase]{id}{}
+ Return a string identifying the specific test case. This is usually
+ the full name of the test method, including the module and class
+ name.
+\end{methoddesc}
+
+\begin{methoddesc}[TestCase]{shortDescription}{}
+ Returns a one-line description of the test, or \constant{None} if no
+ description has been provided. The default implementation of this
+ method returns the first line of the test method's docstring, if
+ available, or \constant{None}.
+\end{methoddesc}
+
+
+\subsection{TestSuite Objects
+ \label{testsuite-objects}}
+
+\class{TestSuite} objects behave much like \class{TestCase} objects,
+except they do not actually implement a test. Instead, they are used
+to aggregate tests into groups of tests that should be run together.
+Some additional methods are available to add tests to \class{TestSuite}
+instances:
+
+\begin{methoddesc}[TestSuite]{addTest}{test}
+ Add a \class{TestCase} or \class{TestSuite} to the suite.
+\end{methoddesc}
+
+\begin{methoddesc}[TestSuite]{addTests}{tests}
+ Add all the tests from an iterable of \class{TestCase} and
+ \class{TestSuite} instances to this test suite.
+
+ This is equivalent to iterating over \var{tests}, calling
+ \method{addTest()} for each element.
+\end{methoddesc}
+
+\class{TestSuite} shares the following methods with \class{TestCase}:
+
+\begin{methoddesc}[TestSuite]{run}{result}
+ Run the tests associated with this suite, collecting the result into
+ the test result object passed as \var{result}. Note that unlike
+ \method{TestCase.run()}, \method{TestSuite.run()} requires the
+ result object to be passed in.
+\end{methoddesc}
+
+\begin{methoddesc}[TestSuite]{debug}{}
+ Run the tests associated with this suite without collecting the result.
+ This allows exceptions raised by the test to be propagated to the caller
+ and can be used to support running tests under a debugger.
+\end{methoddesc}
+
+\begin{methoddesc}[TestSuite]{countTestCases}{}
+ Return the number of tests represented by this test object, including
+ all individual tests and sub-suites.
+\end{methoddesc}
+
+In the typical usage of a \class{TestSuite} object, the \method{run()}
+method is invoked by a \class{TestRunner} rather than by the end-user
+test harness.
+
+
+\subsection{TestResult Objects
+ \label{testresult-objects}}
+
+A \class{TestResult} object stores the results of a set of tests. The
+\class{TestCase} and \class{TestSuite} classes ensure that results are
+properly recorded; test authors do not need to worry about recording the
+outcome of tests.
+
+Testing frameworks built on top of \refmodule{unittest} may want
+access to the \class{TestResult} object generated by running a set of
+tests for reporting purposes; a \class{TestResult} instance is
+returned by the \method{TestRunner.run()} method for this purpose.
+
+\class{TestResult} instances have the following attributes that will
+be of interest when inspecting the results of running a set of tests:
+
+\begin{memberdesc}[TestResult]{errors}
+ A list containing 2-tuples of \class{TestCase} instances and
+ strings holding formatted tracebacks. Each tuple represents a test which
+ raised an unexpected exception.
+ \versionchanged[Contains formatted tracebacks instead of
+ \function{sys.exc_info()} results]{2.2}
+\end{memberdesc}
+
+\begin{memberdesc}[TestResult]{failures}
+ A list containing 2-tuples of \class{TestCase} instances and strings
+ holding formatted tracebacks. Each tuple represents a test where a failure
+ was explicitly signalled using the \method{TestCase.fail*()} or
+ \method{TestCase.assert*()} methods.
+ \versionchanged[Contains formatted tracebacks instead of
+ \function{sys.exc_info()} results]{2.2}
+\end{memberdesc}
+
+\begin{memberdesc}[TestResult]{testsRun}
+ The total number of tests run so far.
+\end{memberdesc}
+
+\begin{methoddesc}[TestResult]{wasSuccessful}{}
+ Returns \constant{True} if all tests run so far have passed,
+ otherwise returns \constant{False}.
+\end{methoddesc}
+
+\begin{methoddesc}[TestResult]{stop}{}
+ This method can be called to signal that the set of tests being run
+ should be aborted by setting the \class{TestResult}'s \code{shouldStop}
+ attribute to \constant{True}. \class{TestRunner} objects should respect
+ this flag and return without running any additional tests.
+
+ For example, this feature is used by the \class{TextTestRunner} class
+ to stop the test framework when the user signals an interrupt from
+ the keyboard. Interactive tools which provide \class{TestRunner}
+ implementations can use this in a similar manner.
+\end{methoddesc}
+
+
+The following methods of the \class{TestResult} class are used to
+maintain the internal data structures, and may be extended in
+subclasses to support additional reporting requirements. This is
+particularly useful in building tools which support interactive
+reporting while tests are being run.
+
+\begin{methoddesc}[TestResult]{startTest}{test}
+ Called when the test case \var{test} is about to be run.
+
+ The default implementation simply increments the instance's
+ \code{testsRun} counter.
+\end{methoddesc}
+
+\begin{methoddesc}[TestResult]{stopTest}{test}
+ Called after the test case \var{test} has been executed, regardless
+ of the outcome.
+
+ The default implementation does nothing.
+\end{methoddesc}
+
+\begin{methoddesc}[TestResult]{addError}{test, err}
+ Called when the test case \var{test} raises an unexpected exception
+ \var{err} is a tuple of the form returned by \function{sys.exc_info()}:
+ \code{(\var{type}, \var{value}, \var{traceback})}.
+
+ The default implementation appends \code{(\var{test}, \var{err})} to
+ the instance's \code{errors} attribute.
+\end{methoddesc}
+
+\begin{methoddesc}[TestResult]{addFailure}{test, err}
+ Called when the test case \var{test} signals a failure.
+ \var{err} is a tuple of the form returned by
+ \function{sys.exc_info()}: \code{(\var{type}, \var{value},
+ \var{traceback})}.
+
+ The default implementation appends \code{(\var{test}, \var{err})} to
+ the instance's \code{failures} attribute.
+\end{methoddesc}
+
+\begin{methoddesc}[TestResult]{addSuccess}{test}
+ Called when the test case \var{test} succeeds.
+
+ The default implementation does nothing.
+\end{methoddesc}
+
+
+
+\subsection{TestLoader Objects
+ \label{testloader-objects}}
+
+The \class{TestLoader} class is used to create test suites from
+classes and modules. Normally, there is no need to create an instance
+of this class; the \refmodule{unittest} module provides an instance
+that can be shared as \code{unittest.defaultTestLoader}.
+Using a subclass or instance, however, allows customization of some
+configurable properties.
+
+\class{TestLoader} objects have the following methods:
+
+\begin{methoddesc}[TestLoader]{loadTestsFromTestCase}{testCaseClass}
+ Return a suite of all tests cases contained in the
+ \class{TestCase}-derived \class{testCaseClass}.
+\end{methoddesc}
+
+\begin{methoddesc}[TestLoader]{loadTestsFromModule}{module}
+ Return a suite of all tests cases contained in the given module.
+ This method searches \var{module} for classes derived from
+ \class{TestCase} and creates an instance of the class for each test
+ method defined for the class.
+
+ \warning{While using a hierarchy of
+ \class{TestCase}-derived classes can be convenient in sharing
+ fixtures and helper functions, defining test methods on base classes
+ that are not intended to be instantiated directly does not play well
+ with this method. Doing so, however, can be useful when the
+ fixtures are different and defined in subclasses.}
+\end{methoddesc}
+
+\begin{methoddesc}[TestLoader]{loadTestsFromName}{name\optional{, module}}
+ Return a suite of all tests cases given a string specifier.
+
+ The specifier \var{name} is a ``dotted name'' that may resolve
+ either to a module, a test case class, a test method within a test
+ case class, a \class{TestSuite} instance, or a callable object which
+ returns a \class{TestCase} or \class{TestSuite} instance. These checks
+ are applied in the order listed here; that is, a method on a possible
+ test case class will be picked up as ``a test method within a test
+ case class'', rather than ``a callable object''.
+
+ For example, if you have a module \module{SampleTests} containing a
+ \class{TestCase}-derived class \class{SampleTestCase} with three test
+ methods (\method{test_one()}, \method{test_two()}, and
+ \method{test_three()}), the specifier \code{'SampleTests.SampleTestCase'}
+ would cause this method to return a suite which will run all three test
+ methods. Using the specifier \code{'SampleTests.SampleTestCase.test_two'}
+ would cause it to return a test suite which will run only the
+ \method{test_two()} test method. The specifier can refer to modules
+ and packages which have not been imported; they will be imported as
+ a side-effect.
+
+ The method optionally resolves \var{name} relative to the given
+ \var{module}.
+\end{methoddesc}
+
+\begin{methoddesc}[TestLoader]{loadTestsFromNames}{names\optional{, module}}
+ Similar to \method{loadTestsFromName()}, but takes a sequence of
+ names rather than a single name. The return value is a test suite
+ which supports all the tests defined for each name.
+\end{methoddesc}
+
+\begin{methoddesc}[TestLoader]{getTestCaseNames}{testCaseClass}
+ Return a sorted sequence of method names found within
+ \var{testCaseClass}; this should be a subclass of \class{TestCase}.
+\end{methoddesc}
+
+
+The following attributes of a \class{TestLoader} can be configured
+either by subclassing or assignment on an instance:
+
+\begin{memberdesc}[TestLoader]{testMethodPrefix}
+ String giving the prefix of method names which will be interpreted
+ as test methods. The default value is \code{'test'}.
+
+ This affects \method{getTestCaseNames()} and all the
+ \method{loadTestsFrom*()} methods.
+\end{memberdesc}
+
+\begin{memberdesc}[TestLoader]{sortTestMethodsUsing}
+ Function to be used to compare method names when sorting them in
+ \method{getTestCaseNames()} and all the \method{loadTestsFrom*()} methods.
+ The default value is the built-in \function{cmp()} function; the attribute
+ can also be set to \constant{None} to disable the sort.
+\end{memberdesc}
+
+\begin{memberdesc}[TestLoader]{suiteClass}
+ Callable object that constructs a test suite from a list of tests.
+ No methods on the resulting object are needed. The default value is
+ the \class{TestSuite} class.
+
+ This affects all the \method{loadTestsFrom*()} methods.
+\end{memberdesc}