diff options
author | cinap_lenrek <cinap_lenrek@localhost> | 2011-05-03 11:25:13 +0000 |
---|---|---|
committer | cinap_lenrek <cinap_lenrek@localhost> | 2011-05-03 11:25:13 +0000 |
commit | 458120dd40db6b4df55a4e96b650e16798ef06a0 (patch) | |
tree | 8f82685be24fef97e715c6f5ca4c68d34d5074ee /sys/src/cmd/python/Doc/lib/libtest.tex | |
parent | 3a742c699f6806c1145aea5149bf15de15a0afd7 (diff) |
add hg and python
Diffstat (limited to 'sys/src/cmd/python/Doc/lib/libtest.tex')
-rw-r--r-- | sys/src/cmd/python/Doc/lib/libtest.tex | 278 |
1 files changed, 278 insertions, 0 deletions
diff --git a/sys/src/cmd/python/Doc/lib/libtest.tex b/sys/src/cmd/python/Doc/lib/libtest.tex new file mode 100644 index 000000000..54a24b1c9 --- /dev/null +++ b/sys/src/cmd/python/Doc/lib/libtest.tex @@ -0,0 +1,278 @@ +\section{\module{test} --- + Regression tests package for Python} + +\declaremodule{standard}{test} +\sectionauthor{Brett Cannon}{brett@python.org} +\modulesynopsis{Regression tests package containing the testing suite + for Python.} + + +The \module{test} package contains all regression tests for Python as +well as the modules \module{test.test_support} and +\module{test.regrtest}. \module{test.test_support} is used to enhance +your tests while \module{test.regrtest} drives the testing suite. + +Each module in the \module{test} package whose name starts with +\samp{test_} is a testing suite for a specific module or feature. +All new tests should be written using the \refmodule{unittest} module; +using \refmodule{unittest} is not required but makes the tests more +flexible and maintenance of the tests easier. Some older tests are +written to use \refmodule{doctest} and a ``traditional'' testing +style; these styles of tests will not be covered. + +\begin{seealso} +\seemodule{unittest}{Writing PyUnit regression tests.} +\seemodule{doctest}{Tests embedded in documentation strings.} +\end{seealso} + + +\subsection{Writing Unit Tests for the \module{test} package% + \label{writing-tests}} + +It is preferred that tests for the \module{test} package use the +\refmodule{unittest} module and follow a few guidelines. +One is to name the test module by starting it with \samp{test_} and end it with +the name of the module being tested. +The test methods in the test module should start with \samp{test_} and end with +a description of what the method is testing. +This is needed so that the methods are recognized by the test driver as +test methods. +Also, no documentation string for the method should be included. +A comment (such as +\samp{\# Tests function returns only True or False}) should be used to provide +documentation for test methods. +This is done because documentation strings get printed out if they exist and +thus what test is being run is not stated. + +A basic boilerplate is often used: + +\begin{verbatim} +import unittest +from test import test_support + +class MyTestCase1(unittest.TestCase): + + # Only use setUp() and tearDown() if necessary + + def setUp(self): + ... code to execute in preparation for tests ... + + def tearDown(self): + ... code to execute to clean up after tests ... + + def test_feature_one(self): + # Test feature one. + ... testing code ... + + def test_feature_two(self): + # Test feature two. + ... testing code ... + + ... more test methods ... + +class MyTestCase2(unittest.TestCase): + ... same structure as MyTestCase1 ... + +... more test classes ... + +def test_main(): + test_support.run_unittest(MyTestCase1, + MyTestCase2, + ... list other tests ... + ) + +if __name__ == '__main__': + test_main() +\end{verbatim} + +This boilerplate code allows the testing suite to be run by +\module{test.regrtest} as well as on its own as a script. + +The goal for regression testing is to try to break code. +This leads to a few guidelines to be followed: + +\begin{itemize} +\item The testing suite should exercise all classes, functions, and + constants. + This includes not just the external API that is to be presented to the + outside world but also "private" code. +\item Whitebox testing (examining the code being tested when the tests are + being written) is preferred. + Blackbox testing (testing only the published user interface) is not + complete enough to make sure all boundary and edge cases are tested. +\item Make sure all possible values are tested including invalid ones. + This makes sure that not only all valid values are acceptable but also + that improper values are handled correctly. +\item Exhaust as many code paths as possible. + Test where branching occurs and thus tailor input to make sure as many + different paths through the code are taken. +\item Add an explicit test for any bugs discovered for the tested code. + This will make sure that the error does not crop up again if the code is + changed in the future. +\item Make sure to clean up after your tests (such as close and remove all + temporary files). +\item If a test is dependent on a specific condition of the operating system + then verify the condition already exists before attempting the test. +\item Import as few modules as possible and do it as soon as possible. + This minimizes external dependencies of tests and also minimizes possible + anomalous behavior from side-effects of importing a module. +\item Try to maximize code reuse. + On occasion, tests will vary by something as small as what type + of input is used. + Minimize code duplication by subclassing a basic test class with a class + that specifies the input: +\begin{verbatim} +class TestFuncAcceptsSequences(unittest.TestCase): + + func = mySuperWhammyFunction + + def test_func(self): + self.func(self.arg) + +class AcceptLists(TestFuncAcceptsSequences): + arg = [1,2,3] + +class AcceptStrings(TestFuncAcceptsSequences): + arg = 'abc' + +class AcceptTuples(TestFuncAcceptsSequences): + arg = (1,2,3) +\end{verbatim} +\end{itemize} + +\begin{seealso} +\seetitle{Test Driven Development} + {A book by Kent Beck on writing tests before code.} +\end{seealso} + + +\subsection{Running tests using \module{test.regrtest} \label{regrtest}} + +\module{test.regrtest} can be used as a script to drive Python's +regression test suite. +Running the script by itself automatically starts running all +regression tests in the \module{test} package. +It does this by finding all modules in the package whose name starts with +\samp{test_}, importing them, and executing the function +\function{test_main()} if present. +The names of tests to execute may also be passed to the script. +Specifying a single regression test (\program{python regrtest.py} +\programopt{test_spam.py}) will minimize output and only print whether +the test passed or failed and thus minimize output. + +Running \module{test.regrtest} directly allows what resources are +available for tests to use to be set. +You do this by using the \programopt{-u} command-line option. +Run \program{python regrtest.py} \programopt{-uall} to turn on all +resources; specifying \programopt{all} as an option for +\programopt{-u} enables all possible resources. +If all but one resource is desired (a more common case), a +comma-separated list of resources that are not desired may be listed after +\programopt{all}. +The command \program{python regrtest.py} +\programopt{-uall,-audio,-largefile} will run \module{test.regrtest} +with all resources except the \programopt{audio} and +\programopt{largefile} resources. +For a list of all resources and more command-line options, run +\program{python regrtest.py} \programopt{-h}. + +Some other ways to execute the regression tests depend on what platform the +tests are being executed on. +On \UNIX{}, you can run \program{make} \programopt{test} at the +top-level directory where Python was built. +On Windows, executing \program{rt.bat} from your \file{PCBuild} +directory will run all regression tests. + + +\section{\module{test.test_support} --- + Utility functions for tests} + +\declaremodule[test.testsupport]{standard}{test.test_support} +\modulesynopsis{Support for Python regression tests.} + +The \module{test.test_support} module provides support for Python's +regression tests. + +This module defines the following exceptions: + +\begin{excdesc}{TestFailed} +Exception to be raised when a test fails. +\end{excdesc} + +\begin{excdesc}{TestSkipped} +Subclass of \exception{TestFailed}. +Raised when a test is skipped. +This occurs when a needed resource (such as a network connection) is not +available at the time of testing. +\end{excdesc} + +\begin{excdesc}{ResourceDenied} +Subclass of \exception{TestSkipped}. +Raised when a resource (such as a network connection) is not available. +Raised by the \function{requires()} function. +\end{excdesc} + + +The \module{test.test_support} module defines the following constants: + +\begin{datadesc}{verbose} +\constant{True} when verbose output is enabled. +Should be checked when more detailed information is desired about a running +test. +\var{verbose} is set by \module{test.regrtest}. +\end{datadesc} + +\begin{datadesc}{have_unicode} +\constant{True} when Unicode support is available. +\end{datadesc} + +\begin{datadesc}{is_jython} +\constant{True} if the running interpreter is Jython. +\end{datadesc} + +\begin{datadesc}{TESTFN} +Set to the path that a temporary file may be created at. +Any temporary that is created should be closed and unlinked (removed). +\end{datadesc} + + +The \module{test.test_support} module defines the following functions: + +\begin{funcdesc}{forget}{module_name} +Removes the module named \var{module_name} from \code{sys.modules} and deletes +any byte-compiled files of the module. +\end{funcdesc} + +\begin{funcdesc}{is_resource_enabled}{resource} +Returns \constant{True} if \var{resource} is enabled and available. +The list of available resources is only set when \module{test.regrtest} +is executing the tests. +\end{funcdesc} + +\begin{funcdesc}{requires}{resource\optional{, msg}} +Raises \exception{ResourceDenied} if \var{resource} is not available. +\var{msg} is the argument to \exception{ResourceDenied} if it is raised. +Always returns true if called by a function whose \code{__name__} is +\code{'__main__'}. +Used when tests are executed by \module{test.regrtest}. +\end{funcdesc} + +\begin{funcdesc}{findfile}{filename} +Return the path to the file named \var{filename}. +If no match is found \var{filename} is returned. +This does not equal a failure since it could be the path to the file. +\end{funcdesc} + +\begin{funcdesc}{run_unittest}{*classes} +Execute \class{unittest.TestCase} subclasses passed to the function. +The function scans the classes for methods starting with the prefix +\samp{test_} and executes the tests individually. +This is the preferred way to execute tests. +\end{funcdesc} + +\begin{funcdesc}{run_suite}{suite\optional{, testclass}} +Execute the \class{unittest.TestSuite} instance \var{suite}. +The optional argument \var{testclass} accepts one of the test classes in the +suite so as to print out more detailed information on where the testing suite +originated from. +\end{funcdesc} |