summaryrefslogtreecommitdiff
path: root/sys/src/cmd/python/Doc/lib/libdoctest.tex
diff options
context:
space:
mode:
authorcinap_lenrek <cinap_lenrek@localhost>2011-05-03 11:25:13 +0000
committercinap_lenrek <cinap_lenrek@localhost>2011-05-03 11:25:13 +0000
commit458120dd40db6b4df55a4e96b650e16798ef06a0 (patch)
tree8f82685be24fef97e715c6f5ca4c68d34d5074ee /sys/src/cmd/python/Doc/lib/libdoctest.tex
parent3a742c699f6806c1145aea5149bf15de15a0afd7 (diff)
add hg and python
Diffstat (limited to 'sys/src/cmd/python/Doc/lib/libdoctest.tex')
-rw-r--r--sys/src/cmd/python/Doc/lib/libdoctest.tex1949
1 files changed, 1949 insertions, 0 deletions
diff --git a/sys/src/cmd/python/Doc/lib/libdoctest.tex b/sys/src/cmd/python/Doc/lib/libdoctest.tex
new file mode 100644
index 000000000..957ecf48c
--- /dev/null
+++ b/sys/src/cmd/python/Doc/lib/libdoctest.tex
@@ -0,0 +1,1949 @@
+\section{\module{doctest} ---
+ Test interactive Python examples}
+
+\declaremodule{standard}{doctest}
+\moduleauthor{Tim Peters}{tim@python.org}
+\sectionauthor{Tim Peters}{tim@python.org}
+\sectionauthor{Moshe Zadka}{moshez@debian.org}
+\sectionauthor{Edward Loper}{edloper@users.sourceforge.net}
+
+\modulesynopsis{A framework for verifying interactive Python examples.}
+
+The \refmodule{doctest} module searches for pieces of text that look like
+interactive Python sessions, and then executes those sessions to
+verify that they work exactly as shown. There are several common ways to
+use doctest:
+
+\begin{itemize}
+\item To check that a module's docstrings are up-to-date by verifying
+ that all interactive examples still work as documented.
+\item To perform regression testing by verifying that interactive
+ examples from a test file or a test object work as expected.
+\item To write tutorial documentation for a package, liberally
+ illustrated with input-output examples. Depending on whether
+ the examples or the expository text are emphasized, this has
+ the flavor of "literate testing" or "executable documentation".
+\end{itemize}
+
+Here's a complete but small example module:
+
+\begin{verbatim}
+"""
+This is the "example" module.
+
+The example module supplies one function, factorial(). For example,
+
+>>> factorial(5)
+120
+"""
+
+def factorial(n):
+ """Return the factorial of n, an exact integer >= 0.
+
+ If the result is small enough to fit in an int, return an int.
+ Else return a long.
+
+ >>> [factorial(n) for n in range(6)]
+ [1, 1, 2, 6, 24, 120]
+ >>> [factorial(long(n)) for n in range(6)]
+ [1, 1, 2, 6, 24, 120]
+ >>> factorial(30)
+ 265252859812191058636308480000000L
+ >>> factorial(30L)
+ 265252859812191058636308480000000L
+ >>> factorial(-1)
+ Traceback (most recent call last):
+ ...
+ ValueError: n must be >= 0
+
+ Factorials of floats are OK, but the float must be an exact integer:
+ >>> factorial(30.1)
+ Traceback (most recent call last):
+ ...
+ ValueError: n must be exact integer
+ >>> factorial(30.0)
+ 265252859812191058636308480000000L
+
+ It must also not be ridiculously large:
+ >>> factorial(1e100)
+ Traceback (most recent call last):
+ ...
+ OverflowError: n too large
+ """
+
+\end{verbatim}
+% allow LaTeX to break here.
+\begin{verbatim}
+
+ import math
+ if not n >= 0:
+ raise ValueError("n must be >= 0")
+ if math.floor(n) != n:
+ raise ValueError("n must be exact integer")
+ if n+1 == n: # catch a value like 1e300
+ raise OverflowError("n too large")
+ result = 1
+ factor = 2
+ while factor <= n:
+ result *= factor
+ factor += 1
+ return result
+
+def _test():
+ import doctest
+ doctest.testmod()
+
+if __name__ == "__main__":
+ _test()
+\end{verbatim}
+
+If you run \file{example.py} directly from the command line,
+\refmodule{doctest} works its magic:
+
+\begin{verbatim}
+$ python example.py
+$
+\end{verbatim}
+
+There's no output! That's normal, and it means all the examples
+worked. Pass \programopt{-v} to the script, and \refmodule{doctest}
+prints a detailed log of what it's trying, and prints a summary at the
+end:
+
+\begin{verbatim}
+$ python example.py -v
+Trying:
+ factorial(5)
+Expecting:
+ 120
+ok
+Trying:
+ [factorial(n) for n in range(6)]
+Expecting:
+ [1, 1, 2, 6, 24, 120]
+ok
+Trying:
+ [factorial(long(n)) for n in range(6)]
+Expecting:
+ [1, 1, 2, 6, 24, 120]
+ok
+\end{verbatim}
+
+And so on, eventually ending with:
+
+\begin{verbatim}
+Trying:
+ factorial(1e100)
+Expecting:
+ Traceback (most recent call last):
+ ...
+ OverflowError: n too large
+ok
+1 items had no tests:
+ __main__._test
+2 items passed all tests:
+ 1 tests in __main__
+ 8 tests in __main__.factorial
+9 tests in 3 items.
+9 passed and 0 failed.
+Test passed.
+$
+\end{verbatim}
+
+That's all you need to know to start making productive use of
+\refmodule{doctest}! Jump in. The following sections provide full
+details. Note that there are many examples of doctests in
+the standard Python test suite and libraries. Especially useful examples
+can be found in the standard test file \file{Lib/test/test_doctest.py}.
+
+\subsection{Simple Usage: Checking Examples in
+ Docstrings\label{doctest-simple-testmod}}
+
+The simplest way to start using doctest (but not necessarily the way
+you'll continue to do it) is to end each module \module{M} with:
+
+\begin{verbatim}
+def _test():
+ import doctest
+ doctest.testmod()
+
+if __name__ == "__main__":
+ _test()
+\end{verbatim}
+
+\refmodule{doctest} then examines docstrings in module \module{M}.
+
+Running the module as a script causes the examples in the docstrings
+to get executed and verified:
+
+\begin{verbatim}
+python M.py
+\end{verbatim}
+
+This won't display anything unless an example fails, in which case the
+failing example(s) and the cause(s) of the failure(s) are printed to stdout,
+and the final line of output is
+\samp{***Test Failed*** \var{N} failures.}, where \var{N} is the
+number of examples that failed.
+
+Run it with the \programopt{-v} switch instead:
+
+\begin{verbatim}
+python M.py -v
+\end{verbatim}
+
+and a detailed report of all examples tried is printed to standard
+output, along with assorted summaries at the end.
+
+You can force verbose mode by passing \code{verbose=True} to
+\function{testmod()}, or
+prohibit it by passing \code{verbose=False}. In either of those cases,
+\code{sys.argv} is not examined by \function{testmod()} (so passing
+\programopt{-v} or not has no effect).
+
+For more information on \function{testmod()}, see
+section~\ref{doctest-basic-api}.
+
+\subsection{Simple Usage: Checking Examples in a Text
+ File\label{doctest-simple-testfile}}
+
+Another simple application of doctest is testing interactive examples
+in a text file. This can be done with the \function{testfile()}
+function:
+
+\begin{verbatim}
+import doctest
+doctest.testfile("example.txt")
+\end{verbatim}
+
+That short script executes and verifies any interactive Python
+examples contained in the file \file{example.txt}. The file content
+is treated as if it were a single giant docstring; the file doesn't
+need to contain a Python program! For example, perhaps \file{example.txt}
+contains this:
+
+\begin{verbatim}
+The ``example`` module
+======================
+
+Using ``factorial``
+-------------------
+
+This is an example text file in reStructuredText format. First import
+``factorial`` from the ``example`` module:
+
+ >>> from example import factorial
+
+Now use it:
+
+ >>> factorial(6)
+ 120
+\end{verbatim}
+
+Running \code{doctest.testfile("example.txt")} then finds the error
+in this documentation:
+
+\begin{verbatim}
+File "./example.txt", line 14, in example.txt
+Failed example:
+ factorial(6)
+Expected:
+ 120
+Got:
+ 720
+\end{verbatim}
+
+As with \function{testmod()}, \function{testfile()} won't display anything
+unless an example fails. If an example does fail, then the failing
+example(s) and the cause(s) of the failure(s) are printed to stdout, using
+the same format as \function{testmod()}.
+
+By default, \function{testfile()} looks for files in the calling
+module's directory. See section~\ref{doctest-basic-api} for a
+description of the optional arguments that can be used to tell it to
+look for files in other locations.
+
+Like \function{testmod()}, \function{testfile()}'s verbosity can be
+set with the \programopt{-v} command-line switch or with the optional
+keyword argument \var{verbose}.
+
+For more information on \function{testfile()}, see
+section~\ref{doctest-basic-api}.
+
+\subsection{How It Works\label{doctest-how-it-works}}
+
+This section examines in detail how doctest works: which docstrings it
+looks at, how it finds interactive examples, what execution context it
+uses, how it handles exceptions, and how option flags can be used to
+control its behavior. This is the information that you need to know
+to write doctest examples; for information about actually running
+doctest on these examples, see the following sections.
+
+\subsubsection{Which Docstrings Are Examined?\label{doctest-which-docstrings}}
+
+The module docstring, and all function, class and method docstrings are
+searched. Objects imported into the module are not searched.
+
+In addition, if \code{M.__test__} exists and "is true", it must be a
+dict, and each entry maps a (string) name to a function object, class
+object, or string. Function and class object docstrings found from
+\code{M.__test__} are searched, and strings are treated as if they
+were docstrings. In output, a key \code{K} in \code{M.__test__} appears
+with name
+
+\begin{verbatim}
+<name of M>.__test__.K
+\end{verbatim}
+
+Any classes found are recursively searched similarly, to test docstrings in
+their contained methods and nested classes.
+
+\versionchanged[A "private name" concept is deprecated and no longer
+ documented]{2.4}
+
+\subsubsection{How are Docstring Examples
+ Recognized?\label{doctest-finding-examples}}
+
+In most cases a copy-and-paste of an interactive console session works
+fine, but doctest isn't trying to do an exact emulation of any specific
+Python shell. All hard tab characters are expanded to spaces, using
+8-column tab stops. If you don't believe tabs should mean that, too
+bad: don't use hard tabs, or write your own \class{DocTestParser}
+class.
+
+\versionchanged[Expanding tabs to spaces is new; previous versions
+ tried to preserve hard tabs, with confusing results]{2.4}
+
+\begin{verbatim}
+>>> # comments are ignored
+>>> x = 12
+>>> x
+12
+>>> if x == 13:
+... print "yes"
+... else:
+... print "no"
+... print "NO"
+... print "NO!!!"
+...
+no
+NO
+NO!!!
+>>>
+\end{verbatim}
+
+Any expected output must immediately follow the final
+\code{'>>>~'} or \code{'...~'} line containing the code, and
+the expected output (if any) extends to the next \code{'>>>~'}
+or all-whitespace line.
+
+The fine print:
+
+\begin{itemize}
+
+\item Expected output cannot contain an all-whitespace line, since such a
+ line is taken to signal the end of expected output. If expected
+ output does contain a blank line, put \code{<BLANKLINE>} in your
+ doctest example each place a blank line is expected.
+ \versionchanged[\code{<BLANKLINE>} was added; there was no way to
+ use expected output containing empty lines in
+ previous versions]{2.4}
+
+\item Output to stdout is captured, but not output to stderr (exception
+ tracebacks are captured via a different means).
+
+\item If you continue a line via backslashing in an interactive session,
+ or for any other reason use a backslash, you should use a raw
+ docstring, which will preserve your backslashes exactly as you type
+ them:
+
+\begin{verbatim}
+>>> def f(x):
+... r'''Backslashes in a raw docstring: m\n'''
+>>> print f.__doc__
+Backslashes in a raw docstring: m\n
+\end{verbatim}
+
+ Otherwise, the backslash will be interpreted as part of the string.
+ For example, the "{\textbackslash}" above would be interpreted as a
+ newline character. Alternatively, you can double each backslash in the
+ doctest version (and not use a raw string):
+
+\begin{verbatim}
+>>> def f(x):
+... '''Backslashes in a raw docstring: m\\n'''
+>>> print f.__doc__
+Backslashes in a raw docstring: m\n
+\end{verbatim}
+
+\item The starting column doesn't matter:
+
+\begin{verbatim}
+ >>> assert "Easy!"
+ >>> import math
+ >>> math.floor(1.9)
+ 1.0
+\end{verbatim}
+
+and as many leading whitespace characters are stripped from the
+expected output as appeared in the initial \code{'>>>~'} line
+that started the example.
+\end{itemize}
+
+\subsubsection{What's the Execution Context?\label{doctest-execution-context}}
+
+By default, each time \refmodule{doctest} finds a docstring to test, it
+uses a \emph{shallow copy} of \module{M}'s globals, so that running tests
+doesn't change the module's real globals, and so that one test in
+\module{M} can't leave behind crumbs that accidentally allow another test
+to work. This means examples can freely use any names defined at top-level
+in \module{M}, and names defined earlier in the docstring being run.
+Examples cannot see names defined in other docstrings.
+
+You can force use of your own dict as the execution context by passing
+\code{globs=your_dict} to \function{testmod()} or
+\function{testfile()} instead.
+
+\subsubsection{What About Exceptions?\label{doctest-exceptions}}
+
+No problem, provided that the traceback is the only output produced by
+the example: just paste in the traceback.\footnote{Examples containing
+ both expected output and an exception are not supported. Trying
+ to guess where one ends and the other begins is too error-prone,
+ and that also makes for a confusing test.}
+Since tracebacks contain details that are likely to change rapidly (for
+example, exact file paths and line numbers), this is one case where doctest
+works hard to be flexible in what it accepts.
+
+Simple example:
+
+\begin{verbatim}
+>>> [1, 2, 3].remove(42)
+Traceback (most recent call last):
+ File "<stdin>", line 1, in ?
+ValueError: list.remove(x): x not in list
+\end{verbatim}
+
+That doctest succeeds if \exception{ValueError} is raised, with the
+\samp{list.remove(x): x not in list} detail as shown.
+
+The expected output for an exception must start with a traceback
+header, which may be either of the following two lines, indented the
+same as the first line of the example:
+
+\begin{verbatim}
+Traceback (most recent call last):
+Traceback (innermost last):
+\end{verbatim}
+
+The traceback header is followed by an optional traceback stack, whose
+contents are ignored by doctest. The traceback stack is typically
+omitted, or copied verbatim from an interactive session.
+
+The traceback stack is followed by the most interesting part: the
+line(s) containing the exception type and detail. This is usually the
+last line of a traceback, but can extend across multiple lines if the
+exception has a multi-line detail:
+
+\begin{verbatim}
+>>> raise ValueError('multi\n line\ndetail')
+Traceback (most recent call last):
+ File "<stdin>", line 1, in ?
+ValueError: multi
+ line
+detail
+\end{verbatim}
+
+The last three lines (starting with \exception{ValueError}) are
+compared against the exception's type and detail, and the rest are
+ignored.
+
+Best practice is to omit the traceback stack, unless it adds
+significant documentation value to the example. So the last example
+is probably better as:
+
+\begin{verbatim}
+>>> raise ValueError('multi\n line\ndetail')
+Traceback (most recent call last):
+ ...
+ValueError: multi
+ line
+detail
+\end{verbatim}
+
+Note that tracebacks are treated very specially. In particular, in the
+rewritten example, the use of \samp{...} is independent of doctest's
+\constant{ELLIPSIS} option. The ellipsis in that example could be left
+out, or could just as well be three (or three hundred) commas or digits,
+or an indented transcript of a Monty Python skit.
+
+Some details you should read once, but won't need to remember:
+
+\begin{itemize}
+
+\item Doctest can't guess whether your expected output came from an
+ exception traceback or from ordinary printing. So, e.g., an example
+ that expects \samp{ValueError: 42 is prime} will pass whether
+ \exception{ValueError} is actually raised or if the example merely
+ prints that traceback text. In practice, ordinary output rarely begins
+ with a traceback header line, so this doesn't create real problems.
+
+\item Each line of the traceback stack (if present) must be indented
+ further than the first line of the example, \emph{or} start with a
+ non-alphanumeric character. The first line following the traceback
+ header indented the same and starting with an alphanumeric is taken
+ to be the start of the exception detail. Of course this does the
+ right thing for genuine tracebacks.
+
+\item When the \constant{IGNORE_EXCEPTION_DETAIL} doctest option is
+ is specified, everything following the leftmost colon is ignored.
+
+\item The interactive shell omits the traceback header line for some
+ \exception{SyntaxError}s. But doctest uses the traceback header
+ line to distinguish exceptions from non-exceptions. So in the rare
+ case where you need to test a \exception{SyntaxError} that omits the
+ traceback header, you will need to manually add the traceback header
+ line to your test example.
+
+\item For some \exception{SyntaxError}s, Python displays the character
+ position of the syntax error, using a \code{\^} marker:
+
+\begin{verbatim}
+>>> 1 1
+ File "<stdin>", line 1
+ 1 1
+ ^
+SyntaxError: invalid syntax
+\end{verbatim}
+
+ Since the lines showing the position of the error come before the
+ exception type and detail, they are not checked by doctest. For
+ example, the following test would pass, even though it puts the
+ \code{\^} marker in the wrong location:
+
+\begin{verbatim}
+>>> 1 1
+Traceback (most recent call last):
+ File "<stdin>", line 1
+ 1 1
+ ^
+SyntaxError: invalid syntax
+\end{verbatim}
+
+\end{itemize}
+
+\versionchanged[The ability to handle a multi-line exception detail,
+ and the \constant{IGNORE_EXCEPTION_DETAIL} doctest option,
+ were added]{2.4}
+
+\subsubsection{Option Flags and Directives\label{doctest-options}}
+
+A number of option flags control various aspects of doctest's
+behavior. Symbolic names for the flags are supplied as module constants,
+which can be or'ed together and passed to various functions. The names
+can also be used in doctest directives (see below).
+
+The first group of options define test semantics, controlling
+aspects of how doctest decides whether actual output matches an
+example's expected output:
+
+\begin{datadesc}{DONT_ACCEPT_TRUE_FOR_1}
+ By default, if an expected output block contains just \code{1},
+ an actual output block containing just \code{1} or just
+ \code{True} is considered to be a match, and similarly for \code{0}
+ versus \code{False}. When \constant{DONT_ACCEPT_TRUE_FOR_1} is
+ specified, neither substitution is allowed. The default behavior
+ caters to that Python changed the return type of many functions
+ from integer to boolean; doctests expecting "little integer"
+ output still work in these cases. This option will probably go
+ away, but not for several years.
+\end{datadesc}
+
+\begin{datadesc}{DONT_ACCEPT_BLANKLINE}
+ By default, if an expected output block contains a line
+ containing only the string \code{<BLANKLINE>}, then that line
+ will match a blank line in the actual output. Because a
+ genuinely blank line delimits the expected output, this is
+ the only way to communicate that a blank line is expected. When
+ \constant{DONT_ACCEPT_BLANKLINE} is specified, this substitution
+ is not allowed.
+\end{datadesc}
+
+\begin{datadesc}{NORMALIZE_WHITESPACE}
+ When specified, all sequences of whitespace (blanks and newlines) are
+ treated as equal. Any sequence of whitespace within the expected
+ output will match any sequence of whitespace within the actual output.
+ By default, whitespace must match exactly.
+ \constant{NORMALIZE_WHITESPACE} is especially useful when a line
+ of expected output is very long, and you want to wrap it across
+ multiple lines in your source.
+\end{datadesc}
+
+\begin{datadesc}{ELLIPSIS}
+ When specified, an ellipsis marker (\code{...}) in the expected output
+ can match any substring in the actual output. This includes
+ substrings that span line boundaries, and empty substrings, so it's
+ best to keep usage of this simple. Complicated uses can lead to the
+ same kinds of "oops, it matched too much!" surprises that \regexp{.*}
+ is prone to in regular expressions.
+\end{datadesc}
+
+\begin{datadesc}{IGNORE_EXCEPTION_DETAIL}
+ When specified, an example that expects an exception passes if
+ an exception of the expected type is raised, even if the exception
+ detail does not match. For example, an example expecting
+ \samp{ValueError: 42} will pass if the actual exception raised is
+ \samp{ValueError: 3*14}, but will fail, e.g., if
+ \exception{TypeError} is raised.
+
+ Note that a similar effect can be obtained using \constant{ELLIPSIS},
+ and \constant{IGNORE_EXCEPTION_DETAIL} may go away when Python releases
+ prior to 2.4 become uninteresting. Until then,
+ \constant{IGNORE_EXCEPTION_DETAIL} is the only clear way to write a
+ doctest that doesn't care about the exception detail yet continues
+ to pass under Python releases prior to 2.4 (doctest directives
+ appear to be comments to them). For example,
+
+\begin{verbatim}
+>>> (1, 2)[3] = 'moo' #doctest: +IGNORE_EXCEPTION_DETAIL
+Traceback (most recent call last):
+ File "<stdin>", line 1, in ?
+TypeError: object doesn't support item assignment
+\end{verbatim}
+
+ passes under Python 2.4 and Python 2.3. The detail changed in 2.4,
+ to say "does not" instead of "doesn't".
+
+\end{datadesc}
+
+\begin{datadesc}{SKIP}
+
+ When specified, do not run the example at all. This can be useful
+ in contexts where doctest examples serve as both documentation and
+ test cases, and an example should be included for documentation
+ purposes, but should not be checked. E.g., the example's output
+ might be random; or the example might depend on resources which
+ would be unavailable to the test driver.
+
+ The SKIP flag can also be used for temporarily "commenting out"
+ examples.
+
+\end{datadesc}
+
+\begin{datadesc}{COMPARISON_FLAGS}
+ A bitmask or'ing together all the comparison flags above.
+\end{datadesc}
+
+The second group of options controls how test failures are reported:
+
+\begin{datadesc}{REPORT_UDIFF}
+ When specified, failures that involve multi-line expected and
+ actual outputs are displayed using a unified diff.
+\end{datadesc}
+
+\begin{datadesc}{REPORT_CDIFF}
+ When specified, failures that involve multi-line expected and
+ actual outputs will be displayed using a context diff.
+\end{datadesc}
+
+\begin{datadesc}{REPORT_NDIFF}
+ When specified, differences are computed by \code{difflib.Differ},
+ using the same algorithm as the popular \file{ndiff.py} utility.
+ This is the only method that marks differences within lines as
+ well as across lines. For example, if a line of expected output
+ contains digit \code{1} where actual output contains letter \code{l},
+ a line is inserted with a caret marking the mismatching column
+ positions.
+\end{datadesc}
+
+\begin{datadesc}{REPORT_ONLY_FIRST_FAILURE}
+ When specified, display the first failing example in each doctest,
+ but suppress output for all remaining examples. This will prevent
+ doctest from reporting correct examples that break because of
+ earlier failures; but it might also hide incorrect examples that
+ fail independently of the first failure. When
+ \constant{REPORT_ONLY_FIRST_FAILURE} is specified, the remaining
+ examples are still run, and still count towards the total number of
+ failures reported; only the output is suppressed.
+\end{datadesc}
+
+\begin{datadesc}{REPORTING_FLAGS}
+ A bitmask or'ing together all the reporting flags above.
+\end{datadesc}
+
+"Doctest directives" may be used to modify the option flags for
+individual examples. Doctest directives are expressed as a special
+Python comment following an example's source code:
+
+\begin{productionlist}[doctest]
+ \production{directive}
+ {"\#" "doctest:" \token{directive_options}}
+ \production{directive_options}
+ {\token{directive_option} ("," \token{directive_option})*}
+ \production{directive_option}
+ {\token{on_or_off} \token{directive_option_name}}
+ \production{on_or_off}
+ {"+" | "-"}
+ \production{directive_option_name}
+ {"DONT_ACCEPT_BLANKLINE" | "NORMALIZE_WHITESPACE" | ...}
+\end{productionlist}
+
+Whitespace is not allowed between the \code{+} or \code{-} and the
+directive option name. The directive option name can be any of the
+option flag names explained above.
+
+An example's doctest directives modify doctest's behavior for that
+single example. Use \code{+} to enable the named behavior, or
+\code{-} to disable it.
+
+For example, this test passes:
+
+\begin{verbatim}
+>>> print range(20) #doctest: +NORMALIZE_WHITESPACE
+[0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
+10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
+\end{verbatim}
+
+Without the directive it would fail, both because the actual output
+doesn't have two blanks before the single-digit list elements, and
+because the actual output is on a single line. This test also passes,
+and also requires a directive to do so:
+
+\begin{verbatim}
+>>> print range(20) # doctest:+ELLIPSIS
+[0, 1, ..., 18, 19]
+\end{verbatim}
+
+Multiple directives can be used on a single physical line, separated
+by commas:
+
+\begin{verbatim}
+>>> print range(20) # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE
+[0, 1, ..., 18, 19]
+\end{verbatim}
+
+If multiple directive comments are used for a single example, then
+they are combined:
+
+\begin{verbatim}
+>>> print range(20) # doctest: +ELLIPSIS
+... # doctest: +NORMALIZE_WHITESPACE
+[0, 1, ..., 18, 19]
+\end{verbatim}
+
+As the previous example shows, you can add \samp{...} lines to your
+example containing only directives. This can be useful when an
+example is too long for a directive to comfortably fit on the same
+line:
+
+\begin{verbatim}
+>>> print range(5) + range(10,20) + range(30,40) + range(50,60)
+... # doctest: +ELLIPSIS
+[0, ..., 4, 10, ..., 19, 30, ..., 39, 50, ..., 59]
+\end{verbatim}
+
+Note that since all options are disabled by default, and directives apply
+only to the example they appear in, enabling options (via \code{+} in a
+directive) is usually the only meaningful choice. However, option flags
+can also be passed to functions that run doctests, establishing different
+defaults. In such cases, disabling an option via \code{-} in a directive
+can be useful.
+
+\versionchanged[Constants \constant{DONT_ACCEPT_BLANKLINE},
+ \constant{NORMALIZE_WHITESPACE}, \constant{ELLIPSIS},
+ \constant{IGNORE_EXCEPTION_DETAIL},
+ \constant{REPORT_UDIFF}, \constant{REPORT_CDIFF},
+ \constant{REPORT_NDIFF}, \constant{REPORT_ONLY_FIRST_FAILURE},
+ \constant{COMPARISON_FLAGS} and \constant{REPORTING_FLAGS}
+ were added; by default \code{<BLANKLINE>} in expected output
+ matches an empty line in actual output; and doctest directives
+ were added]{2.4}
+\versionchanged[Constant \constant{SKIP} was added]{2.5}
+
+There's also a way to register new option flag names, although this
+isn't useful unless you intend to extend \refmodule{doctest} internals
+via subclassing:
+
+\begin{funcdesc}{register_optionflag}{name}
+ Create a new option flag with a given name, and return the new
+ flag's integer value. \function{register_optionflag()} can be
+ used when subclassing \class{OutputChecker} or
+ \class{DocTestRunner} to create new options that are supported by
+ your subclasses. \function{register_optionflag} should always be
+ called using the following idiom:
+
+\begin{verbatim}
+ MY_FLAG = register_optionflag('MY_FLAG')
+\end{verbatim}
+
+ \versionadded{2.4}
+\end{funcdesc}
+
+\subsubsection{Warnings\label{doctest-warnings}}
+
+\refmodule{doctest} is serious about requiring exact matches in expected
+output. If even a single character doesn't match, the test fails. This
+will probably surprise you a few times, as you learn exactly what Python
+does and doesn't guarantee about output. For example, when printing a
+dict, Python doesn't guarantee that the key-value pairs will be printed
+in any particular order, so a test like
+
+% Hey! What happened to Monty Python examples?
+% Tim: ask Guido -- it's his example!
+\begin{verbatim}
+>>> foo()
+{"Hermione": "hippogryph", "Harry": "broomstick"}
+\end{verbatim}
+
+is vulnerable! One workaround is to do
+
+\begin{verbatim}
+>>> foo() == {"Hermione": "hippogryph", "Harry": "broomstick"}
+True
+\end{verbatim}
+
+instead. Another is to do
+
+\begin{verbatim}
+>>> d = foo().items()
+>>> d.sort()
+>>> d
+[('Harry', 'broomstick'), ('Hermione', 'hippogryph')]
+\end{verbatim}
+
+There are others, but you get the idea.
+
+Another bad idea is to print things that embed an object address, like
+
+\begin{verbatim}
+>>> id(1.0) # certain to fail some of the time
+7948648
+>>> class C: pass
+>>> C() # the default repr() for instances embeds an address
+<__main__.C instance at 0x00AC18F0>
+\end{verbatim}
+
+The \constant{ELLIPSIS} directive gives a nice approach for the last
+example:
+
+\begin{verbatim}
+>>> C() #doctest: +ELLIPSIS
+<__main__.C instance at 0x...>
+\end{verbatim}
+
+Floating-point numbers are also subject to small output variations across
+platforms, because Python defers to the platform C library for float
+formatting, and C libraries vary widely in quality here.
+
+\begin{verbatim}
+>>> 1./7 # risky
+0.14285714285714285
+>>> print 1./7 # safer
+0.142857142857
+>>> print round(1./7, 6) # much safer
+0.142857
+\end{verbatim}
+
+Numbers of the form \code{I/2.**J} are safe across all platforms, and I
+often contrive doctest examples to produce numbers of that form:
+
+\begin{verbatim}
+>>> 3./4 # utterly safe
+0.75
+\end{verbatim}
+
+Simple fractions are also easier for people to understand, and that makes
+for better documentation.
+
+\subsection{Basic API\label{doctest-basic-api}}
+
+The functions \function{testmod()} and \function{testfile()} provide a
+simple interface to doctest that should be sufficient for most basic
+uses. For a less formal introduction to these two functions, see
+sections \ref{doctest-simple-testmod} and
+\ref{doctest-simple-testfile}.
+
+\begin{funcdesc}{testfile}{filename\optional{, module_relative}\optional{,
+ name}\optional{, package}\optional{,
+ globs}\optional{, verbose}\optional{,
+ report}\optional{, optionflags}\optional{,
+ extraglobs}\optional{, raise_on_error}\optional{,
+ parser}\optional{, encoding}}
+
+ All arguments except \var{filename} are optional, and should be
+ specified in keyword form.
+
+ Test examples in the file named \var{filename}. Return
+ \samp{(\var{failure_count}, \var{test_count})}.
+
+ Optional argument \var{module_relative} specifies how the filename
+ should be interpreted:
+
+ \begin{itemize}
+ \item If \var{module_relative} is \code{True} (the default), then
+ \var{filename} specifies an OS-independent module-relative
+ path. By default, this path is relative to the calling
+ module's directory; but if the \var{package} argument is
+ specified, then it is relative to that package. To ensure
+ OS-independence, \var{filename} should use \code{/} characters
+ to separate path segments, and may not be an absolute path
+ (i.e., it may not begin with \code{/}).
+ \item If \var{module_relative} is \code{False}, then \var{filename}
+ specifies an OS-specific path. The path may be absolute or
+ relative; relative paths are resolved with respect to the
+ current working directory.
+ \end{itemize}
+
+ Optional argument \var{name} gives the name of the test; by default,
+ or if \code{None}, \code{os.path.basename(\var{filename})} is used.
+
+ Optional argument \var{package} is a Python package or the name of a
+ Python package whose directory should be used as the base directory
+ for a module-relative filename. If no package is specified, then
+ the calling module's directory is used as the base directory for
+ module-relative filenames. It is an error to specify \var{package}
+ if \var{module_relative} is \code{False}.
+
+ Optional argument \var{globs} gives a dict to be used as the globals
+ when executing examples. A new shallow copy of this dict is
+ created for the doctest, so its examples start with a clean slate.
+ By default, or if \code{None}, a new empty dict is used.
+
+ Optional argument \var{extraglobs} gives a dict merged into the
+ globals used to execute examples. This works like
+ \method{dict.update()}: if \var{globs} and \var{extraglobs} have a
+ common key, the associated value in \var{extraglobs} appears in the
+ combined dict. By default, or if \code{None}, no extra globals are
+ used. This is an advanced feature that allows parameterization of
+ doctests. For example, a doctest can be written for a base class, using
+ a generic name for the class, then reused to test any number of
+ subclasses by passing an \var{extraglobs} dict mapping the generic
+ name to the subclass to be tested.
+
+ Optional argument \var{verbose} prints lots of stuff if true, and prints
+ only failures if false; by default, or if \code{None}, it's true
+ if and only if \code{'-v'} is in \code{sys.argv}.
+
+ Optional argument \var{report} prints a summary at the end when true,
+ else prints nothing at the end. In verbose mode, the summary is
+ detailed, else the summary is very brief (in fact, empty if all tests
+ passed).
+
+ Optional argument \var{optionflags} or's together option flags. See
+ section~\ref{doctest-options}.
+
+ Optional argument \var{raise_on_error} defaults to false. If true,
+ an exception is raised upon the first failure or unexpected exception
+ in an example. This allows failures to be post-mortem debugged.
+ Default behavior is to continue running examples.
+
+ Optional argument \var{parser} specifies a \class{DocTestParser} (or
+ subclass) that should be used to extract tests from the files. It
+ defaults to a normal parser (i.e., \code{\class{DocTestParser}()}).
+
+ Optional argument \var{encoding} specifies an encoding that should
+ be used to convert the file to unicode.
+
+ \versionadded{2.4}
+
+ \versionchanged[The parameter \var{encoding} was added]{2.5}
+
+\end{funcdesc}
+
+\begin{funcdesc}{testmod}{\optional{m}\optional{, name}\optional{,
+ globs}\optional{, verbose}\optional{,
+ report}\optional{,
+ optionflags}\optional{, extraglobs}\optional{,
+ raise_on_error}\optional{, exclude_empty}}
+
+ All arguments are optional, and all except for \var{m} should be
+ specified in keyword form.
+
+ Test examples in docstrings in functions and classes reachable
+ from module \var{m} (or module \module{__main__} if \var{m} is not
+ supplied or is \code{None}), starting with \code{\var{m}.__doc__}.
+
+ Also test examples reachable from dict \code{\var{m}.__test__}, if it
+ exists and is not \code{None}. \code{\var{m}.__test__} maps
+ names (strings) to functions, classes and strings; function and class
+ docstrings are searched for examples; strings are searched directly,
+ as if they were docstrings.
+
+ Only docstrings attached to objects belonging to module \var{m} are
+ searched.
+
+ Return \samp{(\var{failure_count}, \var{test_count})}.
+
+ Optional argument \var{name} gives the name of the module; by default,
+ or if \code{None}, \code{\var{m}.__name__} is used.
+
+ Optional argument \var{exclude_empty} defaults to false. If true,
+ objects for which no doctests are found are excluded from consideration.
+ The default is a backward compatibility hack, so that code still
+ using \method{doctest.master.summarize()} in conjunction with
+ \function{testmod()} continues to get output for objects with no tests.
+ The \var{exclude_empty} argument to the newer \class{DocTestFinder}
+ constructor defaults to true.
+
+ Optional arguments \var{extraglobs}, \var{verbose}, \var{report},
+ \var{optionflags}, \var{raise_on_error}, and \var{globs} are the same as
+ for function \function{testfile()} above, except that \var{globs}
+ defaults to \code{\var{m}.__dict__}.
+
+ \versionchanged[The parameter \var{optionflags} was added]{2.3}
+
+ \versionchanged[The parameters \var{extraglobs}, \var{raise_on_error}
+ and \var{exclude_empty} were added]{2.4}
+
+ \versionchanged[The optional argument \var{isprivate}, deprecated
+ in 2.4, was removed]{2.5}
+
+\end{funcdesc}
+
+There's also a function to run the doctests associated with a single object.
+This function is provided for backward compatibility. There are no plans
+to deprecate it, but it's rarely useful:
+
+\begin{funcdesc}{run_docstring_examples}{f, globs\optional{,
+ verbose}\optional{, name}\optional{,
+ compileflags}\optional{, optionflags}}
+
+ Test examples associated with object \var{f}; for example, \var{f} may
+ be a module, function, or class object.
+
+ A shallow copy of dictionary argument \var{globs} is used for the
+ execution context.
+
+ Optional argument \var{name} is used in failure messages, and defaults
+ to \code{"NoName"}.
+
+ If optional argument \var{verbose} is true, output is generated even
+ if there are no failures. By default, output is generated only in case
+ of an example failure.
+
+ Optional argument \var{compileflags} gives the set of flags that should
+ be used by the Python compiler when running the examples. By default, or
+ if \code{None}, flags are deduced corresponding to the set of future
+ features found in \var{globs}.
+
+ Optional argument \var{optionflags} works as for function
+ \function{testfile()} above.
+\end{funcdesc}
+
+\subsection{Unittest API\label{doctest-unittest-api}}
+
+As your collection of doctest'ed modules grows, you'll want a way to run
+all their doctests systematically. Prior to Python 2.4, \refmodule{doctest}
+had a barely documented \class{Tester} class that supplied a rudimentary
+way to combine doctests from multiple modules. \class{Tester} was feeble,
+and in practice most serious Python testing frameworks build on the
+\refmodule{unittest} module, which supplies many flexible ways to combine
+tests from multiple sources. So, in Python 2.4, \refmodule{doctest}'s
+\class{Tester} class is deprecated, and \refmodule{doctest} provides two
+functions that can be used to create \refmodule{unittest} test suites from
+modules and text files containing doctests. These test suites can then be
+run using \refmodule{unittest} test runners:
+
+\begin{verbatim}
+import unittest
+import doctest
+import my_module_with_doctests, and_another
+
+suite = unittest.TestSuite()
+for mod in my_module_with_doctests, and_another:
+ suite.addTest(doctest.DocTestSuite(mod))
+runner = unittest.TextTestRunner()
+runner.run(suite)
+\end{verbatim}
+
+There are two main functions for creating \class{\refmodule{unittest}.TestSuite}
+instances from text files and modules with doctests:
+
+\begin{funcdesc}{DocFileSuite}{\optional{module_relative}\optional{,
+ package}\optional{, setUp}\optional{,
+ tearDown}\optional{, globs}\optional{,
+ optionflags}\optional{, parser}\optional{,
+ encoding}}
+
+ Convert doctest tests from one or more text files to a
+ \class{\refmodule{unittest}.TestSuite}.
+
+ The returned \class{\refmodule{unittest}.TestSuite} is to be run by the
+ unittest framework and runs the interactive examples in each file. If an
+ example in any file fails, then the synthesized unit test fails, and a
+ \exception{failureException} exception is raised showing the name of the
+ file containing the test and a (sometimes approximate) line number.
+
+ Pass one or more paths (as strings) to text files to be examined.
+
+ Options may be provided as keyword arguments:
+
+ Optional argument \var{module_relative} specifies how
+ the filenames in \var{paths} should be interpreted:
+
+ \begin{itemize}
+ \item If \var{module_relative} is \code{True} (the default), then
+ each filename specifies an OS-independent module-relative
+ path. By default, this path is relative to the calling
+ module's directory; but if the \var{package} argument is
+ specified, then it is relative to that package. To ensure
+ OS-independence, each filename should use \code{/} characters
+ to separate path segments, and may not be an absolute path
+ (i.e., it may not begin with \code{/}).
+ \item If \var{module_relative} is \code{False}, then each filename
+ specifies an OS-specific path. The path may be absolute or
+ relative; relative paths are resolved with respect to the
+ current working directory.
+ \end{itemize}
+
+ Optional argument \var{package} is a Python package or the name
+ of a Python package whose directory should be used as the base
+ directory for module-relative filenames. If no package is
+ specified, then the calling module's directory is used as the base
+ directory for module-relative filenames. It is an error to specify
+ \var{package} if \var{module_relative} is \code{False}.
+
+ Optional argument \var{setUp} specifies a set-up function for
+ the test suite. This is called before running the tests in each
+ file. The \var{setUp} function will be passed a \class{DocTest}
+ object. The setUp function can access the test globals as the
+ \var{globs} attribute of the test passed.
+
+ Optional argument \var{tearDown} specifies a tear-down function
+ for the test suite. This is called after running the tests in each
+ file. The \var{tearDown} function will be passed a \class{DocTest}
+ object. The setUp function can access the test globals as the
+ \var{globs} attribute of the test passed.
+
+ Optional argument \var{globs} is a dictionary containing the
+ initial global variables for the tests. A new copy of this
+ dictionary is created for each test. By default, \var{globs} is
+ a new empty dictionary.
+
+ Optional argument \var{optionflags} specifies the default
+ doctest options for the tests, created by or-ing together
+ individual option flags. See section~\ref{doctest-options}.
+ See function \function{set_unittest_reportflags()} below for
+ a better way to set reporting options.
+
+ Optional argument \var{parser} specifies a \class{DocTestParser} (or
+ subclass) that should be used to extract tests from the files. It
+ defaults to a normal parser (i.e., \code{\class{DocTestParser}()}).
+
+ Optional argument \var{encoding} specifies an encoding that should
+ be used to convert the file to unicode.
+
+ \versionadded{2.4}
+
+ \versionchanged[The global \code{__file__} was added to the
+ globals provided to doctests loaded from a text file using
+ \function{DocFileSuite()}]{2.5}
+
+ \versionchanged[The parameter \var{encoding} was added]{2.5}
+
+\end{funcdesc}
+
+\begin{funcdesc}{DocTestSuite}{\optional{module}\optional{,
+ globs}\optional{, extraglobs}\optional{,
+ test_finder}\optional{, setUp}\optional{,
+ tearDown}\optional{, checker}}
+ Convert doctest tests for a module to a
+ \class{\refmodule{unittest}.TestSuite}.
+
+ The returned \class{\refmodule{unittest}.TestSuite} is to be run by the
+ unittest framework and runs each doctest in the module. If any of the
+ doctests fail, then the synthesized unit test fails, and a
+ \exception{failureException} exception is raised showing the name of the
+ file containing the test and a (sometimes approximate) line number.
+
+ Optional argument \var{module} provides the module to be tested. It
+ can be a module object or a (possibly dotted) module name. If not
+ specified, the module calling this function is used.
+
+ Optional argument \var{globs} is a dictionary containing the
+ initial global variables for the tests. A new copy of this
+ dictionary is created for each test. By default, \var{globs} is
+ a new empty dictionary.
+
+ Optional argument \var{extraglobs} specifies an extra set of
+ global variables, which is merged into \var{globs}. By default, no
+ extra globals are used.
+
+ Optional argument \var{test_finder} is the \class{DocTestFinder}
+ object (or a drop-in replacement) that is used to extract doctests
+ from the module.
+
+ Optional arguments \var{setUp}, \var{tearDown}, and \var{optionflags}
+ are the same as for function \function{DocFileSuite()} above.
+
+ \versionadded{2.3}
+
+ \versionchanged[The parameters \var{globs}, \var{extraglobs},
+ \var{test_finder}, \var{setUp}, \var{tearDown}, and
+ \var{optionflags} were added; this function now uses the same search
+ technique as \function{testmod()}]{2.4}
+\end{funcdesc}
+
+Under the covers, \function{DocTestSuite()} creates a
+\class{\refmodule{unittest}.TestSuite} out of \class{doctest.DocTestCase}
+instances, and \class{DocTestCase} is a subclass of
+\class{\refmodule{unittest}.TestCase}. \class{DocTestCase} isn't documented
+here (it's an internal detail), but studying its code can answer questions
+about the exact details of \refmodule{unittest} integration.
+
+Similarly, \function{DocFileSuite()} creates a
+\class{\refmodule{unittest}.TestSuite} out of \class{doctest.DocFileCase}
+instances, and \class{DocFileCase} is a subclass of \class{DocTestCase}.
+
+So both ways of creating a \class{\refmodule{unittest}.TestSuite} run
+instances of \class{DocTestCase}. This is important for a subtle reason:
+when you run \refmodule{doctest} functions yourself, you can control the
+\refmodule{doctest} options in use directly, by passing option flags to
+\refmodule{doctest} functions. However, if you're writing a
+\refmodule{unittest} framework, \refmodule{unittest} ultimately controls
+when and how tests get run. The framework author typically wants to
+control \refmodule{doctest} reporting options (perhaps, e.g., specified by
+command line options), but there's no way to pass options through
+\refmodule{unittest} to \refmodule{doctest} test runners.
+
+For this reason, \refmodule{doctest} also supports a notion of
+\refmodule{doctest} reporting flags specific to \refmodule{unittest}
+support, via this function:
+
+\begin{funcdesc}{set_unittest_reportflags}{flags}
+ Set the \refmodule{doctest} reporting flags to use.
+
+ Argument \var{flags} or's together option flags. See
+ section~\ref{doctest-options}. Only "reporting flags" can be used.
+
+ This is a module-global setting, and affects all future doctests run by
+ module \refmodule{unittest}: the \method{runTest()} method of
+ \class{DocTestCase} looks at the option flags specified for the test case
+ when the \class{DocTestCase} instance was constructed. If no reporting
+ flags were specified (which is the typical and expected case),
+ \refmodule{doctest}'s \refmodule{unittest} reporting flags are or'ed into
+ the option flags, and the option flags so augmented are passed to the
+ \class{DocTestRunner} instance created to run the doctest. If any
+ reporting flags were specified when the \class{DocTestCase} instance was
+ constructed, \refmodule{doctest}'s \refmodule{unittest} reporting flags
+ are ignored.
+
+ The value of the \refmodule{unittest} reporting flags in effect before the
+ function was called is returned by the function.
+
+ \versionadded{2.4}
+\end{funcdesc}
+
+
+\subsection{Advanced API\label{doctest-advanced-api}}
+
+The basic API is a simple wrapper that's intended to make doctest easy
+to use. It is fairly flexible, and should meet most users' needs;
+however, if you require more fine-grained control over testing, or
+wish to extend doctest's capabilities, then you should use the
+advanced API.
+
+The advanced API revolves around two container classes, which are used
+to store the interactive examples extracted from doctest cases:
+
+\begin{itemize}
+\item \class{Example}: A single python statement, paired with its
+ expected output.
+\item \class{DocTest}: A collection of \class{Example}s, typically
+ extracted from a single docstring or text file.
+\end{itemize}
+
+Additional processing classes are defined to find, parse, and run, and
+check doctest examples:
+
+\begin{itemize}
+\item \class{DocTestFinder}: Finds all docstrings in a given module,
+ and uses a \class{DocTestParser} to create a \class{DocTest}
+ from every docstring that contains interactive examples.
+\item \class{DocTestParser}: Creates a \class{DocTest} object from
+ a string (such as an object's docstring).
+\item \class{DocTestRunner}: Executes the examples in a
+ \class{DocTest}, and uses an \class{OutputChecker} to verify
+ their output.
+\item \class{OutputChecker}: Compares the actual output from a
+ doctest example with the expected output, and decides whether
+ they match.
+\end{itemize}
+
+The relationships among these processing classes are summarized in the
+following diagram:
+
+\begin{verbatim}
+ list of:
++------+ +---------+
+|module| --DocTestFinder-> | DocTest | --DocTestRunner-> results
++------+ | ^ +---------+ | ^ (printed)
+ | | | Example | | |
+ v | | ... | v |
+ DocTestParser | Example | OutputChecker
+ +---------+
+\end{verbatim}
+
+\subsubsection{DocTest Objects\label{doctest-DocTest}}
+\begin{classdesc}{DocTest}{examples, globs, name, filename, lineno,
+ docstring}
+ A collection of doctest examples that should be run in a single
+ namespace. The constructor arguments are used to initialize the
+ member variables of the same names.
+ \versionadded{2.4}
+\end{classdesc}
+
+\class{DocTest} defines the following member variables. They are
+initialized by the constructor, and should not be modified directly.
+
+\begin{memberdesc}{examples}
+ A list of \class{Example} objects encoding the individual
+ interactive Python examples that should be run by this test.
+\end{memberdesc}
+
+\begin{memberdesc}{globs}
+ The namespace (aka globals) that the examples should be run in.
+ This is a dictionary mapping names to values. Any changes to the
+ namespace made by the examples (such as binding new variables)
+ will be reflected in \member{globs} after the test is run.
+\end{memberdesc}
+
+\begin{memberdesc}{name}
+ A string name identifying the \class{DocTest}. Typically, this is
+ the name of the object or file that the test was extracted from.
+\end{memberdesc}
+
+\begin{memberdesc}{filename}
+ The name of the file that this \class{DocTest} was extracted from;
+ or \code{None} if the filename is unknown, or if the
+ \class{DocTest} was not extracted from a file.
+\end{memberdesc}
+
+\begin{memberdesc}{lineno}
+ The line number within \member{filename} where this
+ \class{DocTest} begins, or \code{None} if the line number is
+ unavailable. This line number is zero-based with respect to the
+ beginning of the file.
+\end{memberdesc}
+
+\begin{memberdesc}{docstring}
+ The string that the test was extracted from, or `None` if the
+ string is unavailable, or if the test was not extracted from a
+ string.
+\end{memberdesc}
+
+\subsubsection{Example Objects\label{doctest-Example}}
+\begin{classdesc}{Example}{source, want\optional{,
+ exc_msg}\optional{, lineno}\optional{,
+ indent}\optional{, options}}
+ A single interactive example, consisting of a Python statement and
+ its expected output. The constructor arguments are used to
+ initialize the member variables of the same names.
+ \versionadded{2.4}
+\end{classdesc}
+
+\class{Example} defines the following member variables. They are
+initialized by the constructor, and should not be modified directly.
+
+\begin{memberdesc}{source}
+ A string containing the example's source code. This source code
+ consists of a single Python statement, and always ends with a
+ newline; the constructor adds a newline when necessary.
+\end{memberdesc}
+
+\begin{memberdesc}{want}
+ The expected output from running the example's source code (either
+ from stdout, or a traceback in case of exception). \member{want}
+ ends with a newline unless no output is expected, in which case
+ it's an empty string. The constructor adds a newline when
+ necessary.
+\end{memberdesc}
+
+\begin{memberdesc}{exc_msg}
+ The exception message generated by the example, if the example is
+ expected to generate an exception; or \code{None} if it is not
+ expected to generate an exception. This exception message is
+ compared against the return value of
+ \function{traceback.format_exception_only()}. \member{exc_msg}
+ ends with a newline unless it's \code{None}. The constructor adds
+ a newline if needed.
+\end{memberdesc}
+
+\begin{memberdesc}{lineno}
+ The line number within the string containing this example where
+ the example begins. This line number is zero-based with respect
+ to the beginning of the containing string.
+\end{memberdesc}
+
+\begin{memberdesc}{indent}
+ The example's indentation in the containing string, i.e., the
+ number of space characters that precede the example's first
+ prompt.
+\end{memberdesc}
+
+\begin{memberdesc}{options}
+ A dictionary mapping from option flags to \code{True} or
+ \code{False}, which is used to override default options for this
+ example. Any option flags not contained in this dictionary are
+ left at their default value (as specified by the
+ \class{DocTestRunner}'s \member{optionflags}).
+ By default, no options are set.
+\end{memberdesc}
+
+\subsubsection{DocTestFinder objects\label{doctest-DocTestFinder}}
+\begin{classdesc}{DocTestFinder}{\optional{verbose}\optional{,
+ parser}\optional{, recurse}\optional{,
+ exclude_empty}}
+ A processing class used to extract the \class{DocTest}s that are
+ relevant to a given object, from its docstring and the docstrings
+ of its contained objects. \class{DocTest}s can currently be
+ extracted from the following object types: modules, functions,
+ classes, methods, staticmethods, classmethods, and properties.
+
+ The optional argument \var{verbose} can be used to display the
+ objects searched by the finder. It defaults to \code{False} (no
+ output).
+
+ The optional argument \var{parser} specifies the
+ \class{DocTestParser} object (or a drop-in replacement) that is
+ used to extract doctests from docstrings.
+
+ If the optional argument \var{recurse} is false, then
+ \method{DocTestFinder.find()} will only examine the given object,
+ and not any contained objects.
+
+ If the optional argument \var{exclude_empty} is false, then
+ \method{DocTestFinder.find()} will include tests for objects with
+ empty docstrings.
+
+ \versionadded{2.4}
+\end{classdesc}
+
+\class{DocTestFinder} defines the following method:
+
+\begin{methoddesc}{find}{obj\optional{, name}\optional{,
+ module}\optional{, globs}\optional{, extraglobs}}
+ Return a list of the \class{DocTest}s that are defined by
+ \var{obj}'s docstring, or by any of its contained objects'
+ docstrings.
+
+ The optional argument \var{name} specifies the object's name; this
+ name will be used to construct names for the returned
+ \class{DocTest}s. If \var{name} is not specified, then
+ \code{\var{obj}.__name__} is used.
+
+ The optional parameter \var{module} is the module that contains
+ the given object. If the module is not specified or is None, then
+ the test finder will attempt to automatically determine the
+ correct module. The object's module is used:
+
+ \begin{itemize}
+ \item As a default namespace, if \var{globs} is not specified.
+ \item To prevent the DocTestFinder from extracting DocTests
+ from objects that are imported from other modules. (Contained
+ objects with modules other than \var{module} are ignored.)
+ \item To find the name of the file containing the object.
+ \item To help find the line number of the object within its file.
+ \end{itemize}
+
+ If \var{module} is \code{False}, no attempt to find the module
+ will be made. This is obscure, of use mostly in testing doctest
+ itself: if \var{module} is \code{False}, or is \code{None} but
+ cannot be found automatically, then all objects are considered to
+ belong to the (non-existent) module, so all contained objects will
+ (recursively) be searched for doctests.
+
+ The globals for each \class{DocTest} is formed by combining
+ \var{globs} and \var{extraglobs} (bindings in \var{extraglobs}
+ override bindings in \var{globs}). A new shallow copy of the globals
+ dictionary is created for each \class{DocTest}. If \var{globs} is
+ not specified, then it defaults to the module's \var{__dict__}, if
+ specified, or \code{\{\}} otherwise. If \var{extraglobs} is not
+ specified, then it defaults to \code{\{\}}.
+\end{methoddesc}
+
+\subsubsection{DocTestParser objects\label{doctest-DocTestParser}}
+\begin{classdesc}{DocTestParser}{}
+ A processing class used to extract interactive examples from a
+ string, and use them to create a \class{DocTest} object.
+ \versionadded{2.4}
+\end{classdesc}
+
+\class{DocTestParser} defines the following methods:
+
+\begin{methoddesc}{get_doctest}{string, globs, name, filename, lineno}
+ Extract all doctest examples from the given string, and collect
+ them into a \class{DocTest} object.
+
+ \var{globs}, \var{name}, \var{filename}, and \var{lineno} are
+ attributes for the new \class{DocTest} object. See the
+ documentation for \class{DocTest} for more information.
+\end{methoddesc}
+
+\begin{methoddesc}{get_examples}{string\optional{, name}}
+ Extract all doctest examples from the given string, and return
+ them as a list of \class{Example} objects. Line numbers are
+ 0-based. The optional argument \var{name} is a name identifying
+ this string, and is only used for error messages.
+\end{methoddesc}
+
+\begin{methoddesc}{parse}{string\optional{, name}}
+ Divide the given string into examples and intervening text, and
+ return them as a list of alternating \class{Example}s and strings.
+ Line numbers for the \class{Example}s are 0-based. The optional
+ argument \var{name} is a name identifying this string, and is only
+ used for error messages.
+\end{methoddesc}
+
+\subsubsection{DocTestRunner objects\label{doctest-DocTestRunner}}
+\begin{classdesc}{DocTestRunner}{\optional{checker}\optional{,
+ verbose}\optional{, optionflags}}
+ A processing class used to execute and verify the interactive
+ examples in a \class{DocTest}.
+
+ The comparison between expected outputs and actual outputs is done
+ by an \class{OutputChecker}. This comparison may be customized
+ with a number of option flags; see section~\ref{doctest-options}
+ for more information. If the option flags are insufficient, then
+ the comparison may also be customized by passing a subclass of
+ \class{OutputChecker} to the constructor.
+
+ The test runner's display output can be controlled in two ways.
+ First, an output function can be passed to
+ \method{TestRunner.run()}; this function will be called with
+ strings that should be displayed. It defaults to
+ \code{sys.stdout.write}. If capturing the output is not
+ sufficient, then the display output can be also customized by
+ subclassing DocTestRunner, and overriding the methods
+ \method{report_start}, \method{report_success},
+ \method{report_unexpected_exception}, and \method{report_failure}.
+
+ The optional keyword argument \var{checker} specifies the
+ \class{OutputChecker} object (or drop-in replacement) that should
+ be used to compare the expected outputs to the actual outputs of
+ doctest examples.
+
+ The optional keyword argument \var{verbose} controls the
+ \class{DocTestRunner}'s verbosity. If \var{verbose} is
+ \code{True}, then information is printed about each example, as it
+ is run. If \var{verbose} is \code{False}, then only failures are
+ printed. If \var{verbose} is unspecified, or \code{None}, then
+ verbose output is used iff the command-line switch \programopt{-v}
+ is used.
+
+ The optional keyword argument \var{optionflags} can be used to
+ control how the test runner compares expected output to actual
+ output, and how it displays failures. For more information, see
+ section~\ref{doctest-options}.
+
+ \versionadded{2.4}
+\end{classdesc}
+
+\class{DocTestParser} defines the following methods:
+
+\begin{methoddesc}{report_start}{out, test, example}
+ Report that the test runner is about to process the given example.
+ This method is provided to allow subclasses of
+ \class{DocTestRunner} to customize their output; it should not be
+ called directly.
+
+ \var{example} is the example about to be processed. \var{test} is
+ the test containing \var{example}. \var{out} is the output
+ function that was passed to \method{DocTestRunner.run()}.
+\end{methoddesc}
+
+\begin{methoddesc}{report_success}{out, test, example, got}
+ Report that the given example ran successfully. This method is
+ provided to allow subclasses of \class{DocTestRunner} to customize
+ their output; it should not be called directly.
+
+ \var{example} is the example about to be processed. \var{got} is
+ the actual output from the example. \var{test} is the test
+ containing \var{example}. \var{out} is the output function that
+ was passed to \method{DocTestRunner.run()}.
+\end{methoddesc}
+
+\begin{methoddesc}{report_failure}{out, test, example, got}
+ Report that the given example failed. This method is provided to
+ allow subclasses of \class{DocTestRunner} to customize their
+ output; it should not be called directly.
+
+ \var{example} is the example about to be processed. \var{got} is
+ the actual output from the example. \var{test} is the test
+ containing \var{example}. \var{out} is the output function that
+ was passed to \method{DocTestRunner.run()}.
+\end{methoddesc}
+
+\begin{methoddesc}{report_unexpected_exception}{out, test, example, exc_info}
+ Report that the given example raised an unexpected exception.
+ This method is provided to allow subclasses of
+ \class{DocTestRunner} to customize their output; it should not be
+ called directly.
+
+ \var{example} is the example about to be processed.
+ \var{exc_info} is a tuple containing information about the
+ unexpected exception (as returned by \function{sys.exc_info()}).
+ \var{test} is the test containing \var{example}. \var{out} is the
+ output function that was passed to \method{DocTestRunner.run()}.
+\end{methoddesc}
+
+\begin{methoddesc}{run}{test\optional{, compileflags}\optional{,
+ out}\optional{, clear_globs}}
+ Run the examples in \var{test} (a \class{DocTest} object), and
+ display the results using the writer function \var{out}.
+
+ The examples are run in the namespace \code{test.globs}. If
+ \var{clear_globs} is true (the default), then this namespace will
+ be cleared after the test runs, to help with garbage collection.
+ If you would like to examine the namespace after the test
+ completes, then use \var{clear_globs=False}.
+
+ \var{compileflags} gives the set of flags that should be used by
+ the Python compiler when running the examples. If not specified,
+ then it will default to the set of future-import flags that apply
+ to \var{globs}.
+
+ The output of each example is checked using the
+ \class{DocTestRunner}'s output checker, and the results are
+ formatted by the \method{DocTestRunner.report_*} methods.
+\end{methoddesc}
+
+\begin{methoddesc}{summarize}{\optional{verbose}}
+ Print a summary of all the test cases that have been run by this
+ DocTestRunner, and return a tuple \samp{(\var{failure_count},
+ \var{test_count})}.
+
+ The optional \var{verbose} argument controls how detailed the
+ summary is. If the verbosity is not specified, then the
+ \class{DocTestRunner}'s verbosity is used.
+\end{methoddesc}
+
+\subsubsection{OutputChecker objects\label{doctest-OutputChecker}}
+
+\begin{classdesc}{OutputChecker}{}
+ A class used to check the whether the actual output from a doctest
+ example matches the expected output. \class{OutputChecker}
+ defines two methods: \method{check_output}, which compares a given
+ pair of outputs, and returns true if they match; and
+ \method{output_difference}, which returns a string describing the
+ differences between two outputs.
+ \versionadded{2.4}
+\end{classdesc}
+
+\class{OutputChecker} defines the following methods:
+
+\begin{methoddesc}{check_output}{want, got, optionflags}
+ Return \code{True} iff the actual output from an example
+ (\var{got}) matches the expected output (\var{want}). These
+ strings are always considered to match if they are identical; but
+ depending on what option flags the test runner is using, several
+ non-exact match types are also possible. See
+ section~\ref{doctest-options} for more information about option
+ flags.
+\end{methoddesc}
+
+\begin{methoddesc}{output_difference}{example, got, optionflags}
+ Return a string describing the differences between the expected
+ output for a given example (\var{example}) and the actual output
+ (\var{got}). \var{optionflags} is the set of option flags used to
+ compare \var{want} and \var{got}.
+\end{methoddesc}
+
+\subsection{Debugging\label{doctest-debugging}}
+
+Doctest provides several mechanisms for debugging doctest examples:
+
+\begin{itemize}
+\item Several functions convert doctests to executable Python
+ programs, which can be run under the Python debugger, \refmodule{pdb}.
+\item The \class{DebugRunner} class is a subclass of
+ \class{DocTestRunner} that raises an exception for the first
+ failing example, containing information about that example.
+ This information can be used to perform post-mortem debugging on
+ the example.
+\item The \refmodule{unittest} cases generated by \function{DocTestSuite()}
+ support the \method{debug()} method defined by
+ \class{\refmodule{unittest}.TestCase}.
+\item You can add a call to \function{\refmodule{pdb}.set_trace()} in a
+ doctest example, and you'll drop into the Python debugger when that
+ line is executed. Then you can inspect current values of variables,
+ and so on. For example, suppose \file{a.py} contains just this
+ module docstring:
+
+\begin{verbatim}
+"""
+>>> def f(x):
+... g(x*2)
+>>> def g(x):
+... print x+3
+... import pdb; pdb.set_trace()
+>>> f(3)
+9
+"""
+\end{verbatim}
+
+ Then an interactive Python session may look like this:
+
+\begin{verbatim}
+>>> import a, doctest
+>>> doctest.testmod(a)
+--Return--
+> <doctest a[1]>(3)g()->None
+-> import pdb; pdb.set_trace()
+(Pdb) list
+ 1 def g(x):
+ 2 print x+3
+ 3 -> import pdb; pdb.set_trace()
+[EOF]
+(Pdb) print x
+6
+(Pdb) step
+--Return--
+> <doctest a[0]>(2)f()->None
+-> g(x*2)
+(Pdb) list
+ 1 def f(x):
+ 2 -> g(x*2)
+[EOF]
+(Pdb) print x
+3
+(Pdb) step
+--Return--
+> <doctest a[2]>(1)?()->None
+-> f(3)
+(Pdb) cont
+(0, 3)
+>>>
+\end{verbatim}
+
+ \versionchanged[The ability to use \code{\refmodule{pdb}.set_trace()}
+ usefully inside doctests was added]{2.4}
+\end{itemize}
+
+Functions that convert doctests to Python code, and possibly run
+the synthesized code under the debugger:
+
+\begin{funcdesc}{script_from_examples}{s}
+ Convert text with examples to a script.
+
+ Argument \var{s} is a string containing doctest examples. The string
+ is converted to a Python script, where doctest examples in \var{s}
+ are converted to regular code, and everything else is converted to
+ Python comments. The generated script is returned as a string.
+ For example,
+
+ \begin{verbatim}
+ import doctest
+ print doctest.script_from_examples(r"""
+ Set x and y to 1 and 2.
+ >>> x, y = 1, 2
+
+ Print their sum:
+ >>> print x+y
+ 3
+ """)
+ \end{verbatim}
+
+ displays:
+
+ \begin{verbatim}
+ # Set x and y to 1 and 2.
+ x, y = 1, 2
+ #
+ # Print their sum:
+ print x+y
+ # Expected:
+ ## 3
+ \end{verbatim}
+
+ This function is used internally by other functions (see below), but
+ can also be useful when you want to transform an interactive Python
+ session into a Python script.
+
+ \versionadded{2.4}
+\end{funcdesc}
+
+\begin{funcdesc}{testsource}{module, name}
+ Convert the doctest for an object to a script.
+
+ Argument \var{module} is a module object, or dotted name of a module,
+ containing the object whose doctests are of interest. Argument
+ \var{name} is the name (within the module) of the object with the
+ doctests of interest. The result is a string, containing the
+ object's docstring converted to a Python script, as described for
+ \function{script_from_examples()} above. For example, if module
+ \file{a.py} contains a top-level function \function{f()}, then
+
+\begin{verbatim}
+import a, doctest
+print doctest.testsource(a, "a.f")
+\end{verbatim}
+
+ prints a script version of function \function{f()}'s docstring,
+ with doctests converted to code, and the rest placed in comments.
+
+ \versionadded{2.3}
+\end{funcdesc}
+
+\begin{funcdesc}{debug}{module, name\optional{, pm}}
+ Debug the doctests for an object.
+
+ The \var{module} and \var{name} arguments are the same as for function
+ \function{testsource()} above. The synthesized Python script for the
+ named object's docstring is written to a temporary file, and then that
+ file is run under the control of the Python debugger, \refmodule{pdb}.
+
+ A shallow copy of \code{\var{module}.__dict__} is used for both local
+ and global execution context.
+
+ Optional argument \var{pm} controls whether post-mortem debugging is
+ used. If \var{pm} has a true value, the script file is run directly, and
+ the debugger gets involved only if the script terminates via raising an
+ unhandled exception. If it does, then post-mortem debugging is invoked,
+ via \code{\refmodule{pdb}.post_mortem()}, passing the traceback object
+ from the unhandled exception. If \var{pm} is not specified, or is false,
+ the script is run under the debugger from the start, via passing an
+ appropriate \function{execfile()} call to \code{\refmodule{pdb}.run()}.
+
+ \versionadded{2.3}
+
+ \versionchanged[The \var{pm} argument was added]{2.4}
+\end{funcdesc}
+
+\begin{funcdesc}{debug_src}{src\optional{, pm}\optional{, globs}}
+ Debug the doctests in a string.
+
+ This is like function \function{debug()} above, except that
+ a string containing doctest examples is specified directly, via
+ the \var{src} argument.
+
+ Optional argument \var{pm} has the same meaning as in function
+ \function{debug()} above.
+
+ Optional argument \var{globs} gives a dictionary to use as both
+ local and global execution context. If not specified, or \code{None},
+ an empty dictionary is used. If specified, a shallow copy of the
+ dictionary is used.
+
+ \versionadded{2.4}
+\end{funcdesc}
+
+The \class{DebugRunner} class, and the special exceptions it may raise,
+are of most interest to testing framework authors, and will only be
+sketched here. See the source code, and especially \class{DebugRunner}'s
+docstring (which is a doctest!) for more details:
+
+\begin{classdesc}{DebugRunner}{\optional{checker}\optional{,
+ verbose}\optional{, optionflags}}
+
+ A subclass of \class{DocTestRunner} that raises an exception as
+ soon as a failure is encountered. If an unexpected exception
+ occurs, an \exception{UnexpectedException} exception is raised,
+ containing the test, the example, and the original exception. If
+ the output doesn't match, then a \exception{DocTestFailure}
+ exception is raised, containing the test, the example, and the
+ actual output.
+
+ For information about the constructor parameters and methods, see
+ the documentation for \class{DocTestRunner} in
+ section~\ref{doctest-advanced-api}.
+\end{classdesc}
+
+There are two exceptions that may be raised by \class{DebugRunner}
+instances:
+
+\begin{excclassdesc}{DocTestFailure}{test, example, got}
+ An exception thrown by \class{DocTestRunner} to signal that a
+ doctest example's actual output did not match its expected output.
+ The constructor arguments are used to initialize the member
+ variables of the same names.
+\end{excclassdesc}
+\exception{DocTestFailure} defines the following member variables:
+\begin{memberdesc}{test}
+ The \class{DocTest} object that was being run when the example failed.
+\end{memberdesc}
+\begin{memberdesc}{example}
+ The \class{Example} that failed.
+\end{memberdesc}
+\begin{memberdesc}{got}
+ The example's actual output.
+\end{memberdesc}
+
+\begin{excclassdesc}{UnexpectedException}{test, example, exc_info}
+ An exception thrown by \class{DocTestRunner} to signal that a
+ doctest example raised an unexpected exception. The constructor
+ arguments are used to initialize the member variables of the same
+ names.
+\end{excclassdesc}
+\exception{UnexpectedException} defines the following member variables:
+\begin{memberdesc}{test}
+ The \class{DocTest} object that was being run when the example failed.
+\end{memberdesc}
+\begin{memberdesc}{example}
+ The \class{Example} that failed.
+\end{memberdesc}
+\begin{memberdesc}{exc_info}
+ A tuple containing information about the unexpected exception, as
+ returned by \function{sys.exc_info()}.
+\end{memberdesc}
+
+\subsection{Soapbox\label{doctest-soapbox}}
+
+As mentioned in the introduction, \refmodule{doctest} has grown to have
+three primary uses:
+
+\begin{enumerate}
+\item Checking examples in docstrings.
+\item Regression testing.
+\item Executable documentation / literate testing.
+\end{enumerate}
+
+These uses have different requirements, and it is important to
+distinguish them. In particular, filling your docstrings with obscure
+test cases makes for bad documentation.
+
+When writing a docstring, choose docstring examples with care.
+There's an art to this that needs to be learned---it may not be
+natural at first. Examples should add genuine value to the
+documentation. A good example can often be worth many words.
+If done with care, the examples will be invaluable for your users, and
+will pay back the time it takes to collect them many times over as the
+years go by and things change. I'm still amazed at how often one of
+my \refmodule{doctest} examples stops working after a "harmless"
+change.
+
+Doctest also makes an excellent tool for regression testing, especially if
+you don't skimp on explanatory text. By interleaving prose and examples,
+it becomes much easier to keep track of what's actually being tested, and
+why. When a test fails, good prose can make it much easier to figure out
+what the problem is, and how it should be fixed. It's true that you could
+write extensive comments in code-based testing, but few programmers do.
+Many have found that using doctest approaches instead leads to much clearer
+tests. Perhaps this is simply because doctest makes writing prose a little
+easier than writing code, while writing comments in code is a little
+harder. I think it goes deeper than just that: the natural attitude
+when writing a doctest-based test is that you want to explain the fine
+points of your software, and illustrate them with examples. This in
+turn naturally leads to test files that start with the simplest features,
+and logically progress to complications and edge cases. A coherent
+narrative is the result, instead of a collection of isolated functions
+that test isolated bits of functionality seemingly at random. It's
+a different attitude, and produces different results, blurring the
+distinction between testing and explaining.
+
+Regression testing is best confined to dedicated objects or files. There
+are several options for organizing tests:
+
+\begin{itemize}
+\item Write text files containing test cases as interactive examples,
+ and test the files using \function{testfile()} or
+ \function{DocFileSuite()}. This is recommended, although is
+ easiest to do for new projects, designed from the start to use
+ doctest.
+\item Define functions named \code{_regrtest_\textit{topic}} that
+ consist of single docstrings, containing test cases for the
+ named topics. These functions can be included in the same file
+ as the module, or separated out into a separate test file.
+\item Define a \code{__test__} dictionary mapping from regression test
+ topics to docstrings containing test cases.
+\end{itemize}