add hg and python

1 files changed, 208 insertions, 0 deletions

diff --git a/sys/src/cmd/python/Doc/lib/emailparser.tex b/sys/src/cmd/python/Doc/lib/emailparser.tex
new file mode 100644
index 000000000..609fa4008
--- /dev/null
+++ b/sys/src/cmd/python/Doc/lib/emailparser.tex
@@ -0,0 +1,208 @@
+\declaremodule{standard}{email.parser}
+\modulesynopsis{Parse flat text email messages to produce a message
+	        object structure.}
+
+Message object structures can be created in one of two ways: they can be
+created from whole cloth by instantiating \class{Message} objects and
+stringing them together via \method{attach()} and
+\method{set_payload()} calls, or they can be created by parsing a flat text
+representation of the email message.
+
+The \module{email} package provides a standard parser that understands
+most email document structures, including MIME documents.  You can
+pass the parser a string or a file object, and the parser will return
+to you the root \class{Message} instance of the object structure.  For
+simple, non-MIME messages the payload of this root object will likely
+be a string containing the text of the message.  For MIME
+messages, the root object will return \code{True} from its
+\method{is_multipart()} method, and the subparts can be accessed via
+the \method{get_payload()} and \method{walk()} methods.
+
+There are actually two parser interfaces available for use, the classic
+\class{Parser} API and the incremental \class{FeedParser} API.  The classic
+\class{Parser} API is fine if you have the entire text of the message in
+memory as a string, or if the entire message lives in a file on the file
+system.  \class{FeedParser} is more appropriate for when you're reading the
+message from a stream which might block waiting for more input (e.g. reading
+an email message from a socket).  The \class{FeedParser} can consume and parse
+the message incrementally, and only returns the root object when you close the
+parser\footnote{As of email package version 3.0, introduced in
+Python 2.4, the classic \class{Parser} was re-implemented in terms of the
+\class{FeedParser}, so the semantics and results are identical between the two
+parsers.}.
+
+Note that the parser can be extended in limited ways, and of course
+you can implement your own parser completely from scratch.  There is
+no magical connection between the \module{email} package's bundled
+parser and the \class{Message} class, so your custom parser can create
+message object trees any way it finds necessary.
+
+\subsubsection{FeedParser API}
+
+\versionadded{2.4}
+
+The \class{FeedParser}, imported from the \module{email.feedparser} module,
+provides an API that is conducive to incremental parsing of email messages,
+such as would be necessary when reading the text of an email message from a
+source that can block (e.g. a socket).  The
+\class{FeedParser} can of course be used to parse an email message fully
+contained in a string or a file, but the classic \class{Parser} API may be
+more convenient for such use cases.  The semantics and results of the two
+parser APIs are identical.
+
+The \class{FeedParser}'s API is simple; you create an instance, feed it a
+bunch of text until there's no more to feed it, then close the parser to
+retrieve the root message object.  The \class{FeedParser} is extremely
+accurate when parsing standards-compliant messages, and it does a very good
+job of parsing non-compliant messages, providing information about how a
+message was deemed broken.  It will populate a message object's \var{defects}
+attribute with a list of any problems it found in a message.  See the
+\refmodule{email.errors} module for the list of defects that it can find.
+
+Here is the API for the \class{FeedParser}:
+
+\begin{classdesc}{FeedParser}{\optional{_factory}}
+Create a \class{FeedParser} instance.  Optional \var{_factory} is a
+no-argument callable that will be called whenever a new message object is
+needed.  It defaults to the \class{email.message.Message} class.
+\end{classdesc}
+
+\begin{methoddesc}[FeedParser]{feed}{data}
+Feed the \class{FeedParser} some more data.  \var{data} should be a
+string containing one or more lines.  The lines can be partial and the
+\class{FeedParser} will stitch such partial lines together properly.  The
+lines in the string can have any of the common three line endings, carriage
+return, newline, or carriage return and newline (they can even be mixed).
+\end{methoddesc}
+
+\begin{methoddesc}[FeedParser]{close}{}
+Closing a \class{FeedParser} completes the parsing of all previously fed data,
+and returns the root message object.  It is undefined what happens if you feed
+more data to a closed \class{FeedParser}.
+\end{methoddesc}
+
+\subsubsection{Parser class API}
+
+The \class{Parser} class, imported from the \module{email.parser} module,
+provides an API that can be used to parse a message when the complete contents
+of the message are available in a string or file.  The
+\module{email.parser} module also provides a second class, called
+\class{HeaderParser} which can be used if you're only interested in
+the headers of the message. \class{HeaderParser} can be much faster in
+these situations, since it does not attempt to parse the message body,
+instead setting the payload to the raw body as a string.
+\class{HeaderParser} has the same API as the \class{Parser} class.
+
+\begin{classdesc}{Parser}{\optional{_class}}
+The constructor for the \class{Parser} class takes an optional
+argument \var{_class}.  This must be a callable factory (such as a
+function or a class), and it is used whenever a sub-message object
+needs to be created.  It defaults to \class{Message} (see
+\refmodule{email.message}).  The factory will be called without
+arguments.
+
+The optional \var{strict} flag is ignored.  \deprecated{2.4}{Because the
+\class{Parser} class is a backward compatible API wrapper around the
+new-in-Python 2.4 \class{FeedParser}, \emph{all} parsing is effectively
+non-strict.  You should simply stop passing a \var{strict} flag to the
+\class{Parser} constructor.}
+
+\versionchanged[The \var{strict} flag was added]{2.2.2}
+\versionchanged[The \var{strict} flag was deprecated]{2.4}
+\end{classdesc}
+
+The other public \class{Parser} methods are:
+
+\begin{methoddesc}[Parser]{parse}{fp\optional{, headersonly}}
+Read all the data from the file-like object \var{fp}, parse the
+resulting text, and return the root message object.  \var{fp} must
+support both the \method{readline()} and the \method{read()} methods
+on file-like objects.
+
+The text contained in \var{fp} must be formatted as a block of \rfc{2822}
+style headers and header continuation lines, optionally preceded by a
+envelope header.  The header block is terminated either by the
+end of the data or by a blank line.  Following the header block is the
+body of the message (which may contain MIME-encoded subparts).
+
+Optional \var{headersonly} is as with the \method{parse()} method.
+
+\versionchanged[The \var{headersonly} flag was added]{2.2.2}
+\end{methoddesc}
+
+\begin{methoddesc}[Parser]{parsestr}{text\optional{, headersonly}}
+Similar to the \method{parse()} method, except it takes a string
+object instead of a file-like object.  Calling this method on a string
+is exactly equivalent to wrapping \var{text} in a \class{StringIO}
+instance first and calling \method{parse()}.
+
+Optional \var{headersonly} is a flag specifying whether to stop
+parsing after reading the headers or not.  The default is \code{False},
+meaning it parses the entire contents of the file.
+
+\versionchanged[The \var{headersonly} flag was added]{2.2.2}
+\end{methoddesc}
+
+Since creating a message object structure from a string or a file
+object is such a common task, two functions are provided as a
+convenience.  They are available in the top-level \module{email}
+package namespace.
+
+\begin{funcdesc}{message_from_string}{s\optional{, _class\optional{, strict}}}
+Return a message object structure from a string.  This is exactly
+equivalent to \code{Parser().parsestr(s)}.  Optional \var{_class} and
+\var{strict} are interpreted as with the \class{Parser} class constructor.
+
+\versionchanged[The \var{strict} flag was added]{2.2.2}
+\end{funcdesc}
+
+\begin{funcdesc}{message_from_file}{fp\optional{, _class\optional{, strict}}}
+Return a message object structure tree from an open file object.  This
+is exactly equivalent to \code{Parser().parse(fp)}.  Optional
+\var{_class} and \var{strict} are interpreted as with the
+\class{Parser} class constructor.
+
+\versionchanged[The \var{strict} flag was added]{2.2.2}
+\end{funcdesc}
+
+Here's an example of how you might use this at an interactive Python
+prompt:
+
+\begin{verbatim}
+>>> import email
+>>> msg = email.message_from_string(myString)
+\end{verbatim}
+
+\subsubsection{Additional notes}
+
+Here are some notes on the parsing semantics:
+
+\begin{itemize}
+\item Most non-\mimetype{multipart} type messages are parsed as a single
+      message object with a string payload.  These objects will return
+      \code{False} for \method{is_multipart()}.  Their
+      \method{get_payload()} method will return a string object.
+
+\item All \mimetype{multipart} type messages will be parsed as a
+      container message object with a list of sub-message objects for
+      their payload.  The outer container message will return
+      \code{True} for \method{is_multipart()} and their
+      \method{get_payload()} method will return the list of
+      \class{Message} subparts.
+
+\item Most messages with a content type of \mimetype{message/*}
+      (e.g. \mimetype{message/delivery-status} and
+      \mimetype{message/rfc822}) will also be parsed as container
+      object containing a list payload of length 1.  Their
+      \method{is_multipart()} method will return \code{True}.  The
+      single element in the list payload will be a sub-message object.
+
+\item Some non-standards compliant messages may not be internally consistent
+      about their \mimetype{multipart}-edness.  Such messages may have a
+      \mailheader{Content-Type} header of type \mimetype{multipart}, but their
+      \method{is_multipart()} method may return \code{False}.  If such
+      messages were parsed with the \class{FeedParser}, they will have an
+      instance of the \class{MultipartInvariantViolationDefect} class in their
+      \var{defects} attribute list.  See \refmodule{email.errors} for
+      details.
+\end{itemize}