diff options
| author | cinap_lenrek <cinap_lenrek@localhost> | 2011-05-03 11:25:13 +0000 |
|---|---|---|
| committer | cinap_lenrek <cinap_lenrek@localhost> | 2011-05-03 11:25:13 +0000 |
| commit | 458120dd40db6b4df55a4e96b650e16798ef06a0 (patch) | |
| tree | 8f82685be24fef97e715c6f5ca4c68d34d5074ee /sys/src/cmd/python/Misc | |
| parent | 3a742c699f6806c1145aea5149bf15de15a0afd7 (diff) | |
add hg and python
Diffstat (limited to 'sys/src/cmd/python/Misc')
35 files changed, 29031 insertions, 0 deletions
diff --git a/sys/src/cmd/python/Misc/ACKS b/sys/src/cmd/python/Misc/ACKS new file mode 100644 index 000000000..7524baefc --- /dev/null +++ b/sys/src/cmd/python/Misc/ACKS @@ -0,0 +1,698 @@ +Acknowledgements +---------------- + +This list is not complete and not in any useful order, but I would +like to thank everybody who contributed in any way, with code, hints, +bug reports, ideas, moral support, endorsement, or even complaints.... +Without you I would've stopped working on Python long ago! + + --Guido + +PS: In the standard Python distribution this file is encoded in Latin-1. + +David Abrahams +Jim Ahlstrom +Jyrki Alakuijala +Billy G. Allie +Kevin Altis +Mark Anacker +Anders Andersen +Erik Andersén +John Anderson +Oliver Andrich +Ross Andrus +Jason Asbahr +David Ascher +Peter Åstrand +Chris AtLee +John Aycock +Donovan Baarda +Attila Babo +Alfonso Baciero +Stig Bakken +Greg Ball +Luigi Ballabio +Michael J. Barber +Chris Barker +Quentin Barnes +Cesar Eduardo Barros +Des Barry +Ulf Bartelt +Nick Bastin +Jeff Bauer +Michael R Bax +Anthony Baxter +Samuel L. Bayer +Donald Beaudry +David Beazley +Neal Becker +Robin Becker +Bill Bedford +Reimer Behrends +Ben Bell +Thomas Bellman +Juan M. Bello Rivas +Alexander Belopolsky +Andrew Bennetts +Andy Bensky +Michel Van den Bergh +Eric Beser +Steven Bethard +Stephen Bevan +Ron Bickers +Dominic Binks +Philippe Biondi +Stuart Bishop +Roy Bixler +Mike Bland +Martin Bless +Pablo Bleyer +Erik van Blokland +Eric Blossom +Finn Bock +Paul Boddie +Matthew Boedicker +David Bolen +Gregory Bond +Jurjen Bos +Peter Bosch +Eric Bouck +Thierry Bousch +Monty Brandenberg +Georg Brandl +Terrence Brannon +Dave Brennan +Tom Bridgman +Richard Brodie +Gary S. Brown +Daniel Brotsky +Oleg Broytmann +Dave Brueck +Stan Bubrouski +Erik de Bueger +Jan-Hein B"uhrman +Dick Bulterman +Bill Bumgarner +Jimmy Burgett +Tommy Burnette +Roger Burnham +Alastair Burt +Tarn Weisner Burton +Lee Busby +Ralph Butler +Jp Calderone +Daniel Calvelo +Tony Campbell +Brett Cannon +Mike Carlton +Terry Carroll +Luke Kenneth Casson Leighton +Donn Cave +Per Cederqvist +Octavian Cerna +Hye-Shik Chang +Jeffrey Chang +Brad Chapman +Greg Chapman +Mitch Chapman +David Chaum +Nicolas Chauvat +Michael Chermside +Albert Chin-A-Young +Adal Chiriliuc +Tom Christiansen +Vadim Chugunov +David Cinege +Mike Clarkson +Brad Clements +Steve Clift +Nick Coghlan +Josh Cogliati +Dave Cole +Benjamin Collar +Jeffery Collins +Matt Conway +David M. Cooke +Greg Copeland +Aldo Cortesi +David Costanzo +Scott Cotton +Greg Couch +Steve Cousins +Alex Coventry +Matthew Dixon Cowles +Christopher A. Craig +Laura Creighton +Drew Csillag +Tom Culliton +John Cugini +Andrew Dalke +Lars Damerow +Eric Daniel +Scott David Daniels +Ben Darnell +Jonathan Dasteel +John DeGood +Vincent Delft +Roger Dev +Toby Dickenson +Yves Dionne +Daniel Dittmar +Walter Dörwald +Jaromir Dolecek +Dima Dorfman +Cesar Douady +Dean Draayer +Fred L. Drake, Jr. +John DuBois +Paul Dubois +Quinn Dunkan +Robin Dunn +Luke Dunstan +Andy Dustman +Gary Duzan +Eugene Dvurechenski +Maxim Dzumanenko +Hans Eckardt +Grant Edwards +John Ehresman +Andrew Eland +Lance Ellinghaus +David Ely +Jeff Epler +Tom Epperly +Stoffel Erasmus +Jürgen A. Erhard +Michael Ernst +Ben Escoto +Andy Eskilsson +Stefan Esser +Carey Evans +Stephen D Evans +Tim Everett +Paul Everitt +David Everly +Greg Ewing +Martijn Faassen +Andreas Faerber +Bill Fancher +Mark Favas +Niels Ferguson +Sebastian Fernandez +Vincent Fiack +Russell Finn +Nils Fischbeck +Frederik Fix +Matt Fleming +Hernán Martínez Foffani +Doug Fort +John Fouhy +Martin Franklin +Robin Friedrich +Ivan Frohne +Jim Fulton +Tadayoshi Funaba +Gyro Funch +Peter Funk +Geoff Furnish +Lele Gaifax +Yitzchak Gale +Raymund Galvin +Nitin Ganatra +Fred Gansevles +Lars Marius Garshol +Dan Gass +Andrew Gaul +Stephen M. Gava +Harry Henry Gebel +Marius Gedminas +Thomas Gellekum +Christos Georgiou +Ben Gertzfield +Dinu Gherman +Jonathan Giddy +Johannes Gijsbers +Michael Gilfix +Chris Gonnerman +David Goodger +Hans de Graaff +Eddy De Greef +Duncan Grisby +Dag Gruneau +Michael Guravage +Lars Gustäbel +Barry Haddow +Václav Haisman +Paul ten Hagen +Rasmus Hahn +Peter Haight +Bob Halley +Jesse Hallio +Jun Hamano +Mark Hammond +Manus Hand +Milton L. Hankins +Stephen Hansen +Barry Hantman +Lynda Hardman +Derek Harland +Jason Harper +Gerhard Häring +Larry Hastings +Shane Hathaway +Rycharde Hawkes +Jochen Hayek +Thomas Heller +Lance Finn Helsten +Jonathan Hendry +James Henstridge +Chris Herborth +Ivan Herman +Jürgen Hermann +Gary Herron +Bernhard Herzog +Magnus L. Hetland +Raymond Hettinger +Kevan Heydon +Jason Hildebrand +Richie Hindle +Konrad Hinsen +David Hobley +Tim Hochberg +Joerg-Cyril Hoehle +Gregor Hoffleit +Chris Hoffman +Albert Hofkamp +Jonathan Hogg +Gerrit Holl +Rune Holm +Philip Homburg +Naofumi Honda +Jeffrey Honig +Rob Hooft +Brian Hooper +Randall Hopper +Nadav Horesh +Ken Howard +Brad Howes +Chih-Hao Huang +Lawrence Hudson +Michael Hudson +Jim Hugunin +Greg Humphreys +Eric Huss +Jeremy Hylton +Mihai Ibanescu +Juan David Ibáñez Palomar +Lars Immisch +Tony Ingraldi +John Interrante +Bob Ippolito +Ben Jackson +Paul Jackson +David Jacobs +Kevin Jacobs +Kjetil Jacobsen +Geert Jansen +Jack Jansen +Bill Janssen +Drew Jenkins +Flemming Kjær Jensen +Jiba +Orjan Johansen +Gregory K. Johnson +Simon Johnston +Evan Jones +Richard Jones +Irmen de Jong +Lucas de Jonge +Jens B. Jorgensen +John Jorgensen +Andreas Jung +Tattoo Mabonzo K. +Bob Kahn +Kurt B. Kaiser +Tamito Kajiyama +Peter van Kampen +Jacob Kaplan-Moss +Lou Kates +Sebastien Keim +Randall Kern +Robert Kern +Magnus Kessler +Lawrence Kesteloot +Vivek Khera +Mads Kiilerich +Steve Kirsch +Ron Klatchko +Bastian Kleineidam +Bob Kline +Matthias Klose +Kim Knapp +Lenny Kneler +Pat Knight +Greg Kochanski +Joseph Koshy +Bob Kras +Holger Krekel +Hannu Krosing +Andrew Kuchling +Vladimir Kushnir +Arnaud Mazin +Cameron Laird +Tino Lange +Andrew Langmead +Detlef Lannert +Soren Larsen +Piers Lauder +Ben Laurie +Simon Law +Chris Lawrence +Christopher Lee +Inyeol Lee +John J. Lee +Thomas Lee +Luc Lefebvre +Kip Lehman +Joerg Lehmann +Marc-Andre Lemburg +William Lewis +Robert van Liere +Martin Ligr +Christopher Lindblad +Eric Lindvall +Per Lindqvist +Nick Lockwood +Stephanie Lockwood +Martin von Löwis +Anne Lord +Tom Loredo +Jason Lowe +Tony Lownds +Ray Loyzaga +Loren Luke +Fredrik Lundh +Mark Lutz +Jim Lynch +Mikael Lyngvig +Alan McIntyre +Andrew I MacIntyre +Tim MacKenzie +Nick Maclaren +Steve Majewski +Grzegorz Makarewicz +Ken Manheimer +Vladimir Marangozov +Doug Marien +Alex Martelli +Anthony Martin +Roger Masse +Nick Mathewson +Graham Matthews +Dieter Maurer +Greg McFarlane +Michael McLay +Gordon McMillan +Jay T. Miller +Chris McDonough +Andrew McNamara +Caolan McNamara +Craig McPheeters +Lambert Meertens +Bill van Melle +Luke Mewburn +Mike Meyer +Steven Miale +Trent Mick +Chad Miller +Roman Milner +Dom Mitchell +Doug Moen +Paul Moore +The Dragon De Monsyne +Skip Montanaro +James A Morrison +Sape Mullender +Sjoerd Mullender +Michael Muller +Takahiro Nakayama +Travers Naran +Fredrik Nehr +Tony Nelson +Chad Netzer +Max Neunhöffer +George Neville-Neil +Johannes Nicolai +Samuel Nicolary +Gustavo Niemeyer +Oscar Nierstrasz +Hrvoje Niksic +Bill Noon +Stefan Norberg +Tim Northover +Joe Norton +Neal Norwitz +Nigel O'Brian +Kevin O'Connor +Tim O'Malley +Pascal Oberndoerfer +Jeffrey Ollie +Grant Olson +Piet van Oostrum +Jason Orendorff +Douglas Orr +Denis S. Otkidach +Russel Owen +Mike Pall +Todd R. Palmer +Jan Palus +Alexandre Parenteau +Dan Parisien +Harri Pasanen +Randy Pausch +Ondrej Palkovsky +M. Papillon +Marcel van der Peijl +Samuele Pedroni +Steven Pemberton +Eduardo Pérez +Fernando Pérez +Mark Perrego +Trevor Perrin +Tim Peters +Chris Petrilli +Bjorn Pettersen +Geoff Philbrick +Gavrie Philipson +Adrian Phillips +Christopher J. Phoenix +Neale Pickett +Jean-François Piéronne +Dan Pierson +Martijn Pieters +François Pinard +Zach Pincus +Michael Piotrowski +Iustin Pop +John Popplewell +Amrit Prem +Paul Prescod +Donovan Preston +Steve Purcell +Brian Quinlan +Anders Qvist +Burton Radons +Eric Raymond +Edward K. Ream +Marc Recht +John Redford +Terry Reedy +Steve Reeves +Ofir Reichenberg +Sean Reifschneider +Michael P. Reilly +Bernhard Reiter +Steven Reiz +Roeland Rengelink +Tim Rice +Jan Pieter Riegel +Armin Rigo +Nicholas Riley +Jean-Claude Rimbault +Anthony Roach +Andy Robinson +Jim Robinson +Kevin Rodgers +Mike Romberg +Case Roole +Timothy Roscoe +Craig Rowland +Jim Roskind +Erik van Blokland +Just van Rossum +Hugo van Rossum +Saskia van Rossum +Donald Wallace Rouse II +Liam Routt +Sam Ruby +Paul Rubin +Audun S. Runde +Jeff Rush +Sam Rushing +Mark Russell +Nick Russo +Hajime Saitou +Rich Salz +Kevin Samborn +Ty Sarna +Ben Sayer +Michael Scharf +Neil Schemenauer +David Scherer +Gregor Schmid +Ralf Schmitt +Peter Schneider-Kamp +Chad J. Schroeder +Sam Schulenburg +Stefan Schwarzer +Dietmar Schwertberger +Barry Scott +Steven Scott +Nick Seidenman +Žiga Seilnach +Fred Sells +Jiwon Seo +Denis Severson +Ha Shao +Bruce Sherwood +Pete Shinners +Michael Shiplett +John W. Shipman +Joel Shprentz +Itamar Shtull-Trauring +Eric Siegerman +Paul Sijben +Kirill Simonov +Nathan Paul Simons +Janne Sinkkonen +George Sipe +J. Sipprell +Kragen Sitaker +Christopher Smith +Gregory P. Smith +Rafal Smotrzyk +Dirk Soede +Paul Sokolovsky +Clay Spence +Per Spilling +Joshua Spoerri +Noah Spurrier +Nathan Srebro +RajGopal Srinivasan +Jim St. Pierre +Quentin Stafford-Fraser +Frank Stajano +Oliver Steele +Greg Stein +Chris Stern +Richard Stoakley +Peter Stoehr +Casper Stoel +Michael Stone +Ken Stox +Dan Stromberg +Daniel Stutzbach +Nathan Sullivan +Mark Summerfield +Hisao Suzuki +Kalle Svensson +Paul Swartz +Thenault Sylvain +Geoff Talvola +William Tanksley +Christian Tanzer +Steven Taschuk +Amy Taylor +Tobias Thelen +Robin Thomas +Eric Tiedemann +Tracy Tims +Oren Tirosh +Jason Tishler +Christian Tismer +Frank J. Tobin +R Lindsay Todd +Bennett Todd +Richard Townsend +Laurence Tratt +John Tromp +Jason Trowbridge +Anthony Tuininga +Christopher Tur Lesniewski-Laas +Stephen Turner +Bill Tutt +Doobee R. Tzeck +Lionel Ulmer +Michael Urman +Hector Urtubia +Dmitry Vasiliev +Frank Vercruesse +Jaap Vermeulen +Al Vezza +Jacques A. Vidrine +John Viega +Kannan Vijayan +Kurt Vile +Norman Vine +Frank Visser +Niki W. Waibel +Wojtek Walczak +Charles Waldman +Richard Walker +Larry Wall +Greg Ward +Barry Warsaw +Steve Waterbury +Bob Watson +Aaron Watters +Henrik Weber +Corran Webster +Zack Weinberg +Edward Welbourne +Cliff Wells +Rickard Westman +Mats Wichmann +Truida Wiedijk +Felix Wiemann +Gerry Wiener +Bryce "Zooko" Wilcox-O'Hearn +Gerald S. Williams +John Williams +Sue Williams +Frank Willison +Greg V. Wilson +Jody Winston +Collin Winter +Dik Winter +Blake Winton +Jean-Claude Wippler +Lars Wirzenius +Stefan Witzel +Klaus-Juergen Wolf +Dan Wolfe +Richard Wolff +Gordon Worley +Thomas Wouters +Doug Wyatt +Ka-Ping Yee +Bob Yodlowski +Danny Yoo +George Yoshida +Masazumi Yoshikawa +Bernard Yue +Moshe Zadka +Milan Zamazal +Artur Zaprzala +Mike Zarnstorff +Siebren van der Zee +Uwe Zessin diff --git a/sys/src/cmd/python/Misc/AIX-NOTES b/sys/src/cmd/python/Misc/AIX-NOTES new file mode 100644 index 000000000..613d501d4 --- /dev/null +++ b/sys/src/cmd/python/Misc/AIX-NOTES @@ -0,0 +1,155 @@ +Subject: AIX - Misc/AIX-NOTES +From: Vladimir Marangozov <Vladimir.Marangozov@imag.fr> +To: guido@CNRI.Reston.Va.US (Guido van Rossum) +Date: Wed, 6 Aug 1997 11:41:00 +0200 (EET) + +============================================================================== + COMPILER INFORMATION +------------------------------------------------------------------------------ + +(1) A problem has been reported with "make test" failing because of "weird + indentation." Searching the comp.lang.python newsgroup reveals several + threads on this subject, and it seems to be a compiler bug in an old + version of the AIX CC compiler. However, the compiler/OS combination + which has this problem is not identified. In preparation for the 1.4 + release, Vladimir Marangozov (Vladimir.Marangozov@imag.fr) and Manus Hand + (mhand@csn.net) reported no such troubles for the following compilers and + operating system versions: + AIX C compiler version 3.1.2 on AIX 4.1.3 and AIX 4.1.4 + AIX C compiler version 1.3.0 on AIX 3.2.5 + If you have this problem, please report the compiler/OS version. + +(2) Stefan Esser (se@MI.Uni-Koeln.DE), in work done to compile Python + 1.0.0 on AIX 3.2.4, reports that AIX compilers don't like the LANG + environment varaiable set to European locales. This makes the compiler + generate floating point constants using "," as the decimal seperator, + which the assembler doesn't understand (or perhaps it is the other way + around, with the assembler expecting, but not getting "," in float + numbers). "LANG=C; export LANG" solves the problem, as does + "LANG=C $(MAKE) ..." in the master Makefile. + +(3) The cc (or xlc) compiler considers "Python/ceval.c" too complex to + optimize, except when invoked with "-qmaxmem=4000" + +(4) Some problems (due to _AIX not being #defined) when python 1.0.0 was + compiled using 'gcc -ansi' were reported by Stefan Esser, but were not + investigated. + +(5) The cc compiler has internal variables named "__abs" and "__div". These + names are reserved and may not be used as program variables in compiled + source. (As an anecdote in support of this, the implementation of + Python/operator.c had this problem in the 1.4 beta releases, and the + solution was to re#define some core-source variables having these names, + to give these python variables different names if the build is being done + on AIX.) + +(6) As mentioned in the README, builds done immediately after previous builds + (without "make clean" or "make clobber") sometimes fail for mysterious + reasons. There are some unpredictable results when the configuration + is changed (that is, if you "configure" with different parameters) or if + intermediate changes are made to some files. Performing "make clean" or + "make clobber" resolves the problems. + +============================================================================== + THREAD SUPPORT +------------------------------------------------------------------------------ + +As of AIX version 4, there are two (incompatible) types of pthreads on AIX: + a) AIX DCE pthreads (on AIX 3.2.5) + b) AIX 4 pthreads (on AIX 4.1 and up) +Support has been added to Python to handle the distinction. + +The cc and gcc compilers do not initialize pthreads properly. The only +compilers that can initialize pthreads properly are IBM *_r* compilers, +which use the crt0_r.o module, and which invoke ld with the reentrant +version of libc (libc_r). + +In order to enable thread support, follow these steps: + 1. Uncomment the thread module in Modules/Setup + 2. configure --without-gcc --with-thread ... + 3. make CC="cc_r" OPT="-O -qmaxmem=4000" + +For example, to make with both threads and readline, use: + ./configure --without-gcc --with-thread --with-readline=/usr/local/lib + make CC=cc_r OPT="-O2 -qmaxmem=4000" + +If the "make" which is used ignores the "CC=cc_r" directive, one could alias +the cc command to cc_r (for example, in C-shell, perform an "alias cc cc_r"). + +Vladimir Marangozov (Vladimir.Marangozov@imag.fr) provided this information, +and he reports that a cc_r build initializes threads properly and that all +demos on threads run okay with cc_r. + +============================================================================== + SHARED LIBRARY SUPPORT +------------------------------------------------------------------------------ + +AIX shared library support was added to Python in the 1.4 release by Manus +Hand (mhand@csn.net) and Vladimir Marangozov (Vladimir.Marangozov@imag.fr). + +Python modules may now be built as shared libraries on AIX using the normal +process of uncommenting the "*shared*" line in Modules/Setup before the +build. + +AIX shared libraries require that an "export" and "import" file be provided +at compile time to list all extern symbols which may be shared between +modules. The "export" file (named python.exp) for the modules and the +libraries that belong to the Python core is created by the "makexp_aix" +script before performing the link of the python binary. It lists all global +symbols (exported during the link) of the modules and the libraries that +make up the python executable. + +When shared library modules (.so files) are made, a second shell script +is invoked. This script is named "ld_so_aix" and is also provided with +the distribution in the Modules subdirectory. This script acts as an "ld" +wrapper which hides the explicit management of "export" and "import" files; +it adds the appropriate arguments (in the appropriate order) to the link +command that creates the shared module. Among other things, it specifies +that the "python.exp" file is an "import" file for the shared module. + +At the time of this writing, neither the python.exp file nor the makexp_aix +or ld_so_aix scripts are installed by the make procedure, so you should +remember to keep these and/or copy them to a different location for +safekeeping if you wish to use them to add shared extension modules to +python. However, if the make process has been updated since this writing, +these files MAY have been installed for you during the make by the +LIBAINSTALL rule, in which case the need to make safe copies is obviated. + +If you wish to add a shared extension module to the language, you would follow +the steps given in the example below (the example adds the shared extension +module "spam" to python): + 1. Make sure that "ld_so_aix" and "makexp_aix" are in your path. + 2. The "python.exp" file should be in the current directory. + 3. Issue the following commands or include them in your Makefile: + cc -c spammodule.c + ld_so_aix cc spammodule.o -o spammodule.so + +For more detailed information on the shared library support, examine the +contents of the "ld_so_aix" and "makexp_aix" scripts or refer to the AIX +documentation. + +NOTE: If the extension module is written in C++ and contains templates, + an alternative to "ld_so_aix" is the /usr/lpp/xlC/bin/makeC++SharedLib + script. Chris Myers (myers@TC.Cornell.EDU) reports that ld_so_aix + works well for some C++ (including the C++ that is generated + automatically by the Python SWIG package [SWIG can be found at + http://www.cs.utah.edu/~beazley/SWIG/swig.html]). However, it is not + known whether makeC++SharedLib can be used as a complete substitute + for ld_so_aix. + +According to Gary Hook from IBM, the format of the export file changed +in AIX 4.2. For AIX 4.2 and later, a period "." is required on the +first line after "#!". If python crashes while importing a shared +library, you can try modifying the LINKCC variable in the Makefile. +It probably looks like this: + + LINKCC= $(srcdir)/Modules/makexp_aix Modules/python.exp \"\" $(LIBRARY); $(PURIFY) $(CXX) + +You should modify the \"\" to be a period: + + LINKCC= $(srcdir)/Modules/makexp_aix Modules/python.exp . $(LIBRARY); $(PURIFY) $(CXX) + +Using a period fixed the problem in the snake farm. YMMV. +This fix has been incorporated into Python 2.3. + +============================================================================== diff --git a/sys/src/cmd/python/Misc/BeOS-NOTES b/sys/src/cmd/python/Misc/BeOS-NOTES new file mode 100644 index 000000000..41f25a7f2 --- /dev/null +++ b/sys/src/cmd/python/Misc/BeOS-NOTES @@ -0,0 +1,43 @@ +Python for BeOS R5 + +In Python-2.1, the standard version of the new setup.py program +will not build the full complement of modules on BeOS. Instead, +please replace it with the special BeOS version in Misc/BeOS-setup.py. + +To build, + + 1) cp Misc/BeOS-setup.py setup.py + 2) ./configure --prefix=/boot/home/config + 3) make + +The modules will all build, except termios which assumes some flags +we don't have. Put a libreadline.a in /boot/home/config/lib to get +a readline.so for your interactive editing convenience; NB, not +libreadline.so, you want to link a static readline library into the +dynamically loaded Python module. + +Test: + + make test + + The BeOS is Not UNIX category: + - test_select crashed -- select.error : (-2147459072, 'Bad file descriptor') + - test_socket crashed -- exceptions.AttributeError : SOCK_RAW + - test_fcntl crashed -- exceptions.IOError: [Errno -2147483643] Invalid argument + + This one is funny! BeOS does support large files, and that's why + we get this error: the file is too big for my filesystem! + - test_largefile crashed -- exceptions.IOError: [Errno -2147459065] + No space left on device + + - test_pickle crashed. This is apparently a serious problem, "complex" + number objects reconstructed from a pickle don't compare equal to + their ancestors. But it happens on BeOS PPC only, not Intel. + +Install: + + make install + + +- Donn Cave (donn@oz.net) + October 4, 2000 diff --git a/sys/src/cmd/python/Misc/BeOS-setup.py b/sys/src/cmd/python/Misc/BeOS-setup.py new file mode 100644 index 000000000..991e608fa --- /dev/null +++ b/sys/src/cmd/python/Misc/BeOS-setup.py @@ -0,0 +1,574 @@ +# Autodetecting setup.py script for building the Python extensions +# +# Modified for BeOS build. Donn Cave, March 27 2001. + +__version__ = "special BeOS after 1.37" + +import sys, os, getopt +from distutils import sysconfig +from distutils import text_file +from distutils.errors import * +from distutils.core import Extension, setup +from distutils.command.build_ext import build_ext + +# This global variable is used to hold the list of modules to be disabled. +disabled_module_list = ['dbm', 'mmap', 'resource', 'nis'] + +def find_file(filename, std_dirs, paths): + """Searches for the directory where a given file is located, + and returns a possibly-empty list of additional directories, or None + if the file couldn't be found at all. + + 'filename' is the name of a file, such as readline.h or libcrypto.a. + 'std_dirs' is the list of standard system directories; if the + file is found in one of them, no additional directives are needed. + 'paths' is a list of additional locations to check; if the file is + found in one of them, the resulting list will contain the directory. + """ + + # Check the standard locations + for dir in std_dirs: + f = os.path.join(dir, filename) + if os.path.exists(f): return [] + + # Check the additional directories + for dir in paths: + f = os.path.join(dir, filename) + if os.path.exists(f): + return [dir] + + # Not found anywhere + return None + +def find_library_file(compiler, libname, std_dirs, paths): + filename = compiler.library_filename(libname, lib_type='shared') + result = find_file(filename, std_dirs, paths) + if result is not None: return result + + filename = compiler.library_filename(libname, lib_type='static') + result = find_file(filename, std_dirs, paths) + return result + +def module_enabled(extlist, modname): + """Returns whether the module 'modname' is present in the list + of extensions 'extlist'.""" + extlist = [ext for ext in extlist if ext.name == modname] + return len(extlist) + +class PyBuildExt(build_ext): + + def build_extensions(self): + + # Detect which modules should be compiled + self.detect_modules() + + # Remove modules that are present on the disabled list + self.extensions = [ext for ext in self.extensions + if ext.name not in disabled_module_list] + + # Fix up the autodetected modules, prefixing all the source files + # with Modules/ and adding Python's include directory to the path. + (srcdir,) = sysconfig.get_config_vars('srcdir') + + # Figure out the location of the source code for extension modules + moddir = os.path.join(os.getcwd(), srcdir, 'Modules') + moddir = os.path.normpath(moddir) + srcdir, tail = os.path.split(moddir) + srcdir = os.path.normpath(srcdir) + moddir = os.path.normpath(moddir) + + # Fix up the paths for scripts, too + self.distribution.scripts = [os.path.join(srcdir, filename) + for filename in self.distribution.scripts] + + for ext in self.extensions[:]: + ext.sources = [ os.path.join(moddir, filename) + for filename in ext.sources ] + ext.include_dirs.append( '.' ) # to get config.h + ext.include_dirs.append( os.path.join(srcdir, './Include') ) + + # If a module has already been built statically, + # don't build it here + if ext.name in sys.builtin_module_names: + self.extensions.remove(ext) + + # Parse Modules/Setup to figure out which modules are turned + # on in the file. + input = text_file.TextFile('Modules/Setup', join_lines=1) + remove_modules = [] + while 1: + line = input.readline() + if not line: break + line = line.split() + remove_modules.append( line[0] ) + input.close() + + for ext in self.extensions[:]: + if ext.name in remove_modules: + self.extensions.remove(ext) + + # When you run "make CC=altcc" or something similar, you really want + # those environment variables passed into the setup.py phase. Here's + # a small set of useful ones. + compiler = os.environ.get('CC') + linker_so = os.environ.get('LDSHARED') + args = {} + # unfortunately, distutils doesn't let us provide separate C and C++ + # compilers + if compiler is not None: + args['compiler_so'] = compiler + if linker_so is not None: + args['linker_so'] = linker_so + ' -shared' + self.compiler.set_executables(**args) + + build_ext.build_extensions(self) + + def build_extension(self, ext): + + try: + build_ext.build_extension(self, ext) + except (CCompilerError, DistutilsError), why: + self.announce('WARNING: building of extension "%s" failed: %s' % + (ext.name, sys.exc_info()[1])) + + def get_platform (self): + # Get value of sys.platform + platform = sys.platform + if platform[:6] =='cygwin': + platform = 'cygwin' + elif platform[:4] =='beos': + platform = 'beos' + + return platform + + def detect_modules(self): + try: + belibs = os.environ['BELIBRARIES'].split(';') + except KeyError: + belibs = ['/boot/beos/system/lib'] + belibs.append('/boot/home/config/lib') + self.compiler.library_dirs.append('/boot/home/config/lib') + try: + beincl = os.environ['BEINCLUDES'].split(';') + except KeyError: + beincl = [] + beincl.append('/boot/home/config/include') + self.compiler.include_dirs.append('/boot/home/config/include') + # lib_dirs and inc_dirs are used to search for files; + # if a file is found in one of those directories, it can + # be assumed that no additional -I,-L directives are needed. + lib_dirs = belibs + inc_dirs = beincl + exts = [] + + platform = self.get_platform() + + # Check for MacOS X, which doesn't need libm.a at all + math_libs = ['m'] + if platform in ['Darwin1.2', 'beos']: + math_libs = [] + + # XXX Omitted modules: gl, pure, dl, SGI-specific modules + + # + # The following modules are all pretty straightforward, and compile + # on pretty much any POSIXish platform. + # + + # Some modules that are normally always on: + exts.append( Extension('_weakref', ['_weakref.c']) ) + exts.append( Extension('_symtable', ['symtablemodule.c']) ) + + # array objects + exts.append( Extension('array', ['arraymodule.c']) ) + # complex math library functions + exts.append( Extension('cmath', ['cmathmodule.c'], + libraries=math_libs) ) + + # math library functions, e.g. sin() + exts.append( Extension('math', ['mathmodule.c'], + libraries=math_libs) ) + # fast string operations implemented in C + exts.append( Extension('strop', ['stropmodule.c']) ) + # time operations and variables + exts.append( Extension('time', ['timemodule.c'], + libraries=math_libs) ) + # operator.add() and similar goodies + exts.append( Extension('operator', ['operator.c']) ) + # access to the builtin codecs and codec registry + exts.append( Extension('_codecs', ['_codecsmodule.c']) ) + # Python C API test module + exts.append( Extension('_testcapi', ['_testcapimodule.c']) ) + # static Unicode character database + exts.append( Extension('unicodedata', ['unicodedata.c']) ) + # access to ISO C locale support + exts.append( Extension('_locale', ['_localemodule.c']) ) + + # Modules with some UNIX dependencies -- on by default: + # (If you have a really backward UNIX, select and socket may not be + # supported...) + + # fcntl(2) and ioctl(2) + exts.append( Extension('fcntl', ['fcntlmodule.c']) ) + # pwd(3) + exts.append( Extension('pwd', ['pwdmodule.c']) ) + # grp(3) + exts.append( Extension('grp', ['grpmodule.c']) ) + # posix (UNIX) errno values + exts.append( Extension('errno', ['errnomodule.c']) ) + # select(2); not on ancient System V + exts.append( Extension('select', ['selectmodule.c']) ) + + # The md5 module implements the RSA Data Security, Inc. MD5 + # Message-Digest Algorithm, described in RFC 1321. The necessary files + # md5c.c and md5.h are included here. + exts.append( Extension('md5', ['md5module.c', 'md5c.c']) ) + + # The sha module implements the SHA checksum algorithm. + # (NIST's Secure Hash Algorithm.) + exts.append( Extension('sha', ['shamodule.c']) ) + + # Helper module for various ascii-encoders + exts.append( Extension('binascii', ['binascii.c']) ) + + # Fred Drake's interface to the Python parser + exts.append( Extension('parser', ['parsermodule.c']) ) + + # cStringIO and cPickle + exts.append( Extension('cStringIO', ['cStringIO.c']) ) + exts.append( Extension('cPickle', ['cPickle.c']) ) + + # Memory-mapped files (also works on Win32). + exts.append( Extension('mmap', ['mmapmodule.c']) ) + + # Lance Ellinghaus's syslog daemon interface + exts.append( Extension('syslog', ['syslogmodule.c']) ) + + # George Neville-Neil's timing module: + exts.append( Extension('timing', ['timingmodule.c']) ) + + # + # Here ends the simple stuff. From here on, modules need certain + # libraries, are platform-specific, or present other surprises. + # + + # Multimedia modules + # These don't work for 64-bit platforms!!! + # These represent audio samples or images as strings: + + # Disabled on 64-bit platforms + if sys.maxint != 9223372036854775807L: + # Operations on audio samples + exts.append( Extension('audioop', ['audioop.c']) ) + # Operations on images + exts.append( Extension('imageop', ['imageop.c']) ) + # Read SGI RGB image files (but coded portably) + exts.append( Extension('rgbimg', ['rgbimgmodule.c']) ) + + # readline + if self.compiler.find_library_file(lib_dirs, 'readline'): + readline_libs = ['readline'] + if self.compiler.find_library_file(lib_dirs + + ['/usr/lib/termcap'], + 'termcap'): + readline_libs.append('termcap') + exts.append( Extension('readline', ['readline.c'], + library_dirs=['/usr/lib/termcap'], + libraries=readline_libs) ) + + # The crypt module is now disabled by default because it breaks builds + # on many systems (where -lcrypt is needed), e.g. Linux (I believe). + + if self.compiler.find_library_file(lib_dirs, 'crypt'): + libs = ['crypt'] + else: + libs = [] + exts.append( Extension('crypt', ['cryptmodule.c'], libraries=libs) ) + + # socket(2) + # Detect SSL support for the socket module + ssl_incs = find_file('openssl/ssl.h', inc_dirs, + ['/usr/local/ssl/include', + '/usr/contrib/ssl/include/' + ] + ) + ssl_libs = find_library_file(self.compiler, 'ssl',lib_dirs, + ['/usr/local/ssl/lib', + '/usr/contrib/ssl/lib/' + ] ) + + if (ssl_incs is not None and + ssl_libs is not None): + exts.append( Extension('_socket', ['socketmodule.c'], + include_dirs = ssl_incs, + library_dirs = ssl_libs, + libraries = ['ssl', 'crypto'], + define_macros = [('USE_SSL',1)] ) ) + else: + exts.append( Extension('_socket', ['socketmodule.c']) ) + + # Modules that provide persistent dictionary-like semantics. You will + # probably want to arrange for at least one of them to be available on + # your machine, though none are defined by default because of library + # dependencies. The Python module anydbm.py provides an + # implementation independent wrapper for these; dumbdbm.py provides + # similar functionality (but slower of course) implemented in Python. + + # The standard Unix dbm module: + if platform not in ['cygwin']: + if (self.compiler.find_library_file(lib_dirs, 'ndbm')): + exts.append( Extension('dbm', ['dbmmodule.c'], + libraries = ['ndbm'] ) ) + else: + exts.append( Extension('dbm', ['dbmmodule.c']) ) + + # Anthony Baxter's gdbm module. GNU dbm(3) will require -lgdbm: + if (self.compiler.find_library_file(lib_dirs, 'gdbm')): + exts.append( Extension('gdbm', ['gdbmmodule.c'], + libraries = ['gdbm'] ) ) + + # Berkeley DB interface. + # + # This requires the Berkeley DB code, see + # ftp://ftp.cs.berkeley.edu/pub/4bsd/db.1.85.tar.gz + # + # Edit the variables DB and DBPORT to point to the db top directory + # and the subdirectory of PORT where you built it. + # + # (See http://electricrain.com/greg/python/bsddb3/ for an interface to + # BSD DB 3.x.) + + dblib = [] + if self.compiler.find_library_file(lib_dirs, 'db'): + dblib = ['db'] + + db185_incs = find_file('db_185.h', inc_dirs, + ['/usr/include/db3', '/usr/include/db2']) + db_inc = find_file('db.h', inc_dirs, ['/usr/include/db1']) + if db185_incs is not None: + exts.append( Extension('bsddb', ['bsddbmodule.c'], + include_dirs = db185_incs, + define_macros=[('HAVE_DB_185_H',1)], + libraries = dblib ) ) + elif db_inc is not None: + exts.append( Extension('bsddb', ['bsddbmodule.c'], + include_dirs = db_inc, + libraries = dblib) ) + + # Unix-only modules + if platform not in ['mac', 'win32']: + # Steen Lumholt's termios module + exts.append( Extension('termios', ['termios.c']) ) + # Jeremy Hylton's rlimit interface + if platform not in ['cygwin']: + exts.append( Extension('resource', ['resource.c']) ) + + # Generic dynamic loading module + #exts.append( Extension('dl', ['dlmodule.c']) ) + + # Sun yellow pages. Some systems have the functions in libc. + if platform not in ['cygwin']: + if (self.compiler.find_library_file(lib_dirs, 'nsl')): + libs = ['nsl'] + else: + libs = [] + exts.append( Extension('nis', ['nismodule.c'], + libraries = libs) ) + + # Curses support, requring the System V version of curses, often + # provided by the ncurses library. + if (self.compiler.find_library_file(lib_dirs, 'ncurses')): + curses_libs = ['ncurses'] + exts.append( Extension('_curses', ['_cursesmodule.c'], + libraries = curses_libs) ) + elif (self.compiler.find_library_file(lib_dirs, 'curses')): + if (self.compiler.find_library_file(lib_dirs, 'terminfo')): + curses_libs = ['curses', 'terminfo'] + else: + curses_libs = ['curses', 'termcap'] + + exts.append( Extension('_curses', ['_cursesmodule.c'], + libraries = curses_libs) ) + + # If the curses module is enabled, check for the panel module + if (os.path.exists('Modules/_curses_panel.c') and + module_enabled(exts, '_curses') and + self.compiler.find_library_file(lib_dirs, 'panel')): + exts.append( Extension('_curses_panel', ['_curses_panel.c'], + libraries = ['panel'] + curses_libs) ) + + + + # Lee Busby's SIGFPE modules. + # The library to link fpectl with is platform specific. + # Choose *one* of the options below for fpectl: + + if platform == 'irix5': + # For SGI IRIX (tested on 5.3): + exts.append( Extension('fpectl', ['fpectlmodule.c'], + libraries=['fpe']) ) + elif 0: # XXX how to detect SunPro? + # For Solaris with SunPro compiler (tested on Solaris 2.5 with SunPro C 4.2): + # (Without the compiler you don't have -lsunmath.) + #fpectl fpectlmodule.c -R/opt/SUNWspro/lib -lsunmath -lm + pass + else: + # For other systems: see instructions in fpectlmodule.c. + #fpectl fpectlmodule.c ... + exts.append( Extension('fpectl', ['fpectlmodule.c']) ) + + + # Andrew Kuchling's zlib module. + # This require zlib 1.1.3 (or later). + # See http://www.gzip.org/zlib/ + if (self.compiler.find_library_file(lib_dirs, 'z')): + exts.append( Extension('zlib', ['zlibmodule.c'], + libraries = ['z']) ) + + # Interface to the Expat XML parser + # + # Expat is written by James Clark and must be downloaded separately + # (see below). The pyexpat module was written by Paul Prescod after a + # prototype by Jack Jansen. + # + # The Expat dist includes Windows .lib and .dll files. Home page is + # at http://www.jclark.com/xml/expat.html, the current production + # release is always ftp://ftp.jclark.com/pub/xml/expat.zip. + # + # EXPAT_DIR, below, should point to the expat/ directory created by + # unpacking the Expat source distribution. + # + # Note: the expat build process doesn't yet build a libexpat.a; you + # can do this manually while we try convince the author to add it. To + # do so, cd to EXPAT_DIR, run "make" if you have not done so, then + # run: + # + # ar cr libexpat.a xmltok/*.o xmlparse/*.o + # + expat_defs = [] + expat_incs = find_file('expat.h', inc_dirs, []) + if expat_incs is not None: + # expat.h was found + expat_defs = [('HAVE_EXPAT_H', 1)] + else: + expat_incs = find_file('xmlparse.h', inc_dirs, []) + + if (expat_incs is not None and + self.compiler.find_library_file(lib_dirs, 'expat')): + exts.append( Extension('pyexpat', ['pyexpat.c'], + define_macros = expat_defs, + libraries = ['expat']) ) + + # Platform-specific libraries + if platform == 'linux2': + # Linux-specific modules + exts.append( Extension('linuxaudiodev', ['linuxaudiodev.c']) ) + + if platform == 'sunos5': + # SunOS specific modules + exts.append( Extension('sunaudiodev', ['sunaudiodev.c']) ) + + self.extensions.extend(exts) + + # Call the method for detecting whether _tkinter can be compiled + self.detect_tkinter(inc_dirs, lib_dirs) + + + def detect_tkinter(self, inc_dirs, lib_dirs): + # The _tkinter module. + + # Assume we haven't found any of the libraries or include files + tcllib = tklib = tcl_includes = tk_includes = None + for version in ['8.4', '8.3', '8.2', '8.1', '8.0']: + tklib = self.compiler.find_library_file(lib_dirs, + 'tk' + version ) + tcllib = self.compiler.find_library_file(lib_dirs, + 'tcl' + version ) + if tklib and tcllib: + # Exit the loop when we've found the Tcl/Tk libraries + break + + # Now check for the header files + if tklib and tcllib: + # Check for the include files on Debian, where + # they're put in /usr/include/{tcl,tk}X.Y + debian_tcl_include = [ '/usr/include/tcl' + version ] + debian_tk_include = [ '/usr/include/tk' + version ] + debian_tcl_include + tcl_includes = find_file('tcl.h', inc_dirs, debian_tcl_include) + tk_includes = find_file('tk.h', inc_dirs, debian_tk_include) + + if (tcllib is None or tklib is None and + tcl_includes is None or tk_includes is None): + # Something's missing, so give up + return + + # OK... everything seems to be present for Tcl/Tk. + + include_dirs = [] ; libs = [] ; defs = [] ; added_lib_dirs = [] + for dir in tcl_includes + tk_includes: + if dir not in include_dirs: + include_dirs.append(dir) + + # Check for various platform-specific directories + platform = self.get_platform() + if platform == 'sunos5': + include_dirs.append('/usr/openwin/include') + added_lib_dirs.append('/usr/openwin/lib') + elif os.path.exists('/usr/X11R6/include'): + include_dirs.append('/usr/X11R6/include') + added_lib_dirs.append('/usr/X11R6/lib') + elif os.path.exists('/usr/X11R5/include'): + include_dirs.append('/usr/X11R5/include') + added_lib_dirs.append('/usr/X11R5/lib') + else: + # Assume default location for X11 + include_dirs.append('/usr/X11/include') + added_lib_dirs.append('/usr/X11/lib') + + # Check for BLT extension + if self.compiler.find_library_file(lib_dirs + added_lib_dirs, 'BLT8.0'): + defs.append( ('WITH_BLT', 1) ) + libs.append('BLT8.0') + + # Add the Tcl/Tk libraries + libs.append('tk'+version) + libs.append('tcl'+version) + + if platform in ['aix3', 'aix4']: + libs.append('ld') + + # Finally, link with the X11 libraries + libs.append('X11') + + ext = Extension('_tkinter', ['_tkinter.c', 'tkappinit.c'], + define_macros=[('WITH_APPINIT', 1)] + defs, + include_dirs = include_dirs, + libraries = libs, + library_dirs = added_lib_dirs, + ) + self.extensions.append(ext) + + # XXX handle these, but how to detect? + # *** Uncomment and edit for PIL (TkImaging) extension only: + # -DWITH_PIL -I../Extensions/Imaging/libImaging tkImaging.c \ + # *** Uncomment and edit for TOGL extension only: + # -DWITH_TOGL togl.c \ + # *** Uncomment these for TOGL extension only: + # -lGL -lGLU -lXext -lXmu \ + +def main(): + setup(name = 'Python standard library', + version = '%d.%d' % sys.version_info[:2], + cmdclass = {'build_ext':PyBuildExt}, + # The struct module is defined here, because build_ext won't be + # called unless there's at least one extension module defined. + ext_modules=[Extension('struct', ['structmodule.c'])], + + # Scripts to install + scripts = ['Tools/scripts/pydoc'] + ) + +# --install-platlib +if __name__ == '__main__': + sysconfig.set_python_build() + main() diff --git a/sys/src/cmd/python/Misc/HISTORY b/sys/src/cmd/python/Misc/HISTORY new file mode 100644 index 000000000..ea242db3c --- /dev/null +++ b/sys/src/cmd/python/Misc/HISTORY @@ -0,0 +1,15303 @@ +Python History +-------------- + +This file contains the release messages for previous Python releases. +As you read on you go back to the dark ages of Python's history. + + +====================================================================== + + +What's New in Python 2.4 final? +=============================== + +*Release date: 30-NOV-2004* + +Core and builtins +----------------- + +- Bug 875692: Improve signal handling, especially when using threads, by + forcing an early re-execution of PyEval_EvalFrame() "periodic" code when + things_to_do is not cleared by Py_MakePendingCalls(). + + +What's New in Python 2.4 (release candidate 1) +============================================== + +*Release date: 18-NOV-2004* + +Core and builtins +----------------- + +- Bug 1061968: Fixes in 2.4a3 to address thread bug 1010677 reintroduced + the years-old thread shutdown race bug 225673. Numeric history lesson + aside, all bugs in all three reports are fixed now. + + +Library +------- + +- Bug 1052242: If exceptions are raised by an atexit handler function an + attempt is made to execute the remaining handlers. The last exception + raised is re-raised. + +- ``doctest``'s new support for adding ``pdb.set_trace()`` calls to + doctests was broken in a dramatic but shallow way. Fixed. + +- Bug 1065388: ``calendar``'s ``day_name``, ``day_abbr``, ``month_name``, + and ``month_abbr`` attributes emulate sequences of locale-correct + spellings of month and day names. Because the locale can change at + any time, the correct spelling is recomputed whenever one of these is + indexed. In the worst case, the index may be a slice object, so these + recomputed every day or month name each time they were indexed. This is + much slower than necessary in the usual case, when the index is just an + integer. In that case, only the single spelling needed is recomputed + now; and, when the index is a slice object, only the spellings needed + by the slice are recomputed now. + +- Patch 1061679: Added ``__all__`` to pickletools.py. + +Build +----- + +- Bug 1034277 / Patch 1035255: Remove compilation of core against CoreServices + and CoreFoundation on OS X. Involved removing PyMac_GetAppletScriptFile() + which has no known users. Thanks Bob Ippolito. + +C API +----- + +- The PyRange_New() function is deprecated. + + +What's New in Python 2.4 beta 2? +================================ + +*Release date: 03-NOV-2004* + +License +------- + +The Python Software Foundation changed the license under which Python +is released, to remove Python version numbers. There were no other +changes to the license. So, for example, wherever the license for +Python 2.3 said "Python 2.3", the new license says "Python". The +intent is to make it possible to refer to the PSF license in a more +durable way. For example, some people say they're confused by that +the Open Source Initiative's entry for the Python Software Foundation +License:: + + http://www.opensource.org/licenses/PythonSoftFoundation.php + +says "Python 2.1.1" all over it, wondering whether it applies only +to Python 2.1.1. + +The official name of the new license is the Python Software Foundation +License Version 2. + +Core and builtins +----------------- + +- Bug #1055820 Cyclic garbage collection was not protecting against that + calling a live weakref to a piece of cyclic trash could resurrect an + insane mutation of the trash if any Python code ran during gc (via + running a dead object's __del__ method, running another callback on a + weakref to a dead object, or via any Python code run in any other thread + that managed to obtain the GIL while a __del__ or callback was running + in the thread doing gc). The most likely symptom was "impossible" + ``AttributeError`` exceptions, appearing seemingly at random, on weakly + referenced objects. The cure was to clear all weakrefs to unreachable + objects before allowing any callbacks to run. + +- Bug #1054139 _PyString_Resize() now invalidates its cached hash value. + +Extension Modules +----------------- + +- Bug #1048870: the compiler now generates distinct code objects for + functions with identical bodies. This was producing confusing + traceback messages which pointed to the function where the code + object was first defined rather than the function being executed. + +Library +------- + +- Patch #1056967 changes the semantics of Template.safe_substitute() so that + no ValueError is raised on an 'invalid' match group. Now the delimiter is + returned. + +- Bug #1052503 pdb.runcall() was not passing along keyword arguments. + +- Bug #902037: XML.sax.saxutils.prepare_input_source() now combines relative + paths with a base path before checking os.path.isfile(). + +- The whichdb module can now be run from the command line. + +- Bug #1045381: time.strptime() can now infer the date using %U or %W (week of + the year) when the day of the week and year are also specified. + +- Bug #1048816: fix bug in Ctrl-K at start of line in curses.textpad.Textbox + +- Bug #1017553: fix bug in tarfile.filemode() + +- Patch #737473: fix bug that old source code is shown in tracebacks even if + the source code is updated and reloaded. + +Build +----- + +- Patch #1044395: --enable-shared is allowed in FreeBSD also. + +What's New in Python 2.4 beta 1? +================================ + +*Release date: 15-OCT-2004* + +Core and builtins +----------------- + +- Patch #975056: Restartable signals were not correctly disabled on + BSD systems. Consistently use PyOS_setsig() instead of signal(). + +- The internal portable implementation of thread-local storage (TLS), used + by the ``PyGILState_Ensure()``/``PyGILState_Release()`` API, was not + thread-correct. This could lead to a variety of problems, up to and + including segfaults. See bug 1041645 for an example. + +- Added a command line option, -m module, which searches sys.path for the + module and then runs it. (Contributed by Nick Coghlan.) + +- The bytecode optimizer now folds tuples of constants into a single + constant. + +- SF bug #513866: Float/long comparison anomaly. Prior to 2.4b1, when + an integer was compared to a float, the integer was coerced to a float. + That could yield spurious overflow errors (if the integer was very + large), and to anomalies such as + ``long(1e200)+1 == 1e200 == long(1e200)-1``. Coercion to float is no + longer performed, and cases like ``long(1e200)-1 < 1e200``, + ``long(1e200)+1 > 1e200`` and ``(1 << 20000) > 1e200`` are computed + correctly now. + +Extension modules +----------------- + +- ``collections.deque`` objects didn't play quite right with garbage + collection, which could lead to a segfault in a release build, or + an assert failure in a debug build. Also, added overflow checks, + better detection of mutation during iteration, and shielded deque + comparisons from unusual subclass overrides of the __iter__() method. + +Library +------- + +- Patch 1046644: distutils build_ext grew two new options - --swig for + specifying the swig executable to use, and --swig-opts to specify + options to pass to swig. --swig-opts="-c++" is the new way to spell + --swig-cpp. + +- Patch 983206: distutils now obeys environment variable LDSHARED, if + it is set. + +- Added Peter Astrand's subprocess.py module. See PEP 324 for details. + +- time.strptime() now properly escapes timezones and all other locale-specific + strings for regex-specific symbols. Was breaking under Japanese Windows when + the timezone was specified as "Tokyo (standard time)". + Closes bug #1039270. + +- Updates for the email package: + + + email.Utils.formatdate() grew a 'usegmt' argument for HTTP support. + + All deprecated APIs that in email 2.x issued warnings have been removed: + _encoder argument to the MIMEText constructor, Message.add_payload(), + Utils.dump_address_pair(), Utils.decode(), Utils.encode() + + New deprecations: Generator.__call__(), Message.get_type(), + Message.get_main_type(), Message.get_subtype(), the 'strict' argument to + the Parser constructor. These will be removed in email 3.1. + + Support for Python earlier than 2.3 has been removed (see PEP 291). + + All defect classes have been renamed to end in 'Defect'. + + Some FeedParser fixes; also a MultipartInvariantViolationDefect will be + added to messages that claim to be multipart but really aren't. + + Updates to documentation. + +- re's findall() and finditer() functions now take an optional flags argument + just like the compile(), search(), and match() functions. Also, documented + the previously existing start and stop parameters for the findall() and + finditer() methods of regular expression objects. + +- rfc822 Messages now support iterating over the headers. + +- The (undocumented) tarfile.Tarfile.membernames has been removed; + applications should use the getmember function. + +- httplib now offers symbolic constants for the HTTP status codes. + +- SF bug #1028306: Trying to compare a ``datetime.date`` to a + ``datetime.datetime`` mistakenly compared only the year, month and day. + Now it acts like a mixed-type comparison: ``False`` for ``==``, + ``True`` for ``!=``, and raises ``TypeError`` for other comparison + operators. Because datetime is a subclass of date, comparing only the + base class (date) members can still be done, if that's desired, by + forcing using of the approprate date method; e.g., + ``a_date.__eq__(a_datetime)`` is true if and only if the year, month + and day members of ``a_date`` and ``a_datetime`` are equal. + +- bdist_rpm now supports command line options --force-arch, + {pre,post}-install, {pre,post}-uninstall, and + {prep,build,install,clean,verify}-script. + +- SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support + decoding incomplete input (when the input stream is temporarily exhausted). + ``codecs.StreamReader`` now implements buffering, which enables proper + readline support for the UTF-16 decoders. ``codecs.StreamReader.read()`` + has a new argument ``chars`` which specifies the number of characters to + return. ``codecs.StreamReader.readline()`` and + ``codecs.StreamReader.readlines()`` have a new argument ``keepends``. + Trailing "\n"s will be stripped from the lines if ``keepends`` is false. + +- The documentation for doctest is greatly expanded, and now covers all + the new public features (of which there are many). + +- ``doctest.master`` was put back in, and ``doctest.testmod()`` once again + updates it. This isn't good, because every ``testmod()`` call + contributes to bloating the "hidden" state of ``doctest.master``, but + some old code apparently relies on it. For now, all we can do is + encourage people to stitch doctests together via doctest's unittest + integration features instead. + +- httplib now handles ipv6 address/port pairs. + +- SF bug #1017864: ConfigParser now correctly handles default keys, + processing them with ``ConfigParser.optionxform`` when supplied, + consistent with the handling of config file entries and runtime-set + options. + +- SF bug #997050: Document, test, & check for non-string values in + ConfigParser. Moved the new string-only restriction added in + rev. 1.65 to the SafeConfigParser class, leaving existing + ConfigParser & RawConfigParser behavior alone, and documented the + conditions under which non-string values work. + +Build +----- + +- Building on darwin now includes /opt/local/include and /opt/local/lib for + building extension modules. This is so as to include software installed as + a DarwinPorts port <http://darwinports.opendarwin.org/> + +- pyport.h now defines a Py_IS_NAN macro. It works as-is when the + platform C computes true for ``x != x`` if and only if X is a NaN. + Other platforms can override the default definition with a platform- + specific spelling in that platform's pyconfig.h. You can also override + pyport.h's default Py_IS_INFINITY definition now. + +C API +----- + +- SF patch 1044089: New function ``PyEval_ThreadsInitialized()`` returns + non-zero if PyEval_InitThreads() has been called. + +- The undocumented and unused extern int ``_PyThread_Started`` was removed. + +- The C API calls ``PyInterpreterState_New()`` and ``PyThreadState_New()`` + are two of the very few advertised as being safe to call without holding + the GIL. However, this wasn't true in a debug build, as bug 1041645 + demonstrated. In a debug build, Python redirects the ``PyMem`` family + of calls to Python's small-object allocator, to get the benefit of + its extra debugging capabilities. But Python's small-object allocator + isn't threadsafe, relying on the GIL to avoid the expense of doing its + own locking. ``PyInterpreterState_New()`` and ``PyThreadState_New()`` + call the platform ``malloc()`` directly now, regardless of build type. + +- PyLong_AsUnsignedLong[Mask] now support int objects as well. + +- SF patch #998993: ``PyUnicode_DecodeUTF8Stateful`` and + ``PyUnicode_DecodeUTF16Stateful`` have been added, which implement stateful + decoding. + +Tests +----- + +- test__locale ported to unittest + +Mac +--- + +- ``plistlib`` now supports non-dict root objects. There is also a new + interface for reading and writing plist files: ``readPlist(pathOrFile)`` + and ``writePlist(rootObject, pathOrFile)`` + +Tools/Demos +----------- + +- The text file comparison scripts ``ndiff.py`` and ``diff.py`` now + read the input files in universal-newline mode. This spares them + from consuming a great deal of time to deduce the useless result that, + e.g., a file with Windows line ends and a file with Linux line ends + have no lines in common. + + +What's New in Python 2.4 alpha 3? +================================= + +*Release date: 02-SEP-2004* + +Core and builtins +----------------- + +- SF patch #1007189: ``from ... import ...`` statements now allow the name + list to be surrounded by parentheses. + +- Some speedups for long arithmetic, thanks to Trevor Perrin. Gradeschool + multiplication was sped a little by optimizing the C code. Gradeschool + squaring was sped by about a factor of 2, by exploiting that about half + the digit products are duplicates in a square. Because exponentiation + uses squaring often, this also speeds long power. For example, the time + to compute 17**1000000 dropped from about 14 seconds to 9 on my box due + to this much. The cutoff for Karatsuba multiplication was raised, + since gradeschool multiplication got quicker, and the cutoff was + aggressively small regardless. The exponentiation algorithm was switched + from right-to-left to left-to-right, which is more efficient for small + bases. In addition, if the exponent is large, the algorithm now does + 5 bits (instead of 1 bit) at a time. That cut the time to compute + 17**1000000 on my box in half again, down to about 4.5 seconds. + +- OverflowWarning is no longer generated. PEP 237 scheduled this to + occur in Python 2.3, but since OverflowWarning was disabled by default, + nobody realized it was still being generated. On the chance that user + code is still using them, the Python builtin OverflowWarning, and + corresponding C API PyExc_OverflowWarning, will exist until Python 2.5. + +- Py_InitializeEx has been added. + +- Fix the order of application of decorators. The proper order is bottom-up; + the first decorator listed is the last one called. + +- SF patch #1005778. Fix a seg fault if the list size changed while + calling list.index(). This could happen if a rich comparison function + modified the list. + +- The ``func_name`` (a.k.a. ``__name__``) attribute of user-defined + functions is now writable. + +- code_new (a.k.a new.code()) now checks its arguments sufficiently + carefully that passing them on to PyCode_New() won't trigger calls + to Py_FatalError() or PyErr_BadInternalCall(). It is still the case + that the returned code object might be entirely insane. + +- Subclasses of string can no longer be interned. The semantics of + interning were not clear here -- a subclass could be mutable, for + example -- and had bugs. Explicitly interning a subclass of string + via intern() will raise a TypeError. Internal operations that attempt + to intern a string subclass will have no effect. + +- Bug 1003935: xrange() could report bogus OverflowErrors. Documented + what xrange() intends, and repaired tests accordingly. + +Extension modules +----------------- + +- difflib now supports HTML side-by-side diff. + +- os.urandom has been added for systems that support sources of random + data. + +- Patch 1012740: truncate() on a writeable cStringIO now resets the + position to the end of the stream. This is consistent with the original + StringIO module and avoids inadvertently resurrecting data that was + supposed to have been truncated away. + +- Added socket.socketpair(). + +- Added CurrentByteIndex, CurrentColumnNumber, CurrentLineNumber + members to xml.parsers.expat.XMLParser object. + +- The mpz, rotor, and xreadlines modules, all deprecated in earlier + versions of Python, have now been removed. + +Library +------- + +- Patch #934356: if a module defines __all__, believe that rather than using + heuristics for filtering out imported names. + +- Patch #941486: added os.path.lexists(), which returns True for broken + symlinks, unlike os.path.exists(). + +- the random module now uses os.urandom() for seeding if it is available. + Added a new generator based on os.urandom(). + +- difflib and diff.py can now generate HTML. + +- bdist_rpm now includes version and release in the BuildRoot, and + replaces - by ``_`` in version and release. + +- distutils build/build_scripts now has an -e option to specify the + path to the Python interpreter for installed scripts. + +- PEP 292 classes Template and SafeTemplate are added to the string module. + +- tarfile now generates GNU tar files by default. + +- HTTPResponse has now a getheaders method. + +- Patch #1006219: let inspect.getsource handle '@' decorators. Thanks Simon + Percivall. + +- logging.handlers.SMTPHandler.date_time has been removed; + the class now uses email.Utils.formatdate to generate the time stamp. + +- A new function tkFont.nametofont was added to return an existing + font. The Font class constructor now has an additional exists argument + which, if True, requests to return/configure an existing font, rather + than creating a new one. + +- Updated the decimal package's min() and max() methods to match the + latest revision of the General Decimal Arithmetic Specification. + Quiet NaNs are ignored and equal values are sorted based on sign + and exponent. + +- The decimal package's Context.copy() method now returns deep copies. + +- Deprecated sys.exitfunc in favor of the atexit module. The sys.exitfunc + attribute will be kept around for backwards compatibility and atexit + will just become the one preferred way to do it. + +- patch #675551: Add get_history_item and replace_history_item functions + to the readline module. + +- bug #989672: pdb.doc and the help messages for the help_d and help_u methods + of the pdb.Pdb class gives have been corrected. d(own) goes to a newer + frame, u(p) to an older frame, not the other way around. + +- bug #990669: os.path.realpath() will resolve symlinks before normalizing the + path, as normalizing the path may alter the meaning of the path if it + contains symlinks. + +- bug #851123: shutil.copyfile will raise an exception when trying to copy a + file onto a link to itself. Thanks Gregory Ball. + +- bug #570300: Fix inspect to resolve file locations using os.path.realpath() + so as to properly list all functions in a module when the module itself is + reached through a symlink. Thanks Johannes Gijsbers. + +- doctest refactoring continued. See the docs for details. As part of + this effort, some old and little- (never?) used features are now + deprecated: the Tester class, the module is_private() function, and the + isprivate argument to testmod(). The Tester class supplied a feeble + "by hand" way to combine multiple doctests, if you knew exactly what + you were doing. The newer doctest features for unittest integration + already did a better job of that, are stronger now than ever, and the + new DocTestRunner class is a saner foundation if you want to do it by + hand. The "private name" filtering gimmick was a mistake from the + start, and testmod() changed long ago to ignore it by default. If + you want to filter out tests, the new DocTestFinder class can be used + to return a list of all doctests, and you can filter that list by + any computable criteria before passing it to a DocTestRunner instance. + +- Bug #891637, patch #1005466: fix inspect.getargs() crash on def foo((bar)). + +Tools/Demos +----------- + +- IDLE's shortcut keys for windows are now case insensitive so that + Control-V works the same as Control-v. + +- pygettext.py: Generate POT-Creation-Date header in ISO format. + +Build +----- + +- Backward incompatibility: longintrepr.h now triggers a compile-time + error if SHIFT (the number of bits in a Python long "digit") isn't + divisible by 5. This new requirement allows simple code for the new + 5-bits-at-a-time long_pow() implementation. If necessary, the + restriction could be removed (by complicating long_pow(), or by + falling back to the 1-bit-at-a-time algorithm), but there are no + plans to do so. + +- bug #991962: When building with --disable-toolbox-glue on Darwin no + attempt to build Mac-specific modules occurs. + +- The --with-tsc flag to configure to enable VM profiling with the + processor's timestamp counter now works on PPC platforms. + +- patch #1006629: Define _XOPEN_SOURCE to 500 on Solaris 8/9 to match + GCC's definition and avoid redefinition warnings. + +- Detect pthreads support (provided by gnu pth pthread emulation) on + GNU/k*BSD systems. + +- bug #1005737, #1007249: Fixed several build problems and warnings + found on old/legacy C compilers of HP-UX, IRIX and Tru64. + +C API +----- + +.. + +Documentation +------------- + +- patch #1005936, bug #1009373: fix index entries which contain + an underscore when viewed with Acrobat. + +- bug #990669: os.path.normpath may alter the meaning of a path if + it contains symbolic links. This has been documented in a comment + since 1992, but is now in the library reference as well. + +New platforms +------------- + +- FreeBSD 6 is now supported. + +Tests +----- + +.. + +Windows +------- + +- Boosted the stack reservation for python.exe and pythonw.exe from + the default 1MB to 2MB. Stack frames under VC 7.1 for 2.4 are enough + bigger than under VC 6.0 for 2.3.4 that deeply recursive progams + within the default sys.getrecursionlimit() default value of 1000 were + able to suffer undetected C stack overflows. The standard test program + test_compiler was one such program. If a Python process on Windows + "just vanishes" without a trace, and without an error message of any + kind, but with an exit code of 128, undetected stack overflow may be + the problem. + +Mac +--- + +.. + + +What's New in Python 2.4 alpha 2? +================================= + +*Release date: 05-AUG-2004* + +Core and builtins +----------------- + +- Patch #980695: Implements efficient string concatenation for statements + of the form s=s+t and s+=t. This will vary across implementations. + Accordingly, the str.join() method is strongly preferred for performance + sensitive code. + +- PEP-0318, Function Decorators have been added to the language. These are + implemented using the Java-style @decorator syntax, like so:: + + @staticmethod + def foo(bar): + + (The PEP needs to be updated to reflect the current state) + +- When importing a module M raises an exception, Python no longer leaves M + in sys.modules. Before 2.4a2 it did, and a subsequent import of M would + succeed, picking up a module object from sys.modules reflecting as much + of the initialization of M as completed before the exception was raised. + Subsequent imports got no indication that M was in a partially- + initialized state, and the importers could get into arbitrarily bad + trouble as a result (the M they got was in an unintended state, + arbitrarily far removed from M's author's intent). Now subsequent + imports of M will continue raising exceptions (but if, for example, the + source code for M is edited between import attempts, then perhaps later + attempts will succeed, or raise a different exception). + + This can break existing code, but in such cases the code was probably + working before by accident. In the Python source, the only case of + breakage discovered was in a test accidentally relying on a damaged + module remaining in sys.modules. Cases are also known where tests + deliberately provoking import errors remove damaged modules from + sys.modules themselves, and such tests will break now if they do an + unconditional del sys.modules[M]. + +- u'%s' % obj will now try obj.__unicode__() first and fallback to + obj.__str__() if no __unicode__ method can be found. + +- Patch #550732: Add PyArg_VaParseTupleAndKeywords(). Analogous to + PyArg_VaParse(). Both are now documented. Thanks Greg Chapman. + +- Allow string and unicode return types from .encode()/.decode() + methods on string and unicode objects. Added unicode.decode() + which was missing for no apparent reason. + +- An attempt to fix the mess that is Python's behaviour with + signal handlers and threads, complicated by readline's behaviour. + It's quite possible that there are still bugs here. + +- Added C macros Py_CLEAR and Py_VISIT to ease the implementation of + types that support garbage collection. + +- Compiler now treats None as a constant. + +- The type of values returned by __int__, __float__, __long__, + __oct__, and __hex__ are now checked. Returning an invalid type + will cause a TypeError to be raised. This matches the behavior of + Jython. + +- Implemented bind_textdomain_codeset() in locale module. + +- Added a workaround for proper string operations in BSDs. str.split + and str.is* methods can now work correctly with UTF-8 locales. + +- Bug #989185: unicode.iswide() and unicode.width() is dropped and + the East Asian Width support is moved to unicodedata extension + module. + +- Patch #941229: The source code encoding in interactive mode + now refers sys.stdin.encoding not just ISO-8859-1 anymore. This + allows for non-latin-1 users to write unicode strings directly. + +Extension modules +----------------- + +- cpickle now supports the same keyword arguments as pickle. + +Library +------- + +- Added new codecs and aliases for ISO_8859-11, ISO_8859-16 and + TIS-620 + +- Thanks to Edward Loper, doctest has been massively refactored, and + many new features were added. Full docs will appear later. For now + the doctest module comments and new test cases give good coverage. + The refactoring provides many hook points for customizing behavior + (such as how to report errors, and how to compare expected to actual + output). New features include a <BLANKLINE> marker for expected + output containing blank lines, options to produce unified or context + diffs when actual output doesn't match expectations, an option to + normalize whitespace before comparing, and an option to use an + ellipsis to signify "don't care" regions of output. + +- Tkinter now supports the wish -sync and -use options. + +- The following methods in time support passing of None: ctime(), gmtime(), + and localtime(). If None is provided, the current time is used (the + same as when the argument is omitted). + [SF bug 658254, patch 663482] + +- nntplib does now allow to ignore a .netrc file. + +- urllib2 now recognizes Basic authentication even if other authentication + schemes are offered. + +- Bug #1001053. wave.open() now accepts unicode filenames. + +- gzip.GzipFile has a new fileno() method, to retrieve the handle of the + underlying file object (provided it has a fileno() method). This is + needed if you want to use os.fsync() on a GzipFile. + +- imaplib has two new methods: deleteacl and myrights. + +- nntplib has two new methods: description and descriptions. They + use a more RFC-compliant way of getting a newsgroup description. + +- Bug #993394. Fix a possible red herring of KeyError in 'threading' being + raised during interpreter shutdown from a registered function with atexit + when dummy_threading is being used. + +- Bug #857297/Patch #916874. Fix an error when extracting a hard link + from a tarfile. + +- Patch #846659. Fix an error in tarfile.py when using + GNU longname/longlink creation. + +- The obsolete FCNTL.py has been deleted. The builtin fcntl module + has been available (on platforms that support fcntl) since Python + 1.5a3, and all FCNTL.py did is export fcntl's names, after generating + a deprecation warning telling you to use fcntl directly. + +- Several new unicode codecs are added: big5hkscs, euc_jis_2004, + iso2022_jp_2004, shift_jis_2004. + +- Bug #788520. Queue.{get, get_nowait, put, put_nowait} have new + implementations, exploiting Conditions (which didn't exist at the time + Queue was introduced). A minor semantic change is that the Full and + Empty exceptions raised by non-blocking calls now occur only if the + queue truly was full or empty at the instant the queue was checked (of + course the Queue may no longer be full or empty by the time a calling + thread sees those exceptions, though). Before, the exceptions could + also be raised if it was "merely inconvenient" for the implementation + to determine the true state of the Queue (because the Queue was locked + by some other method in progress). + +- Bugs #979794 and #980117: difflib.get_grouped_opcodes() now handles the + case of comparing two empty lists. This affected both context_diff() and + unified_diff(), + +- Bug #980938: smtplib now prints debug output to sys.stderr. + +- Bug #930024: posixpath.realpath() now handles infinite loops in symlinks by + returning the last point in the path that was not part of any loop. Thanks + AM Kuchling. + +- Bug #980327: ntpath not handles compressing erroneous slashes between the + drive letter and the rest of the path. Also clearly handles UNC addresses now + as well. Thanks Paul Moore. + +- bug #679953: zipfile.py should now work for files over 2 GB. The packed data + for file sizes (compressed and uncompressed) was being stored as signed + instead of unsigned. + +- decimal.py now only uses signals in the IBM spec. The other conditions are + no longer part of the public API. + +- codecs module now has two new generic APIs: encode() and decode() + which don't restrict the return types (unlike the unicode and + string methods of the same name). + +- Non-blocking SSL sockets work again; they were broken in Python 2.3. + SF patch 945642. + +- doctest unittest integration improvements: + + o Improved the unitest test output for doctest-based unit tests + + o Can now pass setUp and tearDown functions when creating + DocTestSuites. + +- The threading module has a new class, local, for creating objects + that provide thread-local data. + +- Bug #990307: when keep_empty_values is True, cgi.parse_qsl() + no longer returns spurious empty fields. + +- Implemented bind_textdomain_codeset() in gettext module. + +- Introduced in gettext module the l*gettext() family of functions, + which return translation strings encoded in the preferred encoding, + as informed by locale module's getpreferredencoding(). + +- optparse module (and tests) upgraded to Optik 1.5a1. Changes: + + - Add expansion of default values in help text: the string + "%default" in an option's help string is expanded to str() of + that option's default value, or "none" if no default value. + + - Bug #955889: option default values that happen to be strings are + now processed in the same way as values from the command line; this + allows generation of nicer help when using custom types. Can + be disabled with parser.set_process_default_values(False). + + - Bug #960515: don't crash when generating help for callback + options that specify 'type', but not 'dest' or 'metavar'. + + - Feature #815264: change the default help format for short options + that take an argument from e.g. "-oARG" to "-o ARG"; add + set_short_opt_delimiter() and set_long_opt_delimiter() methods to + HelpFormatter to allow (slight) customization of the formatting. + + - Patch #736940: internationalize Optik: all built-in user- + targeted literal strings are passed through gettext.gettext(). (If + you want translations (.po files), they're not included with Python + -- you'll find them in the Optik source distribution from + http://optik.sourceforge.net/ .) + + - Bug #878453: respect $COLUMNS environment variable for + wrapping help output. + + - Feature #988122: expand "%prog" in the 'description' passed + to OptionParser, just like in the 'usage' and 'version' strings. + (This is *not* done in the 'description' passed to OptionGroup.) + +C API +----- + +- PyImport_ExecCodeModule() and PyImport_ExecCodeModuleEx(): if an + error occurs while loading the module, these now delete the module's + entry from sys.modules. All ways of loading modules eventually call + one of these, so this is an error-case change in semantics for all + ways of loading modules. In rare cases, a module loader may wish + to keep a module object in sys.modules despite that the module's + code cannot be executed. In such cases, the module loader must + arrange to reinsert the name and module object in sys.modules. + PyImport_ReloadModule() has been changed to reinsert the original + module object into sys.modules if the module reload fails, so that + its visible semantics have not changed. + +- A large pile of datetime field-extraction macros is now documented, + thanks to Anthony Tuininga (patch #986010). + +Documentation +------------- + +- Improved the tutorial on creating types in C. + + - point out the importance of reassigning data members before + assigning their values + + - correct my misconception about return values from visitprocs. Sigh. + + - mention the labor saving Py_VISIT and Py_CLEAR macros. + +- Major rewrite of the math module docs, to address common confusions. + +Tests +----- + +- The test data files for the decimal test suite are now installed on + platforms that use the Makefile. + +- SF patch 995225: The test file testtar.tar accidentally contained + CVS keywords (like $Id: HISTORY 43159 2006-03-20 06:30:41Z anthony.baxter $), which could cause spurious failures in + test_tarfile.py depending on how the test file was checked out. + + +What's New in Python 2.4 alpha 1? +================================= + +*Release date: 08-JUL-2004* + +Core and builtins +----------------- + +- weakref.ref is now the type object also known as + weakref.ReferenceType; it can be subclassed like any other new-style + class. There's less per-entry overhead in WeakValueDictionary + objects now (one object instead of three). + +- Bug #951851: Python crashed when reading import table of certain + Windows DLLs. + +- Bug #215126. The locals argument to eval(), execfile(), and exec now + accept any mapping type. + +- marshal now shares interned strings. This change introduces + a new .pyc magic. + +- Bug #966623. classes created with type() in an exec(, {}) don't + have a __module__, but code in typeobject assumed it would always + be there. + +- Python no longer relies on the LC_NUMERIC locale setting to be + the "C" locale; as a result, it no longer tries to prevent changing + the LC_NUMERIC category. + +- Bug #952807: Unpickling pickled instances of subclasses of + datetime.date, datetime.datetime and datetime.time could yield insane + objects. Thanks to Jiwon Seo for a fix. + +- Bug #845802: Python crashes when __init__.py is a directory. + +- Unicode objects received two new methods: iswide() and width(). + These query East Asian width information, as specified in Unicode + TR11. + +- Improved the tuple hashing algorithm to give fewer collisions in + common cases. Fixes bug #942952. + +- Implemented generator expressions (PEP 289). Coded by Jiwon Seo. + +- Enabled the profiling of C extension functions (and builtins) - check + new documentation and modified profile and bdb modules for more details + +- Set file.name to the object passed to open (instead of a new string) + +- Moved tracebackobject into traceback.h and renamed to PyTracebackObject + +- Optimized the byte coding for multiple assignments like "a,b=b,a" and + "a,b,c=1,2,3". Improves their speed by 25% to 30%. + +- Limit the nested depth of a tuple for the second argument to isinstance() + and issubclass() to the recursion limit of the interpreter. + Fixes bug #858016 . + +- Optimized dict iterators, creating separate types for each + and having them reveal their length. Also optimized the + methods: keys(), values(), and items(). + +- Implemented a newcode opcode, LIST_APPEND, that simplifies + the generated bytecode for list comprehensions and further + improves their performance (about 35%). + +- Implemented rich comparisons for floats, which seems to make + comparisons involving NaNs somewhat less surprising when the + underlying C compiler actually implements C99 semantics. + +- Optimized list.extend() to save memory and no longer create + intermediate sequences. Also, extend() now pre-allocates the + needed memory whenever the length of the iterable is known in + advance -- this halves the time to extend the list. + +- Optimized list resize operations to make fewer calls to the system + realloc(). Significantly speeds up list appends, list pops, + list comprehensions, and the list constructor (when the input iterable + length is not known). + +- Changed the internal list over-allocation scheme. For larger lists, + overallocation ranged between 3% and 25%. Now, it is a constant 12%. + For smaller lists (n<8), overallocation was upto eight elements. Now, + the overallocation is no more than three elements -- this improves space + utilization for applications that have large numbers of small lists. + +- Most list bodies now get re-used rather than freed. Speeds up list + instantiation and deletion by saving calls to malloc() and free(). + +- The dict.update() method now accepts all the same argument forms + as the dict() constructor. This now includes item lists and/or + keyword arguments. + +- Support for arbitrary objects supporting the read-only buffer + interface as the co_code field of code objects (something that was + only possible to create from C code) has been removed. + +- Made omitted callback and None equivalent for weakref.ref() and + weakref.proxy(); the None case wasn't handled correctly in all + cases. + +- Fixed problem where PyWeakref_NewRef() and PyWeakref_NewProxy() + assumed that initial existing entries in an object's weakref list + would not be removed while allocating a new weakref object. Since + GC could be invoked at that time, however, that assumption was + invalid. In a truly obscure case of GC being triggered during + creation for a new weakref object for an referent which already + has a weakref without a callback which is only referenced from + cyclic trash, a memory error can occur. This consistently created a + segfault in a debug build, but provided less predictable behavior in + a release build. + +- input() builtin function now respects compiler flags such as + __future__ statements. SF patch 876178. + +- Removed PendingDeprecationWarning from apply(). apply() remains + deprecated, but the nuisance warning will not be issued. + +- At Python shutdown time (Py_Finalize()), 2.3 called cyclic garbage + collection twice, both before and after tearing down modules. The + call after tearing down modules has been disabled, because too much + of Python has been torn down then for __del__ methods and weakref + callbacks to execute sanely. The most common symptom was a sequence + of uninformative messages on stderr when Python shut down, produced + by threads trying to raise exceptions, but unable to report the nature + of their problems because too much of the sys module had already been + destroyed. + +- Removed FutureWarnings related to hex/oct literals and conversions + and left shifts. (Thanks to Kalle Svensson for SF patch 849227.) + This addresses most of the remaining semantic changes promised by + PEP 237, except for repr() of a long, which still shows the trailing + 'L'. The PEP appears to promise warnings for operations that + changed semantics compared to Python 2.3, but this is not + implemented; we've suffered through enough warnings related to + hex/oct literals and I think it's best to be silent now. + +- For str and unicode objects, the ljust(), center(), and rjust() + methods now accept an optional argument specifying a fill + character other than a space. + +- When method objects have an attribute that can be satisfied either + by the function object or by the method object, the function + object's attribute usually wins. Christian Tismer pointed out that + that this is really a mistake, because this only happens for special + methods (like __reduce__) where the method object's version is + really more appropriate than the function's attribute. So from now + on, all method attributes will have precedence over function + attributes with the same name. + +- Critical bugfix, for SF bug 839548: if a weakref with a callback, + its callback, and its weakly referenced object, all became part of + cyclic garbage during a single run of garbage collection, the order + in which they were torn down was unpredictable. It was possible for + the callback to see partially-torn-down objects, leading to immediate + segfaults, or, if the callback resurrected garbage objects, to + resurrect insane objects that caused segfaults (or other surprises) + later. In one sense this wasn't surprising, because Python's cyclic gc + had no knowledge of Python's weakref objects. It does now. When + weakrefs with callbacks become part of cyclic garbage now, those + weakrefs are cleared first. The callbacks don't trigger then, + preventing the problems. If you need callbacks to trigger, then just + as when cyclic gc is not involved, you need to write your code so + that weakref objects outlive the objects they weakly reference. + +- Critical bugfix, for SF bug 840829: if cyclic garbage collection + happened to occur during a weakref callback for a new-style class + instance, subtle memory corruption was the result (in a release build; + in a debug build, a segfault occurred reliably very soon after). + This has been repaired. + +- Compiler flags set in PYTHONSTARTUP are now active in __main__. + +- Added two builtin types, set() and frozenset(). + +- Added a reversed() builtin function that returns a reverse iterator + over a sequence. + +- Added a sorted() builtin function that returns a new sorted list + from any iterable. + +- CObjects are now mutable (on the C level) through PyCObject_SetVoidPtr. + +- list.sort() now supports three keyword arguments: cmp, key, and reverse. + The key argument can be a function of one argument that extracts a + comparison key from the original record: mylist.sort(key=str.lower). + The reverse argument is a boolean value and if True will change the + sort order as if the comparison arguments were reversed. In addition, + the documentation has been amended to provide a guarantee that all sorts + starting with Py2.3 are guaranteed to be stable (the relative order of + records with equal keys is unchanged). + +- Added test whether wchar_t is signed or not. A signed wchar_t is not + usable as internal unicode type base for Py_UNICODE since the + unicode implementation assumes an unsigned type. + +- Fixed a bug in the cache of length-one Unicode strings that could + lead to a seg fault. The specific problem occurred when an earlier, + non-fatal error left an uninitialized Unicode object in the + freelist. + +- The % formatting operator now supports '%F' which is equivalent to + '%f'. This has always been documented but never implemented. + +- complex(obj) could leak a little memory if obj wasn't a string or + number. + +- zip() with no arguments now returns an empty list instead of raising + a TypeError exception. + +- obj.__contains__() now returns True/False instead of 1/0. SF patch + 820195. + +- Python no longer tries to be smart about recursive comparisons. + When comparing containers with cyclic references to themselves it + will now just hit the recursion limit. See SF patch 825639. + +- str and unicode builtin types now have an rsplit() method that is + same as split() except that it scans the string from the end + working towards the beginning. See SF feature request 801847. + +- Fixed a bug in object.__reduce_ex__ when using protocol 2. Failure + to clear the error when attempts to get the __getstate__ attribute + fail caused intermittent errors and odd behavior. + +- buffer objects based on other objects no longer cache a pointer to + the data and the data length. Instead, the appropriate tp_as_buffer + method is called as necessary. + +- fixed: if a file is opened with an explicit buffer size >= 1, repeated + close() calls would attempt to free() the buffer already free()ed on + the first call. + + +Extension modules +----------------- + +- Added socket.getservbyport(), and make the second argument in + getservbyname() and getservbyport() optional. + +- time module code that deals with input POSIX timestamps will now raise + ValueError if more than a second is lost in precision when the + timestamp is cast to the platform C time_t type. There's no chance + that the platform will do anything sensible with the result in such + cases. This includes ctime(), localtime() and gmtime(). Assorted + fromtimestamp() and utcfromtimestamp() methods in the datetime module + were also protected. Closes bugs #919012 and 975996. + +- fcntl.ioctl now warns if the mutate flag is not specified. + +- nt now properly allows to refer to UNC roots, e.g. in nt.stat(). + +- the weakref module now supports additional objects: array.array, + sre.pattern_objects, file objects, and sockets. + +- operator.isMappingType() and operator.isSequenceType() now give + fewer false positives. + +- socket.sslerror is now a subclass of socket.error . Also added + socket.error to the socket module's C API. + +- Bug #920575: A problem where the _locale module segfaults on + nl_langinfo(ERA) caused by GNU libc's illegal NULL return is fixed. + +- array objects now support the copy module. Also, their resizing + scheme has been updated to match that used for list objects. This improves + the performance (speed and memory usage) of append() operations. + Also, array.array() and array.extend() now accept any iterable argument + for repeated appends without needing to create another temporary array. + +- cStringIO.writelines() now accepts any iterable argument and writes + the lines one at a time rather than joining them and writing once. + Made a parallel change to StringIO.writelines(). Saves memory and + makes suitable for use with generator expressions. + +- time.strftime() now checks that the values in its time tuple argument + are within the proper boundaries to prevent possible crashes from the + platform's C library implementation of strftime(). Can possibly + break code that uses values outside the range that didn't cause + problems previously (such as sitting day of year to 0). Fixes bug + #897625. + +- The socket module now supports Bluetooth sockets, if the + system has <bluetooth/bluetooth.h> + +- Added a collections module containing a new datatype, deque(), + offering high-performance, thread-safe, memory friendly appends + and pops on either side of the deque. + +- Several modules now take advantage of collections.deque() for + improved performance: Queue, mutex, shlex, threading, and pydoc. + +- The operator module has two new functions, attrgetter() and + itemgetter() which are useful for creating fast data extractor + functions for map(), list.sort(), itertools.groupby(), and + other functions that expect a function argument. + +- socket.SHUT_{RD,WR,RDWR} was added. + +- os.getsid was added. + +- The pwd module incorrectly advertised its struct type as + struct_pwent; this has been renamed to struct_passwd. (The old name + is still supported for backwards compatibility.) + +- The xml.parsers.expat module now provides Expat 1.95.7. + +- socket.IPPROTO_IPV6 was added. + +- readline.clear_history was added. + +- select.select() now accepts sequences for its first three arguments. + +- cStringIO now supports the f.closed attribute. + +- The signal module now exposes SIGRTMIN and SIGRTMAX (if available). + +- curses module now supports use_default_colors(). [patch #739124] + +- Bug #811028: ncurses.h breakage on FreeBSD/MacOS X + +- Bug #814613: INET_ADDRSTRLEN fix needed for all compilers on SGI + +- Implemented non-recursive SRE matching scheme (#757624). + +- Implemented (?(id/name)yes|no) support in SRE (#572936). + +- random.seed() with no arguments or None uses time.time() as a default + seed. Modified to match Py2.2 behavior and use fractional seconds so + that successive runs are more likely to produce different sequences. + +- random.Random has a new method, getrandbits(k), which returns an int + with k random bits. This method is now an optional part of the API + for user defined generators. Any generator that defines genrandbits() + can now use randrange() for ranges with a length >= 2**53. Formerly, + randrange would return only even numbers for ranges that large (see + SF bug #812202). Generators that do not define genrandbits() now + issue a warning when randrange() is called with a range that large. + +- itertools has a new function, groupby() for aggregating iterables + into groups sharing the same key (as determined by a key function). + It offers some of functionality of SQL's groupby keyword and of + the Unix uniq filter. + +- itertools now has a new tee() function which produces two independent + iterators from a single iterable. + +- itertools.izip() with no arguments now returns an empty iterator instead + of raising a TypeError exception. + +- Fixed #853061: allow BZ2Compressor.compress() to receive an empty string + as parameter. + +Library +------- + +- Added a new module: cProfile, a C profiler with the same interface as the + profile module. cProfile avoids some of the drawbacks of the hotshot + profiler and provides a bit more information than the other two profilers. + Based on "lsprof" (patch #1212837). + +- Bug #1266283: The new function "lexists" is now in os.path.__all__. + +- Bug #981530: Fix UnboundLocalError in shutil.rmtree(). This affects + the documented behavior: the function passed to the onerror() + handler can now also be os.listdir. + +- Bug #754449: threading.Thread objects no longer mask exceptions raised during + interpreter shutdown with another exception from attempting to handle the + original exception. + +- Added decimal.py per PEP 327. + +- Bug #981299: rsync is now a recognized protocol in urlparse that uses a + "netloc" portion of a URL. + +- Bug #919012: shutil.move() will not try to move a directory into itself. + Thanks Johannes Gijsbers. + +- Bug #934282: pydoc.stripid() is now case-insensitive. Thanks Robin Becker. + +- Bug #823209: cmath.log() now takes an optional base argument so that its + API matches math.log(). + +- Bug #957381: distutils bdist_rpm no longer fails on recent RPM versions + that generate a -debuginfo.rpm + +- os.path.devnull has been added for all supported platforms. + +- Fixed #877165: distutils now picks the right C++ compiler command + on cygwin and mingw32. + +- urllib.urlopen().readline() now handles HTTP/0.9 correctly. + +- refactored site.py into functions. Also wrote regression tests for the + module. + +- The distutils install command now supports the --home option and + installation scheme for all platforms. + +- asyncore.loop now has a repeat count parameter that defaults to + looping forever. + +- The distutils sdist command now ignores all .svn directories, in + addition to CVS and RCS directories. .svn directories hold + administrative files for the Subversion source control system. + +- Added a new module: cookielib. Automatic cookie handling for HTTP + clients. Also, support for cookielib has been added to urllib2, so + urllib2.urlopen() can transparently handle cookies. + +- stringprep.py now uses built-in set() instead of sets.Set(). + +- Bug #876278: Unbounded recursion in modulefinder + +- Bug #780300: Swap public and system ID in LexicalHandler.startDTD. + Applications relying on the wrong order need to be corrected. + +- Bug #926075: Fixed a bug that returns a wrong pattern object + for a string or unicode object in sre.compile() when a different + type pattern with the same value exists. + +- Added countcallers arg to trace.Trace class (--trackcalls command line arg + when run from the command prompt). + +- Fixed a caching bug in platform.platform() where the argument of 'terse' was + not taken into consideration when caching value. + +- Added two new command-line arguments for profile (output file and + default sort). + +- Added global runctx function to profile module + +- Add hlist missing entryconfigure and entrycget methods. + +- The ptcp154 codec was added for Kazakh character set support. + +- Support non-anonymous ftp URLs in urllib2. + +- The encodings package will now apply codec name aliases + first before starting to try the import of the codec module. + This simplifies overriding built-in codecs with external + packages, e.g. the included CJK codecs with the JapaneseCodecs + package, by adjusting the aliases dictionary in encodings.aliases + accordingly. + +- base64 now supports RFC 3548 Base16, Base32, and Base64 encoding and + decoding standards. + +- urllib2 now supports processors. A processor is a handler that + implements an xxx_request or xxx_response method. These methods are + called for all requests. + +- distutils compilers now compile source files in the same order as + they are passed to the compiler. + +- pprint.pprint() and pprint.pformat() now have additional parameters + indent, width and depth. + +- Patch #750542: pprint now will pretty print subclasses of list, tuple + and dict too, as long as they don't overwrite __repr__(). + +- Bug #848614: distutils' msvccompiler fails to find the MSVC6 + compiler because of incomplete registry entries. + +- httplib.HTTP.putrequest now offers to omit the implicit Accept-Encoding. + +- Patch #841977: modulefinder didn't find extension modules in packages + +- imaplib.IMAP4.thread was added. + +- Plugged a minor hole in tempfile.mktemp() due to the use of + os.path.exists(), switched to using os.lstat() directly if possible. + +- bisect.py and heapq.py now have underlying C implementations + for better performance. + +- heapq.py has two new functions, nsmallest() and nlargest(). + +- traceback.format_exc has been added (similar to print_exc but it returns + a string). + +- xmlrpclib.MultiCall has been added. + +- poplib.POP3_SSL has been added. + +- tmpfile.mkstemp now returns an absolute path even if dir is relative. + +- urlparse is RFC 2396 compliant. + +- The fieldnames argument to the csv module's DictReader constructor is now + optional. If omitted, the first row of the file will be used as the + list of fieldnames. + +- encodings.bz2_codec was added for access to bz2 compression + using "a long string".encode('bz2') + +- Various improvements to unittest.py, realigned with PyUnit CVS. + +- dircache now passes exceptions to the caller, instead of returning + empty lists. + +- The bsddb module and dbhash module now support the iterator and + mapping protocols which make them more substitutable for dictionaries + and shelves. + +- The csv module's DictReader and DictWriter classes now accept keyword + arguments. This was an omission in the initial implementation. + +- The email package handles some RFC 2231 parameters with missing + CHARSET fields better. It also includes a patch to parameter + parsing when semicolons appear inside quotes. + +- sets.py now runs under Py2.2. In addition, the argument restrictions + for most set methods (but not the operators) have been relaxed to + allow any iterable. + +- _strptime.py now has a behind-the-scenes caching mechanism for the most + recent TimeRE instance used along with the last five unique directive + patterns. The overall module was also made more thread-safe. + +- random.cunifvariate() and random.stdgamma() were deprecated in Py2.3 + and removed in Py2.4. + +- Bug #823328: urllib2.py's HTTP Digest Auth support works again. + +- Patch #873597: CJK codecs are imported into rank of default codecs. + +Tools/Demos +----------- + +- A hotshotmain script was added to the Tools/scripts directory that + makes it easy to run a script under control of the hotshot profiler. + +- The db2pickle and pickle2db scripts can now dump/load gdbm files. + +- The file order on the command line of the pickle2db script was reversed. + It is now [ picklefile ] dbfile. This provides better symmetry with + db2pickle. The file arguments to both scripts are now source followed by + destination in situations where both files are given. + +- The pydoc script will display a link to the module documentation for + modules determined to be part of the core distribution. The documentation + base directory defaults to http://www.python.org/doc/current/lib/ but can + be changed by setting the PYTHONDOCS environment variable. + +- texcheck.py now detects double word errors. + +- md5sum.py mistakenly opened input files in text mode by default, a + silent and dangerous change from previous releases. It once again + opens input files in binary mode by default. The -t and -b flags + remain for compatibility with the 2.3 release, but -b is the default + now. + +- py-electric-colon now works when pending-delete/delete-selection mode is + in effect + +- py-help-at-point is no longer bound to the F1 key - it's still bound to + C-c C-h + +- Pynche was fixed to not crash when there is no ~/.pynche file and no + -d option was given. + +Build +----- + +- Bug #978645: Modules/getpath.c now builds properly in --disable-framework + build under OS X. + +- Profiling using gprof is now available if Python is configured with + --enable-profiling. + +- Profiling the VM using the Pentium TSC is now possible if Python + is configured --with-tsc. + +- In order to find libraries, setup.py now also looks in /lib64, for use + on AMD64. + +- Bug #934635: Fixed a bug where the configure script couldn't detect + getaddrinfo() properly if the KAME stack had SCTP support. + +- Support for missing ANSI C header files (limits.h, stddef.h, etc) was + removed. + +- Systems requiring the D4, D6 or D7 variants of pthreads are no longer + supported (see PEP 11). + +- Universal newline support can no longer be disabled (see PEP 11). + +- Support for DGUX, SunOS 4, IRIX 4 and Minix was removed (see PEP 11). + +- Support for systems requiring --with-dl-dld or --with-sgi-dl was removed + (see PEP 11). + +- Tests for sizeof(char) were removed since ANSI C mandates that + sizeof(char) must be 1. + +C API +----- + +- Thanks to Anthony Tuininga, the datetime module now supplies a C API + containing type-check macros and constructors. See new docs in the + Python/C API Reference Manual for details. + +- Private function _PyTime_DoubleToTimet added, to convert a Python + timestamp (C double) to platform time_t with some out-of-bounds + checking. Declared in new header file timefuncs.h. It would be + good to expose some other internal timemodule.c functions there. + +- New public functions PyEval_EvaluateFrame and PyGen_New to expose + generator objects. + +- New public functions Py_IncRef() and Py_DecRef(), exposing the + functionality of the Py_XINCREF() and Py_XDECREF macros. Useful for + runtime dynamic embedding of Python. See patch #938302, by Bob + Ippolito. + +- Added a new macro, PySequence_Fast_ITEMS, which retrieves a fast sequence's + underlying array of PyObject pointers. Useful for high speed looping. + +- Created a new method flag, METH_COEXIST, which causes a method to be loaded + even if already defined by a slot wrapper. This allows a __contains__ + method, for example, to co-exist with a defined sq_contains slot. This + is helpful because the PyCFunction can take advantage of optimized calls + whenever METH_O or METH_NOARGS flags are defined. + +- Added a new function, PyDict_Contains(d, k) which is like + PySequence_Contains() but is specific to dictionaries and executes + about 10% faster. + +- Added three new macros: Py_RETURN_NONE, Py_RETURN_TRUE, and Py_RETURN_FALSE. + Each return the singleton they mention after Py_INCREF()ing them. + +- Added a new function, PyTuple_Pack(n, ...) for constructing tuples from a + variable length argument list of Python objects without having to invoke + the more complex machinery of Py_BuildValue(). PyTuple_Pack(3, a, b, c) + is equivalent to Py_BuildValue("(OOO)", a, b, c). + +Windows +------- + +- The _winreg module could segfault when reading very large registry + values, due to unchecked alloca() calls (SF bug 851056). The fix is + uses either PyMem_Malloc(n) or PyString_FromStringAndSize(NULL, n), + as appropriate, followed by a size check. + +- file.truncate() could misbehave if the file was open for update + (modes r+, rb+, w+, wb+), and the most recent file operation before + the truncate() call was an input operation. SF bug 801631. + + +What's New in Python 2.3 final? +=============================== + +*Release date: 29-Jul-2003* + +IDLE +---- + +- Bug 778400: IDLE hangs when selecting "Edit with IDLE" from explorer. + This was unique to Windows, and was fixed by adding an -n switch to + the command the Windows installer creates to execute "Edit with IDLE" + context-menu actions. + +- IDLE displays a new message upon startup: some "personal firewall" + kinds of programs (for example, ZoneAlarm) open a dialog of their + own when any program opens a socket. IDLE does use sockets, talking + on the computer's internal loopback interface. This connection is not + visible on any external interface and no data is sent to or received + from the Internet. So, if you get such a dialog when opening IDLE, + asking whether to let pythonw.exe talk to address 127.0.0.1, say yes, + and rest assured no communication external to your machine is taking + place. If you don't allow it, IDLE won't be able to start. + + +What's New in Python 2.3 release candidate 2? +============================================= + +*Release date: 24-Jul-2003* + +Core and builtins +----------------- + +- It is now possible to import from zipfiles containing additional + data bytes before the zip compatible archive. Zipfiles containing a + comment at the end are still unsupported. + +Extension modules +----------------- + +- A longstanding bug in the parser module's initialization could cause + fatal internal refcount confusion when the module got initialized more + than once. This has been fixed. + +- Fixed memory leak in pyexpat; using the parser's ParseFile() method + with open files that aren't instances of the standard file type + caused an instance of the bound .read() method to be leaked on every + call. + +- Fixed some leaks in the locale module. + +Library +------- + +- Lib/encodings/rot_13.py when used as a script, now more properly + uses the first Python interpreter on your path. + +- Removed caching of TimeRE (and thus LocaleTime) in _strptime.py to + fix a locale related bug in the test suite. Although another patch + was needed to actually fix the problem, the cache code was not + restored. + +IDLE +---- + +- Calltips patches. + +Build +----- + +- For MacOSX, added -mno-fused-madd to BASECFLAGS to fix test_coercion + on Panther (OSX 10.3). + +C API +----- + +Windows +------- + +- The tempfile module could do insane imports on Windows if PYTHONCASEOK + was set, making temp file creation impossible. Repaired. + +- Add a patch to workaround pthread_sigmask() bugs in Cygwin. + +Mac +--- + +- Various fixes to pimp. + +- Scripts runs with pythonw no longer had full window manager access. + +- Don't force boot-disk-only install, for reasons unknown it causes + more problems than it solves. + + +What's New in Python 2.3 release candidate 1? +============================================= + +*Release date: 18-Jul-2003* + +Core and builtins +----------------- + +- The new function sys.getcheckinterval() returns the last value set + by sys.setcheckinterval(). + +- Several bugs in the symbol table phase of the compiler have been + fixed. Errors could be lost and compilation could fail without + reporting an error. SF patch 763201. + +- The interpreter is now more robust about importing the warnings + module. In an executable generated by freeze or similar programs, + earlier versions of 2.3 would fail if the warnings module could + not be found on the file system. Fixes SF bug 771097. + +- A warning about assignments to module attributes that shadow + builtins, present in earlier releases of 2.3, has been removed. + +- It is not possible to create subclasses of builtin types like str + and tuple that define an itemsize. Earlier releases of Python 2.3 + allowed this by mistake, leading to crashes and other problems. + +- The thread_id is now initialized to 0 in a non-thread build. SF bug + 770247. + +- SF bug 762891: "del p[key]" on proxy object no longer raises SystemError. + +Extension modules +----------------- + +- weakref.proxy() can now handle "del obj[i]" for proxy objects + defining __delitem__. Formerly, it generated a SystemError. + +- SSL no longer crashes the interpreter when the remote side disconnects. + +- On Unix the mmap module can again be used to map device files. + +- time.strptime now exclusively uses the Python implementation + contained within the _strptime module. + +- The print slot of weakref proxy objects was removed, because it was + not consistent with the object's repr slot. + +- The mmap module only checks file size for regular files, not + character or block devices. SF patch 708374. + +- The cPickle Pickler garbage collection support was fixed to traverse + the find_class attribute, if present. + +- There are several fixes for the bsddb3 wrapper module. + + bsddb3 no longer crashes if an environment is closed before a cursor + (SF bug 763298). + + The DB and DBEnv set_get_returns_none function was extended to take + a level instead of a boolean flag. The new level 2 means that in + addition, cursor.set()/.get() methods return None instead of raising + an exception. + + A typo was fixed in DBCursor.join_item(), preventing a crash. + +Library +------- + +- distutils now supports MSVC 7.1 + +- doctest now examines all docstrings by default. Previously, it would + skip over functions with private names (as indicated by the underscore + naming convention). The old default created too much of a risk that + user tests were being skipped inadvertently. Note, this change could + break code in the unlikely case that someone had intentionally put + failing tests in the docstrings of private functions. The breakage + is easily fixable by specifying the old behavior when calling testmod() + or Tester(). + +- There were several fixes to the way dumbdbms are closed. It's vital + that a dumbdbm database be closed properly, else the on-disk data + and directory files can be left in mutually inconsistent states. + dumbdbm.py's _Database.__del__() method attempted to close the + database properly, but a shutdown race in _Database._commit() could + prevent this from working, so that a program trusting __del__() to + get the on-disk files in synch could be badly surprised. The race + has been repaired. A sync() method was also added so that shelve + can guarantee data is written to disk. + + The close() method can now be called more than once without complaint. + +- The classes in threading.py are now new-style classes. That they + weren't before was an oversight. + +- The urllib2 digest authentication handlers now define the correct + auth_header. The earlier versions would fail at runtime. + +- SF bug 763023: fix uncaught ZeroDivisionError in difflib ratio methods + when there are no lines. + +- SF bug 763637: fix exception in Tkinter with after_cancel + which could occur with Tk 8.4 + +- SF bug 770601: CGIHTTPServer.py now passes the entire environment + to child processes. + +- SF bug 765238: add filter to fnmatch's __all__. + +- SF bug 748201: make time.strptime() error messages more helpful. + +- SF patch 764470: Do not dump the args attribute of a Fault object in + xmlrpclib. + +- SF patch 549151: urllib and urllib2 now redirect POSTs on 301 + responses. + +- SF patch 766650: The whichdb module was fixed to recognize dbm files + generated by gdbm on OS/2 EMX. + +- SF bugs 763047 and 763052: fixes bug of timezone value being left as + -1 when ``time.tzname[0] == time.tzname[1] and not time.daylight`` + is true when it should only when time.daylight is true. + +- SF bug 764548: re now allows subclasses of str and unicode to be + used as patterns. + +- SF bug 763637: In Tkinter, change after_cancel() to handle tuples + of varying sizes. Tk 8.4 returns a different number of values + than Tk 8.3. + +- SF bug 763023: difflib.ratio() did not catch zero division. + +- The Queue module now has an __all__ attribute. + +Tools/Demos +----------- + +- See Lib/idlelib/NEWS.txt for IDLE news. + +- SF bug 753592: webchecker/wsgui now handles user supplied directories. + +- The trace.py script has been removed. It is now in the standard library. + +Build +----- + +- Python now compiles with -fno-strict-aliasing if possible (SF bug 766696). + +- The socket module compiles on IRIX 6.5.10. + +- An irix64 system is treated the same way as an irix6 system (SF + patch 764560). + +- Several definitions were missing on FreeBSD 5.x unless the + __BSD_VISIBLE symbol was defined. configure now defines it as + needed. + +C API +----- + +- Unicode objects now support mbcs as a built-in encoding, so the C + API can use it without deferring to the encodings package. + +Windows +------- + +- The Windows implementation of PyThread_start_new_thread() never + checked error returns from Windows functions correctly. As a result, + it could claim to start a new thread even when the Microsoft + _beginthread() function failed (due to "too many threads" -- this is + on the order of thousands when it happens). In these cases, the + Python exception :: + + thread.error: can't start new thread + + is raised now. + +- SF bug 766669: Prevent a GPF on interpreter exit when sockets are in + use. The interpreter now calls WSACleanup() from Py_Finalize() + instead of from DLL teardown. + +Mac +--- + +- Bundlebuilder now inherits default values in the right way. It was + previously possible for app bundles to get a type of "BNDL" instead + of "APPL." Other improvements include, a --build-id option to + specify the CFBundleIdentifier and using the --python option to set + the executable in the bundle. + +- Fixed two bugs in MacOSX framework handling. + +- pythonw did not allow user interaction in 2.3rc1, this has been fixed. + +- Python is now compiled with -mno-fused-madd, making all tests pass + on Panther. + +What's New in Python 2.3 beta 2? +================================ + +*Release date: 29-Jun-2003* + +Core and builtins +----------------- + +- A program can now set the environment variable PYTHONINSPECT to some + string value in Python, and cause the interpreter to enter the + interactive prompt at program exit, as if Python had been invoked + with the -i option. + +- list.index() now accepts optional start and stop arguments. Similar + changes were made to UserList.index(). SF feature request 754014. + +- SF patch 751998 fixes an unwanted side effect of the previous fix + for SF bug 742860 (the next item). + +- SF bug 742860: "WeakKeyDictionary __delitem__ uses iterkeys". This + wasn't threadsafe, was very inefficient (expected time O(len(dict)) + instead of O(1)), and could raise a spurious RuntimeError if another + thread mutated the dict during __delitem__, or if a comparison function + mutated it. It also neglected to raise KeyError when the key wasn't + present; didn't raise TypeError when the key wasn't of a weakly + referencable type; and broke various more-or-less obscure dict + invariants by using a sequence of equality comparisons over the whole + set of dict keys instead of computing the key's hash code to narrow + the search to those keys with the same hash code. All of these are + considered to be bugs. A new implementation of __delitem__ repairs all + that, but note that fixing these bugs may change visible behavior in + code relying (whether intentionally or accidentally) on old behavior. + +- SF bug 734869: Fixed a compiler bug that caused a fatal error when + compiling a list comprehension that contained another list comprehension + embedded in a lambda expression. + +- SF bug 705231: builtin pow() no longer lets the platform C pow() + raise -1.0 to integer powers, because (at least) glibc gets it wrong + in some cases. The result should be -1.0 if the power is odd and 1.0 + if the power is even, and any float with a sufficiently large exponent + is (mathematically) an exact even integer. + +- SF bug 759227: A new-style class that implements __nonzero__() must + return a bool or int (but not an int subclass) from that method. This + matches the restriction on classic classes. + +- The encoding attribute has been added for file objects, and set to + the terminal encoding on Unix and Windows. + +- The softspace attribute of file objects became read-only by oversight. + It's writable again. + +- Reverted a 2.3 beta 1 change to iterators for subclasses of list and + tuple. By default, the iterators now access data elements directly + instead of going through __getitem__. If __getitem__ access is + preferred, then __iter__ can be overridden. + +- SF bug 735247: The staticmethod and super types participate in + garbage collection. Before this change, it was possible for leaks to + occur in functions with non-global free variables that used these types. + +Extension modules +----------------- + +- the socket module has a new exception, socket.timeout, to allow + timeouts to be handled separately from other socket errors. + +- SF bug 751276: cPickle has fixed to propagate exceptions raised in + user code. In earlier versions, cPickle caught and ignored any + exception when it performed operations that it expected to raise + specific exceptions like AttributeError. + +- cPickle Pickler and Unpickler objects now participate in garbage + collection. + +- mimetools.choose_boundary() could return duplicate strings at times, + especially likely on Windows. The strings returned are now guaranteed + unique within a single program run. + +- thread.interrupt_main() raises KeyboardInterrupt in the main thread. + dummy_thread has also been modified to try to simulate the behavior. + +- array.array.insert() now treats negative indices as being relative + to the end of the array, just like list.insert() does. (SF bug #739313) + +- The datetime module classes datetime, time, and timedelta are now + properly subclassable. + +- _tkinter.{get|set}busywaitinterval was added. + +- itertools.islice() now accepts stop=None as documented. + Fixes SF bug #730685. + +- the bsddb185 module is built in one restricted instance - + /usr/include/db.h exists and defines HASHVERSION to be 2. This is true + for many BSD-derived systems. + + +Library +------- + +- Some happy doctest extensions from Jim Fulton have been added to + doctest.py. These are already being used in Zope3. The two + primary ones: + + doctest.debug(module, name) extracts the doctests from the named object + in the given module, puts them in a temp file, and starts pdb running + on that file. This is great when a doctest fails. + + doctest.DocTestSuite(module=None) returns a synthesized unittest + TestSuite instance, to be run by the unittest framework, which + runs all the doctests in the module. This allows writing tests in + doctest style (which can be clearer and shorter than writing tests + in unittest style), without losing unittest's powerful testing + framework features (which doctest lacks). + +- For compatibility with doctests created before 2.3, if an expected + output block consists solely of "1" and the actual output block + consists solely of "True", it's accepted as a match; similarly + for "0" and "False". This is quite un-doctest-like, but is practical. + The behavior can be disabled by passing the new doctest module + constant DONT_ACCEPT_TRUE_FOR_1 to the new optionflags optional + argument. + +- ZipFile.testzip() now only traps BadZipfile exceptions. Previously, + a bare except caught to much and reported all errors as a problem + in the archive. + +- The logging module now has a new function, makeLogRecord() making + LogHandler easier to interact with DatagramHandler and SocketHandler. + +- The cgitb module has been extended to support plain text display (SF patch + 569574). + +- A brand new version of IDLE (from the IDLEfork project at + SourceForge) is now included as Lib/idlelib. The old Tools/idle is + no more. + +- Added a new module: trace (documentation missing). This module used + to be distributed in Tools/scripts. It uses sys.settrace() to trace + code execution -- either function calls or individual lines. It can + generate tracing output during execution or a post-mortem report of + code coverage. + +- The threading module has new functions settrace() and setprofile() + that cooperate with the functions of the same name in the sys + module. A function registered with the threading module will + be used for all threads it creates. The new trace module uses this + to provide tracing for code running in threads. + +- copy.py: applied SF patch 707900, fixing bug 702858, by Steven + Taschuk. Copying a new-style class that had a reference to itself + didn't work. (The same thing worked fine for old-style classes.) + Builtin functions are now treated as atomic, fixing bug #746304. + +- difflib.py has two new functions: context_diff() and unified_diff(). + +- More fixes to urllib (SF 549151): (a) When redirecting, always use + GET. This is common practice and more-or-less sanctioned by the + HTTP standard. (b) Add a handler for 307 redirection, which becomes + an error for POST, but a regular redirect for GET and HEAD + +- Added optional 'onerror' argument to os.walk(), to control error + handling. + +- inspect.is{method|data}descriptor was added, to allow pydoc display + __doc__ of data descriptors. + +- Fixed socket speed loss caused by use of the _socketobject wrapper class + in socket.py. + +- timeit.py now checks the current directory for imports. + +- urllib2.py now knows how to order proxy classes, so the user doesn't + have to insert it in front of other classes, nor do dirty tricks like + inserting a "dummy" HTTPHandler after a ProxyHandler when building an + opener with proxy support. + +- Iterators have been added for dbm keys. + +- random.Random objects can now be pickled. + +Tools/Demos +----------- + +- pydoc now offers help on keywords and topics. + +- Tools/idle is gone; long live Lib/idlelib. + +- diff.py prints file diffs in context, unified, or ndiff formats, + providing a command line interface to difflib.py. + +- texcheck.py is a new script for making a rough validation of Python LaTeX + files. + +Build +----- + +- Setting DESTDIR during 'make install' now allows specifying a + different root directory. + +C API +----- + +- PyType_Ready(): If a type declares that it participates in gc + (Py_TPFLAGS_HAVE_GC), and its base class does not, and its base class's + tp_free slot is the default _PyObject_Del, and type does not define + a tp_free slot itself, _PyObject_GC_Del is assigned to type->tp_free. + Previously _PyObject_Del was inherited, which could at best lead to a + segfault. In addition, if even after this magic the type's tp_free + slot is _PyObject_Del or NULL, and the type is a base type + (Py_TPFLAGS_BASETYPE), TypeError is raised: since the type is a base + type, its dealloc function must call type->tp_free, and since the type + is gc'able, tp_free must not be NULL or _PyObject_Del. + +- PyThreadState_SetAsyncExc(): A new API (deliberately accessible only + from C) to interrupt a thread by sending it an exception. It is + intentional that you have to write your own C extension to call it + from Python. + + +New platforms +------------- + +None this time. + +Tests +----- + +- test_imp rewritten so that it doesn't raise RuntimeError if run as a + side effect of being imported ("import test.autotest"). + +Windows +------- + +- The Windows installer ships with Tcl/Tk 8.4.3 (upgraded from 8.4.1). + +- The installer always suggested that Python be installed on the C: + drive, due to a hardcoded "C:" generated by the Wise installation + wizard. People with machines where C: is not the system drive + usually want Python installed on whichever drive is their system drive + instead. We removed the hardcoded "C:", and two testers on machines + where C: is not the system drive report that the installer now + suggests their system drive. Note that you can always select the + directory you want in the "Select Destination Directory" dialog -- + that's what it's for. + +Mac +--- + +- There's a new module called "autoGIL", which offers a mechanism to + automatically release the Global Interpreter Lock when an event loop + goes to sleep, allowing other threads to run. It's currently only + supported on OSX, in the Mach-O version. +- The OSA modules now allow direct access to properties of the + toplevel application class (in AppleScript terminology). +- The Package Manager can now update itself. + +SourceForge Bugs and Patches Applied +------------------------------------ + +430160, 471893, 501716, 542562, 549151, 569574, 595837, 596434, +598163, 604210, 604716, 610332, 612627, 614770, 620190, 621891, +622042, 639139, 640236, 644345, 649742, 649742, 658233, 660022, +661318, 661676, 662807, 662923, 666219, 672855, 678325, 682347, +683486, 684981, 685773, 686254, 692776, 692959, 693094, 696777, +697989, 700827, 703666, 708495, 708604, 708901, 710733, 711902, +713722, 715782, 718286, 719359, 719367, 723136, 723831, 723962, +724588, 724767, 724767, 725942, 726150, 726446, 726869, 727051, +727719, 727719, 727805, 728277, 728563, 728656, 729096, 729103, +729293, 729297, 729300, 729317, 729395, 729622, 729817, 730170, +730296, 730594, 730685, 730826, 730963, 731209, 731403, 731504, +731514, 731626, 731635, 731643, 731644, 731644, 731689, 732124, +732143, 732234, 732284, 732284, 732479, 732761, 732783, 732951, +733667, 733781, 734118, 734231, 734869, 735051, 735293, 735527, +735613, 735694, 736962, 736962, 737970, 738066, 739313, 740055, +740234, 740301, 741806, 742126, 742741, 742860, 742860, 742911, +744041, 744104, 744238, 744687, 744877, 745055, 745478, 745525, +745620, 746012, 746304, 746366, 746801, 746953, 747348, 747667, +747954, 748846, 748849, 748973, 748975, 749191, 749210, 749759, +749831, 749911, 750008, 750092, 750542, 750595, 751038, 751107, +751276, 751451, 751916, 751941, 751956, 751998, 752671, 753451, +753602, 753617, 753845, 753925, 754014, 754340, 754447, 755031, +755087, 755147, 755245, 755683, 755987, 756032, 756996, 757058, +757229, 757818, 757821, 757822, 758112, 758910, 759227, 759889, +760257, 760703, 760792, 761104, 761337, 761519, 761830, 762455 + + +What's New in Python 2.3 beta 1? +================================ + +*Release date: 25-Apr-2003* + +Core and builtins +----------------- + +- New format codes B, H, I, k and K have been implemented for + PyArg_ParseTuple and PyBuild_Value. + +- New builtin function sum(seq, start=0) returns the sum of all the + items in iterable object seq, plus start (items are normally numbers, + and cannot be strings). + +- bool() called without arguments now returns False rather than + raising an exception. This is consistent with calling the + constructors for the other builtin types -- called without argument + they all return the false value of that type. (SF patch #724135) + +- In support of PEP 269 (making the pgen parser generator accessible + from Python), some changes to the pgen code structure were made; a + few files that used to be linked only with pgen are now linked with + Python itself. + +- The repr() of a weakref object now shows the __name__ attribute of + the referenced object, if it has one. + +- super() no longer ignores data descriptors, except __class__. See + the thread started at + http://mail.python.org/pipermail/python-dev/2003-April/034338.html + +- list.insert(i, x) now interprets negative i as it would be + interpreted by slicing, so negative values count from the end of the + list. This was the only place where such an interpretation was not + placed on a list index. + +- range() now works even if the arguments are longs with magnitude + larger than sys.maxint, as long as the total length of the sequence + fits. E.g., range(2**100, 2**101, 2**100) is the following list: + [1267650600228229401496703205376L]. (SF patch #707427.) + +- Some horridly obscure problems were fixed involving interaction + between garbage collection and old-style classes with "ambitious" + getattr hooks. If an old-style instance didn't have a __del__ method, + but did have a __getattr__ hook, and the instance became reachable + only from an unreachable cycle, and the hook resurrected or deleted + unreachable objects when asked to resolve "__del__", anything up to + a segfault could happen. That's been repaired. + +- dict.pop now takes an optional argument specifying a default + value to return if the key is not in the dict. If a default is not + given and the key is not found, a KeyError will still be raised. + Parallel changes were made to UserDict.UserDict and UserDict.DictMixin. + [SF patch #693753] (contributed by Michael Stone.) + +- sys.getfilesystemencoding() was added to expose + Py_FileSystemDefaultEncoding. + +- New function sys.exc_clear() clears the current exception. This is + rarely needed, but can sometimes be useful to release objects + referenced by the traceback held in sys.exc_info()[2]. (SF patch + #693195.) + +- On 64-bit systems, a dictionary could contain duplicate long/int keys + if the key value was larger than 2**32. See SF bug #689659. + +- Fixed SF bug #663074. The codec system was using global static + variables to store internal data. As a result, any attempts to use the + unicode system with multiple active interpreters, or successive + interpreter executions, would fail. + +- "%c" % u"a" now returns a unicode string instead of raising a + TypeError. u"%c" % 0xffffffff now raises a OverflowError instead + of a ValueError to be consistent with "%c" % 256. See SF patch #710127. + +Extension modules +----------------- + +- The socket module now provides the functions inet_pton and inet_ntop + for converting between string and packed representation of IP + addresses. There is also a new module variable, has_ipv6, which is + True iff the current Python has IPv6 support. See SF patch #658327. + +- Tkinter wrappers around Tcl variables now pass objects directly + to Tcl, instead of first converting them to strings. + +- The .*? pattern in the re module is now special-cased to avoid the + recursion limit. (SF patch #720991 -- many thanks to Gary Herron + and Greg Chapman.) + +- New function sys.call_tracing() allows pdb to debug code + recursively. + +- New function gc.get_referents(obj) returns a list of objects + directly referenced by obj. In effect, it exposes what the object's + tp_traverse slot does, and can be helpful when debugging memory + leaks. + +- The iconv module has been removed from this release. + +- The platform-independent routines for packing floats in IEEE formats + (struct.pack's <f, >f, <d, and >d codes; pickle and cPickle's protocol 1 + pickling of floats) ignored that rounding can cause a carry to + propagate. The worst consequence was that, in rare cases, <f and >f + could produce strings that, when unpacked again, were a factor of 2 + away from the original float. This has been fixed. See SF bug + #705836. + +- New function time.tzset() provides access to the C library tzset() + function, if supported. (SF patch #675422.) + +- Using createfilehandler, deletefilehandler, createtimerhandler functions + on Tkinter.tkinter (_tkinter module) no longer crashes the interpreter. + See SF bug #692416. + +- Modified the fcntl.ioctl() function to allow modification of a passed + mutable buffer (for details see the reference documentation). + +- Made user requested changes to the itertools module. + Subsumed the times() function into repeat(). + Added chain() and cycle(). + +- The rotor module is now deprecated; the encryption algorithm it uses + is not believed to be secure, and including crypto code with Python + has implications for exporting and importing it in various countries. + +- The socket module now always uses the _socketobject wrapper class, even on + platforms which have dup(2). The makefile() method is built directly + on top of the socket without duplicating the file descriptor, allowing + timeouts to work properly. + +Library +------- + +- New generator function os.walk() is an easy-to-use alternative to + os.path.walk(). See os module docs for details. os.path.walk() + isn't deprecated at this time, but may become deprecated in a + future release. + +- Added new module "platform" which provides a wide range of tools + for querying platform dependent features. + +- netrc now allows ASCII punctuation characters in passwords. + +- shelve now supports the optional writeback argument, and exposes + pickle protocol versions. + +- Several methods of nntplib.NNTP have grown an optional file argument + which specifies a file where to divert the command's output + (already supported by the body() method). (SF patch #720468) + +- The self-documenting XML server library DocXMLRPCServer was added. + +- Support for internationalized domain names has been added through + the 'idna' and 'punycode' encodings, the 'stringprep' module, the + 'mkstringprep' tool, and enhancements to the socket and httplib + modules. + +- htmlentitydefs has two new dictionaries: name2codepoint maps + HTML entity names to Unicode codepoints (as integers). + codepoint2name is the reverse mapping. See SF patch #722017. + +- pdb has a new command, "debug", which lets you step through + arbitrary code from the debugger's (pdb) prompt. + +- unittest.failUnlessEqual and its equivalent unittest.assertEqual now + return 'not a == b' rather than 'a != b'. This gives the desired + result for classes that define __eq__ without defining __ne__. + +- sgmllib now supports SGML marked sections, in particular the + MS Office extensions. + +- The urllib module now offers support for the iterator protocol. + SF patch 698520 contributed by Brett Cannon. + +- New module timeit provides a simple framework for timing the + execution speed of expressions and statements. + +- sets.Set objects now support mixed-type __eq__ and __ne__, instead + of raising TypeError. If x is a Set object and y is a non-Set object, + x == y is False, and x != y is True. This is akin to the change made + for mixed-type comparisons of datetime objects in 2.3a2; more info + about the rationale is in the NEWS entry for that. See also SF bug + report <http://www.python.org/sf/693121>. + +- On Unix platforms, if os.listdir() is called with a Unicode argument, + it now returns Unicode strings. (This behavior was added earlier + to the Windows NT/2k/XP version of os.listdir().) + +- Distutils: both 'py_modules' and 'packages' keywords can now be specified + in core.setup(). Previously you could supply one or the other, but + not both of them. (SF patch #695090 from Bernhard Herzog) + +- New csv package makes it easy to read/write CSV files. + +- Module shlex has been extended to allow posix-like shell parsings, + including a split() function for easy spliting of quoted strings and + commands. An iterator interface was also implemented. + +Tools/Demos +----------- + +- New script combinerefs.py helps analyze new PYTHONDUMPREFS output. + See the module docstring for details. + +Build +----- + +- Fix problem building on OSF1 because the compiler only accepted + preprocessor directives that start in column 1. (SF bug #691793.) + +C API +----- + +- Added PyGC_Collect(), equivalent to calling gc.collect(). + +- PyThreadState_GetDict() was changed not to raise an exception or + issue a fatal error when no current thread state is available. This + makes it possible to print dictionaries when no thread is active. + +- LONG_LONG was renamed to PY_LONG_LONG. Extensions that use this and + need compatibility with previous versions can use this: + + #ifndef PY_LONG_LONG + #define PY_LONG_LONG LONG_LONG + #endif + +- Added PyObject_SelfIter() to fill the tp_iter slot for the + typical case where the method returns its self argument. + +- The extended type structure used for heap types (new-style + classes defined by Python code using a class statement) is now + exported from object.h as PyHeapTypeObject. (SF patch #696193.) + +New platforms +------------- + +None this time. + +Tests +----- + +- test_timeout now requires -u network to be passed to regrtest to run. + See SF bug #692988. + +Windows +------- + +- os.fsync() now exists on Windows, and calls the Microsoft _commit() + function. + +- New function winsound.MessageBeep() wraps the Win32 API + MessageBeep(). + +Mac +--- + +- os.listdir() now returns Unicode strings on MacOS X when called with + a Unicode argument. See the general news item under "Library". + +- A new method MacOS.WMAvailable() returns true if it is safe to access + the window manager, false otherwise. + +- EasyDialogs dialogs are now movable-modal, and if the application is + currently in the background they will ask to be moved to the foreground + before displaying. + +- OSA Scripting support has improved a lot, and gensuitemodule.py can now + be used by mere mortals. The documentation is now also more or less + complete. + +- The IDE (in a framework build) now includes introductory documentation + in Apple Help Viewer format. + + +What's New in Python 2.3 alpha 2? +================================= + +*Release date: 19-Feb-2003* + +Core and builtins +----------------- + +- Negative positions returned from PEP 293 error callbacks are now + treated as being relative to the end of the input string. Positions + that are out of bounds raise an IndexError. + +- sys.path[0] (the directory from which the script is loaded) is now + turned into an absolute pathname, unless it is the empty string. + (SF patch #664376.) + +- Finally fixed the bug in compile() and exec where a string ending + with an indented code block but no newline would raise SyntaxError. + This would have been a four-line change in parsetok.c... Except + codeop.py depends on this behavior, so a compilation flag had to be + invented that causes the tokenizer to revert to the old behavior; + this required extra changes to 2 .h files, 2 .c files, and 2 .py + files. (Fixes SF bug #501622.) + +- If a new-style class defines neither __new__ nor __init__, its + constructor would ignore all arguments. This is changed now: the + constructor refuses arguments in this case. This might break code + that worked under Python 2.2. The simplest fix is to add a no-op + __init__: ``def __init__(self, *args, **kw): pass``. + +- Through a bytecode optimizer bug (and I bet you didn't even know + Python *had* a bytecode optimizer :-), "unsigned" hex/oct constants + with a leading minus sign would come out with the wrong sign. + ("Unsigned" hex/oct constants are those with a face value in the + range sys.maxint+1 through sys.maxint*2+1, inclusive; these have + always been interpreted as negative numbers through sign folding.) + E.g. 0xffffffff is -1, and -(0xffffffff) is 1, but -0xffffffff would + come out as -4294967295. This was the case in Python 2.2 through + 2.2.2 and 2.3a1, and in Python 2.4 it will once again have that + value, but according to PEP 237 it really needs to be 1 now. This + will be backported to Python 2.2.3 a well. (SF #660455) + +- int(s, base) sometimes sign-folds hex and oct constants; it only + does this when base is 0 and s.strip() starts with a '0'. When the + sign is actually folded, as in int("0xffffffff", 0) on a 32-bit + machine, which returns -1, a FutureWarning is now issued; in Python + 2.4, this will return 4294967295L, as do int("+0xffffffff", 0) and + int("0xffffffff", 16) right now. (PEP 347) + +- super(X, x): x may now be a proxy for an X instance, i.e. + issubclass(x.__class__, X) but not issubclass(type(x), X). + +- isinstance(x, X): if X is a new-style class, this is now equivalent + to issubclass(type(x), X) or issubclass(x.__class__, X). Previously + only type(x) was tested. (For classic classes this was already the + case.) + +- compile(), eval() and the exec statement now fully support source code + passed as unicode strings. + +- int subclasses can be initialized with longs if the value fits in an int. + See SF bug #683467. + +- long(string, base) takes time linear in len(string) when base is a power + of 2 now. It used to take time quadratic in len(string). + +- filter returns now Unicode results for Unicode arguments. + +- raw_input can now return Unicode objects. + +- List objects' sort() method now accepts None as the comparison function. + Passing None is semantically identical to calling sort() with no + arguments. + +- Fixed crash when printing a subclass of str and __str__ returned self. + See SF bug #667147. + +- Fixed an invalid RuntimeWarning and an undetected error when trying + to convert a long integer into a float which couldn't fit. + See SF bug #676155. + +- Function objects now have a __module__ attribute that is bound to + the name of the module in which the function was defined. This + applies for C functions and methods as well as functions and methods + defined in Python. This attribute is used by pickle.whichmodule(), + which changes the behavior of whichmodule slightly. In Python 2.2 + whichmodule() returns "__main__" for functions that are not defined + at the top-level of a module (examples: methods, nested functions). + Now whichmodule() will return the proper module name. + +Extension modules +----------------- + +- operator.isNumberType() now checks that the object has a nb_int or + nb_float slot, rather than simply checking whether it has a non-NULL + tp_as_number pointer. + +- The imp module now has ways to acquire and release the "import + lock": imp.acquire_lock() and imp.release_lock(). Note: this is a + reentrant lock, so releasing the lock only truly releases it when + this is the last release_lock() call. You can check with + imp.lock_held(). (SF bug #580952 and patch #683257.) + +- Change to cPickle to match pickle.py (see below and PEP 307). + +- Fix some bugs in the parser module. SF bug #678518. + +- Thanks to Scott David Daniels, a subtle bug in how the zlib + extension implemented flush() was fixed. Scott also rewrote the + zlib test suite using the unittest module. (SF bug #640230 and + patch #678531.) + +- Added an itertools module containing high speed, memory efficient + looping constructs inspired by tools from Haskell and SML. + +- The SSL module now handles sockets with a timeout set correctly (SF + patch #675750, fixing SF bug #675552). + +- os/posixmodule has grown the sysexits.h constants (EX_OK and friends). + +- Fixed broken threadstate swap in readline that could cause fatal + errors when a readline hook was being invoked while a background + thread was active. (SF bugs #660476 and #513033.) + +- fcntl now exposes the strops.h I_* constants. + +- Fix a crash on Solaris that occurred when calling close() on + an mmap'ed file which was already closed. (SF patch #665913) + +- Fixed several serious bugs in the zipimport implementation. + +- datetime changes: + + The date class is now properly subclassable. (SF bug #720908) + + The datetime and datetimetz classes have been collapsed into a single + datetime class, and likewise the time and timetz classes into a single + time class. Previously, a datetimetz object with tzinfo=None acted + exactly like a datetime object, and similarly for timetz. This wasn't + enough of a difference to justify distinct classes, and life is simpler + now. + + today() and now() now round system timestamps to the closest + microsecond <http://www.python.org/sf/661086>. This repairs an + irritation most likely seen on Windows systems. + + In dt.astimezone(tz), if tz.utcoffset(dt) returns a duration, + ValueError is raised if tz.dst(dt) returns None (2.3a1 treated it + as 0 instead, but a tzinfo subclass wishing to participate in + time zone conversion has to take a stand on whether it supports + DST; if you don't care about DST, then code dst() to return 0 minutes, + meaning that DST is never in effect). + + The tzinfo methods utcoffset() and dst() must return a timedelta object + (or None) now. In 2.3a1 they could also return an int or long, but that + was an unhelpfully redundant leftover from an earlier version wherein + they couldn't return a timedelta. TOOWTDI. + + The example tzinfo class for local time had a bug. It was replaced + by a later example coded by Guido. + + datetime.astimezone(tz) no longer raises an exception when the + input datetime has no UTC equivalent in tz. For typical "hybrid" time + zones (a single tzinfo subclass modeling both standard and daylight + time), this case can arise one hour per year, at the hour daylight time + ends. See new docs for details. In short, the new behavior mimics + the local wall clock's behavior of repeating an hour in local time. + + dt.astimezone() can no longer be used to convert between naive and aware + datetime objects. If you merely want to attach, or remove, a tzinfo + object, without any conversion of date and time members, use + dt.replace(tzinfo=whatever) instead, where "whatever" is None or a + tzinfo subclass instance. + + A new method tzinfo.fromutc(dt) can be overridden in tzinfo subclasses + to give complete control over how a UTC time is to be converted to + a local time. The default astimezone() implementation calls fromutc() + as its last step, so a tzinfo subclass can affect that too by overriding + fromutc(). It's expected that the default fromutc() implementation will + be suitable as-is for "almost all" time zone subclasses, but the + creativity of political time zone fiddling appears unbounded -- fromutc() + allows the highly motivated to emulate any scheme expressible in Python. + + datetime.now(): The optional tzinfo argument was undocumented (that's + repaired), and its name was changed to tz ("tzinfo" is overloaded enough + already). With a tz argument, now(tz) used to return the local date + and time, and attach tz to it, without any conversion of date and time + members. This was less than useful. Now now(tz) returns the current + date and time as local time in tz's time zone, akin to :: + + tz.fromutc(datetime.utcnow().replace(tzinfo=utc)) + + where "utc" is an instance of a tzinfo subclass modeling UTC. Without + a tz argument, now() continues to return the current local date and time, + as a naive datetime object. + + datetime.fromtimestamp(): Like datetime.now() above, this had less than + useful behavior when the optional tinzo argument was specified. See + also SF bug report <http://www.python.org/sf/660872>. + + date and datetime comparison: In order to prevent comparison from + falling back to the default compare-object-addresses strategy, these + raised TypeError whenever they didn't understand the other object type. + They still do, except when the other object has a "timetuple" attribute, + in which case they return NotImplemented now. This gives other + datetime objects (e.g., mxDateTime) a chance to intercept the + comparison. + + date, time, datetime and timedelta comparison: When the exception + for mixed-type comparisons in the last paragraph doesn't apply, if + the comparison is == then False is returned, and if the comparison is + != then True is returned. Because dict lookup and the "in" operator + only invoke __eq__, this allows, for example, :: + + if some_datetime in some_sequence: + + and :: + + some_dict[some_timedelta] = whatever + + to work as expected, without raising TypeError just because the + sequence is heterogeneous, or the dict has mixed-type keys. [This + seems like a good idea to implement for all mixed-type comparisons + that don't want to allow falling back to address comparison.] + + The constructors building a datetime from a timestamp could raise + ValueError if the platform C localtime()/gmtime() inserted "leap + seconds". Leap seconds are ignored now. On such platforms, it's + possible to have timestamps that differ by a second, yet where + datetimes constructed from them are equal. + + The pickle format of date, time and datetime objects has changed + completely. The undocumented pickler and unpickler functions no + longer exist. The undocumented __setstate__() and __getstate__() + methods no longer exist either. + +Library +------- + +- The logging module was updated slightly; the WARN level was renamed + to WARNING, and the matching function/method warn() to warning(). + +- The pickle and cPickle modules were updated with a new pickling + protocol (documented by pickletools.py, see below) and several + extensions to the pickle customization API (__reduce__, __setstate__ + etc.). The copy module now uses more of the pickle customization + API to copy objects that don't implement __copy__ or __deepcopy__. + See PEP 307 for details. + +- The distutils "register" command now uses http://www.python.org/pypi + as the default repository. (See PEP 301.) + +- the platform dependent path related variables sep, altsep, extsep, + pathsep, curdir, pardir and defpath are now defined in the platform + dependent path modules (e.g. ntpath.py) rather than os.py, so these + variables are now available via os.path. They continue to be + available from the os module. + (see <http://www.python.org/sf/680789>). + +- array.array was added to the types repr.py knows about (see + <http://www.python.org/sf/680789>). + +- The new pickletools.py contains lots of documentation about pickle + internals, and supplies some helpers for working with pickles, such as + a symbolic pickle disassembler. + +- Xmlrpclib.py now supports the builtin boolean type. + +- py_compile has a new 'doraise' flag and a new PyCompileError + exception. + +- SimpleXMLRPCServer now supports CGI through the CGIXMLRPCRequestHandler + class. + +- The sets module now raises TypeError in __cmp__, to clarify that + sets are not intended to be three-way-compared; the comparison + operators are overloaded as subset/superset tests. + +- Bastion.py and rexec.py are disabled. These modules are not safe in + Python 2.2. or 2.3. + +- realpath is now exported when doing ``from poxixpath import *``. + It is also exported for ntpath, macpath, and os2emxpath. + See SF bug #659228. + +- New module tarfile from Lars Gustäbel provides a comprehensive interface + to tar archive files with transparent gzip and bzip2 compression. + See SF patch #651082. + +- urlparse can now parse imap:// URLs. See SF feature request #618024. + +- Tkinter.Canvas.scan_dragto() provides an optional parameter to support + the gain value which is passed to Tk. SF bug# 602259. + +- Fix logging.handlers.SysLogHandler protocol when using UNIX domain sockets. + See SF patch #642974. + +- The dospath module was deleted. Use the ntpath module when manipulating + DOS paths from other platforms. + +Tools/Demos +----------- + +- Two new scripts (db2pickle.py and pickle2db.py) were added to the + Tools/scripts directory to facilitate conversion from the old bsddb module + to the new one. While the user-visible API of the new module is + compatible with the old one, it's likely that the version of the + underlying database library has changed. To convert from the old library, + run the db2pickle.py script using the old version of Python to convert it + to a pickle file. After upgrading Python, run the pickle2db.py script + using the new version of Python to reconstitute your database. For + example: + + % python2.2 db2pickle.py -h some.db > some.pickle + % python2.3 pickle2db.py -h some.db.new < some.pickle + + Run the scripts without any args to get a usage message. + + +Build +----- + +- The audio driver tests (test_ossaudiodev.py and + test_linuxaudiodev.py) are no longer run by default. This is + because they don't always work, depending on your hardware and + software. To run these tests, you must use an invocation like :: + + ./python Lib/test/regrtest.py -u audio test_ossaudiodev + +- On systems which build using the configure script, compiler flags which + used to be lumped together using the OPT flag have been split into two + groups, OPT and BASECFLAGS. OPT is meant to carry just optimization- and + debug-related flags like "-g" and "-O3". BASECFLAGS is meant to carry + compiler flags that are required to get a clean compile. On some + platforms (many Linux flavors in particular) BASECFLAGS will be empty by + default. On others, such as Mac OS X and SCO, it will contain required + flags. This change allows people building Python to override OPT without + fear of clobbering compiler flags which are required to get a clean build. + +- On Darwin/Mac OS X platforms, /sw/lib and /sw/include are added to the + relevant search lists in setup.py. This allows users building Python to + take advantage of the many packages available from the fink project + <http://fink.sf.net/>. + +- A new Makefile target, scriptsinstall, installs a number of useful scripts + from the Tools/scripts directory. + +C API +----- + +- PyEval_GetFrame() is now declared to return a ``PyFrameObject *`` + instead of a plain ``PyObject *``. (SF patch #686601.) + +- PyNumber_Check() now checks that the object has a nb_int or nb_float + slot, rather than simply checking whether it has a non-NULL + tp_as_number pointer. + +- A C type that inherits from a base type that defines tp_as_buffer + will now inherit the tp_as_buffer pointer if it doesn't define one. + (SF #681367) + +- The PyArg_Parse functions now issue a DeprecationWarning if a float + argument is provided when an integer is specified (this affects the 'b', + 'B', 'h', 'H', 'i', and 'l' codes). Future versions of Python will + raise a TypeError. + +Tests +----- + +- Several tests weren't being run from regrtest.py (test_timeout.py, + test_tarfile.py, test_netrc.py, test_multifile.py, + test_importhooks.py and test_imp.py). Now they are. (Note to + developers: please read Lib/test/README when creating a new test, to + make sure to do it right! All tests need to use either unittest or + pydoc.) + +- Added test_posix.py, a test suite for the posix module. + +- Added test_hexoct.py, a test suite for hex/oct constant folding. + +Windows +------- + +- The timeout code for socket connect() didn't work right; this has + now been fixed. test_timeout.py should pass (at least most of the + time). + +- distutils' msvccompiler class now passes the preprocessor options to + the resource compiler. See SF patch #669198. + +- The bsddb module now ships with Sleepycat's 4.1.25.NC, the latest + release without strong cryptography. + +- sys.path[0], if it contains a directory name, is now always an + absolute pathname. (SF patch #664376.) + +- The new logging package is now installed by the Windows installer. It + wasn't in 2.3a1 due to oversight. + +Mac +--- + +- There are new dialogs EasyDialogs.AskFileForOpen, AskFileForSave + and AskFolder. The old macfs.StandardGetFile and friends are deprecated. + +- Most of the standard library now uses pathnames or FSRefs in preference + of FSSpecs, and use the underlying Carbon.File and Carbon.Folder modules + in stead of macfs. macfs will probably be deprecated in the future. + +- Type Carbon.File.FSCatalogInfo and supporting methods have been implemented. + This also makes macfs.FSSpec.SetDates() work again. + +- There is a new module pimp, the package install manager for Python, and + accompanying applet PackageManager. These allow you to easily download + and install pretested extension packages either in source or binary + form. Only in MacPython-OSX. + +- Applets are now built with bundlebuilder in MacPython-OSX, which should make + them more robust and also provides a path towards BuildApplication. The + downside of this change is that applets can no longer be run from the + Terminal window, this will hopefully be fixed in the 2.3b1. + + +What's New in Python 2.3 alpha 1? +================================= + +*Release date: 31-Dec-2002* + +Type/class unification and new-style classes +-------------------------------------------- + +- One can now assign to __bases__ and __name__ of new-style classes. + +- dict() now accepts keyword arguments so that dict(one=1, two=2) + is the equivalent of {"one": 1, "two": 2}. Accordingly, + the existing (but undocumented) 'items' keyword argument has + been eliminated. This means that dict(items=someMapping) now has + a different meaning than before. + +- int() now returns a long object if the argument is outside the + integer range, so int("4" * 1000), int(1e200) and int(1L<<1000) will + all return long objects instead of raising an OverflowError. + +- Assignment to __class__ is disallowed if either the old or the new + class is a statically allocated type object (such as defined by an + extension module). This prevents anomalies like 2.__class__ = bool. + +- New-style object creation and deallocation have been sped up + significantly; they are now faster than classic instance creation + and deallocation. + +- The __slots__ variable can now mention "private" names, and the + right thing will happen (e.g. __slots__ = ["__foo"]). + +- The built-ins slice() and buffer() are now callable types. The + types classobj (formerly class), code, function, instance, and + instancemethod (formerly instance-method), which have no built-in + names but are accessible through the types module, are now also + callable. The type dict-proxy is renamed to dictproxy. + +- Cycles going through the __class__ link of a new-style instance are + now detected by the garbage collector. + +- Classes using __slots__ are now properly garbage collected. + [SF bug 519621] + +- Tightened the __slots__ rules: a slot name must be a valid Python + identifier. + +- The constructor for the module type now requires a name argument and + takes an optional docstring argument. Previously, this constructor + ignored its arguments. As a consequence, deriving a class from a + module (not from the module type) is now illegal; previously this + created an unnamed module, just like invoking the module type did. + [SF bug 563060] + +- A new type object, 'basestring', is added. This is a common base type + for 'str' and 'unicode', and can be used instead of + types.StringTypes, e.g. to test whether something is "a string": + isinstance(x, basestring) is True for Unicode and 8-bit strings. This + is an abstract base class and cannot be instantiated directly. + +- Changed new-style class instantiation so that when C's __new__ + method returns something that's not a C instance, its __init__ is + not called. [SF bug #537450] + +- Fixed super() to work correctly with class methods. [SF bug #535444] + +- If you try to pickle an instance of a class that has __slots__ but + doesn't define or override __getstate__, a TypeError is now raised. + This is done by adding a bozo __getstate__ to the class that always + raises TypeError. (Before, this would appear to be pickled, but the + state of the slots would be lost.) + +Core and builtins +----------------- + +- Import from zipfiles is now supported. The name of a zipfile placed + on sys.path causes the import statement to look for importable Python + modules (with .py, pyc and .pyo extensions) and packages inside the + zipfile. The zipfile import follows the specification (though not + the sample implementation) of PEP 273. The semantics of __path__ are + compatible with those that have been implemented in Jython since + Jython 2.1. + +- PEP 302 has been accepted. Although it was initially developed to + support zipimport, it offers a new, general import hook mechanism. + Several new variables have been added to the sys module: + sys.meta_path, sys.path_hooks, and sys.path_importer_cache; these + make extending the import statement much more convenient than + overriding the __import__ built-in function. For a description of + these, see PEP 302. + +- A frame object's f_lineno attribute can now be written to from a + trace function to change which line will execute next. A command to + exploit this from pdb has been added. [SF patch #643835] + +- The _codecs support module for codecs.py was turned into a builtin + module to assure that at least the builtin codecs are available + to the Python parser for source code decoding according to PEP 263. + +- issubclass now supports a tuple as the second argument, just like + isinstance does. ``issubclass(X, (A, B))`` is equivalent to + ``issubclass(X, A) or issubclass(X, B)``. + +- Thanks to Armin Rigo, the last known way to provoke a system crash + by cleverly arranging for a comparison function to mutate a list + during a list.sort() operation has been fixed. The effect of + attempting to mutate a list, or even to inspect its contents or + length, while a sort is in progress, is not defined by the language. + The C implementation of Python 2.3 attempts to detect mutations, + and raise ValueError if one occurs, but there's no guarantee that + all mutations will be caught, or that any will be caught across + releases or implementations. + +- Unicode file name processing for Windows (PEP 277) is implemented. + All platforms now have an os.path.supports_unicode_filenames attribute, + which is set to True on Windows NT/2000/XP, and False elsewhere. + +- Codec error handling callbacks (PEP 293) are implemented. + Error handling in unicode.encode or str.decode can now be customized. + +- A subtle change to the semantics of the built-in function intern(): + interned strings are no longer immortal. You must keep a reference + to the return value intern() around to get the benefit. + +- Use of 'None' as a variable, argument or attribute name now + issues a SyntaxWarning. In the future, None may become a keyword. + +- SET_LINENO is gone. co_lnotab is now consulted to determine when to + call the trace function. C code that accessed f_lineno should call + PyCode_Addr2Line instead (f_lineno is still there, but only kept up + to date when there is a trace function set). + +- There's a new warning category, FutureWarning. This is used to warn + about a number of situations where the value or sign of an integer + result will change in Python 2.4 as a result of PEP 237 (integer + unification). The warnings implement stage B0 mentioned in that + PEP. The warnings are about the following situations: + + - Octal and hex literals without 'L' prefix in the inclusive range + [0x80000000..0xffffffff]; these are currently negative ints, but + in Python 2.4 they will be positive longs with the same bit + pattern. + + - Left shifts on integer values that cause the outcome to lose + bits or have a different sign than the left operand. To be + precise: x<<n where this currently doesn't yield the same value + as long(x)<<n; in Python 2.4, the outcome will be long(x)<<n. + + - Conversions from ints to string that show negative values as + unsigned ints in the inclusive range [0x80000000..0xffffffff]; + this affects the functions hex() and oct(), and the string + formatting codes %u, %o, %x, and %X. In Python 2.4, these will + show signed values (e.g. hex(-1) currently returns "0xffffffff"; + in Python 2.4 it will return "-0x1"). + +- The bits manipulated under the cover by sys.setcheckinterval() have + been changed. Both the check interval and the ticker used to be + per-thread values. They are now just a pair of global variables. + In addition, the default check interval was boosted from 10 to 100 + bytecode instructions. This may have some effect on systems that + relied on the old default value. In particular, in multi-threaded + applications which try to be highly responsive, response time will + increase by some (perhaps imperceptible) amount. + +- When multiplying very large integers, a version of the so-called + Karatsuba algorithm is now used. This is most effective if the + inputs have roughly the same size. If they both have about N digits, + Karatsuba multiplication has O(N**1.58) runtime (the exponent is + log_base_2(3)) instead of the previous O(N**2). Measured results may + be better or worse than that, depending on platform quirks. Besides + the O() improvement in raw instruction count, the Karatsuba algorithm + appears to have much better cache behavior on extremely large integers + (starting in the ballpark of a million bits). Note that this is a + simple implementation, and there's no intent here to compete with, + e.g., GMP. It gives a very nice speedup when it applies, but a package + devoted to fast large-integer arithmetic should run circles around it. + +- u'%c' will now raise a ValueError in case the argument is an + integer outside the valid range of Unicode code point ordinals. + +- The tempfile module has been overhauled for enhanced security. The + mktemp() function is now deprecated; new, safe replacements are + mkstemp() (for files) and mkdtemp() (for directories), and the + higher-level functions NamedTemporaryFile() and TemporaryFile(). + Use of some global variables in this module is also deprecated; the + new functions have keyword arguments to provide the same + functionality. All Lib, Tools and Demo modules that used the unsafe + interfaces have been updated to use the safe replacements. Thanks + to Zack Weinberg! + +- When x is an object whose class implements __mul__ and __rmul__, + 1.0*x would correctly invoke __rmul__, but 1*x would erroneously + invoke __mul__. This was due to the sequence-repeat code in the int + type. This has been fixed now. + +- Previously, "str1 in str2" required str1 to be a string of length 1. + This restriction has been relaxed to allow str1 to be a string of + any length. Thus "'el' in 'hello world'" returns True now. + +- File objects are now their own iterators. For a file f, iter(f) now + returns f (unless f is closed), and f.next() is similar to + f.readline() when EOF is not reached; however, f.next() uses a + readahead buffer that messes up the file position, so mixing + f.next() and f.readline() (or other methods) doesn't work right. + Calling f.seek() drops the readahead buffer, but other operations + don't. It so happens that this gives a nice additional speed boost + to "for line in file:"; the xreadlines method and corresponding + module are now obsolete. Thanks to Oren Tirosh! + +- Encoding declarations (PEP 263, phase 1) have been implemented. A + comment of the form "# -*- coding: <encodingname> -*-" in the first + or second line of a Python source file indicates the encoding. + +- list.sort() has a new implementation. While cross-platform results + may vary, and in data-dependent ways, this is much faster on many + kinds of partially ordered lists than the previous implementation, + and reported to be just as fast on randomly ordered lists on + several major platforms. This sort is also stable (if A==B and A + precedes B in the list at the start, A precedes B after the sort too), + although the language definition does not guarantee stability. A + potential drawback is that list.sort() may require temp space of + len(list)*2 bytes (``*4`` on a 64-bit machine). It's therefore possible + for list.sort() to raise MemoryError now, even if a comparison function + does not. See <http://www.python.org/sf/587076> for full details. + +- All standard iterators now ensure that, once StopIteration has been + raised, all future calls to next() on the same iterator will also + raise StopIteration. There used to be various counterexamples to + this behavior, which could caused confusion or subtle program + breakage, without any benefits. (Note that this is still an + iterator's responsibility; the iterator framework does not enforce + this.) + +- Ctrl+C handling on Windows has been made more consistent with + other platforms. KeyboardInterrupt can now reliably be caught, + and Ctrl+C at an interactive prompt no longer terminates the + process under NT/2k/XP (it never did under Win9x). Ctrl+C will + interrupt time.sleep() in the main thread, and any child processes + created via the popen family (on win2k; we can't make win9x work + reliably) are also interrupted (as generally happens on for Linux/Unix.) + [SF bugs 231273, 439992 and 581232] + +- sys.getwindowsversion() has been added on Windows. This + returns a tuple with information about the version of Windows + currently running. + +- Slices and repetitions of buffer objects now consistently return + a string. Formerly, strings would be returned most of the time, + but a buffer object would be returned when the repetition count + was one or when the slice range was all inclusive. + +- Unicode objects in sys.path are no longer ignored but treated + as directory names. + +- Fixed string.startswith and string.endswith builtin methods + so they accept negative indices. [SF bug 493951] + +- Fixed a bug with a continue inside a try block and a yield in the + finally clause. [SF bug 567538] + +- Most builtin sequences now support "extended slices", i.e. slices + with a third "stride" parameter. For example, "hello world"[::-1] + gives "dlrow olleh". + +- A new warning PendingDeprecationWarning was added to provide + direction on features which are in the process of being deprecated. + The warning will not be printed by default. To see the pending + deprecations, use -Walways::PendingDeprecationWarning:: + as a command line option or warnings.filterwarnings() in code. + +- Deprecated features of xrange objects have been removed as + promised. The start, stop, and step attributes and the tolist() + method no longer exist. xrange repetition and slicing have been + removed. + +- New builtin function enumerate(x), from PEP 279. Example: + enumerate("abc") is an iterator returning (0,"a"), (1,"b"), (2,"c"). + The argument can be an arbitrary iterable object. + +- The assert statement no longer tests __debug__ at runtime. This means + that assert statements cannot be disabled by assigning a false value + to __debug__. + +- A method zfill() was added to str and unicode, that fills a numeric + string to the left with zeros. For example, + "+123".zfill(6) -> "+00123". + +- Complex numbers supported divmod() and the // and % operators, but + these make no sense. Since this was documented, they're being + deprecated now. + +- String and unicode methods lstrip(), rstrip() and strip() now take + an optional argument that specifies the characters to strip. For + example, "Foo!!!?!?!?".rstrip("?!") -> "Foo". + +- There's a new dictionary constructor (a class method of the dict + class), dict.fromkeys(iterable, value=None). It constructs a + dictionary with keys taken from the iterable and all values set to a + single value. It can be used for building sets and for removing + duplicates from sequences. + +- Added a new dict method pop(key). This removes and returns the + value corresponding to key. [SF patch #539949] + +- A new built-in type, bool, has been added, as well as built-in + names for its two values, True and False. Comparisons and sundry + other operations that return a truth value have been changed to + return a bool instead. Read PEP 285 for an explanation of why this + is backward compatible. + +- Fixed two bugs reported as SF #535905: under certain conditions, + deallocating a deeply nested structure could cause a segfault in the + garbage collector, due to interaction with the "trashcan" code; + access to the current frame during destruction of a local variable + could access a pointer to freed memory. + +- The optional object allocator ("pymalloc") has been enabled by + default. The recommended practice for memory allocation and + deallocation has been streamlined. A header file is included, + Misc/pymemcompat.h, which can be bundled with 3rd party extensions + and lets them use the same API with Python versions from 1.5.2 + onwards. + +- PyErr_Display will provide file and line information for all exceptions + that have an attribute print_file_and_line, not just SyntaxErrors. + +- The UTF-8 codec will now encode and decode Unicode surrogates + correctly and without raising exceptions for unpaired ones. + +- Universal newlines (PEP 278) is implemented. Briefly, using 'U' + instead of 'r' when opening a text file for reading changes the line + ending convention so that any of '\r', '\r\n', and '\n' is + recognized (even mixed in one file); all three are converted to + '\n', the standard Python line end character. + +- file.xreadlines() now raises a ValueError if the file is closed: + Previously, an xreadlines object was returned which would raise + a ValueError when the xreadlines.next() method was called. + +- sys.exit() inadvertently allowed more than one argument. + An exception will now be raised if more than one argument is used. + +- Changed evaluation order of dictionary literals to conform to the + general left to right evaluation order rule. Now {f1(): f2()} will + evaluate f1 first. + +- Fixed bug #521782: when a file was in non-blocking mode, file.read() + could silently lose data or wrongly throw an unknown error. + +- The sq_repeat, sq_inplace_repeat, sq_concat and sq_inplace_concat + slots are now always tried after trying the corresponding nb_* slots. + This fixes a number of minor bugs (see bug #624807). + +- Fix problem with dynamic loading on 64-bit AIX (see bug #639945). + +Extension modules +----------------- + +- Added three operators to the operator module: + operator.pow(a,b) which is equivalent to: a**b. + operator.is_(a,b) which is equivalent to: a is b. + operator.is_not(a,b) which is equivalent to: a is not b. + +- posix.openpty now works on all systems that have /dev/ptmx. + +- A module zipimport exists to support importing code from zip + archives. + +- The new datetime module supplies classes for manipulating dates and + times. The basic design came from the Zope "fishbowl process", and + favors practical commercial applications over calendar esoterica. See + + http://www.zope.org/Members/fdrake/DateTimeWiki/FrontPage + +- _tkinter now returns Tcl objects, instead of strings. Objects which + have Python equivalents are converted to Python objects, other objects + are wrapped. This can be configured through the wantobjects method, + or Tkinter.wantobjects. + +- The PyBSDDB wrapper around the Sleepycat Berkeley DB library has + been added as the package bsddb. The traditional bsddb module is + still available in source code, but not built automatically anymore, + and is now named bsddb185. This supports Berkeley DB versions from + 3.0 to 4.1. For help converting your databases from the old module (which + probably used an obsolete version of Berkeley DB) to the new module, see + the db2pickle.py and pickle2db.py scripts described in the Tools/Demos + section above. + +- unicodedata was updated to Unicode 3.2. It supports normalization + and names for Hangul syllables and CJK unified ideographs. + +- resource.getrlimit() now returns longs instead of ints. + +- readline now dynamically adjusts its input/output stream if + sys.stdin/stdout changes. + +- The _tkinter module (and hence Tkinter) has dropped support for + Tcl/Tk 8.0 and 8.1. Only Tcl/Tk versions 8.2, 8.3 and 8.4 are + supported. + +- cPickle.BadPickleGet is now a class. + +- The time stamps in os.stat_result are floating point numbers + after stat_float_times has been called. + +- If the size passed to mmap.mmap() is larger than the length of the + file on non-Windows platforms, a ValueError is raised. [SF bug 585792] + +- The xreadlines module is slated for obsolescence. + +- The strptime function in the time module is now always available (a + Python implementation is used when the C library doesn't define it). + +- The 'new' module is no longer an extension, but a Python module that + only exists for backwards compatibility. Its contents are no longer + functions but callable type objects. + +- The bsddb.*open functions can now take 'None' as a filename. + This will create a temporary in-memory bsddb that won't be + written to disk. + +- posix.getloadavg, posix.lchown, posix.killpg, posix.mknod, and + posix.getpgid have been added where available. + +- The locale module now exposes the C library's gettext interface. It + also has a new function getpreferredencoding. + +- A security hole ("double free") was found in zlib-1.1.3, a popular + third party compression library used by some Python modules. The + hole was quickly plugged in zlib-1.1.4, and the Windows build of + Python now ships with zlib-1.1.4. + +- pwd, grp, and resource return enhanced tuples now, with symbolic + field names. + +- array.array is now a type object. A new format character + 'u' indicates Py_UNICODE arrays. For those, .tounicode and + .fromunicode methods are available. Arrays now support __iadd__ + and __imul__. + +- dl now builds on every system that has dlfcn.h. Failure in case + of sizeof(int)!=sizeof(long)!=sizeof(void*) is delayed until dl.open + is called. + +- The sys module acquired a new attribute, api_version, which evaluates + to the value of the PYTHON_API_VERSION macro with which the + interpreter was compiled. + +- Fixed bug #470582: sre module would return a tuple (None, 'a', 'ab') + when applying the regular expression '^((a)c)?(ab)$' on 'ab'. It now + returns (None, None, 'ab'), as expected. Also fixed handling of + lastindex/lastgroup match attributes in similar cases. For example, + when running the expression r'(a)(b)?b' over 'ab', lastindex must be + 1, not 2. + +- Fixed bug #581080: sre scanner was not checking the buffer limit + before increasing the current pointer. This was creating an infinite + loop in the search function, once the pointer exceeded the buffer + limit. + +- The os.fdopen function now enforces a file mode starting with the + letter 'r', 'w' or 'a', otherwise a ValueError is raised. This fixes + bug #623464. + +- The linuxaudiodev module is now deprecated; it is being replaced by + ossaudiodev. The interface has been extended to cover a lot more of + OSS (see www.opensound.com), including most DSP ioctls and the + OSS mixer API. Documentation forthcoming in 2.3a2. + +Library +------- + +- imaplib.py now supports SSL (Tino Lange and Piers Lauder). + +- Freeze's modulefinder.py has been moved to the standard library; + slightly improved so it will issue less false missing submodule + reports (see sf path #643711 for details). Documentation will follow + with Python 2.3a2. + +- os.path exposes getctime. + +- unittest.py now has two additional methods called assertAlmostEqual() + and failIfAlmostEqual(). They implement an approximate comparison + by rounding the difference between the two arguments and comparing + the result to zero. Approximate comparison is essential for + unit tests of floating point results. + +- calendar.py now depends on the new datetime module rather than + the time module. As a result, the range of allowable dates + has been increased. + +- pdb has a new 'j(ump)' command to select the next line to be + executed. + +- The distutils created windows installers now can run a + postinstallation script. + +- doctest.testmod can now be called without argument, which means to + test the current module. + +- When canceling a server that implemented threading with a keyboard + interrupt, the server would shut down but not terminate (waiting on + client threads). A new member variable, daemon_threads, was added to + the ThreadingMixIn class in SocketServer.py to make it explicit that + this behavior needs to be controlled. + +- A new module, optparse, provides a fancy alternative to getopt for + command line parsing. It is a slightly modified version of Greg + Ward's Optik package. + +- UserDict.py now defines a DictMixin class which defines all dictionary + methods for classes that already have a minimum mapping interface. + This greatly simplifies writing classes that need to be substitutable + for dictionaries (such as the shelve module). + +- shelve.py now subclasses from UserDict.DictMixin. Now shelve supports + all dictionary methods. This eases the transition to persistent + storage for scripts originally written with dictionaries in mind. + +- shelve.open and the various classes in shelve.py now accept an optional + binary flag, which defaults to False. If True, the values stored in the + shelf are binary pickles. + +- A new package, logging, implements the logging API defined by PEP + 282. The code is written by Vinay Sajip. + +- StreamReader, StreamReaderWriter and StreamRecoder in the codecs + modules are iterators now. + +- gzip.py now handles files exceeding 2GB. Files over 4GB also work + now (provided the OS supports it, and Python is configured with large + file support), but in that case the underlying gzip file format can + record only the least-significant 32 bits of the file size, so that + some tools working with gzipped files may report an incorrect file + size. + +- xml.sax.saxutils.unescape has been added, to replace entity references + with their entity value. + +- Queue.Queue.{put,get} now support an optional timeout argument. + +- Various features of Tk 8.4 are exposed in Tkinter.py. The multiple + option of tkFileDialog is exposed as function askopenfile{,name}s. + +- Various configure methods of Tkinter have been stream-lined, so that + tag_configure, image_configure, window_configure now return a + dictionary when invoked with no argument. + +- Importing the readline module now no longer has the side effect of + calling setlocale(LC_CTYPE, ""). The initial "C" locale, or + whatever locale is explicitly set by the user, is preserved. If you + want repr() of 8-bit strings in your preferred encoding to preserve + all printable characters of that encoding, you have to add the + following code to your $PYTHONSTARTUP file or to your application's + main(): + + import locale + locale.setlocale(locale.LC_CTYPE, "") + +- shutil.move was added. shutil.copytree now reports errors as an + exception at the end, instead of printing error messages. + +- Encoding name normalization was generalized to not only + replace hyphens with underscores, but also all other non-alphanumeric + characters (with the exception of the dot which is used for Python + package names during lookup). The aliases.py mapping was updated + to the new standard. + +- mimetypes has two new functions: guess_all_extensions() which + returns a list of all known extensions for a mime type, and + add_type() which adds one mapping between a mime type and + an extension to the database. + +- New module: sets, defines the class Set that implements a mutable + set type using the keys of a dict to represent the set. There's + also a class ImmutableSet which is useful when you need sets of sets + or when you need to use sets as dict keys, and a class BaseSet which + is the base class of the two. + +- Added random.sample(population,k) for random sampling without replacement. + Returns a k length list of unique elements chosen from the population. + +- random.randrange(-sys.maxint-1, sys.maxint) no longer raises + OverflowError. That is, it now accepts any combination of 'start' + and 'stop' arguments so long as each is in the range of Python's + bounded integers. + +- Thanks to Raymond Hettinger, random.random() now uses a new core + generator. The Mersenne Twister algorithm is implemented in C, + threadsafe, faster than the previous generator, has an astronomically + large period (2**19937-1), creates random floats to full 53-bit + precision, and may be the most widely tested random number generator + in existence. + + The random.jumpahead(n) method has different semantics for the new + generator. Instead of jumping n steps ahead, it uses n and the + existing state to create a new state. This means that jumpahead() + continues to support multi-threaded code needing generators of + non-overlapping sequences. However, it will break code which relies + on jumpahead moving a specific number of steps forward. + + The attributes random.whseed and random.__whseed have no meaning for + the new generator. Code using these attributes should switch to a + new class, random.WichmannHill which is provided for backward + compatibility and to make an alternate generator available. + +- New "algorithms" module: heapq, implements a heap queue. Thanks to + Kevin O'Connor for the code and François Pinard for an entertaining + write-up explaining the theory and practical uses of heaps. + +- New encoding for the Palm OS character set: palmos. + +- binascii.crc32() and the zipfile module had problems on some 64-bit + platforms. These have been fixed. On a platform with 8-byte C longs, + crc32() now returns a signed-extended 4-byte result, so that its value + as a Python int is equal to the value computed a 32-bit platform. + +- xml.dom.minidom.toxml and toprettyxml now take an optional encoding + argument. + +- Some fixes in the copy module: when an object is copied through its + __reduce__ method, there was no check for a __setstate__ method on + the result [SF patch 565085]; deepcopy should treat instances of + custom metaclasses the same way it treats instances of type 'type' + [SF patch 560794]. + +- Sockets now support timeout mode. After s.settimeout(T), where T is + a float expressing seconds, subsequent operations raise an exception + if they cannot be completed within T seconds. To disable timeout + mode, use s.settimeout(None). There's also a module function, + socket.setdefaulttimeout(T), which sets the default for all sockets + created henceforth. + +- getopt.gnu_getopt was added. This supports GNU-style option + processing, where options can be mixed with non-option arguments. + +- Stop using strings for exceptions. String objects used for + exceptions are now classes deriving from Exception. The objects + changed were: Tkinter.TclError, bdb.BdbQuit, macpath.norm_error, + tabnanny.NannyNag, and xdrlib.Error. + +- Constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, + BOM_UTF32, BOM_UTF32_LE and BOM_UTF32_BE that represent the Byte + Order Mark in UTF-8, UTF-16 and UTF-32 encodings for little and + big endian systems were added to the codecs module. The old names + BOM32_* and BOM64_* were off by a factor of 2. + +- Added conversion functions math.degrees() and math.radians(). + +- math.log() now takes an optional argument: math.log(x[, base]). + +- ftplib.retrlines() now tests for callback is None rather than testing + for False. Was causing an error when given a callback object which + was callable but also returned len() as zero. The change may + create new breakage if the caller relied on the undocumented behavior + and called with callback set to [] or some other False value not + identical to None. + +- random.gauss() uses a piece of hidden state used by nothing else, + and the .seed() and .whseed() methods failed to reset it. In other + words, setting the seed didn't completely determine the sequence of + results produced by random.gauss(). It does now. Programs repeatedly + mixing calls to a seed method with calls to gauss() may see different + results now. + +- The pickle.Pickler class grew a clear_memo() method to mimic that + provided by cPickle.Pickler. + +- difflib's SequenceMatcher class now does a dynamic analysis of + which elements are so frequent as to constitute noise. For + comparing files as sequences of lines, this generally works better + than the IS_LINE_JUNK function, and function ndiff's linejunk + argument defaults to None now as a result. A happy benefit is + that SequenceMatcher may run much faster now when applied + to large files with many duplicate lines (for example, C program + text with lots of repeated "}" and "return NULL;" lines). + +- New Text.dump() method in Tkinter module. + +- New distutils commands for building packagers were added to + support pkgtool on Solaris and swinstall on HP-UX. + +- distutils now has a new abstract binary packager base class + command/bdist_packager, which simplifies writing packagers. + This will hopefully provide the missing bits to encourage + people to submit more packagers, e.g. for Debian, FreeBSD + and other systems. + +- The UTF-16, -LE and -BE stream readers now raise a + NotImplementedError for all calls to .readline(). Previously, they + used to just produce garbage or fail with an encoding error -- + UTF-16 is a 2-byte encoding and the C lib's line reading APIs don't + work well with these. + +- compileall now supports quiet operation. + +- The BaseHTTPServer now implements optional HTTP/1.1 persistent + connections. + +- socket module: the SSL support was broken out of the main + _socket module C helper and placed into a new _ssl helper + which now gets imported by socket.py if available and working. + +- encodings package: added aliases for all supported IANA character + sets + +- ftplib: to safeguard the user's privacy, anonymous login will use + "anonymous@" as default password, rather than the real user and host + name. + +- webbrowser: tightened up the command passed to os.system() so that + arbitrary shell code can't be executed because a bogus URL was + passed in. + +- gettext.translation has an optional fallback argument, and + gettext.find an optional all argument. Translations will now fallback + on a per-message basis. The module supports plural forms, by means + of gettext.[d]ngettext and Translation.[u]ngettext. + +- distutils bdist commands now offer a --skip-build option. + +- warnings.warn now accepts a Warning instance as first argument. + +- The xml.sax.expatreader.ExpatParser class will no longer create + circular references by using itself as the locator that gets passed + to the content handler implementation. [SF bug #535474] + +- The email.Parser.Parser class now properly parses strings regardless + of their line endings, which can be any of \r, \n, or \r\n (CR, LF, + or CRLF). Also, the Header class's constructor default arguments + has changed slightly so that an explicit maxlinelen value is always + honored, and so unicode conversion error handling can be specified. + +- distutils' build_ext command now links C++ extensions with the C++ + compiler available in the Makefile or CXX environment variable, if + running under \*nix. + +- New module bz2: provides a comprehensive interface for the bz2 compression + library. It implements a complete file interface, one-shot (de)compression + functions, and types for sequential (de)compression. + +- New pdb command 'pp' which is like 'p' except that it pretty-prints + the value of its expression argument. + +- Now bdist_rpm distutils command understands a verify_script option in + the config file, including the contents of the referred filename in + the "%verifyscript" section of the rpm spec file. + +- Fixed bug #495695: webbrowser module would run graphic browsers in a + unix environment even if DISPLAY was not set. Also, support for + skipstone browser was included. + +- Fixed bug #636769: rexec would run unallowed code if subclasses of + strings were used as parameters for certain functions. + +Tools/Demos +----------- + +- pygettext.py now supports globbing on Windows, and accepts module + names in addition to accepting file names. + +- The SGI demos (Demo/sgi) have been removed. Nobody thought they + were interesting any more. (The SGI library modules and extensions + are still there; it is believed that at least some of these are + still used and useful.) + +- IDLE supports the new encoding declarations (PEP 263); it can also + deal with legacy 8-bit files if they use the locale's encoding. It + allows non-ASCII strings in the interactive shell and executes them + in the locale's encoding. + +- freeze.py now produces binaries which can import shared modules, + unlike before when this failed due to missing symbol exports in + the generated binary. + +Build +----- + +- On Unix, IDLE is now installed automatically. + +- The fpectl module is not built by default; it's dangerous or useless + except in the hands of experts. + +- The public Python C API will generally be declared using PyAPI_FUNC + and PyAPI_DATA macros, while Python extension module init functions + will be declared with PyMODINIT_FUNC. DL_EXPORT/DL_IMPORT macros + are deprecated. + +- A bug was fixed that could cause COUNT_ALLOCS builds to segfault, or + get into infinite loops, when a new-style class got garbage-collected. + Unfortunately, to avoid this, the way COUNT_ALLOCS works requires + that new-style classes be immortal in COUNT_ALLOCS builds. Note that + COUNT_ALLOCS is not enabled by default, in either release or debug + builds, and that new-style classes are immortal only in COUNT_ALLOCS + builds. + +- Compiling out the cyclic garbage collector is no longer an option. + The old symbol WITH_CYCLE_GC is now ignored, and Python.h arranges + that it's always defined (for the benefit of any extension modules + that may be conditionalizing on it). A bonus is that any extension + type participating in cyclic gc can choose to participate in the + Py_TRASHCAN mechanism now too; in the absence of cyclic gc, this used + to require editing the core to teach the trashcan mechanism about the + new type. + +- According to Annex F of the current C standard, + + The Standard C macro HUGE_VAL and its float and long double analogs, + HUGE_VALF and HUGE_VALL, expand to expressions whose values are + positive infinities. + + Python only uses the double HUGE_VAL, and only to #define its own symbol + Py_HUGE_VAL. Some platforms have incorrect definitions for HUGE_VAL. + pyport.h used to try to worm around that, but the workarounds triggered + other bugs on other platforms, so we gave up. If your platform defines + HUGE_VAL incorrectly, you'll need to #define Py_HUGE_VAL to something + that works on your platform. The only instance of this I'm sure about + is on an unknown subset of Cray systems, described here: + + http://www.cray.com/swpubs/manuals/SN-2194_2.0/html-SN-2194_2.0/x3138.htm + + Presumably 2.3a1 breaks such systems. If anyone uses such a system, help! + +- The configure option --without-doc-strings can be used to remove the + doc strings from the builtin functions and modules; this reduces the + size of the executable. + +- The universal newlines option (PEP 278) is on by default. On Unix + it can be disabled by passing --without-universal-newlines to the + configure script. On other platforms, remove + WITH_UNIVERSAL_NEWLINES from pyconfig.h. + +- On Unix, a shared libpython2.3.so can be created with --enable-shared. + +- All uses of the CACHE_HASH, INTERN_STRINGS, and DONT_SHARE_SHORT_STRINGS + preprocessor symbols were eliminated. The internal decisions they + controlled stopped being experimental long ago. + +- The tools used to build the documentation now work under Cygwin as + well as Unix. + +- The bsddb and dbm module builds have been changed to try and avoid version + skew problems and disable linkage with Berkeley DB 1.85 unless the + installer knows what s/he's doing. See the section on building these + modules in the README file for details. + +C API +----- + +- PyNumber_Check() now returns true for string and unicode objects. + This is a result of these types having a partially defined + tp_as_number slot. (This is not a feature, but an indication that + PyNumber_Check() is not very useful to determine numeric behavior. + It may be deprecated.) + +- The string object's layout has changed: the pointer member + ob_sinterned has been replaced by an int member ob_sstate. On some + platforms (e.g. most 64-bit systems) this may change the offset of + the ob_sval member, so as a precaution the API_VERSION has been + incremented. The apparently unused feature of "indirect interned + strings", supported by the ob_sinterned member, is gone. Interned + strings are now usually mortal; there is a new API, + PyString_InternImmortal() that creates immortal interned strings. + (The ob_sstate member can only take three values; however, while + making it a char saves a few bytes per string object on average, in + it also slowed things down a bit because ob_sval was no longer + aligned.) + +- The Py_InitModule*() functions now accept NULL for the 'methods' + argument. Modules without global functions are becoming more common + now that factories can be types rather than functions. + +- New C API PyUnicode_FromOrdinal() which exposes unichr() at C + level. + +- New functions PyErr_SetExcFromWindowsErr() and + PyErr_SetExcFromWindowsErrWithFilename(). Similar to + PyErr_SetFromWindowsErrWithFilename() and + PyErr_SetFromWindowsErr(), but they allow to specify + the exception type to raise. Available on Windows. + +- Py_FatalError() is now declared as taking a const char* argument. It + was previously declared without const. This should not affect working + code. + +- Added new macro PySequence_ITEM(o, i) that directly calls + sq_item without rechecking that o is a sequence and without + adjusting for negative indices. + +- PyRange_New() now raises ValueError if the fourth argument is not 1. + This is part of the removal of deprecated features of the xrange + object. + +- PyNumber_Coerce() and PyNumber_CoerceEx() now also invoke the type's + coercion if both arguments have the same type but this type has the + CHECKTYPES flag set. This is to better support proxies. + +- The type of tp_free has been changed from "``void (*)(PyObject *)``" to + "``void (*)(void *)``". + +- PyObject_Del, PyObject_GC_Del are now functions instead of macros. + +- A type can now inherit its metatype from its base type. Previously, + when PyType_Ready() was called, if ob_type was found to be NULL, it + was always set to &PyType_Type; now it is set to base->ob_type, + where base is tp_base, defaulting to &PyObject_Type. + +- PyType_Ready() accidentally did not inherit tp_is_gc; now it does. + +- The PyCore_* family of APIs have been removed. + +- The "u#" parser marker will now pass through Unicode objects as-is + without going through the buffer API. + +- The enumerators of cmp_op have been renamed to use the prefix ``PyCmp_``. + +- An old #define of ANY as void has been removed from pyport.h. This + hasn't been used since Python's pre-ANSI days, and the #define has + been marked as obsolete since then. SF bug 495548 says it created + conflicts with other packages, so keeping it around wasn't harmless. + +- Because Python's magic number scheme broke on January 1st, we decided + to stop Python development. Thanks for all the fish! + +- Some of us don't like fish, so we changed Python's magic number + scheme to a new one. See Python/import.c for details. + +New platforms +------------- + +- OpenVMS is now supported. + +- AtheOS is now supported. + +- the EMX runtime environment on OS/2 is now supported. + +- GNU/Hurd is now supported. + +Tests +----- + +- The regrtest.py script's -u option now provides a way to say "allow + all resources except this one." For example, to allow everything + except bsddb, give the option '-uall,-bsddb'. + +Windows +------- + +- The Windows distribution now ships with version 4.0.14 of the + Sleepycat Berkeley database library. This should be a huge + improvement over the previous Berkeley DB 1.85, which had many + bugs. + XXX What are the licensing issues here? + XXX If a user has a database created with a previous version of + XXX Python, what must they do to convert it? + XXX I'm still not sure how to link this thing (see PCbuild/readme.txt). + XXX The version # is likely to change before 2.3a1. + +- The Windows distribution now ships with a Secure Sockets Library (SLL) + module (_ssl.pyd) + +- The Windows distribution now ships with Tcl/Tk version 8.4.1 (it + previously shipped with Tcl/Tk 8.3.2). + +- When Python is built under a Microsoft compiler, sys.version now + includes the compiler version number (_MSC_VER). For example, under + MSVC 6, sys.version contains the substring "MSC v.1200 ". 1200 is + the value of _MSC_VER under MSVC 6. + +- Sometimes the uninstall executable (UNWISE.EXE) vanishes. One cause + of that has been fixed in the installer (disabled Wise's "delete in- + use files" uninstall option). + +- Fixed a bug in urllib's proxy handling in Windows. [SF bug #503031] + +- The installer now installs Start menu shortcuts under (the local + equivalent of) "All Users" when doing an Admin install. + +- file.truncate([newsize]) now works on Windows for all newsize values. + It used to fail if newsize didn't fit in 32 bits, reflecting a + limitation of MS _chsize (which is no longer used). + +- os.waitpid() is now implemented for Windows, and can be used to block + until a specified process exits. This is similar to, but not exactly + the same as, os.waitpid() on POSIX systems. If you're waiting for + a specific process whose pid was obtained from one of the spawn() + functions, the same Python os.waitpid() code works across platforms. + See the docs for details. The docs were changed to clarify that + spawn functions return, and waitpid requires, a process handle on + Windows (not the same thing as a Windows process id). + +- New tempfile.TemporaryFile implementation for Windows: this doesn't + need a TemporaryFileWrapper wrapper anymore, and should be immune + to a nasty problem: before 2.3, if you got a temp file on Windows, it + got wrapped in an object whose close() method first closed the + underlying file, then deleted the file. This usually worked fine. + However, the spawn family of functions on Windows create (at a low C + level) the same set of open files in the spawned process Q as were + open in the spawning process P. If a temp file f was among them, then + doing f.close() in P first closed P's C-level file handle on f, but Q's + C-level file handle on f remained open, so the attempt in P to delete f + blew up with a "Permission denied" error (Windows doesn't allow + deleting open files). This was surprising, subtle, and difficult to + work around. + +- The os module now exports all the symbolic constants usable with the + low-level os.open() on Windows: the new constants in 2.3 are + O_NOINHERIT, O_SHORT_LIVED, O_TEMPORARY, O_RANDOM and O_SEQUENTIAL. + The others were also available in 2.2: O_APPEND, O_BINARY, O_CREAT, + O_EXCL, O_RDONLY, O_RDWR, O_TEXT, O_TRUNC and O_WRONLY. Contrary + to Microsoft docs, O_SHORT_LIVED does not seem to imply O_TEMPORARY + (so specify both if you want both; note that neither is useful unless + specified with O_CREAT too). + +Mac +---- + +- Mac/Relnotes is gone, the release notes are now here. + +- Python (the OSX-only, unix-based version, not the OS9-compatible CFM + version) now fully supports unicode strings as arguments to various file + system calls, eg. open(), file(), os.stat() and os.listdir(). + +- The current naming convention for Python on the Macintosh is that MacPython + refers to the unix-based OSX-only version, and MacPython-OS9 refers to the + CFM-based version that runs on both OS9 and OSX. + +- All MacPython-OS9 functionality is now available in an OSX unix build, + including the Carbon modules, the IDE, OSA support, etc. A lot of this + will only work correctly in a framework build, though, because you cannot + talk to the window manager unless your application is run from a .app + bundle. There is a command line tool "pythonw" that runs your script + with an interpreter living in such a .app bundle, this interpreter should + be used to run any Python script using the window manager (including + Tkinter or wxPython scripts). + +- Most of Mac/Lib has moved to Lib/plat-mac, which is again used both in + MacPython-OSX and MacPython-OS9. The only modules remaining in Mac/Lib + are specifically for MacPython-OS9 (CFM support, preference resources, etc). + +- A new utility PythonLauncher will start a Python interpreter when a .py or + .pyw script is double-clicked in the Finder. By default .py scripts are + run with a normal Python interpreter in a Terminal window and .pyw + files are run with a window-aware pythonw interpreter without a Terminal + window, but all this can be customized. + +- MacPython-OS9 is now Carbon-only, so it runs on Mac OS 9 or Mac OS X and + possibly on Mac OS 8.6 with the right CarbonLib installed, but not on earlier + releases. + +- Many tools such as BuildApplet.py and gensuitemodule.py now support a command + line interface too. + +- All the Carbon classes are now PEP253 compliant, meaning that you can + subclass them from Python. Most of the attributes have gone, you should + now use the accessor function call API, which is also what Apple's + documentation uses. Some attributes such as grafport.visRgn are still + available for convenience. + +- New Carbon modules File (implementing the APIs in Files.h and Aliases.h) + and Folder (APIs from Folders.h). The old macfs builtin module is + gone, and replaced by a Python wrapper around the new modules. + +- Pathname handling should now be fully consistent: MacPython-OSX always uses + unix pathnames and MacPython-OS9 always uses colon-separated Mac pathnames + (also when running on Mac OS X). + +- New Carbon modules Help and AH give access to the Carbon Help Manager. + There are hooks in the IDE to allow accessing the Python documentation + (and Apple's Carbon and Cocoa documentation) through the Help Viewer. + See Mac/OSX/README for converting the Python documentation to a + Help Viewer compatible form and installing it. + +- OSA support has been redesigned and the generated Python classes now + mirror the inheritance defined by the underlying OSA classes. + +- MacPython no longer maps both \r and \n to \n on input for any text file. + This feature has been replaced by universal newline support (PEP278). + +- The default encoding for Python sourcefiles in MacPython-OS9 is no longer + mac-roman (or whatever your local Mac encoding was) but "ascii", like on + other platforms. If you really need sourcefiles with Mac characters in them + you can change this in site.py. + + +What's New in Python 2.2 final? +=============================== + +*Release date: 21-Dec-2001* + +Type/class unification and new-style classes +-------------------------------------------- + +- pickle.py, cPickle: allow pickling instances of new-style classes + with a custom metaclass. + +Core and builtins +----------------- + +- weakref proxy object: when comparing, unwrap both arguments if both + are proxies. + +Extension modules +----------------- + +- binascii.b2a_base64(): fix a potential buffer overrun when encoding + very short strings. + +- cPickle: the obscure "fast" mode was suspected of causing stack + overflows on the Mac. Hopefully fixed this by setting the recursion + limit much smaller. If the limit is too low (it only affects + performance), you can change it by defining PY_CPICKLE_FAST_LIMIT + when compiling cPickle.c (or in pyconfig.h). + +Library +------- + +- dumbdbm.py: fixed a dumb old bug (the file didn't get synched at + close or delete time). + +- rfc822.py: fixed a bug where the address '<>' was converted to None + instead of an empty string (also fixes the email.Utils module). + +- xmlrpclib.py: version 1.0.0; uses precision for doubles. + +- test suite: the pickle and cPickle tests were not executing any code + when run from the standard regression test. + +Tools/Demos +----------- + +Build +----- + +C API +----- + +New platforms +------------- + +Tests +----- + +Windows +------- + +- distutils package: fixed broken Windows installers (bdist_wininst). + +- tempfile.py: prevent mysterious warnings when TemporaryFileWrapper + instances are deleted at process exit time. + +- socket.py: prevent mysterious warnings when socket instances are + deleted at process exit time. + +- posixmodule.c: fix a Windows crash with stat() of a filename ending + in backslash. + +Mac +---- + +- The Carbon toolbox modules have been upgraded to Universal Headers + 3.4, and experimental CoreGraphics and CarbonEvents modules have + been added. All only for framework-enabled MacOSX. + + +What's New in Python 2.2c1? +=========================== + +*Release date: 14-Dec-2001* + +Type/class unification and new-style classes +-------------------------------------------- + +- Guido's tutorial introduction to the new type/class features has + been extensively updated. See + + http://www.python.org/2.2/descrintro.html + + That remains the primary documentation in this area. + +- Fixed a leak: instance variables declared with __slots__ were never + deleted! + +- The "delete attribute" method of descriptor objects is called + __delete__, not __del__. In previous releases, it was mistakenly + called __del__, which created an unfortunate overloading condition + with finalizers. (The "get attribute" and "set attribute" methods + are still called __get__ and __set__, respectively.) + +- Some subtle issues with the super built-in were fixed: + + (a) When super itself is subclassed, its __get__ method would still + return an instance of the base class (i.e., of super). + + (b) super(C, C()).__class__ would return C rather than super. This + is confusing. To fix this, I decided to change the semantics of + super so that it only applies to code attributes, not to data + attributes. After all, overriding data attributes is not + supported anyway. + + (c) The __get__ method didn't check whether the argument was an + instance of the type used in creation of the super instance. + +- Previously, hash() of an instance of a subclass of a mutable type + (list or dictionary) would return some value, rather than raising + TypeError. This has been fixed. Also, directly calling + dict.__hash__ and list.__hash__ now raises the same TypeError + (previously, these were the same as object.__hash__). + +- New-style objects now support deleting their __dict__. This is for + all intents and purposes equivalent to assigning a brand new empty + dictionary, but saves space if the object is not used further. + +Core and builtins +----------------- + +- -Qnew now works as documented in PEP 238: when -Qnew is passed on + the command line, all occurrences of "/" use true division instead + of classic division. See the PEP for details. Note that "all" + means all instances in library and 3rd-party modules, as well as in + your own code. As the PEP says, -Qnew is intended for use only in + educational environments with control over the libraries in use. + Note that test_coercion.py in the standard Python test suite fails + under -Qnew; this is expected, and won't be repaired until true + division becomes the default (in the meantime, test_coercion is + testing the current rules). + +- complex() now only allows the first argument to be a string + argument, and raises TypeError if either the second arg is a string + or if the second arg is specified when the first is a string. + +Extension modules +----------------- + +- gc.get_referents was renamed to gc.get_referrers. + +Library +------- + +- Functions in the os.spawn() family now release the global interpreter + lock around calling the platform spawn. They should always have done + this, but did not before 2.2c1. Multithreaded programs calling + an os.spawn function with P_WAIT will no longer block all Python threads + until the spawned program completes. It's possible that some programs + relies on blocking, although more likely by accident than by design. + +- webbrowser defaults to netscape.exe on OS/2 now. + +- Tix.ResizeHandle exposes detach_widget, hide, and show. + +- The charset alias windows_1252 has been added. + +- types.StringTypes is a tuple containing the defined string types; + usually this will be (str, unicode), but if Python was compiled + without Unicode support it will be just (str,). + +- The pulldom and minidom modules were synchronized to PyXML. + +Tools/Demos +----------- + +- A new script called Tools/scripts/google.py was added, which fires + off a search on Google. + +Build +----- + +- Note that release builds of Python should arrange to define the + preprocessor symbol NDEBUG on the command line (or equivalent). + In the 2.2 pre-release series we tried to define this by magic in + Python.h instead, but it proved to cause problems for extension + authors. The Unix, Windows and Mac builds now all define NDEBUG in + release builds via cmdline (or equivalent) instead. Ports to + other platforms should do likewise. + +- It is no longer necessary to use --with-suffix when building on a + case-insensitive file system (such as Mac OS X HFS+). In the build + directory an extension is used, but not in the installed python. + +C API +----- + +- New function PyDict_MergeFromSeq2() exposes the builtin dict + constructor's logic for updating a dictionary from an iterable object + producing key-value pairs. + +- PyArg_ParseTupleAndKeywords() requires that the number of entries in + the keyword list equal the number of argument specifiers. This + wasn't checked correctly, and PyArg_ParseTupleAndKeywords could even + dump core in some bad cases. This has been repaired. As a result, + PyArg_ParseTupleAndKeywords may raise RuntimeError in bad cases that + previously went unchallenged. + +New platforms +------------- + +Tests +----- + +Windows +------- + +Mac +---- + +- In unix-Python on Mac OS X (and darwin) sys.platform is now "darwin", + without any trailing digits. + +- Changed logic for finding python home in Mac OS X framework Pythons. + Now sys.executable points to the executable again, in stead of to + the shared library. The latter is used only for locating the python + home. + + +What's New in Python 2.2b2? +=========================== + +*Release date: 16-Nov-2001* + +Type/class unification and new-style classes +-------------------------------------------- + +- Multiple inheritance mixing new-style and classic classes in the + list of base classes is now allowed, so this works now: + + class Classic: pass + class Mixed(Classic, object): pass + + The MRO (method resolution order) for each base class is respected + according to its kind, but the MRO for the derived class is computed + using new-style MRO rules if any base class is a new-style class. + This needs to be documented. + +- The new builtin dictionary() constructor, and dictionary type, have + been renamed to dict. This reflects a decade of common usage. + +- dict() now accepts an iterable object producing 2-sequences. For + example, dict(d.items()) == d for any dictionary d. The argument, + and the elements of the argument, can be any iterable objects. + +- New-style classes can now have a __del__ method, which is called + when the instance is deleted (just like for classic classes). + +- Assignment to object.__dict__ is now possible, for objects that are + instances of new-style classes that have a __dict__ (unless the base + class forbids it). + +- Methods of built-in types now properly check for keyword arguments + (formerly these were silently ignored). The only built-in methods + that take keyword arguments are __call__, __init__ and __new__. + +- The socket function has been converted to a type; see below. + +Core and builtins +----------------- + +- Assignment to __debug__ raises SyntaxError at compile-time. This + was promised when 2.1c1 was released as "What's New in Python 2.1c1" + (see below) says. + +- Clarified the error messages for unsupported operands to an operator + (like 1 + ''). + +Extension modules +----------------- + +- mmap has a new keyword argument, "access", allowing a uniform way for + both Windows and Unix users to create read-only, write-through and + copy-on-write memory mappings. This was previously possible only on + Unix. A new keyword argument was required to support this in a + uniform way because the mmap() signatures had diverged across + platforms. Thanks to Jay T Miller for repairing this! + +- By default, the gc.garbage list now contains only those instances in + unreachable cycles that have __del__ methods; in 2.1 it contained all + instances in unreachable cycles. "Instances" here has been generalized + to include instances of both new-style and old-style classes. + +- The socket module defines a new method for socket objects, + sendall(). This is like send() but may make multiple calls to + send() until all data has been sent. Also, the socket function has + been converted to a subclassable type, like list and tuple (etc.) + before it; socket and SocketType are now the same thing. + +- Various bugfixes to the curses module. There is now a test suite + for the curses module (you have to run it manually). + +- binascii.b2a_base64 no longer places an arbitrary restriction of 57 + bytes on its input. + +Library +------- + +- tkFileDialog exposes a Directory class and askdirectory + convenience function. + +- Symbolic group names in regular expressions must be unique. For + example, the regexp r'(?P<abc>)(?P<abc>)' is not allowed, because a + single name can't mean both "group 1" and "group 2" simultaneously. + Python 2.2 detects this error at regexp compilation time; + previously, the error went undetected, and results were + unpredictable. Also in sre, the pattern.split(), pattern.sub(), and + pattern.subn() methods have been rewritten in C. Also, an + experimental function/method finditer() has been added, which works + like findall() but returns an iterator. + +- Tix exposes more commands through the classes DirSelectBox, + DirSelectDialog, ListNoteBook, Meter, CheckList, and the + methods tix_addbitmapdir, tix_cget, tix_configure, tix_filedialog, + tix_getbitmap, tix_getimage, tix_option_get, and tix_resetoptions. + +- Traceback objects are now scanned by cyclic garbage collection, so + cycles created by casual use of sys.exc_info() no longer cause + permanent memory leaks (provided garbage collection is enabled). + +- os.extsep -- a new variable needed by the RISCOS support. It is the + separator used by extensions, and is '.' on all platforms except + RISCOS, where it is '/'. There is no need to use this variable + unless you have a masochistic desire to port your code to RISCOS. + +- mimetypes.py has optional support for non-standard, but commonly + found types. guess_type() and guess_extension() now accept an + optional 'strict' flag, defaulting to true, which controls whether + recognize non-standard types or not. A few non-standard types we + know about have been added. Also, when run as a script, there are + new -l and -e options. + +- statcache is now deprecated. + +- email.Utils.formatdate() now produces the preferred RFC 2822 style + dates with numeric timezones (it used to produce obsolete dates + hard coded to "GMT" timezone). An optional 'localtime' flag is + added to produce dates in the local timezone, with daylight savings + time properly taken into account. + +- In pickle and cPickle, instead of masking errors in load() by + transforming them into SystemError, we let the original exception + propagate out. Also, implement support for __safe_for_unpickling__ + in pickle, as it already was supported in cPickle. + +Tools/Demos +----------- + +Build +----- + +- The dbm module is built using libdb1 if available. The bsddb module + is built with libdb3 if available. + +- Misc/Makefile.pre.in has been removed by BDFL pronouncement. + +C API +----- + +- New function PySequence_Fast_GET_SIZE() returns the size of a non- + NULL result from PySequence_Fast(), more quickly than calling + PySequence_Size(). + +- New argument unpacking function PyArg_UnpackTuple() added. + +- New functions PyObject_CallFunctionObjArgs() and + PyObject_CallMethodObjArgs() have been added to make it more + convenient and efficient to call functions and methods from C. + +- PyArg_ParseTupleAndKeywords() no longer masks errors, so it's + possible that this will propagate errors it didn't before. + +- New function PyObject_CheckReadBuffer(), which returns true if its + argument supports the single-segment readable buffer interface. + +New platforms +------------- + +- We've finally confirmed that this release builds on HP-UX 11.00, + *with* threads, and passes the test suite. + +- Thanks to a series of patches from Michael Muller, Python may build + again under OS/2 Visual Age C++. + +- Updated RISCOS port by Dietmar Schwertberger. + +Tests +----- + +- Added a test script for the curses module. It isn't run automatically; + regrtest.py must be run with '-u curses' to enable it. + +Windows +------- + +Mac +---- + +- PythonScript has been moved to unsupported and is slated to be + removed completely in the next release. + +- It should now be possible to build applets that work on both OS9 and + OSX. + +- The core is now linked with CoreServices not Carbon; as a side + result, default 8bit encoding on OSX is now ASCII. + +- Python should now build on OSX 10.1.1 + + +What's New in Python 2.2b1? +=========================== + +*Release date: 19-Oct-2001* + +Type/class unification and new-style classes +-------------------------------------------- + +- New-style classes are now always dynamic (except for built-in and + extension types). There is no longer a performance penalty, and I + no longer see another reason to keep this baggage around. One relic + remains: the __dict__ of a new-style class is a read-only proxy; you + must set the class's attribute to modify it. As a consequence, the + __defined__ attribute of new-style types no longer exists, for lack + of need: there is once again only one __dict__ (although in the + future a __cache__ may be resurrected with a similar function, if I + can prove that it actually speeds things up). + +- C.__doc__ now works as expected for new-style classes (in 2.2a4 it + always returned None, even when there was a class docstring). + +- doctest now finds and runs docstrings attached to new-style classes, + class methods, static methods, and properties. + +Core and builtins +----------------- + +- A very subtle syntactical pitfall in list comprehensions was fixed. + For example: [a+b for a in 'abc', for b in 'def']. The comma in + this example is a mistake. Previously, this would silently let 'a' + iterate over the singleton tuple ('abc',), yielding ['abcd', 'abce', + 'abcf'] rather than the intended ['ad', 'ae', 'af', 'bd', 'be', + 'bf', 'cd', 'ce', 'cf']. Now, this is flagged as a syntax error. + Note that [a for a in <singleton>] is a convoluted way to say + [<singleton>] anyway, so it's not like any expressiveness is lost. + +- getattr(obj, name, default) now only catches AttributeError, as + documented, rather than returning the default value for all + exceptions (which could mask bugs in a __getattr__ hook, for + example). + +- Weak reference objects are now part of the core and offer a C API. + A bug which could allow a core dump when binary operations involved + proxy reference has been fixed. weakref.ReferenceError is now a + built-in exception. + +- unicode(obj) now behaves more like str(obj), accepting arbitrary + objects, and calling a __unicode__ method if it exists. + unicode(obj, encoding) and unicode(obj, encoding, errors) still + require an 8-bit string or character buffer argument. + +- isinstance() now allows any object as the first argument and a + class, a type or something with a __bases__ tuple attribute for the + second argument. The second argument may also be a tuple of a + class, type, or something with __bases__, in which case isinstance() + will return true if the first argument is an instance of any of the + things contained in the second argument tuple. E.g. + + isinstance(x, (A, B)) + + returns true if x is an instance of A or B. + +Extension modules +----------------- + +- thread.start_new_thread() now returns the thread ID (previously None). + +- binascii has now two quopri support functions, a2b_qp and b2a_qp. + +- readline now supports setting the startup_hook and the + pre_event_hook, and adds the add_history() function. + +- os and posix supports chroot(), setgroups() and unsetenv() where + available. The stat(), fstat(), statvfs() and fstatvfs() functions + now return "pseudo-sequences" -- the various fields can now be + accessed as attributes (e.g. os.stat("/").st_mtime) but for + backwards compatibility they also behave as a fixed-length sequence. + Some platform-specific fields (e.g. st_rdev) are only accessible as + attributes. + +- time: localtime(), gmtime() and strptime() now return a + pseudo-sequence similar to the os.stat() return value, with + attributes like tm_year etc. + +- Decompression objects in the zlib module now accept an optional + second parameter to decompress() that specifies the maximum amount + of memory to use for the uncompressed data. + +- optional SSL support in the socket module now exports OpenSSL + functions RAND_add(), RAND_egd(), and RAND_status(). These calls + are useful on platforms like Solaris where OpenSSL does not + automatically seed its PRNG. Also, the keyfile and certfile + arguments to socket.ssl() are now optional. + +- posixmodule (and by extension, the os module on POSIX platforms) now + exports O_LARGEFILE, O_DIRECT, O_DIRECTORY, and O_NOFOLLOW. + +Library +------- + +- doctest now excludes functions and classes not defined by the module + being tested, thanks to Tim Hochberg. + +- HotShot, a new profiler implemented using a C-based callback, has + been added. This substantially reduces the overhead of profiling, + but it is still quite preliminary. Support modules and + documentation will be added in upcoming releases (before 2.2 final). + +- profile now produces correct output in situations where an exception + raised in Python is cleared by C code (e.g. hasattr()). This used + to cause wrong output, including spurious claims of recursive + functions and attribution of time spent to the wrong function. + + The code and documentation for the derived OldProfile and HotProfile + profiling classes was removed. The code hasn't worked for years (if + you tried to use them, they raised exceptions). OldProfile + intended to reproduce the behavior of the profiler Python used more + than 7 years ago, and isn't interesting anymore. HotProfile intended + to provide a faster profiler (but producing less information), and + that's a worthy goal we intend to meet via a different approach (but + without losing information). + +- Profile.calibrate() has a new implementation that should deliver + a much better system-specific calibration constant. The constant can + now be specified in an instance constructor, or as a Profile class or + instance variable, instead of by editing profile.py's source code. + Calibration must still be done manually (see the docs for the profile + module). + + Note that Profile.calibrate() must be overridden by subclasses. + Improving the accuracy required exploiting detailed knowledge of + profiler internals; the earlier method abstracted away the details + and measured a simplified model instead, but consequently computed + a constant too small by a factor of 2 on some modern machines. + +- quopri's encode and decode methods take an optional header parameter, + which indicates whether output is intended for the header 'Q' + encoding. + +- The SocketServer.ThreadingMixIn class now closes the request after + finish_request() returns. (Not when it errors out though.) + +- The nntplib module's NNTP.body() method has grown a 'file' argument + to allow saving the message body to a file. + +- The email package has added a class email.Parser.HeaderParser which + only parses headers and does not recurse into the message's body. + Also, the module/class MIMEAudio has been added for representing + audio data (contributed by Anthony Baxter). + +- ftplib should be able to handle files > 2GB. + +- ConfigParser.getboolean() now also interprets TRUE, FALSE, YES, NO, + ON, and OFF. + +- xml.dom.minidom NodeList objects now support the length attribute + and item() method as required by the DOM specifications. + +Tools/Demos +----------- + +- Demo/dns was removed. It no longer serves any purpose; a package + derived from it is now maintained by Anthony Baxter, see + http://PyDNS.SourceForge.net. + +- The freeze tool has been made more robust, and two new options have + been added: -X and -E. + +Build +----- + +- configure will use CXX in LINKCC if CXX is used to build main() and + the system requires to link a C++ main using the C++ compiler. + +C API +----- + +- The documentation for the tp_compare slot is updated to require that + the return value must be -1, 0, 1; an arbitrary number <0 or >0 is + not correct. This is not yet enforced but will be enforced in + Python 2.3; even later, we may use -2 to indicate errors and +2 for + "NotImplemented". Right now, -1 should be used for an error return. + +- PyLong_AsLongLong() now accepts int (as well as long) arguments. + Consequently, PyArg_ParseTuple's 'L' code also accepts int (as well + as long) arguments. + +- PyThread_start_new_thread() now returns a long int giving the thread + ID, if one can be calculated; it returns -1 for error, 0 if no + thread ID is calculated (this is an incompatible change, but only + the thread module used this API). This code has only really been + tested on Linux and Windows; other platforms please beware (and + report any bugs or strange behavior). + +- PyUnicode_FromEncodedObject() no longer accepts Unicode objects as + input. + +New platforms +------------- + +Tests +----- + +Windows +------- + +- Installer: If you install IDLE, and don't disable file-extension + registration, a new "Edit with IDLE" context (right-click) menu entry + is created for .py and .pyw files. + +- The signal module now supports SIGBREAK on Windows, thanks to Steven + Scott. Note that SIGBREAK is unique to Windows. The default SIGBREAK + action remains to call Win32 ExitProcess(). This can be changed via + signal.signal(). For example:: + + # Make Ctrl+Break raise KeyboardInterrupt, like Python's default Ctrl+C + # (SIGINT) behavior. + import signal + signal.signal(signal.SIGBREAK, signal.default_int_handler) + + try: + while 1: + pass + except KeyboardInterrupt: + # We get here on Ctrl+C or Ctrl+Break now; if we had not changed + # SIGBREAK, only on Ctrl+C (and Ctrl+Break would terminate the + # program without the possibility for any Python-level cleanup). + print "Clean exit" + + +What's New in Python 2.2a4? +=========================== + +*Release date: 28-Sep-2001* + +Type/class unification and new-style classes +-------------------------------------------- + +- pydoc and inspect are now aware of new-style classes; + e.g. help(list) at the interactive prompt now shows proper + documentation for all operations on list objects. + +- Applications using Jim Fulton's ExtensionClass module can now safely + be used with Python 2.2. In particular, Zope 2.4.1 now works with + Python 2.2 (as well as with Python 2.1.1). The Demo/metaclass + examples also work again. It is hoped that Gtk and Boost also work + with 2.2a4 and beyond. (If you can confirm this, please write + webmaster@python.org; if there are still problems, please open a bug + report on SourceForge.) + +- property() now takes 4 keyword arguments: fget, fset, fdel and doc. + These map to read-only attributes 'fget', 'fset', 'fdel', and '__doc__' + in the constructed property object. fget, fset and fdel weren't + discoverable from Python in 2.2a3. __doc__ is new, and allows to + associate a docstring with a property. + +- Comparison overloading is now more completely implemented. For + example, a str subclass instance can properly be compared to a str + instance, and it can properly overload comparison. Ditto for most + other built-in object types. + +- The repr() of new-style classes has changed; instead of <type + 'M.Foo'> a new-style class is now rendered as <class 'M.Foo'>, + *except* for built-in types, which are still rendered as <type + 'Foo'> (to avoid upsetting existing code that might parse or + otherwise rely on repr() of certain type objects). + +- The repr() of new-style objects is now always <Foo object at XXX>; + previously, it was sometimes <Foo instance at XXX>. + +- For new-style classes, what was previously called __getattr__ is now + called __getattribute__. This method, if defined, is called for + *every* attribute access. A new __getattr__ hook more similar to the + one in classic classes is defined which is called only if regular + attribute access raises AttributeError; to catch *all* attribute + access, you can use __getattribute__ (for new-style classes). If + both are defined, __getattribute__ is called first, and if it raises + AttributeError, __getattr__ is called. + +- The __class__ attribute of new-style objects can be assigned to. + The new class must have the same C-level object layout as the old + class. + +- The builtin file type can be subclassed now. In the usual pattern, + "file" is the name of the builtin type, and file() is a new builtin + constructor, with the same signature as the builtin open() function. + file() is now the preferred way to open a file. + +- Previously, __new__ would only see sequential arguments passed to + the type in a constructor call; __init__ would see both sequential + and keyword arguments. This made no sense whatsoever any more, so + now both __new__ and __init__ see all arguments. + +- Previously, hash() applied to an instance of a subclass of str or + unicode always returned 0. This has been repaired. + +- Previously, an operation on an instance of a subclass of an + immutable type (int, long, float, complex, tuple, str, unicode), + where the subtype didn't override the operation (and so the + operation was handled by the builtin type), could return that + instance instead a value of the base type. For example, if s was of + a str subclass type, s[:] returned s as-is. Now it returns a str + with the same value as s. + +- Provisional support for pickling new-style objects has been added. + +Core +---- + +- file.writelines() now accepts any iterable object producing strings. + +- PyUnicode_FromEncodedObject() now works very much like + PyObject_Str(obj) in that it tries to use __str__/tp_str + on the object if the object is not a string or buffer. This + makes unicode() behave like str() when applied to non-string/buffer + objects. + +- PyFile_WriteObject now passes Unicode objects to the file's write + method. As a result, all file-like objects which may be the target + of a print statement must support Unicode objects, i.e. they must + at least convert them into ASCII strings. + +- Thread scheduling on Solaris should be improved; it is no longer + necessary to insert a small sleep at the start of a thread in order + to let other runnable threads be scheduled. + +Library +------- + +- StringIO.StringIO instances and cStringIO.StringIO instances support + read character buffer compatible objects for their .write() methods. + These objects are converted to strings and then handled as such + by the instances. + +- The "email" package has been added. This is basically a port of the + mimelib package <http://sf.net/projects/mimelib> with API changes + and some implementations updated to use iterators and generators. + +- difflib.ndiff() and difflib.Differ.compare() are generators now. This + restores the ability of Tools/scripts/ndiff.py to start producing output + before the entire comparison is complete. + +- StringIO.StringIO instances and cStringIO.StringIO instances support + iteration just like file objects (i.e. their .readline() method is + called for each iteration until it returns an empty string). + +- The codecs module has grown four new helper APIs to access + builtin codecs: getencoder(), getdecoder(), getreader(), + getwriter(). + +- SimpleXMLRPCServer: a new module (based upon SimpleHTMLServer) + simplifies writing XML RPC servers. + +- os.path.realpath(): a new function that returns the absolute pathname + after interpretation of symbolic links. On non-Unix systems, this + is an alias for os.path.abspath(). + +- operator.indexOf() (PySequence_Index() in the C API) now works with any + iterable object. + +- smtplib now supports various authentication and security features of + the SMTP protocol through the new login() and starttls() methods. + +- hmac: a new module implementing keyed hashing for message + authentication. + +- mimetypes now recognizes more extensions and file types. At the + same time, some mappings not sanctioned by IANA were removed. + +- The "compiler" package has been brought up to date to the state of + Python 2.2 bytecode generation. It has also been promoted from a + Tool to a standard library package. (Tools/compiler still exists as + a sample driver.) + +Build +----- + +- Large file support (LFS) is now automatic when the platform supports + it; no more manual configuration tweaks are needed. On Linux, at + least, it's possible to have a system whose C library supports large + files but whose kernel doesn't; in this case, large file support is + still enabled but doesn't do you any good unless you upgrade your + kernel or share your Python executable with another system whose + kernel has large file support. + +- The configure script now supplies plausible defaults in a + cross-compilation environment. This doesn't mean that the supplied + values are always correct, or that cross-compilation now works + flawlessly -- but it's a first step (and it shuts up most of + autoconf's warnings about AC_TRY_RUN). + +- The Unix build is now a bit less chatty, courtesy of the parser + generator. The build is completely silent (except for errors) when + using "make -s", thanks to a -q option to setup.py. + +C API +----- + +- The "structmember" API now supports some new flag bits to deny read + and/or write access to attributes in restricted execution mode. + +New platforms +------------- + +- Compaq's iPAQ handheld, running the "familiar" Linux distribution + (http://familiar.handhelds.org). + +Tests +----- + +- The "classic" standard tests, which work by comparing stdout to + an expected-output file under Lib/test/output/, no longer stop at + the first mismatch. Instead the test is run to completion, and a + variant of ndiff-style comparison is used to report all differences. + This is much easier to understand than the previous style of reporting. + +- The unittest-based standard tests now use regrtest's test_main() + convention, instead of running as a side-effect of merely being + imported. This allows these tests to be run in more natural and + flexible ways as unittests, outside the regrtest framework. + +- regrtest.py is much better integrated with unittest and doctest now, + especially in regard to reporting errors. + +Windows +------- + +- Large file support now also works for files > 4GB, on filesystems + that support it (NTFS under Windows 2000). See "What's New in + Python 2.2a3" for more detail. + + +What's New in Python 2.2a3? +=========================== + +*Release Date: 07-Sep-2001* + +Core +---- + +- Conversion of long to float now raises OverflowError if the long is too + big to represent as a C double. + +- The 3-argument builtin pow() no longer allows a third non-None argument + if either of the first two arguments is a float, or if both are of + integer types and the second argument is negative (in which latter case + the arguments are converted to float, so this is really the same + restriction). + +- The builtin dir() now returns more information, and sometimes much + more, generally naming all attributes of an object, and all attributes + reachable from the object via its class, and from its class's base + classes, and so on from them too. Example: in 2.2a2, dir([]) returned + an empty list. In 2.2a3, + + >>> dir([]) + ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', + '__eq__', '__ge__', '__getattr__', '__getitem__', '__getslice__', + '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__le__', + '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__repr__', + '__rmul__', '__setattr__', '__setitem__', '__setslice__', '__str__', + 'append', 'count', 'extend', 'index', 'insert', 'pop', 'remove', + 'reverse', 'sort'] + + dir(module) continues to return only the module's attributes, though. + +- Overflowing operations on plain ints now return a long int rather + than raising OverflowError. This is a partial implementation of PEP + 237. You can use -Wdefault::OverflowWarning to enable a warning for + this situation, and -Werror::OverflowWarning to revert to the old + OverflowError exception. + +- A new command line option, -Q<arg>, is added to control run-time + warnings for the use of classic division. (See PEP 238.) Possible + values are -Qold, -Qwarn, -Qwarnall, and -Qnew. The default is + -Qold, meaning the / operator has its classic meaning and no + warnings are issued. Using -Qwarn issues a run-time warning about + all uses of classic division for int and long arguments; -Qwarnall + also warns about classic division for float and complex arguments + (for use with fixdiv.py). + [Note: the remainder of this item (preserved below) became + obsolete in 2.2c1 -- -Qnew has global effect in 2.2] :: + + Using -Qnew is questionable; it turns on new division by default, but + only in the __main__ module. You can usefully combine -Qwarn or + -Qwarnall and -Qnew: this gives the __main__ module new division, and + warns about classic division everywhere else. + +- Many built-in types can now be subclassed. This applies to int, + long, float, str, unicode, and tuple. (The types complex, list and + dictionary can also be subclassed; this was introduced earlier.) + Note that restrictions apply when subclassing immutable built-in + types: you can only affect the value of the instance by overloading + __new__. You can add mutable attributes, and the subclass instances + will have a __dict__ attribute, but you cannot change the "value" + (as implemented by the base class) of an immutable subclass instance + once it is created. + +- The dictionary constructor now takes an optional argument, a + mapping-like object, and initializes the dictionary from its + (key, value) pairs. + +- A new built-in type, super, has been added. This facilitates making + "cooperative super calls" in a multiple inheritance setting. For an + explanation, see http://www.python.org/2.2/descrintro.html#cooperation + +- A new built-in type, property, has been added. This enables the + creation of "properties". These are attributes implemented by + getter and setter functions (or only one of these for read-only or + write-only attributes), without the need to override __getattr__. + See http://www.python.org/2.2/descrintro.html#property + +- The syntax of floating-point and imaginary literals has been + liberalized, to allow leading zeroes. Examples of literals now + legal that were SyntaxErrors before: + + 00.0 0e3 0100j 07.5 00000000000000000008. + +- An old tokenizer bug allowed floating point literals with an incomplete + exponent, such as 1e and 3.1e-. Such literals now raise SyntaxError. + +Library +------- + +- telnetlib includes symbolic names for the options, and support for + setting an option negotiation callback. It also supports processing + of suboptions. + +- The new C standard no longer requires that math libraries set errno to + ERANGE on overflow. For platform libraries that exploit this new + freedom, Python's overflow-checking was wholly broken. A new overflow- + checking scheme attempts to repair that, but may not be reliable on all + platforms (C doesn't seem to provide anything both useful and portable + in this area anymore). + +- Asynchronous timeout actions are available through the new class + threading.Timer. + +- math.log and math.log10 now return sensible results for even huge + long arguments. For example, math.log10(10 ** 10000) ~= 10000.0. + +- A new function, imp.lock_held(), returns 1 when the import lock is + currently held. See the docs for the imp module. + +- pickle, cPickle and marshal on 32-bit platforms can now correctly read + dumps containing ints written on platforms where Python ints are 8 bytes. + When read on a box where Python ints are 4 bytes, such values are + converted to Python longs. + +- In restricted execution mode (using the rexec module), unmarshalling + code objects is no longer allowed. This plugs a security hole. + +- unittest.TestResult instances no longer store references to tracebacks + generated by test failures. This prevents unexpected dangling references + to objects that should be garbage collected between tests. + +Tools +----- + +- Tools/scripts/fixdiv.py has been added which can be used to fix + division operators as per PEP 238. + +Build +----- + +- If you are an adventurous person using Mac OS X you may want to look at + Mac/OSX. There is a Makefile there that will build Python as a real Mac + application, which can be used for experimenting with Carbon or Cocoa. + Discussion of this on pythonmac-sig, please. + +C API +----- + +- New function PyObject_Dir(obj), like Python __builtin__.dir(obj). + +- Note that PyLong_AsDouble can fail! This has always been true, but no + callers checked for it. It's more likely to fail now, because overflow + errors are properly detected now. The proper way to check:: + + double x = PyLong_AsDouble(some_long_object); + if (x == -1.0 && PyErr_Occurred()) { + /* The conversion failed. */ + } + +- The GC API has been changed. Extensions that use the old API will still + compile but will not participate in GC. To upgrade an extension + module: + + - rename Py_TPFLAGS_GC to PyTPFLAGS_HAVE_GC + + - use PyObject_GC_New or PyObject_GC_NewVar to allocate objects and + PyObject_GC_Del to deallocate them + + - rename PyObject_GC_Init to PyObject_GC_Track and PyObject_GC_Fini + to PyObject_GC_UnTrack + + - remove PyGC_HEAD_SIZE from object size calculations + + - remove calls to PyObject_AS_GC and PyObject_FROM_GC + +- Two new functions: PyString_FromFormat() and PyString_FromFormatV(). + These can be used safely to construct string objects from a + sprintf-style format string (similar to the format string supported + by PyErr_Format()). + +New platforms +------------- + +- Stephen Hansen contributed patches sufficient to get a clean compile + under Borland C (Windows), but he reports problems running it and ran + out of time to complete the port. Volunteers? Expect a MemoryError + when importing the types module; this is probably shallow, and + causing later failures too. + +Tests +----- + +Windows +------- + +- Large file support is now enabled on Win32 platforms as well as on + Win64. This means that, for example, you can use f.tell() and f.seek() + to manipulate files larger than 2 gigabytes (provided you have enough + disk space, and are using a Windows filesystem that supports large + partitions). Windows filesystem limits: FAT has a 2GB (gigabyte) + filesize limit, and large file support makes no difference there. + FAT32's limit is 4GB, and files >= 2GB are easier to use from Python now. + NTFS has no practical limit on file size, and files of any size can be + used from Python now. + +- The w9xpopen hack is now used on Windows NT and 2000 too when COMPSPEC + points to command.com (patch from Brian Quinlan). + + +What's New in Python 2.2a2? +=========================== + +*Release Date: 22-Aug-2001* + +Build +----- + +- Tim Peters developed a brand new Windows installer using Wise 8.1, + generously donated to us by Wise Solutions. + +- configure supports a new option --enable-unicode, with the values + ucs2 and ucs4 (new in 2.2a1). With --disable-unicode, the Unicode + type and supporting code is completely removed from the interpreter. + +- A new configure option --enable-framework builds a Mac OS X framework, + which "make frameworkinstall" will install. This provides a starting + point for more mac-like functionality, join pythonmac-sig@python.org + if you are interested in helping. + +- The NeXT platform is no longer supported. + +- The 'new' module is now statically linked. + +Tools +----- + +- The new Tools/scripts/cleanfuture.py can be used to automatically + edit out obsolete future statements from Python source code. See + the module docstring for details. + +Tests +----- + +- regrtest.py now knows which tests are expected to be skipped on some + platforms, allowing to give clearer test result output. regrtest + also has optional --use/-u switch to run normally disabled tests + which require network access or consume significant disk resources. + +- Several new tests in the standard test suite, with special thanks to + Nick Mathewson. + +Core +---- + +- The floor division operator // has been added as outlined in PEP + 238. The / operator still provides classic division (and will until + Python 3.0) unless "from __future__ import division" is included, in + which case the / operator will provide true division. The operator + module provides truediv() and floordiv() functions. Augmented + assignment variants are included, as are the equivalent overloadable + methods and C API methods. See the PEP for a full discussion: + <http://python.sf.net/peps/pep-0238.html> + +- Future statements are now effective in simulated interactive shells + (like IDLE). This should "just work" by magic, but read Michael + Hudson's "Future statements in simulated shells" PEP 264 for full + details: <http://python.sf.net/peps/pep-0264.html>. + +- The type/class unification (PEP 252-253) was integrated into the + trunk and is not so tentative any more (the exact specification of + some features is still tentative). A lot of work has done on fixing + bugs and adding robustness and features (performance still has to + come a long way). + +- Warnings about a mismatch in the Python API during extension import + now use the Python warning framework (which makes it possible to + write filters for these warnings). + +- A function's __dict__ (aka func_dict) will now always be a + dictionary. It used to be possible to delete it or set it to None, + but now both actions raise TypeErrors. It is still legal to set it + to a dictionary object. Getting func.__dict__ before any attributes + have been assigned now returns an empty dictionary instead of None. + +- A new command line option, -E, was added which disables the use of + all environment variables, or at least those that are specifically + significant to Python. Usually those have a name starting with + "PYTHON". This was used to fix a problem where the tests fail if + the user happens to have PYTHONHOME or PYTHONPATH pointing to an + older distribution. + +Library +------- + +- New class Differ and new functions ndiff() and restore() in difflib.py. + These package the algorithms used by the popular Tools/scripts/ndiff.py, + for programmatic reuse. + +- New function xml.sax.saxutils.quoteattr(): Quote an XML attribute + value using the minimal quoting required for the value; more + reliable than using xml.sax.saxutils.escape() for attribute values. + +- Readline completion support for cmd.Cmd was added. + +- Calling os.tempnam() or os.tmpnam() generate RuntimeWarnings. + +- Added function threading.BoundedSemaphore() + +- Added Ka-Ping Yee's cgitb.py module. + +- The 'new' module now exposes the CO_xxx flags. + +- The gc module offers the get_referents function. + +New platforms +------------- + +C API +----- + +- Two new APIs PyOS_snprintf() and PyOS_vsnprintf() were added + which provide a cross-platform implementations for the + relatively new snprintf()/vsnprintf() C lib APIs. In contrast to + the standard sprintf() and vsprintf() C lib APIs, these versions + apply bounds checking on the used buffer which enhances protection + against buffer overruns. + +- Unicode APIs now use name mangling to assure that mixing interpreters + and extensions using different Unicode widths is rendered next to + impossible. Trying to import an incompatible Unicode-aware extension + will result in an ImportError. Unicode extensions writers must make + sure to check the Unicode width compatibility in their extensions by + using at least one of the mangled Unicode APIs in the extension. + +- Two new flags METH_NOARGS and METH_O are available in method definition + tables to simplify implementation of methods with no arguments and a + single untyped argument. Calling such methods is more efficient than + calling corresponding METH_VARARGS methods. METH_OLDARGS is now + deprecated. + +Windows +------- + +- "import module" now compiles module.pyw if it exists and nothing else + relevant is found. + + +What's New in Python 2.2a1? +=========================== + +*Release date: 18-Jul-2001* + +Core +---- + +- TENTATIVELY, a large amount of code implementing much of what's + described in PEP 252 (Making Types Look More Like Classes) and PEP + 253 (Subtyping Built-in Types) was added. This will be released + with Python 2.2a1. Documentation will be provided separately + through http://www.python.org/2.2/. The purpose of releasing this + with Python 2.2a1 is to test backwards compatibility. It is + possible, though not likely, that a decision is made not to release + this code as part of 2.2 final, if any serious backwards + incompatibilities are found during alpha testing that cannot be + repaired. + +- Generators were added; this is a new way to create an iterator (see + below) using what looks like a simple function containing one or + more 'yield' statements. See PEP 255. Since this adds a new + keyword to the language, this feature must be enabled by including a + future statement: "from __future__ import generators" (see PEP 236). + Generators will become a standard feature in a future release + (probably 2.3). Without this future statement, 'yield' remains an + ordinary identifier, but a warning is issued each time it is used. + (These warnings currently don't conform to the warnings framework of + PEP 230; we intend to fix this in 2.2a2.) + +- The UTF-16 codec was modified to be more RFC compliant. It will now + only remove BOM characters at the start of the string and then + only if running in native mode (UTF-16-LE and -BE won't remove a + leading BMO character). + +- Strings now have a new method .decode() to complement the already + existing .encode() method. These two methods provide direct access + to the corresponding decoders and encoders of the registered codecs. + + To enhance the usability of the .encode() method, the special + casing of Unicode object return values was dropped (Unicode objects + were auto-magically converted to string using the default encoding). + + Both methods will now return whatever the codec in charge of the + requested encoding returns as object, e.g. Unicode codecs will + return Unicode objects when decoding is requested ("äöü".decode("latin-1") + will return u"äöü"). This enables codec writer to create codecs + for various simple to use conversions. + + New codecs were added to demonstrate these new features (the .encode() + and .decode() columns indicate the type of the returned objects): + + +---------+-----------+-----------+-----------------------------+ + |Name | .encode() | .decode() | Description | + +=========+===========+===========+=============================+ + |uu | string | string | UU codec (e.g. for email) | + +---------+-----------+-----------+-----------------------------+ + |base64 | string | string | base64 codec | + +---------+-----------+-----------+-----------------------------+ + |quopri | string | string | quoted-printable codec | + +---------+-----------+-----------+-----------------------------+ + |zlib | string | string | zlib compression | + +---------+-----------+-----------+-----------------------------+ + |hex | string | string | 2-byte hex codec | + +---------+-----------+-----------+-----------------------------+ + |rot-13 | string | Unicode | ROT-13 Unicode charmap codec| + +---------+-----------+-----------+-----------------------------+ + +- Some operating systems now support the concept of a default Unicode + encoding for file system operations. Notably, Windows supports 'mbcs' + as the default. The Macintosh will also adopt this concept in the medium + term, although the default encoding for that platform will be other than + 'mbcs'. + + On operating system that support non-ASCII filenames, it is common for + functions that return filenames (such as os.listdir()) to return Python + string objects pre-encoded using the default file system encoding for + the platform. As this encoding is likely to be different from Python's + default encoding, converting this name to a Unicode object before passing + it back to the Operating System would result in a Unicode error, as Python + would attempt to use its default encoding (generally ASCII) rather than + the default encoding for the file system. + + In general, this change simply removes surprises when working with + Unicode and the file system, making these operations work as you expect, + increasing the transparency of Unicode objects in this context. + See [????] for more details, including examples. + +- Float (and complex) literals in source code were evaluated to full + precision only when running from a .py file; the same code loaded from a + .pyc (or .pyo) file could suffer numeric differences starting at about the + 12th significant decimal digit. For example, on a machine with IEEE-754 + floating arithmetic, + + x = 9007199254740992.0 + print long(x) + + printed 9007199254740992 if run directly from .py, but 9007199254740000 + if from a compiled (.pyc or .pyo) file. This was due to marshal using + str(float) instead of repr(float) when building code objects. marshal + now uses repr(float) instead, which should reproduce floats to full + machine precision (assuming the platform C float<->string I/O conversion + functions are of good quality). + + This may cause floating-point results to change in some cases, and + usually for the better, but may also cause numerically unstable + algorithms to break. + +- The implementation of dicts suffers fewer collisions, which has speed + benefits. However, the order in which dict entries appear in dict.keys(), + dict.values() and dict.items() may differ from previous releases for a + given dict. Nothing is defined about this order, so no program should + rely on it. Nevertheless, it's easy to write test cases that rely on the + order by accident, typically because of printing the str() or repr() of a + dict to an "expected results" file. See Lib/test/test_support.py's new + sortdict(dict) function for a simple way to display a dict in sorted + order. + +- Many other small changes to dicts were made, resulting in faster + operation along the most common code paths. + +- Dictionary objects now support the "in" operator: "x in dict" means + the same as dict.has_key(x). + +- The update() method of dictionaries now accepts generic mapping + objects. Specifically the argument object must support the .keys() + and __getitem__() methods. This allows you to say, for example, + {}.update(UserDict()) + +- Iterators were added; this is a generalized way of providing values + to a for loop. See PEP 234. There's a new built-in function iter() + to return an iterator. There's a new protocol to get the next value + from an iterator using the next() method (in Python) or the + tp_iternext slot (in C). There's a new protocol to get iterators + using the __iter__() method (in Python) or the tp_iter slot (in C). + Iterating (i.e. a for loop) over a dictionary generates its keys. + Iterating over a file generates its lines. + +- The following functions were generalized to work nicely with iterator + arguments:: + + map(), filter(), reduce(), zip() + list(), tuple() (PySequence_Tuple() and PySequence_Fast() in C API) + max(), min() + join() method of strings + extend() method of lists + 'x in y' and 'x not in y' (PySequence_Contains() in C API) + operator.countOf() (PySequence_Count() in C API) + right-hand side of assignment statements with multiple targets, such as :: + x, y, z = some_iterable_object_returning_exactly_3_values + +- Accessing module attributes is significantly faster (for example, + random.random or os.path or yourPythonModule.yourAttribute). + +- Comparing dictionary objects via == and != is faster, and now works even + if the keys and values don't support comparisons other than ==. + +- Comparing dictionaries in ways other than == and != is slower: there were + insecurities in the dict comparison implementation that could cause Python + to crash if the element comparison routines for the dict keys and/or + values mutated the dicts. Making the code bulletproof slowed it down. + +- Collisions in dicts are resolved via a new approach, which can help + dramatically in bad cases. For example, looking up every key in a dict + d with d.keys() == [i << 16 for i in range(20000)] is approximately 500x + faster now. Thanks to Christian Tismer for pointing out the cause and + the nature of an effective cure (last December! better late than never). + +- repr() is much faster for large containers (dict, list, tuple). + + +Library +------- + +- The constants ascii_letters, ascii_lowercase. and ascii_uppercase + were added to the string module. These a locale-independent + constants, unlike letters, lowercase, and uppercase. These are now + use in appropriate locations in the standard library. + +- The flags used in dlopen calls can now be configured using + sys.setdlopenflags and queried using sys.getdlopenflags. + +- Fredrik Lundh's xmlrpclib is now a standard library module. This + provides full client-side XML-RPC support. In addition, + Demo/xmlrpc/ contains two server frameworks (one SocketServer-based, + one asyncore-based). Thanks to Eric Raymond for the documentation. + +- The xrange() object is simplified: it no longer supports slicing, + repetition, comparisons, efficient 'in' checking, the tolist() + method, or the start, stop and step attributes. See PEP 260. + +- A new function fnmatch.filter to filter lists of file names was added. + +- calendar.py uses month and day names based on the current locale. + +- strop is now *really* obsolete (this was announced before with 1.6), + and issues DeprecationWarning when used (except for the four items + that are still imported into string.py). + +- Cookie.py now sorts key+value pairs by key in output strings. + +- pprint.isrecursive(object) didn't correctly identify recursive objects. + Now it does. + +- pprint functions now much faster for large containers (tuple, list, dict). + +- New 'q' and 'Q' format codes in the struct module, corresponding to C + types "long long" and "unsigned long long" (on Windows, __int64). In + native mode, these can be used only when the platform C compiler supports + these types (when HAVE_LONG_LONG is #define'd by the Python config + process), and then they inherit the sizes and alignments of the C types. + In standard mode, 'q' and 'Q' are supported on all platforms, and are + 8-byte integral types. + +- The site module installs a new built-in function 'help' that invokes + pydoc.help. It must be invoked as 'help()'; when invoked as 'help', + it displays a message reminding the user to use 'help()' or + 'help(object)'. + +Tests +----- + +- New test_mutants.py runs dict comparisons where the key and value + comparison operators mutate the dicts randomly during comparison. This + rapidly causes Python to crash under earlier releases (not for the faint + of heart: it can also cause Win9x to freeze or reboot!). + +- New test_pprint.py verifies that pprint.isrecursive() and + pprint.isreadable() return sensible results. Also verifies that simple + cases produce correct output. + +C API +----- + +- Removed the unused last_is_sticky argument from the internal + _PyTuple_Resize(). If this affects you, you were cheating. + +What's New in Python 2.1 (final)? +================================= + +We only changed a few things since the last release candidate, all in +Python library code: + +- A bug in the locale module was fixed that affected locales which + define no grouping for numeric formatting. + +- A few bugs in the weakref module's implementations of weak + dictionaries (WeakValueDictionary and WeakKeyDictionary) were fixed, + and the test suite was updated to check for these bugs. + +- An old bug in the os.path.walk() function (introduced in Python + 2.0!) was fixed: a non-existent file would cause an exception + instead of being ignored. + +- Fixed a few bugs in the new symtable module found by Neil Norwitz's + PyChecker. + + +What's New in Python 2.1c2? +=========================== + +A flurry of small changes, and one showstopper fixed in the nick of +time made it necessary to release another release candidate. The list +here is the *complete* list of patches (except version updates): + +Core + +- Tim discovered a nasty bug in the dictionary code, caused by + PyDict_Next() calling dict_resize(), and the GC code's use of + PyDict_Next() violating an assumption in dict_items(). This was + fixed with considerable amounts of band-aid, but the net effect is a + saner and more robust implementation. + +- Made a bunch of symbols static that were accidentally global. + +Build and Ports + +- The setup.py script didn't check for a new enough version of zlib + (1.1.3 is needed). Now it does. + +- Changed "make clean" target to also remove shared libraries. + +- Added a more general warning about the SGI Irix optimizer to README. + +Library + +- Fix a bug in urllib.basejoin("http://host", "../file.html") which + omitted the slash between host and file.html. + +- The mailbox module's _Mailbox class contained a completely broken + and undocumented seek() method. Ripped it out. + +- Fixed a bunch of typos in various library modules (urllib2, smtpd, + sgmllib, netrc, chunk) found by Neil Norwitz's PyChecker. + +- Fixed a few last-minute bugs in unittest. + +Extensions + +- Reverted the patch to the OpenSSL code in socketmodule.c to support + RAND_status() and the EGD, and the subsequent patch that tried to + fix it for pre-0.9.5 versions; the problem with the patch is that on + some systems it issues a warning whenever socket is imported, and + that's unacceptable. + +Tests + +- Fixed the pickle tests to work with "import test.test_pickle". + +- Tweaked test_locale.py to actually run the test Windows. + +- In distutils/archive_util.py, call zipfile.ZipFile() with mode "w", + not "wb" (which is not a valid mode at all). + +- Fix pstats browser crashes. Import readline if it exists to make + the user interface nicer. + +- Add "import thread" to the top of test modules that import the + threading module (test_asynchat and test_threadedtempfile). This + prevents test failures caused by a broken threading module resulting + from a previously caught failed import. + +- Changed test_asynchat.py to set the SO_REUSEADDR option; this was + needed on some platforms (e.g. Solaris 8) when the tests are run + twice in succession. + +- Skip rather than fail test_sunaudiodev if no audio device is found. + + +What's New in Python 2.1c1? +=========================== + +This list was significantly updated when 2.1c2 was released; the 2.1c1 +release didn't mention most changes that were actually part of 2.1c1: + +Legal + +- Copyright was assigned to the Python Software Foundation (PSF) and a + PSF license (very similar to the CNRI license) was added. + +- The CNRI copyright notice was updated to include 2001. + +Core + +- After a public outcry, assignment to __debug__ is no longer illegal; + instead, a warning is issued. It will become illegal in 2.2. + +- Fixed a core dump with "%#x" % 0, and changed the semantics so that + "%#x" now always prepends "0x", even if the value is zero. + +- Fixed some nits in the bytecode compiler. + +- Fixed core dumps when calling certain kinds of non-functions. + +- Fixed various core dumps caused by reference count bugs. + +Build and Ports + +- Use INSTALL_SCRIPT to install script files. + +- New port: SCO Unixware 7, by Billy G. Allie. + +- Updated RISCOS port. + +- Updated BeOS port and notes. + +- Various other porting problems resolved. + +Library + +- The TERMIOS and SOCKET modules are now truly obsolete and + unnecessary. Their symbols are incorporated in the termios and + socket modules. + +- Fixed some 64-bit bugs in pickle, cPickle, and struct, and added + better tests for pickling. + +- threading: make Condition.wait() robust against KeyboardInterrupt. + +- zipfile: add support to zipfile to support opening an archive + represented by an open file rather than a file name. Fix bug where + the archive was not properly closed. Fixed a bug in this bugfix + where flush() was called for a read-only file. + +- imputil: added an uninstall() method to the ImportManager. + +- Canvas: fixed bugs in lower() and tkraise() methods. + +- SocketServer: API change (added overridable close_request() method) + so that the TCP server can explicitly close the request. + +- pstats: Eric Raymond added a simple interactive statistics browser, + invoked when the module is run as a script. + +- locale: fixed a problem in format(). + +- webbrowser: made it work when the BROWSER environment variable has a + value like "/usr/bin/netscape". Made it auto-detect Konqueror for + KDE 2. Fixed some other nits. + +- unittest: changes to allow using a different exception than + AssertionError, and added a few more function aliases. Some other + small changes. + +- urllib, urllib2: fixed redirect problems and a coupleof other nits. + +- asynchat: fixed a critical bug in asynchat that slipped through the + 2.1b2 release. Fixed another rare bug. + +- Fix some unqualified except: clauses (always a bad code example). + +XML + +- pyexpat: new API get_version_string(). + +- Fixed some minidom bugs. + +Extensions + +- Fixed a core dump in _weakref. Removed the weakref.mapping() + function (it adds nothing to the API). + +- Rationalized the use of header files in the readline module, to make + it compile (albeit with some warnings) with the very recent readline + 4.2, without breaking for earlier versions. + +- Hopefully fixed a buffering problem in linuxaudiodev. + +- Attempted a fix to make the OpenSSL support in the socket module + work again with pre-0.9.5 versions of OpenSSL. + +Tests + +- Added a test case for asynchat and asyncore. + +- Removed coupling between tests where one test failing could break + another. + +Tools + +- Ping added an interactive help browser to pydoc, fixed some nits + in the rest of the pydoc code, and added some features to his + inspect module. + +- An updated python-mode.el version 4.1 which integrates Ken + Manheimer's pdbtrack.el. This makes debugging Python code via pdb + much nicer in XEmacs and Emacs. When stepping through your program + with pdb, in either the shell window or the *Python* window, the + source file and line will be tracked by an arrow. Very cool! + +- IDLE: syntax warnings in interactive mode are changed into errors. + +- Some improvements to Tools/webchecker (ignore some more URL types, + follow some more links). + +- Brought the Tools/compiler package up to date. + + +What's New in Python 2.1 beta 2? +================================ + +(Unlisted are many fixed bugs, more documentation, etc.) + +Core language, builtins, and interpreter + +- The nested scopes work (enabled by "from __future__ import + nested_scopes") is completed; in particular, the future now extends + into code executed through exec, eval() and execfile(), and into the + interactive interpreter. + +- When calling a base class method (e.g. BaseClass.__init__(self)), + this is now allowed even if self is not strictly spoken a class + instance (e.g. when using metaclasses or the Don Beaudry hook). + +- Slice objects are now comparable but not hashable; this prevents + dict[:] from being accepted but meaningless. + +- Complex division is now calculated using less braindead algorithms. + This doesn't change semantics except it's more likely to give useful + results in extreme cases. Complex repr() now uses full precision + like float repr(). + +- sgmllib.py now calls handle_decl() for simple <!...> declarations. + +- It is illegal to assign to the name __debug__, which is set when the + interpreter starts. It is effectively a compile-time constant. + +- A warning will be issued if a global statement for a variable + follows a use or assignment of that variable. + +Standard library + +- unittest.py, a unit testing framework by Steve Purcell (PyUNIT, + inspired by JUnit), is now part of the standard library. You now + have a choice of two testing frameworks: unittest requires you to + write testcases as separate code, doctest gathers them from + docstrings. Both approaches have their advantages and + disadvantages. + +- A new module Tix was added, which wraps the Tix extension library + for Tk. With that module, it is not necessary to statically link + Tix with _tkinter, since Tix will be loaded with Tcl's "package + require" command. See Demo/tix/. + +- tzparse.py is now obsolete. + +- In gzip.py, the seek() and tell() methods are removed -- they were + non-functional anyway, and it's better if callers can test for their + existence with hasattr(). + +Python/C API + +- PyDict_Next(): it is now safe to call PyDict_SetItem() with a key + that's already in the dictionary during a PyDict_Next() iteration. + This used to fail occasionally when a dictionary resize operation + could be triggered that would rehash all the keys. All other + modifications to the dictionary are still off-limits during a + PyDict_Next() iteration! + +- New extended APIs related to passing compiler variables around. + +- New abstract APIs PyObject_IsInstance(), PyObject_IsSubclass() + implement isinstance() and issubclass(). + +- Py_BuildValue() now has a "D" conversion to create a Python complex + number from a Py_complex C value. + +- Extensions types which support weak references must now set the + field allocated for the weak reference machinery to NULL themselves; + this is done to avoid the cost of checking each object for having a + weakly referencable type in PyObject_INIT(), since most types are + not weakly referencable. + +- PyFrame_FastToLocals() and PyFrame_LocalsToFast() copy bindings for + free variables and cell variables to and from the frame's f_locals. + +- Variants of several functions defined in pythonrun.h have been added + to support the nested_scopes future statement. The variants all end + in Flags and take an extra argument, a PyCompilerFlags *; examples: + PyRun_AnyFileExFlags(), PyRun_InteractiveLoopFlags(). These + variants may be removed in Python 2.2, when nested scopes are + mandatory. + +Distutils + +- the sdist command now writes a PKG-INFO file, as described in PEP 241, + into the release tree. + +- several enhancements to the bdist_wininst command from Thomas Heller + (an uninstaller, more customization of the installer's display) + +- from Jack Jansen: added Mac-specific code to generate a dialog for + users to specify the command-line (because providing a command-line with + MacPython is awkward). Jack also made various fixes for the Mac + and the Metrowerks compiler. + +- added 'platforms' and 'keywords' to the set of metadata that can be + specified for a distribution. + +- applied patches from Jason Tishler to make the compiler class work with + Cygwin. + + +What's New in Python 2.1 beta 1? +================================ + +Core language, builtins, and interpreter + +- Following an outcry from the community about the amount of code + broken by the nested scopes feature introduced in 2.1a2, we decided + to make this feature optional, and to wait until Python 2.2 (or at + least 6 months) to make it standard. The option can be enabled on a + per-module basis by adding "from __future__ import nested_scopes" at + the beginning of a module (before any other statements, but after + comments and an optional docstring). See PEP 236 (Back to the + __future__) for a description of the __future__ statement. PEP 227 + (Statically Nested Scopes) has been updated to reflect this change, + and to clarify the semantics in a number of endcases. + +- The nested scopes code, when enabled, has been hardened, and most + bugs and memory leaks in it have been fixed. + +- Compile-time warnings are now generated for a number of conditions + that will break or change in meaning when nested scopes are enabled: + + - Using "from...import *" or "exec" without in-clause in a function + scope that also defines a lambda or nested function with one or + more free (non-local) variables. The presence of the import* or + bare exec makes it impossible for the compiler to determine the + exact set of local variables in the outer scope, which makes it + impossible to determine the bindings for free variables in the + inner scope. To avoid the warning about import *, change it into + an import of explicitly name object, or move the import* statement + to the global scope; to avoid the warning about bare exec, use + exec...in... (a good idea anyway -- there's a possibility that + bare exec will be deprecated in the future). + + - Use of a global variable in a nested scope with the same name as a + local variable in a surrounding scope. This will change in + meaning with nested scopes: the name in the inner scope will + reference the variable in the outer scope rather than the global + of the same name. To avoid the warning, either rename the outer + variable, or use a global statement in the inner function. + +- An optional object allocator has been included. This allocator is + optimized for Python objects and should be faster and use less memory + than the standard system allocator. It is not enabled by default + because of possible thread safety problems. The allocator is only + protected by the Python interpreter lock and it is possible that some + extension modules require a thread safe allocator. The object + allocator can be enabled by providing the "--with-pymalloc" option to + configure. + +Standard library + +- pyexpat now detects the expat version if expat.h defines it. A + number of additional handlers are provided, which are only available + since expat 1.95. In addition, the methods SetParamEntityParsing and + GetInputContext of Parser objects are available with 1.95.x + only. Parser objects now provide the ordered_attributes and + specified_attributes attributes. A new module expat.model was added, + which offers a number of additional constants if 1.95.x is used. + +- xml.dom offers the new functions registerDOMImplementation and + getDOMImplementation. + +- xml.dom.minidom offers a toprettyxml method. A number of DOM + conformance issues have been resolved. In particular, Element now + has an hasAttributes method, and the handling of namespaces was + improved. + +- Ka-Ping Yee contributed two new modules: inspect.py, a module for + getting information about live Python code, and pydoc.py, a module + for interactively converting docstrings to HTML or text. + Tools/scripts/pydoc, which is now automatically installed into + <prefix>/bin, uses pydoc.py to display documentation; try running + "pydoc -h" for instructions. "pydoc -g" pops up a small GUI that + lets you browse the module docstrings using a web browser. + +- New library module difflib.py, primarily packaging the SequenceMatcher + class at the heart of the popular ndiff.py file-comparison tool. + +- doctest.py (a framework for verifying Python code examples in docstrings) + is now part of the std library. + +Windows changes + +- A new entry in the Start menu, "Module Docs", runs "pydoc -g" -- a + small GUI that lets you browse the module docstrings using your + default web browser. + +- Import is now case-sensitive. PEP 235 (Import on Case-Insensitive + Platforms) is implemented. See + + http://python.sourceforge.net/peps/pep-0235.html + + for full details, especially the "Current Lower-Left Semantics" section. + The new Windows import rules are simpler than before: + + A. If the PYTHONCASEOK environment variable exists, same as + before: silently accept the first case-insensitive match of any + kind; raise ImportError if none found. + + B. Else search sys.path for the first case-sensitive match; raise + ImportError if none found. + + The same rules have been implemented on other platforms with case- + insensitive but case-preserving filesystems too (including Cygwin, and + several flavors of Macintosh operating systems). + +- winsound module: Under Win9x, winsound.Beep() now attempts to simulate + what it's supposed to do (and does do under NT and 2000) via direct + port manipulation. It's unknown whether this will work on all systems, + but it does work on my Win98SE systems now and was known to be useless on + all Win9x systems before. + +- Build: Subproject _test (effectively) renamed to _testcapi. + +New platforms + +- 2.1 should compile and run out of the box under MacOS X, even using HFS+. + Thanks to Steven Majewski! + +- 2.1 should compile and run out of the box on Cygwin. Thanks to Jason + Tishler! + +- 2.1 contains new files and patches for RISCOS, thanks to Dietmar + Schwertberger! See RISCOS/README for more information -- it seems + that because of the bizarre filename conventions on RISCOS, no port + to that platform is easy. + + +What's New in Python 2.1 alpha 2? +================================= + +Core language, builtins, and interpreter + +- Scopes nest. If a name is used in a function or class, but is not + local, the definition in the nearest enclosing function scope will + be used. One consequence of this change is that lambda statements + could reference variables in the namespaces where the lambda is + defined. In some unusual cases, this change will break code. + + In all previous version of Python, names were resolved in exactly + three namespaces -- the local namespace, the global namespace, and + the builtin namespace. According to this old definition, if a + function A is defined within a function B, the names bound in B are + not visible in A. The new rules make names bound in B visible in A, + unless A contains a name binding that hides the binding in B. + + Section 4.1 of the reference manual describes the new scoping rules + in detail. The test script in Lib/test/test_scope.py demonstrates + some of the effects of the change. + + The new rules will cause existing code to break if it defines nested + functions where an outer function has local variables with the same + name as globals or builtins used by the inner function. Example: + + def munge(str): + def helper(x): + return str(x) + if type(str) != type(''): + str = helper(str) + return str.strip() + + Under the old rules, the name str in helper() is bound to the + builtin function str(). Under the new rules, it will be bound to + the argument named str and an error will occur when helper() is + called. + +- The compiler will report a SyntaxError if "from ... import *" occurs + in a function or class scope. The language reference has documented + that this case is illegal, but the compiler never checked for it. + The recent introduction of nested scope makes the meaning of this + form of name binding ambiguous. In a future release, the compiler + may allow this form when there is no possibility of ambiguity. + +- repr(string) is easier to read, now using hex escapes instead of octal, + and using \t, \n and \r instead of \011, \012 and \015 (respectively): + + >>> "\texample \r\n" + chr(0) + chr(255) + '\texample \r\n\x00\xff' # in 2.1 + '\011example \015\012\000\377' # in 2.0 + +- Functions are now compared and hashed by identity, not by value, since + the func_code attribute is writable. + +- Weak references (PEP 205) have been added. This involves a few + changes in the core, an extension module (_weakref), and a Python + module (weakref). The weakref module is the public interface. It + includes support for "explicit" weak references, proxy objects, and + mappings with weakly held values. + +- A 'continue' statement can now appear in a try block within the body + of a loop. It is still not possible to use continue in a finally + clause. + +Standard library + +- mailbox.py now has a new class, PortableUnixMailbox which is + identical to UnixMailbox but uses a more portable scheme for + determining From_ separators. Also, the constructors for all the + classes in this module have a new optional `factory' argument, which + is a callable used when new message classes must be instantiated by + the next() method. + +- random.py is now self-contained, and offers all the functionality of + the now-deprecated whrandom.py. See the docs for details. random.py + also supports new functions getstate() and setstate(), for saving + and restoring the internal state of the generator; and jumpahead(n), + for quickly forcing the internal state to be the same as if n calls to + random() had been made. The latter is particularly useful for multi- + threaded programs, creating one instance of the random.Random() class for + each thread, then using .jumpahead() to force each instance to use a + non-overlapping segment of the full period. + +- random.py's seed() function is new. For bit-for-bit compatibility with + prior releases, use the whseed function instead. The new seed function + addresses two problems: (1) The old function couldn't produce more than + about 2**24 distinct internal states; the new one about 2**45 (the best + that can be done in the Wichmann-Hill generator). (2) The old function + sometimes produced identical internal states when passed distinct + integers, and there was no simple way to predict when that would happen; + the new one guarantees to produce distinct internal states for all + arguments in [0, 27814431486576L). + +- The socket module now supports raw packets on Linux. The socket + family is AF_PACKET. + +- test_capi.py is a start at running tests of the Python C API. The tests + are implemented by the new Modules/_testmodule.c. + +- A new extension module, _symtable, provides provisional access to the + internal symbol table used by the Python compiler. A higher-level + interface will be added on top of _symtable in a future release. + +- Removed the obsolete soundex module. + +- xml.dom.minidom now uses the standard DOM exceptions. Node supports + the isSameNode method; NamedNodeMap the get method. + +- xml.sax.expatreader supports the lexical handler property; it + generates comment, startCDATA, and endCDATA events. + +Windows changes + +- Build procedure: the zlib project is built in a different way that + ensures the zlib header files used can no longer get out of synch with + the zlib binary used. See PCbuild\readme.txt for details. Your old + zlib-related directories can be deleted; you'll need to download fresh + source for zlib and unpack it into a new directory. + +- Build: New subproject _test for the benefit of test_capi.py (see above). + +- Build: New subproject _symtable, for new DLL _symtable.pyd (a nascent + interface to some Python compiler internals). + +- Build: Subproject ucnhash is gone, since the code was folded into the + unicodedata subproject. + +What's New in Python 2.1 alpha 1? +================================= + +Core language, builtins, and interpreter + +- There is a new Unicode companion to the PyObject_Str() API + called PyObject_Unicode(). It behaves in the same way as the + former, but assures that the returned value is an Unicode object + (applying the usual coercion if necessary). + +- The comparison operators support "rich comparison overloading" (PEP + 207). C extension types can provide a rich comparison function in + the new tp_richcompare slot in the type object. The cmp() function + and the C function PyObject_Compare() first try the new rich + comparison operators before trying the old 3-way comparison. There + is also a new C API PyObject_RichCompare() (which also falls back on + the old 3-way comparison, but does not constrain the outcome of the + rich comparison to a Boolean result). + + The rich comparison function takes two objects (at least one of + which is guaranteed to have the type that provided the function) and + an integer indicating the opcode, which can be Py_LT, Py_LE, Py_EQ, + Py_NE, Py_GT, Py_GE (for <, <=, ==, !=, >, >=), and returns a Python + object, which may be NotImplemented (in which case the tp_compare + slot function is used as a fallback, if defined). + + Classes can overload individual comparison operators by defining one + or more of the methods__lt__, __le__, __eq__, __ne__, __gt__, + __ge__. There are no explicit "reflected argument" versions of + these; instead, __lt__ and __gt__ are each other's reflection, + likewise for__le__ and __ge__; __eq__ and __ne__ are their own + reflection (similar at the C level). No other implications are + made; in particular, Python does not assume that == is the Boolean + inverse of !=, or that < is the Boolean inverse of >=. This makes + it possible to define types with partial orderings. + + Classes or types that want to implement (in)equality tests but not + the ordering operators (i.e. unordered types) should implement == + and !=, and raise an error for the ordering operators. + + It is possible to define types whose rich comparison results are not + Boolean; e.g. a matrix type might want to return a matrix of bits + for A < B, giving elementwise comparisons. Such types should ensure + that any interpretation of their value in a Boolean context raises + an exception, e.g. by defining __nonzero__ (or the tp_nonzero slot + at the C level) to always raise an exception. + +- Complex numbers use rich comparisons to define == and != but raise + an exception for <, <=, > and >=. Unfortunately, this also means + that cmp() of two complex numbers raises an exception when the two + numbers differ. Since it is not mathematically meaningful to compare + complex numbers except for equality, I hope that this doesn't break + too much code. + +- The outcome of comparing non-numeric objects of different types is + not defined by the language, other than that it's arbitrary but + consistent (see the Reference Manual). An implementation detail changed + in 2.1a1 such that None now compares less than any other object. Code + relying on this new behavior (like code that relied on the previous + behavior) does so at its own risk. + +- Functions and methods now support getting and setting arbitrarily + named attributes (PEP 232). Functions have a new __dict__ + (a.k.a. func_dict) which hold the function attributes. Methods get + and set attributes on their underlying im_func. It is a TypeError + to set an attribute on a bound method. + +- The xrange() object implementation has been improved so that + xrange(sys.maxint) can be used on 64-bit platforms. There's still a + limitation that in this case len(xrange(sys.maxint)) can't be + calculated, but the common idiom "for i in xrange(sys.maxint)" will + work fine as long as the index i doesn't actually reach 2**31. + (Python uses regular ints for sequence and string indices; fixing + that is much more work.) + +- Two changes to from...import: + + 1) "from M import X" now works even if (after loading module M) + sys.modules['M'] is not a real module; it's basically a getattr() + operation with AttributeError exceptions changed into ImportError. + + 2) "from M import *" now looks for M.__all__ to decide which names to + import; if M.__all__ doesn't exist, it uses M.__dict__.keys() but + filters out names starting with '_' as before. Whether or not + __all__ exists, there's no restriction on the type of M. + +- File objects have a new method, xreadlines(). This is the fastest + way to iterate over all lines in a file: + + for line in file.xreadlines(): + ...do something to line... + + See the xreadlines module (mentioned below) for how to do this for + other file-like objects. + +- Even if you don't use file.xreadlines(), you may expect a speedup on + line-by-line input. The file.readline() method has been optimized + quite a bit in platform-specific ways: on systems (like Linux) that + support flockfile(), getc_unlocked(), and funlockfile(), those are + used by default. On systems (like Windows) without getc_unlocked(), + a complicated (but still thread-safe) method using fgets() is used by + default. + + You can force use of the fgets() method by #define'ing + USE_FGETS_IN_GETLINE at build time (it may be faster than + getc_unlocked()). + + You can force fgets() not to be used by #define'ing + DONT_USE_FGETS_IN_GETLINE (this is the first thing to try if std test + test_bufio.py fails -- and let us know if it does!). + +- In addition, the fileinput module, while still slower than the other + methods on most platforms, has been sped up too, by using + file.readlines(sizehint). + +- Support for run-time warnings has been added, including a new + command line option (-W) to specify the disposition of warnings. + See the description of the warnings module below. + +- Extensive changes have been made to the coercion code. This mostly + affects extension modules (which can now implement mixed-type + numerical operators without having to use coercion), but + occasionally, in boundary cases the coercion semantics have changed + subtly. Since this was a terrible gray area of the language, this + is considered an improvement. Also note that __rcmp__ is no longer + supported -- instead of calling __rcmp__, __cmp__ is called with + reflected arguments. + +- In connection with the coercion changes, a new built-in singleton + object, NotImplemented is defined. This can be returned for + operations that wish to indicate they are not implemented for a + particular combination of arguments. From C, this is + Py_NotImplemented. + +- The interpreter accepts now bytecode files on the command line even + if they do not have a .pyc or .pyo extension. On Linux, after executing + +import imp,sys,string +magic = string.join(["\\x%.2x" % ord(c) for c in imp.get_magic()],"") +reg = ':pyc:M::%s::%s:' % (magic, sys.executable) +open("/proc/sys/fs/binfmt_misc/register","wb").write(reg) + + any byte code file can be used as an executable (i.e. as an argument + to execve(2)). + +- %[xXo] formats of negative Python longs now produce a sign + character. In 1.6 and earlier, they never produced a sign, + and raised an error if the value of the long was too large + to fit in a Python int. In 2.0, they produced a sign if and + only if too large to fit in an int. This was inconsistent + across platforms (because the size of an int varies across + platforms), and inconsistent with hex() and oct(). Example: + + >>> "%x" % -0x42L + '-42' # in 2.1 + 'ffffffbe' # in 2.0 and before, on 32-bit machines + >>> hex(-0x42L) + '-0x42L' # in all versions of Python + + The behavior of %d formats for negative Python longs remains + the same as in 2.0 (although in 1.6 and before, they raised + an error if the long didn't fit in a Python int). + + %u formats don't make sense for Python longs, but are allowed + and treated the same as %d in 2.1. In 2.0, a negative long + formatted via %u produced a sign if and only if too large to + fit in an int. In 1.6 and earlier, a negative long formatted + via %u raised an error if it was too big to fit in an int. + +- Dictionary objects have an odd new method, popitem(). This removes + an arbitrary item from the dictionary and returns it (in the form of + a (key, value) pair). This can be useful for algorithms that use a + dictionary as a bag of "to do" items and repeatedly need to pick one + item. Such algorithms normally end up running in quadratic time; + using popitem() they can usually be made to run in linear time. + +Standard library + +- In the time module, the time argument to the functions strftime, + localtime, gmtime, asctime and ctime is now optional, defaulting to + the current time (in the local timezone). + +- The ftplib module now defaults to passive mode, which is deemed a + more useful default given that clients are often inside firewalls + these days. Note that this could break if ftplib is used to connect + to a *server* that is inside a firewall, from outside; this is + expected to be a very rare situation. To fix that, you can call + ftp.set_pasv(0). + +- The module site now treats .pth files not only for path configuration, + but also supports extensions to the initialization code: Lines starting + with import are executed. + +- There's a new module, warnings, which implements a mechanism for + issuing and filtering warnings. There are some new built-in + exceptions that serve as warning categories, and a new command line + option, -W, to control warnings (e.g. -Wi ignores all warnings, -We + turns warnings into errors). warnings.warn(message[, category]) + issues a warning message; this can also be called from C as + PyErr_Warn(category, message). + +- A new module xreadlines was added. This exports a single factory + function, xreadlines(). The intention is that this code is the + absolutely fastest way to iterate over all lines in an open + file(-like) object: + + import xreadlines + for line in xreadlines.xreadlines(file): + ...do something to line... + + This is equivalent to the previous the speed record holder using + file.readlines(sizehint). Note that if file is a real file object + (as opposed to a file-like object), this is equivalent: + + for line in file.xreadlines(): + ...do something to line... + +- The bisect module has new functions bisect_left, insort_left, + bisect_right and insort_right. The old names bisect and insort + are now aliases for bisect_right and insort_right. XXX_right + and XXX_left methods differ in what happens when the new element + compares equal to one or more elements already in the list: the + XXX_left methods insert to the left, the XXX_right methods to the + right. Code that doesn't care where equal elements end up should + continue to use the old, short names ("bisect" and "insort"). + +- The new curses.panel module wraps the panel library that forms part + of SYSV curses and ncurses. Contributed by Thomas Gellekum. + +- The SocketServer module now sets the allow_reuse_address flag by + default in the TCPServer class. + +- A new function, sys._getframe(), returns the stack frame pointer of + the caller. This is intended only as a building block for + higher-level mechanisms such as string interpolation. + +- The pyexpat module supports a number of new handlers, which are + available only in expat 1.2. If invocation of a callback fails, it + will report an additional frame in the traceback. Parser objects + participate now in garbage collection. If expat reports an unknown + encoding, pyexpat will try to use a Python codec; that works only + for single-byte charsets. The parser type objects is exposed as + XMLParserObject. + +- xml.dom now offers standard definitions for symbolic node type and + exception code constants, and a hierarchy of DOM exceptions. minidom + was adjusted to use them. + +- The conformance of xml.dom.minidom to the DOM specification was + improved. It detects a number of additional error cases; the + previous/next relationship works even when the tree is modified; + Node supports the normalize() method; NamedNodeMap, DocumentType and + DOMImplementation classes were added; Element supports the + hasAttribute and hasAttributeNS methods; and Text supports the splitText + method. + +Build issues + +- For Unix (and Unix-compatible) builds, configuration and building of + extension modules is now greatly automated. Rather than having to + edit the Modules/Setup file to indicate which modules should be + built and where their include files and libraries are, a + distutils-based setup.py script now takes care of building most + extension modules. All extension modules built this way are built + as shared libraries. Only a few modules that must be linked + statically are still listed in the Setup file; you won't need to + edit their configuration. + +- Python should now build out of the box on Cygwin. If it doesn't, + mail to Jason Tishler (jlt63 at users.sourceforge.net). + +- Python now always uses its own (renamed) implementation of getopt() + -- there's too much variation among C library getopt() + implementations. + +- C++ compilers are better supported; the CXX macro is always set to a + C++ compiler if one is found. + +Windows changes + +- select module: By default under Windows, a select() call + can specify no more than 64 sockets. Python now boosts + this Microsoft default to 512. If you need even more than + that, see the MS docs (you'll need to #define FD_SETSIZE + and recompile Python from source). + +- Support for Windows 3.1, DOS and OS/2 is gone. The Lib/dos-8x3 + subdirectory is no more! + + +What's New in Python 2.0? +========================= + +Below is a list of all relevant changes since release 1.6. Older +changes are in the file HISTORY. If you are making the jump directly +from Python 1.5.2 to 2.0, make sure to read the section for 1.6 in the +HISTORY file! Many important changes listed there. + +Alternatively, a good overview of the changes between 1.5.2 and 2.0 is +the document "What's New in Python 2.0" by Kuchling and Moshe Zadka: +http://www.amk.ca/python/2.0/. + +--Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) + +====================================================================== + +What's new in 2.0 (since release candidate 1)? +============================================== + +Standard library + +- The copy_reg module was modified to clarify its intended use: to + register pickle support for extension types, not for classes. + pickle() will raise a TypeError if it is passed a class. + +- Fixed a bug in gettext's "normalize and expand" code that prevented + it from finding an existing .mo file. + +- Restored support for HTTP/0.9 servers in httplib. + +- The math module was changed to stop raising OverflowError in case of + underflow, and return 0 instead in underflow cases. Whether Python + used to raise OverflowError in case of underflow was platform- + dependent (it did when the platform math library set errno to ERANGE + on underflow). + +- Fixed a bug in StringIO that occurred when the file position was not + at the end of the file and write() was called with enough data to + extend past the end of the file. + +- Fixed a bug that caused Tkinter error messages to get lost on + Windows. The bug was fixed by replacing direct use of + interp->result with Tcl_GetStringResult(interp). + +- Fixed bug in urllib2 that caused it to fail when it received an HTTP + redirect response. + +- Several changes were made to distutils: Some debugging code was + removed from util. Fixed the installer used when an external zip + program (like WinZip) is not found; the source code for this + installer is in Misc/distutils. check_lib() was modified to behave + more like AC_CHECK_LIB by add other_libraries() as a parameter. The + test for whether installed modules are on sys.path was changed to + use both normcase() and normpath(). + +- Several minor bugs were fixed in the xml package (the minidom, + pulldom, expatreader, and saxutils modules). + +- The regression test driver (regrtest.py) behavior when invoked with + -l changed: It now reports a count of objects that are recognized as + garbage but not freed by the garbage collector. + +- The regression test for the math module was changed to test + exceptional behavior when the test is run in verbose mode. Python + cannot yet guarantee consistent exception behavior across platforms, + so the exception part of test_math is run only in verbose mode, and + may fail on your platform. + +Internals + +- PyOS_CheckStack() has been disabled on Win64, where it caused + test_sre to fail. + +Build issues + +- Changed compiler flags, so that gcc is always invoked with -Wall and + -Wstrict-prototypes. Users compiling Python with GCC should see + exactly one warning, except if they have passed configure the + --with-pydebug flag. The expected warning is for getopt() in + Modules/main.c. This warning will be fixed for Python 2.1. + +- Fixed configure to add -threads argument during linking on OSF1. + +Tools and other miscellany + +- The compiler in Tools/compiler was updated to support the new + language features introduced in 2.0: extended print statement, list + comprehensions, and augmented assignments. The new compiler should + also be backwards compatible with Python 1.5.2; the compiler will + always generate code for the version of the interpreter it runs + under. + +What's new in 2.0 release candidate 1 (since beta 2)? +===================================================== + +What is release candidate 1? + +We believe that release candidate 1 will fix all known bugs that we +intend to fix for the 2.0 final release. This release should be a bit +more stable than the previous betas. We would like to see even more +widespread testing before the final release, so we are producing this +release candidate. The final release will be exactly the same unless +any show-stopping (or brown bag) bugs are found by testers of the +release candidate. + +All the changes since the last beta release are bug fixes or changes +to support building Python for specific platforms. + +Core language, builtins, and interpreter + +- A bug that caused crashes when __coerce__ was used with augmented + assignment, e.g. +=, was fixed. + +- Raise ZeroDivisionError when raising zero to a negative number, + e.g. 0.0 ** -2.0. Note that math.pow is unrelated to the builtin + power operator and the result of math.pow(0.0, -2.0) will vary by + platform. On Linux, it raises a ValueError. + +- A bug in Unicode string interpolation was fixed that occasionally + caused errors with formats including "%%". For example, the + following expression "%% %s" % u"abc" no longer raises a TypeError. + +- Compilation of deeply nested expressions raises MemoryError instead + of SyntaxError, e.g. eval("[" * 50 + "]" * 50). + +- In 2.0b2 on Windows, the interpreter wrote .pyc files in text mode, + rendering them useless. They are now written in binary mode again. + +Standard library + +- Keyword arguments are now accepted for most pattern and match object + methods in SRE, the standard regular expression engine. + +- In SRE, fixed error with negative lookahead and lookbehind that + manifested itself as a runtime error in patterns like "(?<!abc)(def)". + +- Several bugs in the Unicode handling and error handling in _tkinter + were fixed. + +- Fix memory management errors in Merge() and Tkapp_Call() routines. + +- Several changes were made to cStringIO to make it compatible with + the file-like object interface and with StringIO. If operations are + performed on a closed object, an exception is raised. The truncate + method now accepts a position argument and readline accepts a size + argument. + +- There were many changes made to the linuxaudiodev module and its + test suite; as a result, a short, unexpected audio sample should now + play when the regression test is run. + + Note that this module is named poorly, because it should work + correctly on any platform that supports the Open Sound System + (OSS). + + The module now raises exceptions when errors occur instead of + crashing. It also defines the AFMT_A_LAW format (logarithmic A-law + audio) and defines a getptr() method that calls the + SNDCTL_DSP_GETxPTR ioctl defined in the OSS Programmer's Guide. + +- The library_version attribute, introduced in an earlier beta, was + removed because it can not be supported with early versions of the C + readline library, which provides no way to determine the version at + compile-time. + +- The binascii module is now enabled on Win64. + +- tokenize.py no longer suffers "recursion depth" errors when parsing + programs with very long string literals. + +Internals + +- Fixed several buffer overflow vulnerabilities in calculate_path(), + which is called when the interpreter starts up to determine where + the standard library is installed. These vulnerabilities affect all + previous versions of Python and can be exploited by setting very + long values for PYTHONHOME or argv[0]. The risk is greatest for a + setuid Python script, although use of the wrapper in + Misc/setuid-prog.c will eliminate the vulnerability. + +- Fixed garbage collection bugs in instance creation that were + triggered when errors occurred during initialization. The solution, + applied in cPickle and in PyInstance_New(), is to call + PyObject_GC_Init() after the initialization of the object's + container attributes is complete. + +- pyexpat adds definitions of PyModule_AddStringConstant and + PyModule_AddObject if the Python version is less than 2.0, which + provides compatibility with PyXML on Python 1.5.2. + +- If the platform has a bogus definition for LONG_BIT (the number of + bits in a long), an error will be reported at compile time. + +- Fix bugs in _PyTuple_Resize() which caused hard-to-interpret garbage + collection crashes and possibly other, unreported crashes. + +- Fixed a memory leak in _PyUnicode_Fini(). + +Build issues + +- configure now accepts a --with-suffix option that specifies the + executable suffix. This is useful for builds on Cygwin and Mac OS + X, for example. + +- The mmap.PAGESIZE constant is now initialized using sysconf when + possible, which eliminates a dependency on -lucb for Reliant UNIX. + +- The md5 file should now compile on all platforms. + +- The select module now compiles on platforms that do not define + POLLRDNORM and related constants. + +- Darwin (Mac OS X): Initial support for static builds on this + platform. + +- BeOS: A number of changes were made to the build and installation + process. ar-fake now operates on a directory of object files. + dl_export.h is gone, and its macros now appear on the mwcc command + line during build on PPC BeOS. + +- Platform directory in lib/python2.0 is "plat-beos5" (or + "plat-beos4", if building on BeOS 4.5), rather than "plat-beos". + +- Cygwin: Support for shared libraries, Tkinter, and sockets. + +- SunOS 4.1.4_JL: Fix test for directory existence in configure. + +Tools and other miscellany + +- Removed debugging prints from main used with freeze. + +- IDLE auto-indent no longer crashes when it encounters Unicode + characters. + +What's new in 2.0 beta 2 (since beta 1)? +======================================== + +Core language, builtins, and interpreter + +- Add support for unbounded ints in %d,i,u,x,X,o formats; for example + "%d" % 2L**64 == "18446744073709551616". + +- Add -h and -V command line options to print the usage message and + Python version number and exit immediately. + +- eval() and exec accept Unicode objects as code parameters. + +- getattr() and setattr() now also accept Unicode objects for the + attribute name, which are converted to strings using the default + encoding before lookup. + +- Multiplication on string and Unicode now does proper bounds + checking; e.g. 'a' * 65536 * 65536 will raise ValueError, "repeated + string is too long." + +- Better error message when continue is found in try statement in a + loop. + + +Standard library and extensions + +- socket module: the OpenSSL code now adds support for RAND_status() + and EGD (Entropy Gathering Device). + +- array: reverse() method of array now works. buffer_info() now does + argument checking; it still takes no arguments. + +- asyncore/asynchat: Included most recent version from Sam Rushing. + +- cgi: Accept '&' or ';' as separator characters when parsing form data. + +- CGIHTTPServer: Now works on Windows (and perhaps even Mac). + +- ConfigParser: When reading the file, options spelled in upper case + letters are now correctly converted to lowercase. + +- copy: Copy Unicode objects atomically. + +- cPickle: Fail gracefully when copy_reg can't be imported. + +- cStringIO: Implemented readlines() method. + +- dbm: Add get() and setdefault() methods to dbm object. Add constant + `library' to module that names the library used. Added doc strings + and method names to error messages. Uses configure to determine + which ndbm.h file to include; Berkeley DB's nbdm and GDBM's ndbm is + now available options. + +- distutils: Update to version 0.9.3. + +- dl: Add several dl.RTLD_ constants. + +- fpectl: Now supported on FreeBSD. + +- gc: Add DEBUG_SAVEALL option. When enabled all garbage objects + found by the collector will be saved in gc.garbage. This is useful + for debugging a program that creates reference cycles. + +- httplib: Three changes: Restore support for set_debuglevel feature + of HTTP class. Do not close socket on zero-length response. Do not + crash when server sends invalid content-length header. + +- mailbox: Mailbox class conforms better to qmail specifications. + +- marshal: When reading a short, sign-extend on platforms where shorts + are bigger than 16 bits. When reading a long, repair the unportable + sign extension that was being done for 64-bit machines. (It assumed + that signed right shift sign-extends.) + +- operator: Add contains(), invert(), __invert__() as aliases for + __contains__(), inv(), and __inv__() respectively. + +- os: Add support for popen2() and popen3() on all platforms where + fork() exists. (popen4() is still in the works.) + +- os: (Windows only:) Add startfile() function that acts like double- + clicking on a file in Explorer (or passing the file name to the + DOS "start" command). + +- os.path: (Windows, DOS:) Treat trailing colon correctly in + os.path.join. os.path.join("a:", "b") yields "a:b". + +- pickle: Now raises ValueError when an invalid pickle that contains + a non-string repr where a string repr was expected. This behavior + matches cPickle. + +- posixfile: Remove broken __del__() method. + +- py_compile: support CR+LF line terminators in source file. + +- readline: Does not immediately exit when ^C is hit when readline and + threads are configured. Adds definition of rl_library_version. (The + latter addition requires GNU readline 2.2 or later.) + +- rfc822: Domain literals returned by AddrlistClass method + getdomainliteral() are now properly wrapped in brackets. + +- site: sys.setdefaultencoding() should only be called in case the + standard default encoding ("ascii") is changed. This saves quite a + few cycles during startup since the first call to + setdefaultencoding() will initialize the codec registry and the + encodings package. + +- socket: Support for size hint in readlines() method of object returned + by makefile(). + +- sre: Added experimental expand() method to match objects. Does not + use buffer interface on Unicode strings. Does not hang if group id + is followed by whitespace. + +- StringIO: Size hint in readlines() is now supported as documented. + +- struct: Check ranges for bytes and shorts. + +- urllib: Improved handling of win32 proxy settings. Fixed quote and + quote_plus functions so that the always encode a comma. + +- Tkinter: Image objects are now guaranteed to have unique ids. Set + event.delta to zero if Tk version doesn't support mousewheel. + Removed some debugging prints. + +- UserList: now implements __contains__(). + +- webbrowser: On Windows, use os.startfile() instead of os.popen(), + which works around a bug in Norton AntiVirus 2000 that leads directly + to a Blue Screen freeze. + +- xml: New version detection code allows PyXML to override standard + XML package if PyXML version is greater than 0.6.1. + +- xml.dom: DOM level 1 support for basic XML. Includes xml.dom.minidom + (conventional DOM), and xml.dom.pulldom, which allows building the DOM + tree only for nodes which are sufficiently interesting to a specific + application. Does not provide the HTML-specific extensions. Still + undocumented. + +- xml.sax: SAX 2 support for Python, including all the handler + interfaces needed to process XML 1.0 compliant XML. Some + documentation is already available. + +- pyexpat: Renamed to xml.parsers.expat since this is part of the new, + packagized XML support. + + +C API + +- Add three new convenience functions for module initialization -- + PyModule_AddObject(), PyModule_AddIntConstant(), and + PyModule_AddStringConstant(). + +- Cleaned up definition of NULL in C source code; all definitions were + removed and add #error to Python.h if NULL isn't defined after + #include of stdio.h. + +- Py_PROTO() macros that were removed in 2.0b1 have been restored for + backwards compatibility (at the source level) with old extensions. + +- A wrapper API was added for signal() and sigaction(). Instead of + either function, always use PyOS_getsig() to get a signal handler + and PyOS_setsig() to set one. A new convenience typedef + PyOS_sighandler_t is defined for the type of signal handlers. + +- Add PyString_AsStringAndSize() function that provides access to the + internal data buffer and size of a string object -- or the default + encoded version of a Unicode object. + +- PyString_Size() and PyString_AsString() accept Unicode objects. + +- The standard header <limits.h> is now included by Python.h (if it + exists). INT_MAX and LONG_MAX will always be defined, even if + <limits.h> is not available. + +- PyFloat_FromString takes a second argument, pend, that was + effectively useless. It is now officially useless but preserved for + backwards compatibility. If the pend argument is not NULL, *pend is + set to NULL. + +- PyObject_GetAttr() and PyObject_SetAttr() now accept Unicode objects + for the attribute name. See note on getattr() above. + +- A few bug fixes to argument processing for Unicode. + PyArg_ParseTupleAndKeywords() now accepts "es#" and "es". + PyArg_Parse() special cases "s#" for Unicode objects; it returns a + pointer to the default encoded string data instead of to the raw + UTF-16. + +- Py_BuildValue accepts B format (for bgen-generated code). + + +Internals + +- On Unix, fix code for finding Python installation directory so that + it works when argv[0] is a relative path. + +- Added a true unicode_internal_encode() function and fixed the + unicode_internal_decode function() to support Unicode objects directly + rather than by generating a copy of the object. + +- Several of the internal Unicode tables are much smaller now, and + the source code should be much friendlier to weaker compilers. + +- In the garbage collector: Fixed bug in collection of tuples. Fixed + bug that caused some instances to be removed from the container set + while they were still live. Fixed parsing in gc.set_debug() for + platforms where sizeof(long) > sizeof(int). + +- Fixed refcount problem in instance deallocation that only occurred + when Py_REF_DEBUG was defined and Py_TRACE_REFS was not. + +- On Windows, getpythonregpath is now protected against null data in + registry key. + +- On Unix, create .pyc/.pyo files with O_EXCL flag to avoid a race + condition. + + +Build and platform-specific issues + +- Better support of GNU Pth via --with-pth configure option. + +- Python/C API now properly exposed to dynamically-loaded extension + modules on Reliant UNIX. + +- Changes for the benefit of SunOS 4.1.4 (really!). mmapmodule.c: + Don't define MS_SYNC to be zero when it is undefined. Added missing + prototypes in posixmodule.c. + +- Improved support for HP-UX build. Threads should now be correctly + configured (on HP-UX 10.20 and 11.00). + +- Fix largefile support on older NetBSD systems and OpenBSD by adding + define for TELL64. + + +Tools and other miscellany + +- ftpmirror: Call to main() is wrapped in if __name__ == "__main__". + +- freeze: The modulefinder now works with 2.0 opcodes. + +- IDLE: + Move hackery of sys.argv until after the Tk instance has been + created, which allows the application-specific Tkinter + initialization to be executed if present; also pass an explicit + className parameter to the Tk() constructor. + + +What's new in 2.0 beta 1? +========================= + +Source Incompatibilities +------------------------ + +None. Note that 1.6 introduced several incompatibilities with 1.5.2, +such as single-argument append(), connect() and bind(), and changes to +str(long) and repr(float). + + +Binary Incompatibilities +------------------------ + +- Third party extensions built for Python 1.5.x or 1.6 cannot be used +with Python 2.0; these extensions will have to be rebuilt for Python +2.0. + +- On Windows, attempting to import a third party extension built for +Python 1.5.x or 1.6 results in an immediate crash; there's not much we +can do about this. Check your PYTHONPATH environment variable! + +- Python bytecode files (*.pyc and *.pyo) are not compatible between +releases. + + +Overview of Changes Since 1.6 +----------------------------- + +There are many new modules (including brand new XML support through +the xml package, and i18n support through the gettext module); a list +of all new modules is included below. Lots of bugs have been fixed. + +The process for making major new changes to the language has changed +since Python 1.6. Enhancements must now be documented by a Python +Enhancement Proposal (PEP) before they can be accepted. + +There are several important syntax enhancements, described in more +detail below: + + - Augmented assignment, e.g. x += 1 + + - List comprehensions, e.g. [x**2 for x in range(10)] + + - Extended import statement, e.g. import Module as Name + + - Extended print statement, e.g. print >> file, "Hello" + +Other important changes: + + - Optional collection of cyclical garbage + +Python Enhancement Proposal (PEP) +--------------------------------- + +PEP stands for Python Enhancement Proposal. A PEP is a design +document providing information to the Python community, or describing +a new feature for Python. The PEP should provide a concise technical +specification of the feature and a rationale for the feature. + +We intend PEPs to be the primary mechanisms for proposing new +features, for collecting community input on an issue, and for +documenting the design decisions that have gone into Python. The PEP +author is responsible for building consensus within the community and +documenting dissenting opinions. + +The PEPs are available at http://python.sourceforge.net/peps/. + +Augmented Assignment +-------------------- + +This must have been the most-requested feature of the past years! +Eleven new assignment operators were added: + + += -= *= /= %= **= <<= >>= &= ^= |= + +For example, + + A += B + +is similar to + + A = A + B + +except that A is evaluated only once (relevant when A is something +like dict[index].attr). + +However, if A is a mutable object, A may be modified in place. Thus, +if A is a number or a string, A += B has the same effect as A = A+B +(except A is only evaluated once); but if a is a list, A += B has the +same effect as A.extend(B)! + +Classes and built-in object types can override the new operators in +order to implement the in-place behavior; the not-in-place behavior is +used automatically as a fallback when an object doesn't implement the +in-place behavior. For classes, the method name is derived from the +method name for the corresponding not-in-place operator by inserting +an 'i' in front of the name, e.g. __iadd__ implements in-place +__add__. + +Augmented assignment was implemented by Thomas Wouters. + + +List Comprehensions +------------------- + +This is a flexible new notation for lists whose elements are computed +from another list (or lists). The simplest form is: + + [<expression> for <variable> in <sequence>] + +For example, [i**2 for i in range(4)] yields the list [0, 1, 4, 9]. +This is more efficient than a for loop with a list.append() call. + +You can also add a condition: + + [<expression> for <variable> in <sequence> if <condition>] + +For example, [w for w in words if w == w.lower()] would yield the list +of words that contain no uppercase characters. This is more efficient +than a for loop with an if statement and a list.append() call. + +You can also have nested for loops and more than one 'if' clause. For +example, here's a function that flattens a sequence of sequences:: + + def flatten(seq): + return [x for subseq in seq for x in subseq] + + flatten([[0], [1,2,3], [4,5], [6,7,8,9], []]) + +This prints + + [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] + +List comprehensions originated as a patch set from Greg Ewing; Skip +Montanaro and Thomas Wouters also contributed. Described by PEP 202. + + +Extended Import Statement +------------------------- + +Many people have asked for a way to import a module under a different +name. This can be accomplished like this: + + import foo + bar = foo + del foo + +but this common idiom gets old quickly. A simple extension of the +import statement now allows this to be written as follows: + + import foo as bar + +There's also a variant for 'from ... import': + + from foo import bar as spam + +This also works with packages; e.g. you can write this: + + import test.regrtest as regrtest + +Note that 'as' is not a new keyword -- it is recognized only in this +context (this is only possible because the syntax for the import +statement doesn't involve expressions). + +Implemented by Thomas Wouters. Described by PEP 221. + + +Extended Print Statement +------------------------ + +Easily the most controversial new feature, this extension to the print +statement adds an option to make the output go to a different file +than the default sys.stdout. + +For example, to write an error message to sys.stderr, you can now +write: + + print >> sys.stderr, "Error: bad dog!" + +As a special feature, if the expression used to indicate the file +evaluates to None, the current value of sys.stdout is used. Thus: + + print >> None, "Hello world" + +is equivalent to + + print "Hello world" + +Design and implementation by Barry Warsaw. Described by PEP 214. + + +Optional Collection of Cyclical Garbage +--------------------------------------- + +Python is now equipped with a garbage collector that can hunt down +cyclical references between Python objects. It's no replacement for +reference counting; in fact, it depends on the reference counts being +correct, and decides that a set of objects belong to a cycle if all +their reference counts can be accounted for from their references to +each other. This devious scheme was first proposed by Eric Tiedemann, +and brought to implementation by Neil Schemenauer. + +There's a module "gc" that lets you control some parameters of the +garbage collection. There's also an option to the configure script +that lets you enable or disable the garbage collection. In 2.0b1, +it's on by default, so that we (hopefully) can collect decent user +experience with this new feature. There are some questions about its +performance. If it proves to be too much of a problem, we'll turn it +off by default in the final 2.0 release. + + +Smaller Changes +--------------- + +A new function zip() was added. zip(seq1, seq2, ...) is equivalent to +map(None, seq1, seq2, ...) when the sequences have the same length; +i.e. zip([1,2,3], [10,20,30]) returns [(1,10), (2,20), (3,30)]. When +the lists are not all the same length, the shortest list wins: +zip([1,2,3], [10,20]) returns [(1,10), (2,20)]. See PEP 201. + +sys.version_info is a tuple (major, minor, micro, level, serial). + +Dictionaries have an odd new method, setdefault(key, default). +dict.setdefault(key, default) returns dict[key] if it exists; if not, +it sets dict[key] to default and returns that value. Thus: + + dict.setdefault(key, []).append(item) + +does the same work as this common idiom: + + if not dict.has_key(key): + dict[key] = [] + dict[key].append(item) + +There are two new variants of SyntaxError that are raised for +indentation-related errors: IndentationError and TabError. + +Changed \x to consume exactly two hex digits; see PEP 223. Added \U +escape that consumes exactly eight hex digits. + +The limits on the size of expressions and file in Python source code +have been raised from 2**16 to 2**32. Previous versions of Python +were limited because the maximum argument size the Python VM accepted +was 2**16. This limited the size of object constructor expressions, +e.g. [1,2,3] or {'a':1, 'b':2}, and the size of source files. This +limit was raised thanks to a patch by Charles Waldman that effectively +fixes the problem. It is now much more likely that you will be +limited by available memory than by an arbitrary limit in Python. + +The interpreter's maximum recursion depth can be modified by Python +programs using sys.getrecursionlimit and sys.setrecursionlimit. This +limit is the maximum number of recursive calls that can be made by +Python code. The limit exists to prevent infinite recursion from +overflowing the C stack and causing a core dump. The default value is +1000. The maximum safe value for a particular platform can be found +by running Misc/find_recursionlimit.py. + +New Modules and Packages +------------------------ + +atexit - for registering functions to be called when Python exits. + +imputil - Greg Stein's alternative API for writing custom import +hooks. + +pyexpat - an interface to the Expat XML parser, contributed by Paul +Prescod. + +xml - a new package with XML support code organized (so far) in three +subpackages: xml.dom, xml.sax, and xml.parsers. Describing these +would fill a volume. There's a special feature whereby a +user-installed package named _xmlplus overrides the standard +xmlpackage; this is intended to give the XML SIG a hook to distribute +backwards-compatible updates to the standard xml package. + +webbrowser - a platform-independent API to launch a web browser. + + +Changed Modules +--------------- + +array -- new methods for array objects: count, extend, index, pop, and +remove + +binascii -- new functions b2a_hex and a2b_hex that convert between +binary data and its hex representation + +calendar -- Many new functions that support features including control +over which day of the week is the first day, returning strings instead +of printing them. Also new symbolic constants for days of week, +e.g. MONDAY, ..., SUNDAY. + +cgi -- FieldStorage objects have a getvalue method that works like a +dictionary's get method and returns the value attribute of the object. + +ConfigParser -- The parser object has new methods has_option, +remove_section, remove_option, set, and write. They allow the module +to be used for writing config files as well as reading them. + +ftplib -- ntransfercmd(), transfercmd(), and retrbinary() all now +optionally support the RFC 959 REST command. + +gzip -- readline and readlines now accept optional size arguments + +httplib -- New interfaces and support for HTTP/1.1 by Greg Stein. See +the module doc strings for details. + +locale -- implement getdefaultlocale for Win32 and Macintosh + +marshal -- no longer dumps core when marshaling deeply nested or +recursive data structures + +os -- new functions isatty, seteuid, setegid, setreuid, setregid + +os/popen2 -- popen2/popen3/popen4 support under Windows. popen2/popen3 +support under Unix. + +os/pty -- support for openpty and forkpty + +os.path -- fix semantics of os.path.commonprefix + +smtplib -- support for sending very long messages + +socket -- new function getfqdn() + +readline -- new functions to read, write and truncate history files. +The readline section of the library reference manual contains an +example. + +select -- add interface to poll system call + +shutil -- new copyfileobj function + +SimpleHTTPServer, CGIHTTPServer -- Fix problems with buffering in the +HTTP server. + +Tkinter -- optimization of function flatten + +urllib -- scans environment variables for proxy configuration, +e.g. http_proxy. + +whichdb -- recognizes dumbdbm format + + +Obsolete Modules +---------------- + +None. However note that 1.6 made a whole slew of modules obsolete: +stdwin, soundex, cml, cmpcache, dircache, dump, find, grep, packmail, +poly, zmod, strop, util, whatsound. + + +Changed, New, Obsolete Tools +---------------------------- + +None. + + +C-level Changes +--------------- + +Several cleanup jobs were carried out throughout the source code. + +All C code was converted to ANSI C; we got rid of all uses of the +Py_PROTO() macro, which makes the header files a lot more readable. + +Most of the portability hacks were moved to a new header file, +pyport.h; several other new header files were added and some old +header files were removed, in an attempt to create a more rational set +of header files. (Few of these ever need to be included explicitly; +they are all included by Python.h.) + +Trent Mick ensured portability to 64-bit platforms, under both Linux +and Win64, especially for the new Intel Itanium processor. Mick also +added large file support for Linux64 and Win64. + +The C APIs to return an object's size have been update to consistently +use the form PyXXX_Size, e.g. PySequence_Size and PyDict_Size. In +previous versions, the abstract interfaces used PyXXX_Length and the +concrete interfaces used PyXXX_Size. The old names, +e.g. PyObject_Length, are still available for backwards compatibility +at the API level, but are deprecated. + +The PyOS_CheckStack function has been implemented on Windows by +Fredrik Lundh. It prevents Python from failing with a stack overflow +on Windows. + +The GC changes resulted in creation of two new slots on object, +tp_traverse and tp_clear. The augmented assignment changes result in +the creation of a new slot for each in-place operator. + +The GC API creates new requirements for container types implemented in +C extension modules. See Include/objimpl.h for details. + +PyErr_Format has been updated to automatically calculate the size of +the buffer needed to hold the formatted result string. This change +prevents crashes caused by programmer error. + +New C API calls: PyObject_AsFileDescriptor, PyErr_WriteUnraisable. + +PyRun_AnyFileEx, PyRun_SimpleFileEx, PyRun_FileEx -- New functions +that are the same as their non-Ex counterparts except they take an +extra flag argument that tells them to close the file when done. + +XXX There were other API changes that should be fleshed out here. + + +Windows Changes +--------------- + +New popen2/popen3/peopen4 in os module (see Changed Modules above). + +os.popen is much more usable on Windows 95 and 98. See Microsoft +Knowledge Base article Q150956. The Win9x workaround described there +is implemented by the new w9xpopen.exe helper in the root of your +Python installation. Note that Python uses this internally; it is not +a standalone program. + +Administrator privileges are no longer required to install Python +on Windows NT or Windows 2000. If you have administrator privileges, +Python's registry info will be written under HKEY_LOCAL_MACHINE. +Otherwise the installer backs off to writing Python's registry info +under HKEY_CURRENT_USER. The latter is sufficient for all "normal" +uses of Python, but will prevent some advanced uses from working +(for example, running a Python script as an NT service, or possibly +from CGI). + +[This was new in 1.6] The installer no longer runs a separate Tcl/Tk +installer; instead, it installs the needed Tcl/Tk files directly in the +Python directory. If you already have a Tcl/Tk installation, this +wastes some disk space (about 4 Megs) but avoids problems with +conflicting Tcl/Tk installations, and makes it much easier for Python +to ensure that Tcl/Tk can find all its files. + +[This was new in 1.6] The Windows installer now installs by default in +\Python20\ on the default volume, instead of \Program Files\Python-2.0\. + + +Updates to the changes between 1.5.2 and 1.6 +-------------------------------------------- + +The 1.6 NEWS file can't be changed after the release is done, so here +is some late-breaking news: + +New APIs in locale.py: normalize(), getdefaultlocale(), resetlocale(), +and changes to getlocale() and setlocale(). + +The new module is now enabled per default. + +It is not true that the encodings codecs cannot be used for normal +strings: the string.encode() (which is also present on 8-bit strings +!) allows using them for 8-bit strings too, e.g. to convert files from +cp1252 (Windows) to latin-1 or vice-versa. + +Japanese codecs are available from Tamito KAJIYAMA: +http://pseudo.grad.sccs.chukyo-u.ac.jp/~kajiyama/python/ + + +====================================================================== + + +======================================= +==> Release 1.6 (September 5, 2000) <== +======================================= + +What's new in release 1.6? +========================== + +Below is a list of all relevant changes since release 1.5.2. + + +Source Incompatibilities +------------------------ + +Several small incompatible library changes may trip you up: + + - The append() method for lists can no longer be invoked with more + than one argument. This used to append a single tuple made out of + all arguments, but was undocumented. To append a tuple, use + e.g. l.append((a, b, c)). + + - The connect(), connect_ex() and bind() methods for sockets require + exactly one argument. Previously, you could call s.connect(host, + port), but this was undocumented. You must now write + s.connect((host, port)). + + - The str() and repr() functions are now different more often. For + long integers, str() no longer appends a 'L'. Thus, str(1L) == '1', + which used to be '1L'; repr(1L) is unchanged and still returns '1L'. + For floats, repr() now gives 17 digits of precision, to ensure no + precision is lost (on all current hardware). + + - The -X option is gone. Built-in exceptions are now always + classes. Many more library modules also have been converted to + class-based exceptions. + + +Binary Incompatibilities +------------------------ + +- Third party extensions built for Python 1.5.x cannot be used with +Python 1.6; these extensions will have to be rebuilt for Python 1.6. + +- On Windows, attempting to import a third party extension built for +Python 1.5.x results in an immediate crash; there's not much we can do +about this. Check your PYTHONPATH environment variable! + + +Overview of Changes since 1.5.2 +------------------------------- + +For this overview, I have borrowed from the document "What's New in +Python 2.0" by Andrew Kuchling and Moshe Zadka: +http://www.amk.ca/python/2.0/ . + +There are lots of new modules and lots of bugs have been fixed. A +list of all new modules is included below. + +Probably the most pervasive change is the addition of Unicode support. +We've added a new fundamental datatype, the Unicode string, a new +build-in function unicode(), an numerous C APIs to deal with Unicode +and encodings. See the file Misc/unicode.txt for details, or +http://starship.python.net/crew/lemburg/unicode-proposal.txt. + +Two other big changes, related to the Unicode support, are the +addition of string methods and (yet another) new regular expression +engine. + + - String methods mean that you can now say s.lower() etc. instead of + importing the string module and saying string.lower(s) etc. One + peculiarity is that the equivalent of string.join(sequence, + delimiter) is delimiter.join(sequence). Use " ".join(sequence) for + the effect of string.join(sequence); to make this more readable, try + space=" " first. Note that the maxsplit argument defaults in + split() and replace() have changed from 0 to -1. + + - The new regular expression engine, SRE by Fredrik Lundh, is fully + backwards compatible with the old engine, and is in fact invoked + using the same interface (the "re" module). You can explicitly + invoke the old engine by import pre, or the SRE engine by importing + sre. SRE is faster than pre, and supports Unicode (which was the + main reason to put effort in yet another new regular expression + engine -- this is at least the fourth!). + + +Other Changes +------------- + +Other changes that won't break code but are nice to know about: + +Deleting objects is now safe even for deeply nested data structures. + +Long/int unifications: long integers can be used in seek() calls, as +slice indexes. + +String formatting (s % args) has a new formatting option, '%r', which +acts like '%s' but inserts repr(arg) instead of str(arg). (Not yet in +alpha 1.) + +Greg Ward's "distutils" package is included: this will make +installing, building and distributing third party packages much +simpler. + +There's now special syntax that you can use instead of the apply() +function. f(*args, **kwds) is equivalent to apply(f, args, kwds). +You can also use variations f(a1, a2, *args, **kwds) and you can leave +one or the other out: f(*args), f(**kwds). + +The built-ins int() and long() take an optional second argument to +indicate the conversion base -- of course only if the first argument +is a string. This makes string.atoi() and string.atol() obsolete. +(string.atof() was already obsolete). + +When a local variable is known to the compiler but undefined when +used, a new exception UnboundLocalError is raised. This is a class +derived from NameError so code catching NameError should still work. +The purpose is to provide better diagnostics in the following example: + x = 1 + def f(): + print x + x = x+1 +This used to raise a NameError on the print statement, which confused +even experienced Python programmers (especially if there are several +hundreds of lines of code between the reference and the assignment to +x :-). + +You can now override the 'in' operator by defining a __contains__ +method. Note that it has its arguments backwards: x in a causes +a.__contains__(x) to be called. That's why the name isn't __in__. + +The exception AttributeError will have a more friendly error message, +e.g.: <code>'Spam' instance has no attribute 'eggs'</code>. This may +<b>break code</b> that expects the message to be exactly the attribute +name. + + +New Modules in 1.6 +------------------ + +UserString - base class for deriving from the string type. + +distutils - tools for distributing Python modules. + +robotparser - parse a robots.txt file, for writing web spiders. +(Moved from Tools/webchecker/.) + +linuxaudiodev - audio for Linux. + +mmap - treat a file as a memory buffer. (Windows and Unix.) + +sre - regular expressions (fast, supports unicode). Currently, this +code is very rough. Eventually, the re module will be reimplemented +using sre (without changes to the re API). + +filecmp - supersedes the old cmp.py and dircmp.py modules. + +tabnanny - check Python sources for tab-width dependance. (Moved from +Tools/scripts/.) + +urllib2 - new and improved but incompatible version of urllib (still +experimental). + +zipfile - read and write zip archives. + +codecs - support for Unicode encoders/decoders. + +unicodedata - provides access to the Unicode 3.0 database. + +_winreg - Windows registry access. + +encodings - package which provides a large set of standard codecs -- +currently only for the new Unicode support. It has a drop-in extension +mechanism which allows you to add new codecs by simply copying them +into the encodings package directory. Asian codec support will +probably be made available as separate distribution package built upon +this technique and the new distutils package. + + +Changed Modules +--------------- + +readline, ConfigParser, cgi, calendar, posix, readline, xmllib, aifc, +chunk, wave, random, shelve, nntplib - minor enhancements. + +socket, httplib, urllib - optional OpenSSL support (Unix only). + +_tkinter - support for 8.0 up to 8.3. Support for versions older than +8.0 has been dropped. + +string - most of this module is deprecated now that strings have +methods. This no longer uses the built-in strop module, but takes +advantage of the new string methods to provide transparent support for +both Unicode and ordinary strings. + + +Changes on Windows +------------------ + +The installer no longer runs a separate Tcl/Tk installer; instead, it +installs the needed Tcl/Tk files directly in the Python directory. If +you already have a Tcl/Tk installation, this wastes some disk space +(about 4 Megs) but avoids problems with conflincting Tcl/Tk +installations, and makes it much easier for Python to ensure that +Tcl/Tk can find all its files. Note: the alpha installers don't +include the documentation. + +The Windows installer now installs by default in \Python16\ on the +default volume, instead of \Program Files\Python-1.6\. + + +Changed Tools +------------- + +IDLE - complete overhaul. See the <a href="../idle/">IDLE home +page</a> for more information. (Python 1.6 alpha 1 will come with +IDLE 0.6.) + +Tools/i18n/pygettext.py - Python equivalent of xgettext(1). A message +text extraction tool used for internationalizing applications written +in Python. + + +Obsolete Modules +---------------- + +stdwin and everything that uses it. (Get Python 1.5.2 if you need +it. :-) + +soundex. (Skip Montanaro has a version in Python but it won't be +included in the Python release.) + +cmp, cmpcache, dircmp. (Replaced by filecmp.) + +dump. (Use pickle.) + +find. (Easily coded using os.walk().) + +grep. (Not very useful as a library module.) + +packmail. (No longer has any use.) + +poly, zmod. (These were poor examples at best.) + +strop. (No longer needed by the string module.) + +util. (This functionality was long ago built in elsewhere). + +whatsound. (Use sndhdr.) + + +Detailed Changes from 1.6b1 to 1.6 +---------------------------------- + +- Slight changes to the CNRI license. A copyright notice has been +added; the requirement to indicate the nature of modifications now +applies when making a derivative work available "to others" instead of +just "to the public"; the version and date are updated. The new +license has a new handle. + +- Added the Tools/compiler package. This is a project led by Jeremy +Hylton to write the Python bytecode generator in Python. + +- The function math.rint() is removed. + +- In Python.h, "#define _GNU_SOURCE 1" was added. + +- Version 0.9.1 of Greg Ward's distutils is included (instead of +version 0.9). + +- A new version of SRE is included. It is more stable, and more +compatible with the old RE module. Non-matching ranges are indicated +by -1, not None. (The documentation said None, but the PRE +implementation used -1; changing to None would break existing code.) + +- The winreg module has been renamed to _winreg. (There are plans for +a higher-level API called winreg, but this has not yet materialized in +a form that is acceptable to the experts.) + +- The _locale module is enabled by default. + +- Fixed the configuration line for the _curses module. + +- A few crashes have been fixed, notably <file>.writelines() with a +list containing non-string objects would crash, and there were +situations where a lost SyntaxError could dump core. + +- The <list>.extend() method now accepts an arbitrary sequence +argument. + +- If __str__() or __repr__() returns a Unicode object, this is +converted to an 8-bit string. + +- Unicode string comparisons is no longer aware of UTF-16 +encoding peculiarities; it's a straight 16-bit compare. + +- The Windows installer now installs the LICENSE file and no longer +registers the Python DLL version in the registry (this is no longer +needed). It now uses Tcl/Tk 8.3.2. + +- A few portability problems have been fixed, in particular a +compilation error involving socklen_t. + +- The PC configuration is slightly friendlier to non-Microsoft +compilers. + + +====================================================================== + + +====================================== +==> Release 1.5.2 (April 13, 1999) <== +====================================== + +From 1.5.2c1 to 1.5.2 (final) +============================= + +Tue Apr 13 15:44:49 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * PCbuild/python15.wse: Bump version to 1.5.2 (final) + + * PCbuild/python15.dsp: Added shamodule.c + + * PC/config.c: Added sha module! + + * README, Include/patchlevel.h: Prepare for final release. + + * Misc/ACKS: + More (Cameron Laird is honorary; the others are 1.5.2c1 testers). + + * Python/thread_solaris.h: + While I can't really test this thoroughly, Pat Knight and the Solaris + man pages suggest that the proper thing to do is to add THR_NEW_LWP to + the flags on thr_create(), and that there really isn't a downside, so + I'll do that. + + * Misc/ACKS: + Bunch of new names who helped iron out the last wrinkles of 1.5.2. + + * PC/python_nt.rc: + Bump the myusterious M$ version number from 1,5,2,1 to 1,5,2,3. + (I can't even display this on NT, maybe Win/98 can?) + + * Lib/pstats.py: + Fix mysterious references to jprofile that were in the source since + its creation. I'm assuming these were once valid references to "Jim + Roskind's profile"... + + * Lib/Attic/threading_api.py: + Removed; since long subsumed in Doc/lib/libthreading.tex + + * Modules/socketmodule.c: + Put back __osf__ support for gethostbyname_r(); the real bug was that + it was being used even without threads. This of course might be an + all-platform problem so now we only use the _r variant when we are + using threads. + +Mon Apr 12 22:51:20 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Modules/cPickle.c: + Fix accidentally reversed NULL test in load_mark(). Suggested by + Tamito Kajiyama. (This caused a bug only on platforms where malloc(0) + returns NULL.) + + * README: + Add note about popen2 problem on Linux noticed by Pablo Bleyer. + + * README: Add note about -D_REENTRANT for HP-UX 10.20. + + * Modules/Makefile.pre.in: 'clean' target should remove hassignal. + + * PC/Attic/vc40.mak, PC/readme.txt: + Remove all VC++ info (except VC 1.5) from readme.txt; + remove the VC++ 4.0 project file; remove the unused _tkinter extern defs. + + * README: Clarify PC build instructions (point to PCbuild). + + * Modules/zlibmodule.c: Cast added by Jack Jansen (for Mac port). + + * Lib/plat-sunos5/CDIO.py, Lib/plat-linux2/CDROM.py: + Forgot to add this file. CDROM device parameters. + + * Lib/gzip.py: Two different changes. + + 1. Jack Jansen reports that on the Mac, the time may be negative, and + solves this by adding a write32u() function that writes an unsigned + long. + + 2. On 64-bit platforms the CRC comparison fails; I've fixed this by + casting both values to be compared to "unsigned long" i.e. modulo + 0x100000000L. + +Sat Apr 10 18:42:02 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * PC/Attic/_tkinter.def: No longer needed. + + * Misc/ACKS: Correct missed character in Andrew Dalke's name. + + * README: Add DEC Ultrix notes (from Donn Cave's email). + + * configure: The usual + + * configure.in: + Quote a bunch of shell variables used in test, related to long-long. + + * Objects/fileobject.c, Modules/shamodule.c, Modules/regexpr.c: + casts for picky compilers. + + * Modules/socketmodule.c: + 3-arg gethostbyname_r doesn't really work on OSF/1. + + * PC/vc15_w31/_.c, PC/vc15_lib/_.c, Tools/pynche/__init__.py: + Avoid totally empty files. + +Fri Apr 9 14:56:35 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Tools/scripts/fixps.py: Use re instead of regex. + Don't rewrite the file in place. + (Reported by Andy Dustman.) + + * Lib/netrc.py, Lib/shlex.py: Get rid of #! line + +Thu Apr 8 23:13:37 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * PCbuild/python15.wse: Use the Tcl 8.0.5 installer. + Add a variable %_TCL_% that makes it easier to switch to a different version. + + +====================================================================== + + +From 1.5.2b2 to 1.5.2c1 +======================= + +Thu Apr 8 23:13:37 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * PCbuild/python15.wse: + Release 1.5.2c1. Add IDLE and Uninstall to program group. + Don't distribute zlib.dll. Tweak some comments. + + * PCbuild/zlib.dsp: Now using static zlib 1.1.3 + + * Lib/dos-8x3/userdict.py, Lib/dos-8x3/userlist.py, Lib/dos-8x3/test_zli.py, Lib/dos-8x3/test_use.py, Lib/dos-8x3/test_pop.py, Lib/dos-8x3/test_pic.py, Lib/dos-8x3/test_ntp.py, Lib/dos-8x3/test_gzi.py, Lib/dos-8x3/test_fcn.py, Lib/dos-8x3/test_cpi.py, Lib/dos-8x3/test_bsd.py, Lib/dos-8x3/posixfil.py, Lib/dos-8x3/mimetype.py, Lib/dos-8x3/nturl2pa.py, Lib/dos-8x3/compilea.py, Lib/dos-8x3/exceptio.py, Lib/dos-8x3/basehttp.py: + The usual + + * Include/patchlevel.h: Release 1.5.2c1 + + * README: Release 1.5.2c1. + + * Misc/NEWS: News for the 1.5.2c1 release. + + * Lib/test/test_strftime.py: + On Windows, we suddenly find, strftime() may return "" for an + unsupported format string. (I guess this is because the logic for + deciding whether to reallocate the buffer or not has been improved.) + This caused the test code to crash on result[0]. Fix this by assuming + an empty result also means the format is not supported. + + * Demo/tkinter/matt/window-creation-w-location.py: + This demo imported some private code from Matt. Make it cripple along. + + * Lib/lib-tk/Tkinter.py: + Delete an accidentally checked-in feature that actually broke more + than was worth it: when deleting a canvas item, it would try to + automatically delete the bindings for that item. Since there's + nothing that says you can't reuse the tag and still have the bindings, + this is not correct. Also, it broke at least one demo + (Demo/tkinter/matt/rubber-band-box-demo-1.py). + + * Python/thread_wince.h: Win/CE thread support by Mark Hammond. + +Wed Apr 7 20:23:17 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Modules/zlibmodule.c: + Patch by Andrew Kuchling to unflush() (flush() for deflating). + Without this, if inflate() returned Z_BUF_ERROR asking for more output + space, we would report the error; now, we increase the buffer size and + try again, just as for Z_OK. + + * Lib/test/test_gzip.py: Use binary mode for all gzip files we open. + + * Tools/idle/ChangeLog: New change log. + + * Tools/idle/README.txt, Tools/idle/NEWS.txt: New version. + + * Python/pythonrun.c: + Alas, get rid of the Win specific hack to ask the user to press Return + before exiting when an error happened. This didn't work right when + Python is invoked from a daemon. + + * Tools/idle/idlever.py: Version bump awaiting impending new release. + (Not much has changed :-( ) + + * Lib/lib-tk/Tkinter.py: + lower, tkraise/lift hide Misc.lower, Misc.tkraise/lift, + so the preferred name for them is tag_lower, tag_raise + (similar to tag_bind, and similar to the Text widget); + unfortunately can't delete the old ones yet (maybe in 1.6) + + * Python/thread.c, Python/strtod.c, Python/mystrtoul.c, Python/import.c, Python/ceval.c: + Changes by Mark Hammond for Windows CE. Mostly of the form + #ifdef DONT_HAVE_header_H ... #endif around #include <header.h>. + + * Python/bltinmodule.c: + Remove unused variable from complex_from_string() code. + + * Include/patchlevel.h: + Add the possibility of a gamma release (release candidate). + Add '+' to string version number to indicate we're beyond b2 now. + + * Modules/posixmodule.c: Add extern decl for fsync() for SunOS 4.x. + + * Lib/smtplib.py: Changes by Per Cederquist and The Dragon. + + Per writes: + + """ + The application where Signum Support uses smtplib needs to be able to + report good error messages to the user when sending email fails. To + help in diagnosing problems it is useful to be able to report the + entire message sent by the server, not only the SMTP error code of the + offending command. + + A lot of the functions in sendmail.py unfortunately discards the + message, leaving only the code. The enclosed patch fixes that + problem. + + The enclosed patch also introduces a base class for exceptions that + include an SMTP error code and error message, and make the code and + message available on separate attributes, so that surrounding code can + deal with them in whatever way it sees fit. I've also added some + documentation to the exception classes. + + The constructor will now raise an exception if it cannot connect to + the SMTP server. + + The data() method will raise an SMTPDataError if it doesn't receive + the expected 354 code in the middle of the exchange. + + According to section 5.2.10 of RFC 1123 a smtp client must accept "any + text, including no text at all" after the error code. If the response + of a HELO command contains no text self.helo_resp will be set to the + empty string (""). The patch fixes the test in the sendmail() method + so that helo_resp is tested against None; if it has the empty string + as value the sendmail() method would invoke the helo() method again. + + The code no longer accepts a -1 reply from the ehlo() method in + sendmail(). + + [Text about removing SMTPRecipientsRefused deleted --GvR] + """ + + and also: + + """ + smtplib.py appends an extra blank line to the outgoing mail if the + `msg' argument to the sendmail method already contains a trailing + newline. This patch should fix the problem. + """ + + The Dragon writes: + + """ + Mostly I just re-added the SMTPRecipientsRefused exception + (the exeption object now has the appropriate info in it ) [Per had + removed this in his patch --GvR] and tweaked the behavior of the + sendmail method whence it throws the newly added SMTPHeloException (it + was closing the connection, which it shouldn't. whatever catches the + exception should do that. ) + + I pondered the change of the return values to tuples all around, + and after some thinking I decided that regularizing the return values was + too much of the Right Thing (tm) to not do. + + My one concern is that code expecting an integer & getting a tuple + may fail silently. + + (i.e. if it's doing : + + x.somemethod() >= 400: + expecting an integer, the expression will always be true if it gets a + tuple instead. ) + + However, most smtplib code I've seen only really uses the + sendmail() method, so this wouldn't bother it. Usually code I've seen + that calls the other methods usually only calls helo() and ehlo() for + doing ESMTP, a feature which was not in the smtplib included with 1.5.1, + and thus I would think not much code uses it yet. + """ + +Tue Apr 6 19:38:18 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/test/test_ntpath.py: + Fix the tests now that splitdrive() no longer treats UNC paths special. + (Some tests converted to splitunc() tests.) + + * Lib/ntpath.py: + Withdraw the UNC support from splitdrive(). Instead, a new function + splitunc() parses UNC paths. The contributor of the UNC parsing in + splitdrive() doesn't like it, but I haven't heard a good reason to + keep it, and it causes some problems. (I think there's a + philosophical problem -- to me, the split*() functions are purely + syntactical, and the fact that \\foo is not a valid path doesn't mean + that it shouldn't be considered an absolute path.) + + Also (quite separately, but strangely related to the philosophical + issue above) fix abspath() so that if win32api exists, it doesn't fail + when the path doesn't actually exist -- if GetFullPathName() fails, + fall back on the old strategy (join with getcwd() if neccessary, and + then use normpath()). + + * configure.in, configure, config.h.in, acconfig.h: + For BeOS PowerPC. Chris Herborth. + +Mon Apr 5 21:54:14 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Modules/timemodule.c: + Jonathan Giddy notes, and Chris Lawrence agrees, that some comments on + #else/#endif are wrong, and that #if HAVE_TM_ZONE should be #ifdef. + + * Misc/ACKS: + Bunch of new contributors, including 9 who contributed to the Docs, + reported by Fred. + +Mon Apr 5 18:37:59 1999 Fred Drake <fdrake@eric.cnri.reston.va.us> + + * Lib/gzip.py: + Oops, missed mode parameter to open(). + + * Lib/gzip.py: + Made the default mode 'rb' instead of 'r', for better cross-platform + support. (Based on comment on the documentation by Bernhard Reiter + <bernhard@csd.uwm.edu>). + +Fri Apr 2 22:18:25 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Tools/scripts/dutree.py: + For reasons I dare not explain, this script should always execute + main() when imported (in other words, it is not usable as a module). + +Thu Apr 1 15:32:30 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/test/test_cpickle.py: Jonathan Giddy write: + + In test_cpickle.py, the module os got imported, but the line to remove + the temp file has gone missing. + +Tue Mar 30 20:17:31 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/BaseHTTPServer.py: Per Cederqvist writes: + + If you send something like "PUT / HTTP/1.0" to something derived from + BaseHTTPServer that doesn't define do_PUT, you will get a response + that begins like this: + + HTTP/1.0 501 Unsupported method ('do_PUT') + Server: SimpleHTTP/0.3 Python/1.5 + Date: Tue, 30 Mar 1999 18:53:53 GMT + + The server should complain about 'PUT' instead of 'do_PUT'. This + patch should fix the problem. + +Mon Mar 29 20:33:21 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/smtplib.py: Patch by Per Cederqvist, who writes: + + """ + - It needlessly used the makefile() method for each response that is + read from the SMTP server. + + - If the remote SMTP server closes the connection unexpectedly the + code raised an IndexError. It now raises an SMTPServerDisconnected + exception instead. + + - The code now checks that all lines in a multiline response actually + contains an error code. + """ + + The Dragon approves. + +Mon Mar 29 20:25:40 1999 Fred Drake <fdrake@eric.cnri.reston.va.us> + + * Lib/compileall.py: + When run as a script, report failures in the exit code as well. + Patch largely based on changes by Andrew Dalke, as discussed in the + distutils-sig. + +Mon Mar 29 20:23:41 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/urllib.py: + Hack so that if a 302 or 301 redirect contains a relative URL, the + right thing "just happens" (basejoin() with old URL). + + * Modules/cPickle.c: + Protection against picling to/from closed (real) file. + The problem was reported by Moshe Zadka. + + * Lib/test/test_cpickle.py: + Test protection against picling to/from closed (real) file. + + * Modules/timemodule.c: Chris Lawrence writes: + + """ + The GNU folks, in their infinite wisdom, have decided not to implement + altzone in libc6; this would not be horrible, except that timezone + (which is implemented) includes the current DST setting (i.e. timezone + for Central is 18000 in summer and 21600 in winter). So Python's + timezone and altzone variables aren't set correctly during DST. + + Here's a patch relative to 1.5.2b2 that (a) makes timezone and altzone + show the "right" thing on Linux (by using the tm_gmtoff stuff + available in BSD, which is how the GLIBC manual claims things should + be done) and (b) should cope with the southern hemisphere. In pursuit + of (b), I also took the liberty of renaming the "summer" and "winter" + variables to "july" and "jan". This patch should also make certain + time calculations on Linux actually work right (like the tz-aware + functions in the rfc822 module). + + (It's hard to find DST that's currently being used in the southern + hemisphere; I tested using Africa/Windhoek.) + """ + + * Lib/test/output/test_gzip: + Jonathan Giddy discovered this file was missing. + + * Modules/shamodule.c: + Avoid warnings from AIX compiler. Reported by Vladimir (AIX is my + middlename) Marangozov, patch coded by Greg Stein. + + * Tools/idle/ScriptBinding.py, Tools/idle/PyShell.py: + At Tim Peters' recommendation, add a dummy flush() method to PseudoFile. + +Sun Mar 28 17:55:32 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Tools/scripts/ndiff.py: Tim Peters writes: + + I should have waited overnight <wink/sigh>. Nothing wrong with the one I + sent, but I couldn't resist going on to add new -r1 / -r2 cmdline options + for recreating the original files from ndiff's output. That's attached, if + you're game! Us Windows guys don't usually have a sed sitting around + <wink>. + +Sat Mar 27 13:34:01 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Tools/scripts/ndiff.py: Tim Peters writes: + + Attached is a cleaned-up version of ndiff (added useful module + docstring, now echo'ed in case of cmd line mistake); added -q option + to suppress initial file identification lines; + other minor cleanups, + & a slightly faster match engine. + +Fri Mar 26 22:36:00 1999 Fred Drake <fdrake@eric.cnri.reston.va.us> + + * Tools/scripts/dutree.py: + During display, if EPIPE is raised, it's probably because a pager was + killed. Discard the error in that case, but propogate it otherwise. + +Fri Mar 26 16:20:45 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/test/output/test_userlist, Lib/test/test_userlist.py: + Test suite for UserList. + + * Lib/UserList.py: Use isinstance() where appropriate. + Reformatted with 4-space indent. + +Fri Mar 26 16:11:40 1999 Barry Warsaw <bwarsaw@eric.cnri.reston.va.us> + + * Tools/pynche/PyncheWidget.py: + Helpwin.__init__(): The text widget should get focus. + + * Tools/pynche/pyColorChooser.py: + Removed unnecessary import `from PyncheWidget import PyncheWidget' + +Fri Mar 26 15:32:05 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/test/output/test_userdict, Lib/test/test_userdict.py: + Test suite for UserDict + + * Lib/UserDict.py: Improved a bunch of things. + The constructor now takes an optional dictionary. + Use isinstance() where appropriate. + +Thu Mar 25 22:38:49 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/test/output/test_pickle, Lib/test/output/test_cpickle, Lib/test/test_pickle.py, Lib/test/test_cpickle.py: + Basic regr tests for pickle/cPickle + + * Lib/pickle.py: + Don't use "exec" in find_class(). It's slow, unnecessary, and (as AMK + points out) it doesn't work in JPython Applets. + +Thu Mar 25 21:50:27 1999 Andrew Kuchling <akuchlin@eric.cnri.reston.va.us> + + * Lib/test/test_gzip.py: + Added a simple test suite for gzip. It simply opens a temp file, + writes a chunk of compressed data, closes it, writes another chunk, and + reads the contents back to verify that they are the same. + + * Lib/gzip.py: + Based on a suggestion from bruce@hams.com, make a trivial change to + allow using the 'a' flag as a mode for opening a GzipFile. gzip + files, surprisingly enough, can be concatenated and then decompressed; + the effect is to concatenate the two chunks of data. + + If we support it on writing, it should also be supported on reading. + This *wasn't* trivial, and required rearranging the code in the + reading path, particularly the _read() method. + + Raise IOError instead of RuntimeError in two cases, 'Not a gzipped file' + and 'Unknown compression method' + +Thu Mar 25 21:25:01 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/test/test_b1.py: + Add tests for float() and complex() with string args (Nick/Stephanie + Lockwood). + +Thu Mar 25 21:21:08 1999 Andrew Kuchling <akuchlin@eric.cnri.reston.va.us> + + * Modules/zlibmodule.c: + Add an .unused_data attribute to decompressor objects. If .unused_data + is not an empty string, this means that you have arrived at the + end of the stream of compressed data, and the contents of .unused_data are + whatever follows the compressed stream. + +Thu Mar 25 21:16:07 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Python/bltinmodule.c: + Patch by Nick and Stephanie Lockwood to implement complex() with a string + argument. This closes TODO item 2.19. + +Wed Mar 24 19:09:00 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Tools/webchecker/wcnew.py: Added Samuel Bayer's new webchecker. + Unfortunately his code breaks wcgui.py in a way that's not easy + to fix. I expect that this is a temporary situation -- + eventually Sam's changes will be merged back in. + (The changes add a -t option to specify exceptions to the -x + option, and explicit checking for #foo style fragment ids.) + + * Objects/dictobject.c: + Vladimir Marangozov contributed updated comments. + + * Objects/bufferobject.c: Folded long lines. + + * Lib/test/output/test_sha, Lib/test/test_sha.py: + Added Jeremy's test code for the sha module. + + * Modules/shamodule.c, Modules/Setup.in: + Added Greg Stein and Andrew Kuchling's sha module. + Fix comments about zlib version and URL. + + * Lib/test/test_bsddb.py: Remove the temp file when we're done. + + * Include/pythread.h: Conform to standard boilerplate. + + * configure.in, configure, BeOS/linkmodule, BeOS/ar-fake: + Chris Herborth: the new compiler in R4.1 needs some new options to work... + + * Modules/socketmodule.c: + Implement two suggestions by Jonathan Giddy: (1) in AIX, clear the + data struct before calling gethostby{name,addr}_r(); (2) ignore the + 3/5/6 args determinations made by the configure script and switch on + platform identifiers instead: + + AIX, OSF have 3 args + Sun, SGI have 5 args + Linux has 6 args + + On all other platforms, undef HAVE_GETHOSTBYNAME_R altogether. + + * Modules/socketmodule.c: + Vladimir Marangozov implements the AIX 3-arg gethostbyname_r code. + + * Lib/mailbox.py: + Add readlines() to _Subfile class. Not clear who would need it, but + Chris Lawrence sent me a broken version; this one is a tad simpler and + more conforming to the standard. + +Tue Mar 23 23:05:34 1999 Jeremy Hylton <jhylton@eric.cnri.reston.va.us> + + * Lib/gzip.py: use struct instead of bit-manipulate in Python + +Tue Mar 23 19:00:55 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Modules/Makefile.pre.in: + Add $(EXE) to various occurrences of python so it will work on Cygwin + with egcs (after setting EXE=.exe). Patch by Norman Vine. + + * configure, configure.in: + Ack! It never defined HAVE_GETHOSTBYNAME_R so that code was never tested! + +Mon Mar 22 22:25:39 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Include/thread.h: + Adding thread.h -- unused but for b/w compatibility. + As requested by Bill Janssen. + + * configure.in, configure: + Add code to test for all sorts of gethostbyname_r variants, + donated by David Arnold. + + * config.h.in, acconfig.h: + Add symbols for gethostbyname_r variants (sigh). + + * Modules/socketmodule.c: Clean up pass for the previous patches. + + - Use HAVE_GETHOSTBYNAME_R_6_ARG instead of testing for Linux and + glibc2. + + - If gethostbyname takes 3 args, undefine HAVE_GETHOSTBYNAME_R -- + don't know what code should be used. + + - New symbol USE_GETHOSTBYNAME_LOCK defined iff the lock should be used. + + - Modify the gethostbyaddr() code to also hold on to the lock until + after it is safe to release, overlapping with the Python lock. + + (Note: I think that it could in theory be possible that Python code + executed while gethostbyname_lock is held could attempt to reacquire + the lock -- e.g. in a signal handler or destructor. I will simply say + "don't do that then.") + + * Modules/socketmodule.c: Jonathan Giddy writes: + + Here's a patch to fix the race condition, which wasn't fixed by Rob's + patch. It holds the gethostbyname lock until the results are copied out, + which means that this lock and the Python global lock are held at the same + time. This shouldn't be a problem as long as the gethostbyname lock is + always acquired when the global lock is not held. + +Mon Mar 22 19:25:30 1999 Andrew Kuchling <akuchlin@eric.cnri.reston.va.us> + + * Modules/zlibmodule.c: + Fixed the flush() method of compression objects; the test for + the end of loop was incorrect, and failed when the flushmode != Z_FINISH. + Logic cleaned up and commented. + + * Lib/test/test_zlib.py: + Added simple test for the flush() method of compression objects, trying the + different flush values Z_NO_FLUSH, Z_SYNC_FLUSH, Z_FULL_FLUSH. + +Mon Mar 22 15:28:08 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/shlex.py: + Bug reported by Tobias Thelen: missing "self." in assignment target. + +Fri Mar 19 21:50:11 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Modules/arraymodule.c: + Use an unsigned cast to avoid a warning in VC++. + + * Lib/dospath.py, Lib/ntpath.py: + New code for split() by Tim Peters, behaves more like posixpath.split(). + + * Objects/floatobject.c: + Fix a problem with Vladimir's PyFloat_Fini code: clear the free list; if + a block cannot be freed, add its free items back to the free list. + This is necessary to avoid leaking when Python is reinitialized later. + + * Objects/intobject.c: + Fix a problem with Vladimir's PyInt_Fini code: clear the free list; if + a block cannot be freed, add its free items back to the free list, and + add its valid ints back to the small_ints array if they are in range. + This is necessary to avoid leaking when Python is reinitialized later. + + * Lib/types.py: + Added BufferType, the type returned by the new builtin buffer(). Greg Stein. + + * Python/bltinmodule.c: + New builtin buffer() creates a derived read-only buffer from any + object that supports the buffer interface (e.g. strings, arrays). + + * Objects/bufferobject.c: + Added check for negative offset for PyBuffer_FromObject and check for + negative size for PyBuffer_FromMemory. Greg Stein. + +Thu Mar 18 15:10:44 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/urlparse.py: Sjoerd Mullender writes: + + If a filename on Windows starts with \\, it is converted to a URL + which starts with ////. If this URL is passed to urlparse.urlparse + you get a path that starts with // (and an empty netloc). If you pass + the result back to urlparse.urlunparse, you get a URL that starts with + //, which is parsed differently by urlparse.urlparse. The fix is to + add the (empty) netloc with accompanying slashes if the path in + urlunparse starts with //. Do this for all schemes that use a netloc. + + * Lib/nturl2path.py: Sjoerd Mullender writes: + + Pathnames of files on other hosts in the same domain + (\\host\path\to\file) are not translated correctly to URLs and back. + The URL should be something like file:////host/path/to/file. + Note that a combination of drive letter and remote host is not + possible. + +Wed Mar 17 22:30:10 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/urlparse.py: + Delete non-standard-conforming code in urljoin() that would use the + netloc from the base url as the default netloc for the resulting url + even if the schemes differ. + + Once upon a time, when the web was wild, this was a valuable hack + because some people had a URL referencing an ftp server colocated with + an http server without having the host in the ftp URL (so they could + replicate it or change the hostname easily). + + More recently, after the file: scheme got added back to the list of + schemes that accept a netloc, it turns out that this caused weirdness + when joining an http: URL with a file: URL -- the resulting file: URL + would always inherit the host from the http: URL because the file: + scheme supports a netloc but in practice never has one. + + There are two reasons to get rid of the old, once-valuable hack, + instead of removing the file: scheme from the uses_netloc list. One, + the RFC says that file: uses the netloc syntax, and does not endorse + the old hack. Two, neither netscape 4.5 nor IE 4.0 support the old + hack. + + * Include/ceval.h, Include/abstract.h: + Add DLL level b/w compat for PySequence_In and PyEval_CallObject + +Tue Mar 16 21:54:50 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/lib-tk/Tkinter.py: Bug reported by Jim Robinson: + + An attempt to execute grid_slaves with arguments (0,0) results in + *all* of the slaves being returned, not just the slave associated with + row 0, column 0. This is because the test for arguments in the method + does not test to see if row (and column) does not equal None, but + rather just whether is evaluates to non-false. A value of 0 fails + this test. + +Tue Mar 16 14:17:48 1999 Fred Drake <fdrake@eric.cnri.reston.va.us> + + * Modules/cmathmodule.c: + Docstring fix: acosh() returns the hyperbolic arccosine, not the + hyperbolic cosine. Problem report via David Ascher by one of his + students. + +Mon Mar 15 21:40:59 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * configure.in: + Should test for gethost*by*name_r, not for gethostname_r (which + doesn't exist and doesn't make sense). + + * Modules/socketmodule.c: + Patch by Rob Riggs for Linux -- glibc2 has a different argument + converntion for gethostbyname_r() etc. than Solaris! + + * Python/thread_pthread.h: Rob Riggs wrote: + + """ + Spec says that on success pthread_create returns 0. It does not say + that an error code will be < 0. Linux glibc2 pthread_create() returns + ENOMEM (12) when one exceed process limits. (It looks like it should + return EAGAIN, but that's another story.) + + For reference, see: + http://www.opengroup.org/onlinepubs/7908799/xsh/pthread_create.html + """ + + [I have a feeling that similar bugs were fixed before; perhaps someone + could check that all error checks no check for != 0?] + + * Tools/bgen/bgen/bgenObjectDefinition.py: + New mixin class that defines cmp and hash that use + the ob_itself pointer. This allows (when using the mixin) + different Python objects pointing to the same C object and + behaving well as dictionary keys. + + Or so sez Jack Jansen... + + * Lib/urllib.py: Yet another patch by Sjoerd Mullender: + + Don't convert URLs to URLs using pathname2url. + +Fri Mar 12 22:15:43 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/cmd.py: Patch by Michael Scharf. He writes: + + The module cmd requires for each do_xxx command a help_xxx + function. I think this is a little old fashioned. + + Here is a patch: use the docstring as help if no help_xxx + function can be found. + + [I'm tempted to rip out all the help_* functions from pdb, but I'll + resist it. Any takers? --Guido] + + * Tools/freeze/freeze.py: Bug submitted by Wayne Knowles, who writes: + + Under Windows, python freeze.py -o hello hello.py + creates all the correct files in the hello subdirectory, but the + Makefile has the directory prefix in it for frozen_extensions.c + nmake fails because it tries to locate hello/frozen_extensions.c + + (His fix adds a call to os.path.basename() in the appropriate place.) + + * Objects/floatobject.c, Objects/intobject.c: + Vladimir has restructured his code somewhat so that the blocks are now + represented by an explicit structure. (There are still too many casts + in the code, but that may be unavoidable.) + + Also added code so that with -vv it is very chatty about what it does. + + * Demo/zlib/zlibdemo.py, Demo/zlib/minigzip.py: + Change #! line to modern usage; also chmod +x + + * Demo/pdist/rrcs, Demo/pdist/rcvs, Demo/pdist/rcsbump: + Change #! line to modern usage + + * Lib/nturl2path.py, Lib/urllib.py: From: Sjoerd Mullender + + The filename to URL conversion didn't properly quote special + characters. + The URL to filename didn't properly unquote special chatacters. + + * Objects/floatobject.c: + OK, try again. Vladimir gave me a fix for the alignment bus error, + so here's his patch again. This time it works (at least on Solaris, + Linux and Irix). + +Thu Mar 11 23:21:23 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Tools/idle/PathBrowser.py: + Don't crash when sys.path contains an empty string. + + * Tools/idle/PathBrowser.py: + - Don't crash in the case where a superclass is a string instead of a + pyclbr.Class object; this can happen when the superclass is + unrecognizable (to pyclbr), e.g. when module renaming is used. + + - Show a watch cursor when calling pyclbr (since it may take a while + recursively parsing imported modules!). + +Thu Mar 11 16:04:04 1999 Fred Drake <fdrake@eric.cnri.reston.va.us> + + * Lib/mimetypes.py: + Added .rdf and .xsl as application/xml types. (.rdf is for the + Resource Description Framework, a metadata encoding, and .xsl is for + the Extensible Stylesheet Language.) + +Thu Mar 11 13:26:23 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/test/output/test_popen2, Lib/test/test_popen2.py: + Test for popen2 module, by Chris Tismer. + + * Objects/floatobject.c: + Alas, Vladimir's patch caused a bus error (probably double + alignment?), and I didn't test it. Withdrawing it for now. + +Wed Mar 10 22:55:47 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Objects/floatobject.c: + Patch by Vladimir Marangoz to allow freeing of the allocated blocks of + floats on finalization. + + * Objects/intobject.c: + Patch by Vladimir Marangoz to allow freeing of the allocated blocks of + integers on finalization. + + * Tools/idle/EditorWindow.py, Tools/idle/Bindings.py: + Add PathBrowser to File module + + * Tools/idle/PathBrowser.py: + "Path browser" - 4 scrolled lists displaying: + directories on sys.path + modules in selected directory + classes in selected module + methods of selected class + + Sinlge clicking in a directory, module or class item updates the next + column with info about the selected item. Double clicking in a + module, class or method item opens the file (and selects the clicked + item if it is a class or method). + + I guess eventually I should be using a tree widget for this, but the + ones I've seen don't work well enough, so for now I use the old + Smalltalk or NeXT style multi-column hierarchical browser. + + * Tools/idle/MultiScrolledLists.py: + New utility: multiple scrolled lists in parallel + + * Tools/idle/ScrolledList.py: - White background. + - Display "(None)" (or text of your choosing) when empty. + - Don't set the focus. + +Tue Mar 9 19:31:21 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/urllib.py: + open_http also had the 'data is None' test backwards. don't call with the + extra argument if data is None. + + * Demo/embed/demo.c: + Call Py_SetProgramName() instead of redefining getprogramname(), + reflecting changes in the runtime around 1.5 or earlier. + + * Python/ceval.c: + Always test for an error return (usually NULL or -1) without setting + an exception. + + * Modules/timemodule.c: Patch by Chris Herborth for BeOS code. + He writes: + + I had an off-by-1000 error in floatsleep(), + and the problem with time.clock() is that it's not implemented properly + on QNX... ANSI says it's supposed to return _CPU_ time used by the + process, but on QNX it returns the amount of real time used... so I was + confused. + + * Tools/bgen/bgen/macsupport.py: Small change by Jack Jansen. + Test for self.returntype behaving like OSErr rather than being it. + +Thu Feb 25 16:14:58 1999 Jeremy Hylton <jhylton@eric.cnri.reston.va.us> + + * Lib/urllib.py: + http_error had the 'data is None' test backwards. don't call with the + extra argument if data is None. + + * Lib/urllib.py: change indentation from 8 spaces to 4 spaces + + * Lib/urllib.py: pleasing the tabnanny + +Thu Feb 25 14:26:02 1999 Fred Drake <fdrake@eric.cnri.reston.va.us> + + * Lib/colorsys.py: + Oops, one more "x, y, z" to convert... + + * Lib/colorsys.py: + Adjusted comment at the top to be less confusing, following Fredrik + Lundh's example. + + Converted comment to docstring. + +Wed Feb 24 18:49:15 1999 Fred Drake <fdrake@eric.cnri.reston.va.us> + + * Lib/toaiff.py: + Use sndhdr instead of the obsolete whatsound module. + +Wed Feb 24 18:42:38 1999 Jeremy Hylton <jhylton@eric.cnri.reston.va.us> + + * Lib/urllib.py: + When performing a POST request, i.e. when the second argument to + urlopen is used to specify form data, make sure the second argument is + threaded through all of the http_error_NNN calls. This allows error + handlers like the redirect and authorization handlers to properly + re-start the connection. + +Wed Feb 24 16:25:17 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/mhlib.py: Patch by Lars Wirzenius: + + o the initial comment is wrong: creating messages is already + implemented + + o Message.getbodytext: if the mail or it's part contains an + empty content-transfer-encoding header, the code used to + break; the change below treats an empty encoding value the same + as the other types that do not need decoding + + o SubMessage.getbodytext was missing the decode argument; the + change below adds it; I also made it unconditionally return + the raw text if decoding was not desired, because my own + routines needed that (and it was easier than rewriting my + own routines ;-) + +Wed Feb 24 00:35:43 1999 Barry Warsaw <bwarsaw@eric.cnri.reston.va.us> + + * Python/bltinmodule.c (initerrors): + Make sure that the exception tuples ("base-classes" when + string-based exceptions are used) reflect the real class hierarchy, + i.e. that SystemExit derives from Exception not StandardError. + + * Lib/exceptions.py: + Document the correct class hierarchy for SystemExit. It is not an + error and so it derives from Exception and not SystemError. The + docstring was incorrect but the implementation was fine. + +Tue Feb 23 23:07:51 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/shutil.py: + Add import sys, needed by reference to sys.exc_info() in rmtree(). + Discovered by Mitch Chapman. + + * config.h.in: + Now that we don't have AC_CHECK_LIB(m, pow), the HAVE_LIBM symbol + disappears. It wasn't used anywhere anyway... + + * Modules/arraymodule.c: + Carefully check for overflow when allocating the memory for fromfile + -- someone tried to pass in sys.maxint and got bitten by the bogus + calculations. + + * configure.in: + Get rid of AC_CHECK_LIB(m, pow) since this is taken care of later with + LIBM (from --with-libm=...); this actually broke the customizability + offered by the latter option. Thanks go to Clay Spence for reporting + this. + + * Lib/test/test_dl.py: + 1. Print the error message (carefully) when a dl.open() fails in verbose mode. + 2. When no test case worked, raise ImportError instead of failing. + + * Python/bltinmodule.c: + Patch by Tim Peters to improve the range checks for range() and + xrange(), especially for platforms where int and long are different + sizes (so sys.maxint isn't actually the theoretical limit for the + length of a list, but the largest C int is -- sys.maxint is the + largest Python int, which is actually a C long). + + * Makefile.in: + 1. Augment the DG/UX rule so it doesn't break the BeOS build. + 2. Add $(EXE) to various occurrences of python so it will work on + Cygwin with egcs (after setting EXE=.exe). These patches by + Norman Vine. + + * Lib/posixfile.py: + According to Jeffrey Honig, bsd/os 2.0 - 4.0 should be added to the + list (of bsd variants that have a different lock structure). + + * Lib/test/test_fcntl.py: + According to Jeffrey Honig, bsd/os 4.0 should be added to the list. + + * Modules/timemodule.c: + Patch by Tadayoshi Funaba (with some changes) to be smarter about + guessing what happened when strftime() returns 0. Is it buffer + overflow or was the result simply 0 bytes long? (This happens for an + empty format string, or when the format string is a single %Z and the + timezone is unknown.) if the buffer is at least 256 times as long as + the format, assume the latter. + +Mon Feb 22 19:01:42 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/urllib.py: + As Des Barry points out, we need to call pathname2url(file) in two + calls to addinfourl() in open_file(). + + * Modules/Setup.in: Document *static* -- in two places! + + * Modules/timemodule.c: + We don't support leap seconds, so the seconds field of a time 9-tuple + should be in the range [0-59]. Noted by Tadayoshi Funaba. + + * Modules/stropmodule.c: + In atoi(), don't use isxdigit() to test whether the last character + converted was a "digit" -- use isalnum(). This test is there only to + guard against "+" or "-" being interpreted as a valid int literal. + Reported by Takahiro Nakayama. + + * Lib/os.py: + As Finn Bock points out, _P_WAIT etc. don't have a leading underscore + so they don't need to be treated specially here. + +Mon Feb 22 15:38:58 1999 Fred Drake <fdrake@eric.cnri.reston.va.us> + + * Misc/NEWS: + Typo: "apparentlt" --> "apparently" + +Mon Feb 22 15:38:46 1999 Guido van Rossum <guido@eric.cnri.reston.va.us> + + * Lib/urlparse.py: Steve Clift pointed out that 'file' allows a netloc. + + * Modules/posixmodule.c: + The docstring for ttyname(..) claims a second "mode" argument. The + actual code does not allow such an argument. (Finn Bock.) + + * Lib/lib-old/poly.py: + Dang. Even though this is obsolete code, somebody found a bug, and I + fix it. Oh well. + +Thu Feb 18 20:51:50 1999 Fred Drake <fdrake@eric.cnri.reston.va.us> + + * Lib/pyclbr.py: + Bow to font-lock at the end of the docstring, since it throws stuff + off. + + Make sure the path paramter to readmodule() is a list before adding it + with sys.path, or the addition could fail. + + +====================================================================== + + +From 1.5.2b1 to 1.5.2b2 +======================= + +General +------- + +- Many memory leaks fixed. + +- Many small bugs fixed. + +- Command line option -OO (or -O -O) suppresses inclusion of doc +strings in resulting bytecode. + +Windows-specific changes +------------------------ + +- New built-in module winsound provides an interface to the Win32 +PlaySound() call. + +- Re-enable the audioop module in the config.c file. + +- On Windows, support spawnv() and associated P_* symbols. + +- Fixed the conversion of times() return values on Windows. + +- Removed freeze from the installer -- it doesn't work without the +source tree. (See FAQ 8.11.) + +- On Windows 95/98, the Tkinter module now is smart enough to find +Tcl/Tk even when the PATH environment variable hasn't been set -- when +the import of _tkinter fails, it searches in a standard locations, +patches os.environ["PATH"], and tries again. When it still fails, a +clearer error message is produced. This should avoid most +installation problems with Tkinter use (e.g. in IDLE). + +- The -i option doesn't make any calls to set[v]buf() for stdin -- +this apparently screwed up _kbhit() and the _tkinter main loop. + +- The ntpath module (and hence, os.path on Windows) now parses out UNC +paths (e.g. \\host\mountpoint\dir\file) as "drive letters", so that +splitdrive() will \\host\mountpoint as the drive and \dir\file as the +path. ** EXPERIMENTAL ** + +- Added a hack to the exit code so that if (1) the exit status is +nonzero and (2) we think we have our own DOS box (i.e. we're not +started from a command line shell), we print a message and wait for +the user to hit a key before the DOS box is closed. + +- Updated the installer to WISE 5.0g. Added a dialog warning about +the imminent Tcl installation. Added a dialog to specify the program +group name in the start menu. Upgraded the Tcl installer to Tcl +8.0.4. + +Changes to intrinsics +--------------------- + +- The repr() or str() of a module object now shows the __file__ +attribute (i.e., the file which it was loaded), or the string +"(built-in)" if there is no __file__ attribute. + +- The range() function now avoids overflow during its calculations (if +at all possible). + +- New info string sys.hexversion, which is an integer encoding the +version in hexadecimal. In other words, hex(sys.hexversion) == +0x010502b2 for Python 1.5.2b2. + +New or improved ports +--------------------- + +- Support for Nextstep descendants (future Mac systems). + +- Improved BeOS support. + +- Support dynamic loading of shared libraries on NetBSD platforms that +use ELF (i.e., MIPS and Alpha systems). + +Configuration/build changes +--------------------------- + +- The Lib/test directory is no longer included in the default module +search path (sys.path) -- "test" has been a package ever since 1.5. + +- Now using autoconf 2.13. + +New library modules +------------------- + +- New library modules asyncore and asynchat: these form Sam Rushing's +famous asynchronous socket library. Sam has gracefully allowed me to +incorporate these in the standard Python library. + +- New module statvfs contains indexing constants for [f]statvfs() +return tuple. + +Changes to the library +---------------------- + +- The wave module (platform-independent support for Windows sound +files) has been fixed to actually make it work. + +- The sunau module (platform-independent support for Sun/NeXT sound +files) has been fixed to work across platforms. Also, a weird +encoding bug in the header of the audio test data file has been +corrected. + +- Fix a bug in the urllib module that occasionally tripped up +webchecker and other ftp retrieves. + +- ConfigParser's get() method now accepts an optional keyword argument +(vars) that is substituted on top of the defaults that were setup in +__init__. You can now also have recusive references in your +configuration file. + +- Some improvements to the Queue module, including a put_nowait() +module and an optional "block" second argument, to get() and put(), +defaulting to 1. + +- The updated xmllib module is once again compatible with the version +present in Python 1.5.1 (this was accidentally broken in 1.5.2b1). + +- The bdb module (base class for the debugger) now supports +canonicalizing pathnames used in breakpoints. The derived class must +override the new canonical() method for this to work. Also changed +clear_break() to the backwards compatible old signature, and added +clear_bpbynumber() for the new functionality. + +- In sgmllib (and hence htmllib), recognize attributes even if they +don't have space in front of them. I.e. '<a +name="foo"href="bar.html">' will now have two attributes recognized. + +- In the debugger (pdb), change clear syntax to support three +alternatives: clear; clear file:line; clear bpno bpno ... + +- The os.path module now pretends to be a submodule within the os +"package", so you can do things like "from os.path import exists". + +- The standard exceptions now have doc strings. + +- In the smtplib module, exceptions are now classes. Also avoid +inserting a non-standard space after "TO" in rcpt() command. + +- The rfc822 module's getaddrlist() method now uses all occurrences of +the specified header instead of just the first. Some other bugfixes +too (to handle more weird addresses found in a very large test set, +and to avoid crashes on certain invalid dates), and a small test +module has been added. + +- Fixed bug in urlparse in the common-case code for HTTP URLs; it +would lose the query, fragment, and/or parameter information. + +- The sndhdr module no longer supports whatraw() -- it depended on a +rare extenral program. + +- The UserList module/class now supports the extend() method, like +real list objects. + +- The uu module now deals better with trailing garbage generated by +some broke uuencoders. + +- The telnet module now has an my_interact() method which uses threads +instead of select. The interact() method uses this by default on +Windows (where the single-threaded version doesn't work). + +- Add a class to mailbox.py for dealing with qmail directory +mailboxes. The test code was extended to notice these being used as +well. + +Changes to extension modules +---------------------------- + +- Support for the [f]statvfs() system call, where it exists. + +- Fixed some bugs in cPickle where bad input could cause it to dump +core. + +- Fixed cStringIO to make the writelines() function actually work. + +- Added strop.expandtabs() so string.expandtabs() is now much faster. + +- Added fsync() and fdatasync(), if they appear to exist. + +- Support for "long files" (64-bit seek pointers). + +- Fixed a bug in the zlib module's flush() function. + +- Added access() system call. It returns 1 if access granted, 0 if +not. + +- The curses module implements an optional nlines argument to +w.scroll(). (It then calls wscrl(win, nlines) instead of scoll(win).) + +Changes to tools +---------------- + +- Some changes to IDLE; see Tools/idle/NEWS.txt. + +- Latest version of Misc/python-mode.el included. + +Changes to Tkinter +------------------ + +- Avoid tracebacks when an image is deleted after its root has been +destroyed. + +Changes to the Python/C API +--------------------------- + +- When parentheses are used in a PyArg_Parse[Tuple]() call, any +sequence is now accepted, instead of requiring a tuple. This is in +line with the general trend towards accepting arbitrary sequences. + +- Added PyModule_GetFilename(). + +- In PyNumber_Power(), remove unneeded and even harmful test for float +to the negative power (which is already and better done in +floatobject.c). + +- New version identification symbols; read patchlevel.h for info. The +version numbers are now exported by Python.h. + +- Rolled back the API version change -- it's back to 1007! + +- The frozenmain.c function calls PyInitFrozenExtensions(). + +- Added 'N' format character to Py_BuildValue -- like 'O' but doesn't +INCREF. + + +====================================================================== + + +From 1.5.2a2 to 1.5.2b1 +======================= + +Changes to intrinsics +--------------------- + +- New extension NotImplementedError, derived from RuntimeError. Not +used, but recommended use is for "abstract" methods to raise this. + +- The parser will now spit out a warning or error when -t or -tt is +used for parser input coming from a string, too. + +- The code generator now inserts extra SET_LINENO opcodes when +compiling multi-line argument lists. + +- When comparing bound methods, use identity test on the objects, not +equality test. + +New or improved ports +--------------------- + +- Chris Herborth has redone his BeOS port; it now works on PowerPC +(R3/R4) and x86 (R4 only). Threads work too in this port. + +Renaming +-------- + +- Thanks to Chris Herborth, the thread primitives now have proper Py* +names in the source code (they already had those for the linker, +through some smart macros; but the source still had the old, un-Py +names). + +Configuration/build changes +--------------------------- + +- Improved support for FreeBSD/3. + +- Check for pthread_detach instead of pthread_create in libc. + +- The makesetup script now searches EXECINCLUDEPY before INCLUDEPY. + +- Misc/Makefile.pre.in now also looks at Setup.thread and Setup.local. +Otherwise modules such as thread didn't get incorporated in extensions. + +New library modules +------------------- + +- shlex.py by Eric Raymond provides a lexical analyzer class for +simple shell-like syntaxes. + +- netrc.py by Eric Raymond provides a parser for .netrc files. (The +undocumented Netrc class in ftplib.py is now obsolete.) + +- codeop.py is a new module that contains the compile_command() +function that was previously in code.py. This is so that JPython can +provide its own version of this function, while still sharing the +higher-level classes in code.py. + +- turtle.py is a new module for simple turtle graphics. I'm still +working on it; let me know if you use this to teach Python to children +or other novices without prior programming experience. + +Obsoleted library modules +------------------------- + +- poly.py and zmod.py have been moved to Lib/lib-old to emphasize +their status of obsoleteness. They don't do a particularly good job +and don't seem particularly relevant to the Python core. + +New tools +--------- + +- I've added IDLE: my Integrated DeveLopment Environment for Python. +Requires Tcl/Tk (and Tkinter). Works on Windows and Unix (and should +work on Macintosh, but I haven't been able to test it there; it does +depend on new features in 1.5.2 and perhaps even new features in +1.5.2b1, especially the new code module). This is very much a work in +progress. I'd like to hear how people like it compared to PTUI (or +any other IDE they are familiar with). + +- New tools by Barry Warsaw: + + = audiopy: controls the Solaris Audio device + = pynche: The PYthonically Natural Color and Hue Editor + = world: Print mappings between country names and DNS country codes + +New demos +--------- + +- Demo/scripts/beer.py prints the lyrics to an arithmetic drinking +song. + +- Demo/tkinter/guido/optionmenu.py shows how to do an option menu in +Tkinter. (By Fredrik Lundh -- not by me!) + +Changes to the library +---------------------- + +- compileall.py now avoids recompiling .py files that haven't changed; +it adds a -f option to force recompilation. + +- New version of xmllib.py by Sjoerd Mullender (0.2 with latest +patches). + +- nntplib.py: statparse() no longer lowercases the message-id. + +- types.py: use type(__stdin__) for FileType. + +- urllib.py: fix translations for filenames with "funny" characters. +Patch by Sjoerd Mullender. Note that if you subclass one of the +URLopener classes, and you have copied code from the old urllib.py, +your subclass may stop working. A long-term solution is to provide +more methods so that you don't have to copy code. + +- cgi.py: In read_multi, allow a subclass to override the class we +instantiate when we create a recursive instance, by setting the class +variable 'FieldStorageClass' to the desired class. By default, this +is set to None, in which case we use self.__class__ (as before). +Also, a patch by Jim Fulton to pass additional arguments to recursive +calls to the FieldStorage constructor from its read_multi method. + +- UserList.py: In __getslice__, use self.__class__ instead of +UserList. + +- In SimpleHTTPServer.py, the server specified in test() should be +BaseHTTPServer.HTTPServer, in case the request handler should want to +reference the two attributes added by BaseHTTPServer.server_bind. (By +Jeff Rush, for Bobo). Also open the file in binary mode, so serving +images from a Windows box might actually work. + +- In CGIHTTPServer.py, the list of acceptable formats is -split- +on spaces but -joined- on commas, resulting in double commas +in the joined text. (By Jeff Rush.) + +- SocketServer.py, patch by Jeff Bauer: a minor change to declare two +new threaded versions of Unix Server classes, using the ThreadingMixIn +class: ThreadingUnixStreamServer, ThreadingUnixDatagramServer. + +- bdb.py: fix bomb on deleting a temporary breakpoint: there's no +method do_delete(); do_clear() was meant. By Greg Ward. + +- getopt.py: accept a non-list sequence for the long options (request +by Jack Jansen). Because it might be a common mistake to pass a +single string, this situation is treated separately. Also added +docstrings (copied from the library manual) and removed the (now +redundant) module comments. + +- tempfile.py: improvements to avoid security leaks. + +- code.py: moved compile_command() to new module codeop.py. + +- pickle.py: support pickle format 1.3 (binary float added). By Jim +Fulton. Also get rid of the undocumented obsolete Pickler dump_special +method. + +- uu.py: Move 'import sys' to top of module, as noted by Tim Peters. + +- imaplib.py: fix problem with some versions of IMAP4 servers that +choose to mix the case in their CAPABILITIES response. + +- cmp.py: use (f1, f2) as cache key instead of f1 + ' ' + f2. Noted +by Fredrik Lundh. + +Changes to extension modules +---------------------------- + +- More doc strings for several modules were contributed by Chris +Petrilli: math, cmath, fcntl. + +- Fixed a bug in zlibmodule.c that could cause core dumps on +decompression of rarely occurring input. + +- cPickle.c: new version from Jim Fulton, with Open Source copyright +notice. Also, initialize self->safe_constructors early on to prevent +crash in early dealloc. + +- cStringIO.c: new version from Jim Fulton, with Open Source copyright +notice. Also fixed a core dump in cStringIO.c when doing seeks. + +- mpzmodule.c: fix signed character usage in mpz.mpz(stringobjecty). + +- readline.c: Bernard Herzog pointed out that rl_parse_and_bind +modifies its argument string (bad function!), so we make a temporary +copy. + +- sunaudiodev.c: Barry Warsaw added more smarts to get the device and +control pseudo-device, per audio(7I). + +Changes to tools +---------------- + +- New, improved version of Barry Warsaw's Misc/python-mode.el (editing +support for Emacs). + +- tabnanny.py: added a -q ('quiet') option to tabnanny, which causes +only the names of offending files to be printed. + +- freeze: when printing missing modules, also print the module they +were imported from. + +- untabify.py: patch by Detlef Lannert to implement -t option +(set tab size). + +Changes to Tkinter +------------------ + +- grid_bbox(): support new Tk API: grid bbox ?column row? ?column2 +row2? + +- _tkinter.c: RajGopal Srinivasan noted that the latest code (1.5.2a2) +doesn't work when running in a non-threaded environment. He added +some #ifdefs that fix this. + +Changes to the Python/C API +--------------------------- + +- Bumped API version number to 1008 -- enough things have changed! + +- There's a new macro, PyThreadState_GET(), which does the same work +as PyThreadState_Get() without the overhead of a function call (it +also avoids the error check). The two top calling locations of +PyThreadState_Get() have been changed to use this macro. + +- All symbols intended for export from a DLL or shared library are now +marked as such (with the DL_IMPORT() macro) in the header file that +declares them. This was needed for the BeOS port, and should also +make some other ports easier. The PC port no longer needs the file +with exported symbols (PC/python_nt.def). There's also a DL_EXPORT +macro which is only used for init methods in extension modules, and +for Py_Main(). + +Invisible changes to internals +------------------------------ + +- Fixed a bug in new_buffersize() in fileobject.c which could +return a buffer size that was way too large. + +- Use PySys_WriteStderr instead of fprintf in most places. + +- dictobject.c: remove dead code discovered by Vladimir Marangozov. + +- tupleobject.c: make tuples less hungry -- an extra item was +allocated but never used. Tip by Vladimir Marangozov. + +- mymath.h: Metrowerks PRO4 finally fixes the hypot snafu. (Jack +Jansen) + +- import.c: Jim Fulton fixes a reference count bug in +PyEval_GetGlobals. + +- glmodule.c: check in the changed version after running the stubber +again -- this solves the conflict with curses over the 'clear' entry +point much nicer. (Jack Jansen had checked in the changes to cstubs +eons ago, but I never regenrated glmodule.c :-( ) + +- frameobject.c: fix reference count bug in PyFrame_New. Vladimir +Marangozov. + +- stropmodule.c: add a missing DECREF in an error exit. Submitted by +Jonathan Giddy. + + +====================================================================== + + +From 1.5.2a1 to 1.5.2a2 +======================= + +General +------- + +- It is now a syntax error to have a function argument without a +default following one with a default. + +- __file__ is now set to the .py file if it was parsed (it used to +always be the .pyc/.pyo file). + +- Don't exit with a fatal error during initialization when there's a +problem with the exceptions.py module. + +- New environment variable PYTHONOPTIMIZE can be used to set -O. + +- New version of python-mode.el for Emacs. + +Miscellaneous fixed bugs +------------------------ + +- No longer print the (confusing) error message about stack underflow +while compiling. + +- Some threading and locking bugs fixed. + +- When errno is zero, report "Error", not "Success". + +Documentation +------------- + +- Documentation will be released separately. + +- Doc strings added to array and md5 modules by Chris Petrilli. + +Ports and build procedure +------------------------- + +- Stop installing when a move or copy fails. + +- New version of the OS/2 port code by Jeff Rush. + +- The makesetup script handles absolute filenames better. + +- The 'new' module is now enabled by default in the Setup file. + +- I *think* I've solved the problem with the Linux build blowing up +sometimes due to a conflict between sigcheck/intrcheck and +signalmodule. + +Built-in functions +------------------ + +- The second argument to apply() can now be any sequence, not just a +tuple. + +Built-in types +-------------- + +- Lists have a new method: L1.extend(L2) is equivalent to the common +idiom L1[len(L1):] = L2. + +- Better error messages when a sequence is indexed with a non-integer. + +- Bettter error message when calling a non-callable object (include +the type in the message). + +Python services +--------------- + +- New version of cPickle.c fixes some bugs. + +- pickle.py: improved instantiation error handling. + +- code.py: reworked quite a bit. New base class +InteractiveInterpreter and derived class InteractiveConsole. Fixed +several problems in compile_command(). + +- py_compile.py: print error message and continue on syntax errors. +Also fixed an old bug with the fstat code (it was never used). + +- pyclbr.py: support submodules of packages. + +String Services +--------------- + +- StringIO.py: raise the right exception (ValueError) for attempted +I/O on closed StringIO objects. + +- re.py: fixed a bug in subn(), which caused .groups() to fail inside +the replacement function called by sub(). + +- The struct module has a new format 'P': void * in native mode. + +Generic OS Services +------------------- + +- Module time: Y2K robustness. 2-digit year acceptance depends on +value of time.accept2dyear, initialized from env var PYTHONY2K, +default 0. Years 00-68 mean 2000-2068, while 69-99 mean 1969-1999 +(POSIX or X/Open recommendation). + +- os.path: normpath(".//x") should return "x", not "/x". + +- getpass.py: fall back on default_getpass() when sys.stdin.fileno() +doesn't work. + +- tempfile.py: regenerate the template after a fork() call. + +Optional OS Services +-------------------- + +- In the signal module, disable restarting interrupted system calls +when we have siginterrupt(). + +Debugger +-------- + +- No longer set __args__; this feature is no longer supported and can +affect the debugged code. + +- cmd.py, pdb.py and bdb.py have been overhauled by Richard Wolff, who +added aliases and some other useful new features, e.g. much better +breakpoint support: temporary breakpoint, disabled breakpoints, +breakpoints with ignore counts, and conditions; breakpoints can be set +on a file before it is loaded. + +Profiler +-------- + +- Changes so that JPython can use it. Also fix the calibration code +so it actually works again +. +Internet Protocols and Support +------------------------------ + +- imaplib.py: new version from Piers Lauder. + +- smtplib.py: change sendmail() method to accept a single string or a +list or strings as the destination (commom newbie mistake). + +- poplib.py: LIST with a msg argument fixed. + +- urlparse.py: some optimizations for common case (http). + +- urllib.py: support content-length in info() for ftp protocol; +support for a progress meter through a third argument to +urlretrieve(); commented out gopher test (the test site is dead). + +Internet Data handling +---------------------- + +- sgmllib.py: support tags with - or . in their name. + +- mimetypes.py: guess_type() understands 'data' URLs. + +Restricted Execution +-------------------- + +- The classes rexec.RModuleLoader and rexec.RModuleImporter no +longer exist. + +Tkinter +------- + +- When reporting an exception, store its info in sys.last_*. Also, +write all of it to stderr. + +- Added NS, EW, and NSEW constants, for grid's sticky option. + +- Fixed last-minute bug in 1.5.2a1 release: need to include "mytime.h". + +- Make bind variants without a sequence return a tuple of sequences +(formerly it returned a string, which wasn't very convenient). + +- Add image commands to the Text widget (these are new in Tk 8.0). + +- Added new listbox and canvas methods: {xview,yview}_{scroll,moveto}.) + +- Improved the thread code (but you still can't call update() from +another thread on Windows). + +- Fixed unnecessary references to _default_root in the new dialog +modules. + +- Miscellaneous problems fixed. + + +Windows General +--------------- + +- Call LoadLibraryEx(..., ..., LOAD_WITH_ALTERED_SEARCH_PATH) to +search for dependent dlls in the directory containing the .pyd. + +- In debugging mode, call DebugBreak() in Py_FatalError(). + +Windows Installer +----------------- + +- Install zlib.dll in the DLLs directory instead of in the win32 +system directory, to avoid conflicts with other applications that have +their own zlib.dll. + +Test Suite +---------- + +- test_long.py: new test for long integers, by Tim Peters. + +- regrtest.py: improved so it can be used for other test suites as +well. + +- test_strftime.py: use re to compare test results, to support legal +variants (e.g. on Linux). + +Tools and Demos +--------------- + +- Four new scripts in Tools/scripts: crlf.py and lfcr.py (to +remove/add Windows style '\r\n' line endings), untabify.py (to remove +tabs), and rgrep.yp (reverse grep). + +- Improvements to Tools/freeze/. Each Python module is now written to +its own C file. This prevents some compilers or assemblers from +blowing up on large frozen programs, and saves recompilation time if +only a few modules are changed. Other changes too, e.g. new command +line options -x and -i. + +- Much improved (and smaller!) version of Tools/scripts/mailerdaemon.py. + +Python/C API +------------ + +- New mechanism to support extensions of the type object while +remaining backward compatible with extensions compiled for previous +versions of Python 1.5. A flags field indicates presence of certain +fields. + +- Addition to the buffer API to differentiate access to bytes and +8-bit characters (in anticipation of Unicode characters). + +- New argument parsing format t# ("text") to indicate 8-bit +characters; s# simply means 8-bit bytes, for backwards compatibility. + +- New object type, bufferobject.c is an example and can be used to +create buffers from memory. + +- Some support for 64-bit longs, including some MS platforms. + +- Many calls to fprintf(stderr, ...) have been replaced with calls to +PySys_WriteStderr(...). + +- The calling context for PyOS_Readline() has changed: it must now be +called with the interpreter lock held! It releases the lock around +the call to the function pointed to by PyOS_ReadlineFunctionPointer +(default PyOS_StdioReadline()). + +- New APIs PyLong_FromVoidPtr() and PyLong_AsVoidPtr(). + +- Renamed header file "thread.h" to "pythread.h". + +- The code string of code objects may now be anything that supports the +buffer API. + + +====================================================================== + + +From 1.5.1 to 1.5.2a1 +===================== + +General +------- + +- When searching for the library, a landmark that is a compiled module +(string.pyc or string.pyo) is also accepted. + +- When following symbolic links to the python executable, use a loop +so that a symlink to a symlink can work. + +- Added a hack so that when you type 'quit' or 'exit' at the +interpreter, you get a friendly explanation of how to press Ctrl-D (or +Ctrl-Z) to exit. + +- New and improved Misc/python-mode.el (Python mode for Emacs). + +- Revert a new feature in Unix dynamic loading: for one or two +revisions, modules were loaded using the RTLD_GLOBAL flag. It turned +out to be a bad idea. + +Miscellaneous fixed bugs +------------------------ + +- All patches on the patch page have been integrated. (But much more +has been done!) + +- Several memory leaks plugged (e.g. the one for classes with a +__getattr__ method). + +- Removed the only use of calloc(). This triggered an obscure bug on +multiprocessor Sparc Solaris 2.6. + +- Fix a peculiar bug that would allow "import sys.time" to succeed +(believing the built-in time module to be a part of the sys package). + +- Fix a bug in the overflow checking when converting a Python long to +a C long (failed to convert -2147483648L, and some other cases). + +Documentation +------------- + +- Doc strings have been added to many extension modules: __builtin__, +errno, select, signal, socket, sys, thread, time. Also to methods of +list objects (try [].append.__doc__). A doc string on a type will now +automatically be propagated to an instance if the instance has methods +that are accessed in the usual way. + +- The documentation has been expanded and the formatting improved. +(Remember that the documentation is now unbundled and has its own +release cycle though; see http://www.python.org/doc/.) + +- Added Misc/Porting -- a mini-FAQ on porting to a new platform. + +Ports and build procedure +------------------------- + +- The BeOS port is now integrated. Courtesy Chris Herborth. + +- Symbol files for FreeBSD 2.x and 3.x have been contributed +(Lib/plat-freebsd[23]/*). + +- Support HPUX 10.20 DCE threads. + +- Finally fixed the configure script so that (on SGI) if -OPT:Olimit=0 +works, it won't also use -Olimit 1500 (which gives a warning for every +file). Also support the SGI_ABI environment variable better. + +- The makesetup script now understands absolute pathnames ending in .o +in the module -- it assumes it's a file for which we have no source. + +- Other miscellaneous improvements to the configure script and +Makefiles. + +- The test suite now uses a different sound sample. + +Built-in functions +------------------ + +- Better checks for invalid input to int(), long(), string.atoi(), +string.atol(). (Formerly, a sign without digits would be accepted as +a legal ways to spell zero.) + +- Changes to map() and filter() to use the length of a sequence only +as a hint -- if an IndexError happens earlier, take that. (Formerly, +this was considered an error.) + +- Experimental feature in getattr(): a third argument can specify a +default (instead of raising AttributeError). + +- Implement round() slightly different, so that for negative ndigits +no additional errors happen in the last step. + +- The open() function now adds the filename to the exception when it +fails. + +Built-in exceptions +------------------- + +- New standard exceptions EnvironmentError and PosixError. +EnvironmentError is the base class for IOError and PosixError; +PosixError is the same as os.error. All this so that either exception +class can be instantiated with a third argument indicating a filename. +The built-in function open() and most os/posix functions that take a +filename argument now use this. + +Built-in types +-------------- + +- List objects now have an experimental pop() method; l.pop() returns +and removes the last item; l.pop(i) returns and removes the item at +i. Also, the sort() method is faster again. Sorting is now also +safer: it is impossible for the sorting function to modify the list +while the sort is going on (which could cause core dumps). + +- Changes to comparisons: numbers are now smaller than any other type. +This is done to prevent the circularity where [] < 0L < 1 < [] is +true. As a side effect, cmp(None, 0) is now positive instead of +negative. This *shouldn't* affect any working code, but I've found +that the change caused several "sleeping" bugs to become active, so +beware! + +- Instance methods may now have other callable objects than just +Python functions as their im_func. Use new.instancemethod() or write +your own C code to create them; new.instancemethod() may be called +with None for the instance to create an unbound method. + +- Assignment to __name__, __dict__ or __bases__ of a class object is +now allowed (with stringent type checks); also allow assignment to +__getattr__ etc. The cached values for __getattr__ etc. are +recomputed after such assignments (but not for derived classes :-( ). + +- Allow assignment to some attributes of function objects: func_code, +func_defaults and func_doc / __doc__. (With type checks except for +__doc__ / func_doc .) + +Python services +--------------- + +- New tests (in Lib/test): reperf.py (regular expression benchmark), +sortperf.py (list sorting benchmark), test_MimeWriter.py (test case +for the MimeWriter module). + +- Generalized test/regrtest.py so that it is useful for testing other +packages. + +- The ihooks.py module now understands package imports. + +- In code.py, add a class that subsumes Fredrik Lundh's +PythonInterpreter class. The interact() function now uses this. + +- In rlcompleter.py, in completer(), return None instead of raising an +IndexError when there are no more completions left. + +- Fixed the marshal module to test for certain common kinds of invalid +input. (It's still not foolproof!) + +- In the operator module, add an alias (now the preferred name) +"contains" for "sequenceincludes". + +String Services +--------------- + +- In the string and strop modules, in the replace() function, treat an +empty pattern as an error (since it's not clear what was meant!). + +- Some speedups to re.py, especially the string substitution and split +functions. Also added new function/method findall(), to find all +occurrences of a given substring. + +- In cStringIO, add better argument type checking and support the +readonly 'closed' attribute (like regular files). + +- In the struct module, unsigned 1-2 byte sized formats no longer +result in long integer values. + +Miscellaneous services +---------------------- + +- In whrandom.py, added new method and function randrange(), same as +choice(range(start, stop, step)) but faster. This addresses the +problem that randint() was accidentally defined as taking an inclusive +range. Also, randint(a, b) is now redefined as randrange(a, b+1), +adding extra range and type checking to its arguments! + +- Add some semi-thread-safety to random.gauss() (it used to be able to +crash when invoked from separate threads; now the worst it can do is +give a duplicate result occasionally). + +- Some restructuring and generalization done to cmd.py. + +- Major upgrade to ConfigParser.py; converted to using 're', added new +exceptions, support underscore in section header and option name. No +longer add 'name' option to every section; instead, add '__name__'. + +- In getpass.py, don't use raw_input() to ask for the password -- we +don't want it to show up in the readline history! Also don't catch +interrupts (the try-finally already does all necessary cleanup). + +Generic OS Services +------------------- + +- New functions in os.py: makedirs(), removedirs(), renames(). New +variable: linesep (the line separator as found in binary files, +i.e. '\n' on Unix, '\r\n' on DOS/Windows, '\r' on Mac. Do *not* use +this with files opened in (default) text mode; the line separator used +will always be '\n'! + +- Changes to the 'os.path' submodule of os.py: added getsize(), +getmtime(), getatime() -- these fetch the most popular items from the +stat return tuple. + +- In the time module, add strptime(), if it exists. (This parses a +time according to a format -- the inverse of strftime().) Also, +remove the call to mktime() from strftime() -- it messed up the +formatting of some non-local times. + +- In the socket module, added a new function gethostbyname_ex(). +Also, don't use #ifdef to test for some symbols that are enums on some +platforms (and should exist everywhere). + +Optional OS Services +-------------------- + +- Some fixes to gzip.py. In particular, the readlines() method now +returns the lines *with* trailing newline characters, like readlines() +of regular file objects. Also, it didn't work together with cPickle; +fixed that. + +- In whichdb.py, support byte-swapped dbhash (bsddb) files. + +- In anydbm.py, look at the type of an existing database to determine +which module to use to open it. (The anydbm.error exception is now a +tuple.) + +Unix Services +------------- + +- In the termios module, in tcsetattr(), initialize the structure vy +calling tcgetattr(). + +- Added some of the "wait status inspection" macros as functions to +the posix module (and thus to the os module): WEXITSTATUS(), +WIFEXITED(), WIFSIGNALED(), WIFSTOPPED(), WSTOPSIG(), WTERMSIG(). + +- In the syslog module, make the default facility more intuitive +(matching the docs). + +Debugger +-------- + +- In pdb.py, support for setting breaks on files/modules that haven't +been loaded yet. + +Internet Protocols and Support +------------------------------ + +- Changes in urllib.py; sped up unquote() and quote(). Fixed an +obscure bug in quote_plus(). Added urlencode(dict) -- convenience +function for sending a POST request with urlopen(). Use the getpass +module to ask for a password. Rewrote the (test) main program so that +when used as a script, it can retrieve one or more URLs to stdout. +Use -t to run the self-test. Made the proxy code work again. + +- In cgi.py, treat "HEAD" the same as "GET", so that CGI scripts don't +fail when someone asks for their HEAD. Also, for POST, set the +default content-type to application/x-www-form-urlencoded. Also, in +FieldStorage.__init__(), when method='GET', always get the query +string from environ['QUERY_STRING'] or sys.argv[1] -- ignore an +explicitly passed in fp. + +- The smtplib.py module now supports ESMTP and has improved standard +compliance, for picky servers. + +- Improved imaplib.py. + +- Fixed UDP support in SocketServer.py (it never worked). + +- Fixed a small bug in CGIHTTPServer.py. + +Internet Data handling +---------------------- + +- In rfc822.py, add a new class AddressList. Also support a new +overridable method, isheader(). Also add a get() method similar to +dictionaries (and make getheader() an alias for it). Also, be smarter +about seekable (test whether fp.tell() works) and test for presence of +unread() method before trying seeks. + +- In sgmllib.py, restore the call to report_unbalanced() that was lost +long ago. Also some other improvements: handle <? processing +instructions >, allow . and - in entity names, and allow \r\n as line +separator. + +- Some restructuring and generalization done to multifile.py; support +a 'seekable' flag. + +Restricted Execution +-------------------- + +- Improvements to rexec.py: package support; support a (minimal) +sys.exc_info(). Also made the (test) main program a bit fancier (you +can now use it to run arbitrary Python scripts in restricted mode). + +Tkinter +------- + +- On Unix, Tkinter can now safely be used from a multi-threaded +application. (Formerly, no threads would make progress while +Tkinter's mainloop() was active, because it didn't release the Python +interpreter lock.) Unfortunately, on Windows, threads other than the +main thread should not call update() or update_idletasks() because +this will deadlock the application. + +- An interactive interpreter that uses readline and Tkinter no longer +uses up all available CPU time. + +- Even if readline is not used, Tk windows created in an interactive +interpreter now get continuously updated. (This even works in Windows +as long as you don't hit a key.) + +- New demos in Demo/tkinter/guido/: brownian.py, redemo.py, switch.py. + +- No longer register Tcl_finalize() as a low-level exit handler. It +may call back into Python, and that's a bad idea. + +- Allow binding of Tcl commands (given as a string). + +- Some minor speedups; replace explicitly coded getint() with int() in +most places. + +- In FileDialog.py, remember the directory of the selected file, if +given. + +- Change the names of all methods in the Wm class: they are now +wm_title(), etc. The old names (title() etc.) are still defined as +aliases. + +- Add a new method of interpreter objects, interpaddr(). This returns +the address of the Tcl interpreter object, as an integer. Not very +useful for the Python programmer, but this can be called by another C +extension that needs to make calls into the Tcl/Tk C API and needs to +get the address of the Tcl interpreter object. A simple cast of the +return value to (Tcl_Interp *) will do the trick. + +Windows General +--------------- + +- Don't insist on proper case for module source files if the filename +is all uppercase (e.g. FOO.PY now matches foo; but FOO.py still +doesn't). This should address problems with this feature on +oldfashioned filesystems (Novell servers?). + +Windows Library +--------------- + +- os.environ is now all uppercase, but accesses are case insensitive, +and the putenv() calls made as a side effect of changing os.environ +are case preserving. + +- Removed samefile(), sameopenfile(), samestat() from os.path (aka +ntpath.py) -- these cannot be made to work reliably (at least I +wouldn't know how). + +- Fixed os.pipe() so that it returns file descriptors acceptable to +os.read() and os.write() (like it does on Unix), rather than Windows +file handles. + +- Added a table of WSA error codes to socket.py. + +- In the select module, put the (huge) file descriptor arrays on the +heap. + +- The getpass module now raises KeyboardInterrupt when it sees ^C. + +- In mailbox.py, fix tell/seek when using files opened in text mode. + +- In rfc822.py, fix tell/seek when using files opened in text mode. + +- In the msvcrt extension module, release the interpreter lock for +calls that may block: _locking(), _getch(), _getche(). Also fix a +bogus error return when open_osfhandle() doesn't have the right +argument list. + +Windows Installer +----------------- + +- The registry key used is now "1.5" instead of "1.5.x" -- so future +versions of 1.5 and Mark Hammond's win32all installer don't need to be +resynchronized. + +Windows Tools +------------- + +- Several improvements to freeze specifically for Windows. + +Windows Build Procedure +----------------------- + +- The VC++ project files and the WISE installer have been moved to the +PCbuild subdirectory, so they are distributed in the same subdirectory +where they must be used. This avoids confusion. + +- New project files for Windows 3.1 port by Jim Ahlstrom. + +- Got rid of the obsolete subdirectory PC/setup_nt/. + +- The projects now use distinct filenames for the .exe, .dll, .lib and +.pyd files built in debug mode (by appending "_d" to the base name, +before the extension). This makes it easier to switch between the two +and get the right versions. There's a pragma in config.h that directs +the linker to include the appropriate .lib file (so python15.lib no +longer needs to be explicit in your project). + +- The installer now installs more files (e.g. config.h). The idea is +that you shouldn't need the source distribution if you want build your +own extensions in C or C++. + +Tools and Demos +--------------- + +- New script nm2def.py by Marc-Andre Lemburg, to construct +PC/python_nt.def automatically (some hand editing still required). + +- New tool ndiff.py: Tim Peters' text diffing tool. + +- Various and sundry improvements to the freeze script. + +- The script texi2html.py (which was part of the Doc tree but is no +longer used there) has been moved to the Tools/scripts subdirectory. + +- Some generalizations in the webchecker code. There's now a +primnitive gui for websucker.py: wsgui.py. (In Tools/webchecker/.) + +- The ftpmirror.py script now handles symbolic links properly, and +also files with multiple spaces in their names. + +- The 1.5.1 tabnanny.py suffers an assert error if fed a script whose +last line is both indented and lacks a newline. This is now fixed. + +Python/C API +------------ + +- Added missing prototypes for PyEval_CallFunction() and +PyEval_CallMethod(). + +- New macro PyList_SET_ITEM(). + +- New macros to access object members for PyFunction, PyCFunction +objects. + +- New APIs PyImport_AppendInittab() an PyImport_ExtendInittab() to +dynamically add one or many entries to the table of built-in modules. + +- New macro Py_InitModule3(name, methods, doc) which calls +Py_InitModule4() with appropriate arguments. (The -4 variant requires +you to pass an obscure version number constant which is always the same.) + +- New APIs PySys_WriteStdout() and PySys_WriteStderr() to write to +sys.stdout or sys.stderr using a printf-like interface. (Used in +_tkinter.c, for example.) + +- New APIs for conversion between Python longs and C 'long long' if +your compiler supports it. + +- PySequence_In() is now called PySequence_Contains(). +(PySequence_In() is still supported for b/w compatibility; it is +declared obsolete because its argument order is confusing.) + +- PyDict_GetItem() and PyDict_GetItemString() are changed so that they +*never* raise an exception -- (even if the hash() fails, simply clear +the error). This was necessary because there is lots of code out +there that already assumes this. + +- Changes to PySequence_Tuple() and PySequence_List() to use the +length of a sequence only as a hint -- if an IndexError happens +earlier, take that. (Formerly, this was considered an error.) + +- Reformatted abstract.c to give it a more familiar "look" and fixed +many error checking bugs. + +- Add NULL pointer checks to all calls of a C function through a type +object and extensions (e.g. nb_add). + +- The code that initializes sys.path now calls Py_GetPythonHome() +instead of getenv("PYTHONHOME"). This, together with the new API +Py_SetPythonHome(), makes it easier for embedding applications to +change the notion of Python's "home" directory (where the libraries +etc. are sought). + +- Fixed a very old bug in the parsing of "O?" format specifiers. + + +====================================================================== + + +======================================== +==> Release 1.5.1 (October 31, 1998) <== +======================================== + +From 1.5 to 1.5.1 +================= + +General +------- + +- The documentation is now unbundled. It has also been extensively +modified (mostly to implement a new and more uniform formatting +style). We figure that most people will prefer to download one of the +preformatted documentation sets (HTML, PostScript or PDF) and that +only a minority have a need for the LaTeX or FrameMaker sources. Of +course, the unbundled documentation sources still released -- just not +in the same archive file, and perhaps not on the same date. + +- All bugs noted on the errors page (and many unnoted) are fixed. All +new bugs take their places. + +- No longer a core dump when attempting to print (or repr(), or str()) +a list or dictionary that contains an instance of itself; instead, the +recursive entry is printed as [...] or {...}. See Py_ReprEnter() and +Py_ReprLeave() below. Comparisons of such objects still go beserk, +since this requires a different kind of fix; fortunately, this is a +less common scenario in practice. + +Syntax change +------------- + +- The raise statement can now be used without arguments, to re-raise +a previously set exception. This should be used after catching an +exception with an except clause only, either in the except clause or +later in the same function. + +Import and module handling +-------------------------- + +- The implementation of import has changed to use a mutex (when +threading is supported). This means that when two threads +simultaneously import the same module, the import statements are +serialized. Recursive imports are not affected. + +- Rewrote the finalization code almost completely, to be much more +careful with the order in which modules are destroyed. Destructors +will now generally be able to reference built-in names such as None +without trouble. + +- Case-insensitive platforms such as Mac and Windows require the case +of a module's filename to match the case of the module name as +specified in the import statement (see below). + +- The code for figuring out the default path now distinguishes between +files, modules, executable files, and directories. When expecting a +module, we also look for the .pyc or .pyo file. + +Parser/tokenizer changes +------------------------ + +- The tokenizer can now warn you when your source code mixes tabs and +spaces for indentation in a manner that depends on how much a tab is +worth in spaces. Use "python -t" or "python -v" to enable this +option. Use "python -tt" to turn the warnings into errors. (See also +tabnanny.py and tabpolice.py below.) + +- Return unsigned characters from tok_nextc(), so '\377' isn't +mistaken for an EOF character. + +- Fixed two pernicious bugs in the tokenizer that only affected AIX. +One was actually a general bug that was triggered by AIX's smaller I/O +buffer size. The other was a bug in the AIX optimizer's loop +unrolling code; swapping two statements made the problem go away. + +Tools, demos and miscellaneous files +------------------------------------ + +- There's a new version of Misc/python-mode.el (the Emacs mode for +Python) which is much smarter about guessing the indentation style +used in a particular file. Lots of other cool features too! + +- There are two new tools in Tools/scripts: tabnanny.py and +tabpolice.py, implementing two different ways of checking whether a +file uses indentation in a way that is sensitive to the interpretation +of a tab. The preferred module is tabnanny.py (by Tim Peters). + +- Some new demo programs: + + Demo/tkinter/guido/paint.py -- Dave Mitchell + Demo/sockets/unixserver.py -- Piet van Oostrum + + +- Much better freeze support. The freeze script can now freeze +hierarchical module names (with a corresponding change to import.c), +and has a few extra options (e.g. to suppress freezing specific +modules). It also does much more on Windows NT. + +- Version 1.0 of the faq wizard is included (only very small changes +since version 0.9.0). + +- New feature for the ftpmirror script: when removing local files +(i.e., only when -r is used), do a recursive delete. + +Configuring and building Python +------------------------------- + +- Get rid of the check for -linet -- recent Sequent Dynix systems don't +need this any more and apparently it screws up their configuration. + +- Some changes because gcc on SGI doesn't support '-all'. + +- Changed the build rules to use $(LIBRARY) instead of + -L.. -lpython$(VERSION) +since the latter trips up the SunOS 4.1.x linker (sigh). + +- Fix the bug where the '# dgux is broken' comment in the Makefile +tripped over Make on some platforms. + +- Changes for AIX: install the python.exp file; properly use +$(srcdir); the makexp_aix script now removes C++ entries of the form +Class::method. + +- Deleted some Makefile targets only used by the (long obsolete) +gMakefile hacks. + +Extension modules +----------------- + +- Performance and threading improvements to the socket and bsddb +modules, by Christopher Lindblad of Infoseek. + +- Added operator.__not__ and operator.not_. + +- In the thread module, when a thread exits due to an unhandled +exception, don't store the exception information in sys.last_*; it +prevents proper calling of destructors of local variables. + +- Fixed a number of small bugs in the cPickle module. + +- Changed find() and rfind() in the strop module so that +find("x","",2) returns -1, matching the implementation in string.py. + +- In the time module, be more careful with the result of ctime(), and +test for HAVE_MKTIME before usinmg mktime(). + +- Doc strings contributed by Mitch Chapman to the termios, pwd, gdbm +modules. + +- Added the LOG_SYSLOG constant to the syslog module, if defined. + +Standard library modules +------------------------ + +- All standard library modules have been converted to an indentation +style using either only tabs or only spaces -- never a mixture -- if +they weren't already consistent according to tabnanny. This means +that the new -t option (see above) won't complain about standard +library modules. + +- New standard library modules: + + threading -- GvR and the thread-sig + Java style thread objects -- USE THIS!!! + + getpass -- Piers Lauder + simple utilities to prompt for a password and to + retrieve the current username + + imaplib -- Piers Lauder + interface for the IMAP4 protocol + + poplib -- David Ascher, Piers Lauder + interface for the POP3 protocol + + smtplib -- Dragon De Monsyne + interface for the SMTP protocol + +- Some obsolete modules moved to a separate directory (Lib/lib-old) +which is *not* in the default module search path: + + Para + addpack + codehack + fmt + lockfile + newdir + ni + rand + tb + +- New version of the PCRE code (Perl Compatible Regular Expressions -- +the re module and the supporting pcre extension) by Andrew Kuchling. +Incompatible new feature in re.sub(): the handling of escapes in the +replacement string has changed. + +- Interface change in the copy module: a __deepcopy__ method is now +called with the memo dictionary as an argument. + +- Feature change in the tokenize module: differentiate between NEWLINE +token (an official newline) and NL token (a newline that the grammar +ignores). + +- Several bugfixes to the urllib module. It is now truly thread-safe, +and several bugs and a portability problem have been fixed. New +features, all due to Sjoerd Mullender: When creating a temporary file, +it gives it an appropriate suffix. Support the "data:" URL scheme. +The open() method uses the tempcache. + +- New version of the xmllib module (this time with a test suite!) by +Sjoerd Mullender. + +- Added debugging code to the telnetlib module, to be able to trace +the actual traffic. + +- In the rfc822 module, added support for deleting a header (still no +support for adding headers, though). Also fixed a bug where an +illegal address would cause a crash in getrouteaddr(), fixed a +sign reversal in mktime_tz(), and use the local timezone by default +(the latter two due to Bill van Melle). + +- The normpath() function in the dospath and ntpath modules no longer +does case normalization -- for that, use the separate function +normcase() (which always existed); normcase() has been sped up and +fixed (it was the cause of a crash in Mark Hammond's installer in +certain locales). + +- New command supported by the ftplib module: rmd(); also fixed some +minor bugs. + +- The profile module now uses a different timer function by default -- +time.clock() is generally better than os.times(). This makes it work +better on Windows NT, too. + +- The tempfile module now recovers when os.getcwd() raises an +exception. + +- Fixed some bugs in the random module; gauss() was subtly wrong, and +vonmisesvariate() should return a full circle. Courtesy Mike Miller, +Lambert Meertens (gauss()), and Magnus Kessler (vonmisesvariate()). + +- Better default seed in the whrandom module, courtesy Andrew Kuchling. + +- Fix slow close() in shelve module. + +- The Unix mailbox class in the mailbox module is now more robust when +a line begins with the string "From " but is definitely not the start +of a new message. The pattern used can be changed by overriding a +method or class variable. + +- Added a rmtree() function to the copy module. + +- Fixed several typos in the pickle module. Also fixed problems when +unpickling in restricted execution environments. + +- Added docstrings and fixed a typo in the py_compile and compileall +modules. At Mark Hammond's repeated request, py_compile now append a +newline to the source if it needs one. Both modules support an extra +parameter to specify the purported source filename (to be used in +error messages). + +- Some performance tweaks by Jeremy Hylton to the gzip module. + +- Fixed a bug in the merge order of dictionaries in the ConfigParser +module. Courtesy Barry Warsaw. + +- In the multifile module, support the optional second parameter to +seek() when possible. + +- Several fixes to the gopherlib module by Lars Marius Garshol. Also, +urlparse now correctly handles Gopher URLs with query strings. + +- Fixed a tiny bug in format_exception() in the traceback module. +Also rewrite tb_lineno() to be compatible with JPython (and not +disturb the current exception!); by Jim Hugunin. + +- The httplib module is more robust when servers send a short response +-- courtesy Tim O'Malley. + +Tkinter and friends +------------------- + +- Various typos and bugs fixed. + +- New module Tkdnd implements a drag-and-drop protocol (within one +application only). + +- The event_*() widget methods have been restructured slightly -- they +no longer use the default root. + +- The interfaces for the bind*() and unbind() widget methods have been +redesigned; the bind*() methods now return the name of the Tcl command +created for the callback, and this can be passed as a optional +argument to unbind() in order to delete the command (normally, such +commands are automatically unbound when the widget is destroyed, but +for some applications this isn't enough). + +- Variable objects now have trace methods to interface to Tcl's +variable tracing facilities. + +- Image objects now have an optional keyword argument, 'master', to +specify a widget (tree) to which they belong. The image_names() and +image_types() calls are now also widget methods. + +- There's a new global call, Tkinter.NoDefaultRoot(), which disables +all use of the default root by the Tkinter library. This is useful to +debug applications that are in the process of being converted from +relying on the default root to explicit specification of the root +widget. + +- The 'exit' command is deleted from the Tcl interpreter, since it +provided a loophole by which one could (accidentally) exit the Python +interpreter without invoking any cleanup code. + +- Tcl_Finalize() is now registered as a Python low-level exit handle, +so Tcl will be finalized when Python exits. + +The Python/C API +---------------- + +- New function PyThreadState_GetDict() returns a per-thread dictionary +intended for storing thread-local global variables. + +- New functions Py_ReprEnter() and Py_ReprLeave() use the per-thread +dictionary to allow recursive container types to detect recursion in +their repr(), str() and print implementations. + +- New function PyObject_Not(x) calculates (not x) according to Python's +standard rules (basically, it negates the outcome PyObject_IsTrue(x). + +- New function _PyModule_Clear(), which clears a module's dictionary +carefully without removing the __builtins__ entry. This is implied +when a module object is deallocated (this used to clear the dictionary +completely). + +- New function PyImport_ExecCodeModuleEx(), which extends +PyImport_ExecCodeModule() by adding an extra parameter to pass it the +true file. + +- New functions Py_GetPythonHome() and Py_SetPythonHome(), intended to +allow embedded applications to force a different value for PYTHONHOME. + +- New global flag Py_FrozenFlag is set when this is a "frozen" Python +binary; it suppresses warnings about not being able to find the +standard library directories. + +- New global flag Py_TabcheckFlag is incremented by the -t option and +causes the tokenizer to issue warnings or errors about inconsistent +mixing of tabs and spaces for indentation. + +Miscellaneous minor changes and bug fixes +----------------------------------------- + +- Improved the error message when an attribute of an attribute-less +object is requested -- include the name of the attribute and the type +of the object in the message. + +- Sped up int(), long(), float() a bit. + +- Fixed a bug in list.sort() that would occasionally dump core. + +- Fixed a bug in PyNumber_Power() that caused numeric arrays to fail +when taken tothe real power. + +- Fixed a number of bugs in the file reading code, at least one of +which could cause a core dump on NT, and one of which would +occasionally cause file.read() to return less than the full contents +of the file. + +- Performance hack by Vladimir Marangozov for stack frame creation. + +- Make sure setvbuf() isn't used unless HAVE_SETVBUF is defined. + +Windows 95/NT +------------- + +- The .lib files are now part of the distribution; they are collected +in the subdirectory "libs" of the installation directory. + +- The extension modules (.pyd files) are now collected in a separate +subdirectory of the installation directory named "DLLs". + +- The case of a module's filename must now match the case of the +module name as specified in the import statement. This is an +experimental feature -- if it turns out to break in too many +situations, it will be removed (or disabled by default) in the future. +It can be disabled on a per-case basis by setting the environment +variable PYTHONCASEOK (to any value). + + +====================================================================== + + +===================================== +==> Release 1.5 (January 3, 1998) <== +===================================== + + +From 1.5b2 to 1.5 +================= + +- Newly documentated module: BaseHTTPServer.py, thanks to Greg Stein. + +- Added doc strings to string.py, stropmodule.c, structmodule.c, +thanks to Charles Waldman. + +- Many nits fixed in the manuals, thanks to Fred Drake and many others +(especially Rob Hooft and Andrew Kuchling). The HTML version now uses +HTML markup instead of inline GIF images for tables; only two images +are left (for obsure bits of math). The index of the HTML version has +also been much improved. Finally, it is once again possible to +generate an Emacs info file from the library manual (but I don't +commit to supporting this in future versions). + +- New module: telnetlib.py (a simple telnet client library). + +- New tool: Tools/versioncheck/, by Jack Jansen. + +- Ported zlibmodule.c and bsddbmodule.c to NT; The project file for MS +DevStudio 5.0 now includes new subprojects to build the zlib and bsddb +extension modules. + +- Many small changes again to Tkinter.py -- mostly bugfixes and adding +missing routines. Thanks to Greg McFarlane for reporting a bunch of +problems and proofreading my fixes. + +- The re module and its documentation are up to date with the latest +version released to the string-sig (Dec. 22). + +- Stop test_grp.py from failing when the /etc/group file is empty +(yes, this happens!). + +- Fix bug in integer conversion (mystrtoul.c) that caused +4294967296==0 to be true! + +- The VC++ 4.2 project file should be complete again. + +- In tempfile.py, use a better template on NT, and add a new optional +argument "suffix" with default "" to specify a specific extension for +the temporary filename (needed sometimes on NT but perhaps also handy +elsewhere). + +- Fixed some bugs in the FAQ wizard, and converted it to use re +instead of regex. + +- Fixed a mysteriously undetected error in dlmodule.c (it was using a +totally bogus routine name to raise an exception). + +- Fixed bug in import.c which wasn't using the new "dos-8x3" name yet. + +- Hopefully harmless changes to the build process to support shared +libraries on DG/UX. This adds a target to create +libpython$(VERSION).so; however this target is *only* for DG/UX. + +- Fixed a bug in the new format string error checking in getargs.c. + +- A simple fix for infinite recursion when printing __builtins__: +reset '_' to None before printing and set it to the printed variable +*after* printing (and only when printing is successful). + +- Fixed lib-tk/SimpleDialog.py to keep the dialog visible even if the +parent window is not (Skip Montanaro). + +- Fixed the two most annoying problems with ftp URLs in +urllib.urlopen(); an empty file now correctly raises an error, and it +is no longer required to explicitly close the returned "file" object +before opening another ftp URL to the same host and directory. + + +====================================================================== + + +From 1.5b1 to 1.5b2 +=================== + +- Fixed a bug in cPickle.c that caused it to crash right away because +the version string had a different format. + +- Changes in pickle.py and cPickle.c: when unpickling an instance of a +class that doesn't define the __getinitargs__() method, the __init__() +constructor is no longer called. This makes a much larger group of +classes picklable by default, but may occasionally change semantics. +To force calling __init__() on unpickling, define a __getinitargs__() +method. Other changes too, in particular cPickle now handles classes +defined in packages correctly. The same change applies to copying +instances with copy.py. The cPickle.c changes and some pickle.py +changes are courtesy Jim Fulton. + +- Locale support in he "re" (Perl regular expressions) module. Use +the flag re.L (or re.LOCALE) to enable locale-specific matching +rules for \w and \b. The in-line syntax for this flag is (?L). + +- The built-in function isinstance(x, y) now also succeeds when y is +a type object and type(x) is y. + +- repr() and str() of class and instance objects now reflect the +package/module in which the class is defined. + +- Module "ni" has been removed. (If you really need it, it's been +renamed to "ni1". Let me know if this causes any problems for you. +Package authors are encouraged to write __init__.py files that +support both ni and 1.5 package support, so the same version can be +used with Python 1.4 as well as 1.5.) + +- The thread module is now automatically included when threads are +configured. (You must remove it from your existing Setup file, +since it is now in its own Setup.thread file.) + +- New command line option "-x" to skip the first line of the script; +handy to make executable scripts on non-Unix platforms. + +- In importdl.c, add the RTLD_GLOBAL to the dlopen() flags. I +haven't checked how this affects things, but it should make symbols +in one shared library available to the next one. + +- The Windows installer now installs in the "Program Files" folder on +the proper volume by default. + +- The Windows configuration adds a new main program, "pythonw", and +registers a new extension, ".pyw" that invokes this. This is a +pstandard Python interpreter that does not pop up a console window; +handy for pure Tkinter applications. All output to the original +stdout and stderr is lost; reading from the original stdin yields +EOF. Also, both python.exe and pythonw.exe now have a pretty icon +(a green snake in a box, courtesy Mark Hammond). + +- Lots of improvements to emacs-mode.el again. See Barry's web page: +http://www.python.org/ftp/emacs/pmdetails.html. + +- Lots of improvements and additions to the library reference manual; +many by Fred Drake. + +- Doc strings for the following modules: rfc822.py, posixpath.py, +ntpath.py, httplib.py. Thanks to Mitch Chapman and Charles Waldman. + +- Some more regression testing. + +- An optional 4th (maxsplit) argument to strop.replace(). + +- Fixed handling of maxsplit in string.splitfields(). + +- Tweaked os.environ so it can be pickled and copied. + +- The portability problems caused by indented preprocessor commands +and C++ style comments should be gone now. + +- In random.py, added Pareto and Weibull distributions. + +- The crypt module is now disabled in Modules/Setup.in by default; it +is rarely needed and causes errors on some systems where users often +don't know how to deal with those. + +- Some improvements to the _tkinter build line suggested by Case Roole. + +- A full suite of platform specific files for NetBSD 1.x, submitted by +Anders Andersen. + +- New Solaris specific header STROPTS.py. + +- Moved a confusing occurrence of *shared* from the comments in +Modules/Setup.in (people would enable this one instead of the real +one, and get disappointing results). + +- Changed the default mode for directories to be group-writable when +the installation process creates them. + +- Check for pthread support in "-l_r" for FreeBSD/NetBSD, and support +shared libraries for both. + +- Support FreeBSD and NetBSD in posixfile.py. + +- Support for the "event" command, new in Tk 4.2. By Case Roole. + +- Add Tix_SafeInit() support to tkappinit.c. + +- Various bugs fixed in "re.py" and "pcre.c". + +- Fixed a bug (broken use of the syntax table) in the old "regexpr.c". + +- In frozenmain.c, stdin is made unbuffered too when PYTHONUNBUFFERED +is set. + +- Provide default blocksize for retrbinary in ftplib.py (Skip +Montanaro). + +- In NT, pick the username up from different places in user.py (Jeff +Bauer). + +- Patch to urlparse.urljoin() for ".." and "..#1", Marc Lemburg. + +- Many small improvements to Jeff Rush' OS/2 support. + +- ospath.py is gone; it's been obsolete for so many years now... + +- The reference manual is now set up to prepare better HTML (still +using webmaker, alas). + +- Add special handling to /Tools/freeze for Python modules that are +imported implicitly by the Python runtime: 'site' and 'exceptions'. + +- Tools/faqwiz 0.8.3 -- add an option to suppress URL processing +inside <PRE>, by "Scott". + +- Added ConfigParser.py, a generic parser for sectioned configuration +files. + +- In _localemodule.c, LC_MESSAGES is not always defined; put it +between #ifdefs. + +- Typo in resource.c: RUSAGE_CHILDERN -> RUSAGE_CHILDREN. + +- Demo/scripts/newslist.py: Fix the way the version number is gotten +out of the RCS revision. + +- PyArg_Parse[Tuple] now explicitly check for bad characters at the +end of the format string. + +- Revamped PC/example_nt to support VC++ 5.x. + +- <listobject>.sort() now uses a modified quicksort by Raymund Galvin, +after studying the GNU libg++ quicksort. This should be much faster +if there are lots of duplicates, and otherwise at least as good. + +- Added "uue" as an alias for "uuencode" to mimetools.py. (Hm, the +uudecode bug where it complaints about trailing garbage is still there +:-( ). + +- pickle.py requires integers in text mode to be in decimal notation +(it used to accept octal and hex, even though it would only generate +decimal numbers). + +- In string.atof(), don't fail when the "re" module is unavailable. +Plug the ensueing security leak by supplying an empty __builtins__ +directory to eval(). + +- A bunch of small fixes and improvements to Tkinter.py. + +- Fixed a buffer overrun in PC/getpathp.c. + + +====================================================================== + + +From 1.5a4 to 1.5b1 +=================== + +- The Windows NT/95 installer now includes full HTML of all manuals. +It also has a checkbox that lets you decide whether to install the +interpreter and library. The WISE installer script for the installer +is included in the source tree as PC/python15.wse, and so are the +icons used for Python files. The config.c file for the Windows build +is now complete with the pcre module. + +- sys.ps1 and sys.ps2 can now arbitrary objects; their str() is +evaluated for the prompt. + +- The reference manual is brought up to date (more or less -- it still +needs work, e.g. in the area of package import). + +- The icons used by latex2html are now included in the Doc +subdirectory (mostly so that tarring up the HTML files can be fully +automated). A simple index.html is also added to Doc (it only works +after you have successfully run latex2html). + +- For all you would-be proselytizers out there: a new version of +Misc/BLURB describes Python more concisely, and Misc/comparisons +compares Python to several other languages. Misc/BLURB.WINDOWS +contains a blurb specifically aimed at Windows programmers (by Mark +Hammond). + +- A new version of the Python mode for Emacs is included as +Misc/python-mode.el. There are too many new features to list here. +See http://www.python.org/ftp/emacs/pmdetails.html for more info. + +- New module fileinput makes iterating over the lines of a list of +files easier. (This still needs some more thinking to make it more +extensible.) + +- There's full OS/2 support, courtesy Jeff Rush. To build the OS/2 +version, see PC/readme.txt and PC/os2vacpp. This is for IBM's Visual +Age C++ compiler. I expect that Jeff will also provide a binary +release for this platform. + +- On Linux, the configure script now uses '-Xlinker -export-dynamic' +instead of '-rdynamic' to link the main program so that it exports its +symbols to shared libraries it loads dynamically. I hope this doesn't +break on older Linux versions; it is needed for mklinux and appears to +work on Linux 2.0.30. + +- Some Tkinter resstructuring: the geometry methods that apply to a +master are now properly usable on toplevel master widgets. There's a +new (internal) widget class, BaseWidget. New, longer "official" names +for the geometry manager methods have been added, +e.g. "grid_columnconfigure()" instead of "columnconfigure()". The old +shorter names still work, and where there's ambiguity, pack wins over +place wins over grid. Also, the bind_class method now returns its +value. + +- New, RFC-822 conformant parsing of email addresses and address lists +in the rfc822 module, courtesy Ben Escoto. + +- New, revamped tkappinit.c with support for popular packages (PIL, +TIX, BLT, TOGL). For the last three, you need to execute the Tcl +command "load {} Tix" (or Blt, or Togl) to gain access to them. +The Modules/Setup line for the _tkinter module has been rewritten +using the cool line-breaking feature of most Bourne shells. + +- New socket method connect_ex() returns the error code from connect() +instead of raising an exception on errors; this makes the logic +required for asynchronous connects simpler and more efficient. + +- New "locale" module with (still experimental) interface to the +standard C library locale interface, courtesy Martin von Loewis. This +does not repeat my mistake in 1.5a4 of always calling +setlocale(LC_ALL, ""). In fact, we've pretty much decided that +Python's standard numerical formatting operations should always use +the conventions for the C locale; the locale module contains utility +functions to format numbers according to the user specified locale. +(All this is accomplished by an explicit call to setlocale(LC_NUMERIC, +"C") after locale-changing calls.) See the library manual. (Alas, the +promised changes to the "re" module for locale support have not been +materialized yet. If you care, volunteer!) + +- Memory leak plugged in Py_BuildValue when building a dictionary. + +- Shared modules can now live inside packages (hierarchical module +namespaces). No changes to the shared module itself are needed. + +- Improved policy for __builtins__: this is a module in __main__ and a +dictionary everywhere else. + +- Python no longer catches SIGHUP and SIGTERM by default. This was +impossible to get right in the light of thread contexts. If you want +your program to clean up when a signal happens, use the signal module +to set up your own signal handler. + +- New Python/C API PyNumber_CoerceEx() does not return an exception +when no coercion is possible. This is used to fix a problem where +comparing incompatible numbers for equality would raise an exception +rather than return false as in Python 1.4 -- it once again will return +false. + +- The errno module is changed again -- the table of error messages +(errorstr) is removed. Instead, you can use os.strerror(). This +removes redundance and a potential locale dependency. + +- New module xmllib, to parse XML files. By Sjoerd Mullender. + +- New C API PyOS_AfterFork() is called after fork() in posixmodule.c. +It resets the signal module's notion of what the current process ID +and thread are, so that signal handlers will work after (and across) +calls to os.fork(). + +- Fixed most occurrences of fatal errors due to missing thread state. + +- For vgrind (a flexible source pretty printer) fans, there's a simple +Python definition in Misc/vgrindefs, courtesy Neale Pickett. + +- Fixed memory leak in exec statement. + +- The test.pystone module has a new function, pystones(loops=LOOPS), +which returns a (benchtime, stones) tuple. The main() function now +calls this and prints the report. + +- Package directories now *require* the presence of an __init__.py (or +__init__.pyc) file before they are considered as packages. This is +done to prevent accidental subdirectories with common names from +overriding modules with the same name. + +- Fixed some strange exceptions in __del__ methods in library modules +(e.g. urllib). This happens because the builtin names are already +deleted by the time __del__ is called. The solution (a hack, but it +works) is to set some instance variables to 0 instead of None. + +- The table of built-in module initializers is replaced by a pointer +variable. This makes it possible to switch to a different table at +run time, e.g. when a collection of modules is loaded from a shared +library. (No example code of how to do this is given, but it is +possible.) The table is still there of course, its name prefixed with +an underscore and used to initialize the pointer. + +- The warning about a thread still having a frame now only happens in +verbose mode. + +- Change the signal finialization so that it also resets the signal +handlers. After this has been called, our signal handlers are no +longer active! + +- New version of tokenize.py (by Ka-Ping Yee) recognizes raw string +literals. There's now also a test fort this module. + +- The copy module now also uses __dict__.update(state) instead of +going through individual attribute assignments, for class instances +without a __setstate__ method. + +- New module reconvert translates old-style (regex module) regular +expressions to new-style (re module, Perl-style) regular expressions. + +- Most modules that used to use the regex module now use the re +module. The grep module has a new pgrep() function which uses +Perl-style regular expressions. + +- The (very old, backwards compatibility) regexp.py module has been +deleted. + +- Restricted execution (rexec): added the pcre module (support for the +re module) to the list of trusted extension modules. + +- New version of Jim Fulton's CObject object type, adds +PyCObject_FromVoidPtrAndDesc() and PyCObject_GetDesc() APIs. + +- Some patches to Lee Busby's fpectl mods that accidentally didn't +make it into 1.5a4. + +- In the string module, add an optional 4th argument to count(), +matching find() etc. + +- Patch for the nntplib module by Charles Waldman to add optional user +and password arguments to NNTP.__init__(), for nntp servers that need +them. + +- The str() function for class objects now returns +"modulename.classname" instead of returning the same as repr(). + +- The parsing of \xXX escapes no longer relies on sscanf(). + +- The "sharedmodules" subdirectory of the installation is renamed to +"lib-dynload". (You may have to edit your Modules/Setup file to fix +this in an existing installation!) + +- Fixed Don Beaudry's mess-up with the OPT test in the configure +script. Certain SGI platforms will still issue a warning for each +compile; there's not much I can do about this since the compiler's +exit status doesn't indicate that I was using an obsolete option. + +- Fixed Barry's mess-up with {}.get(), and added test cases for it. + +- Shared libraries didn't quite work under AIX because of the change +in status of the GNU readline interface. Fix due to by Vladimir +Marangozov. + + +====================================================================== + + +From 1.5a3 to 1.5a4 +=================== + +- faqwiz.py: version 0.8; Recognize https:// as URL; <html>...</html> +feature; better install instructions; removed faqmain.py (which was an +older version). + +- nntplib.py: Fixed some bugs reported by Lars Wirzenius (to Debian) +about the treatment of lines starting with '.'. Added a minimal test +function. + +- struct module: ignore most whitespace in format strings. + +- urllib.py: close the socket and temp file in URLopener.retrieve() so +that multiple retrievals using the same connection work. + +- All standard exceptions are now classes by default; use -X to make +them strings (for backward compatibility only). + +- There's a new standard exception hierarchy, defined in the standard +library module exceptions.py (which you never need to import +explicitly). See +http://grail.cnri.reston.va.us/python/essays/stdexceptions.html for +more info. + +- Three new C API functions: + + - int PyErr_GivenExceptionMatches(obj1, obj2) + + Returns 1 if obj1 and obj2 are the same object, or if obj1 is an + instance of type obj2, or of a class derived from obj2 + + - int PyErr_ExceptionMatches(obj) + + Higher level wrapper around PyErr_GivenExceptionMatches() which uses + PyErr_Occurred() as obj1. This will be the more commonly called + function. + + - void PyErr_NormalizeException(typeptr, valptr, tbptr) + + Normalizes exceptions, and places the normalized values in the + arguments. If type is not a class, this does nothing. If type is a + class, then it makes sure that value is an instance of the class by: + + 1. if instance is of the type, or a class derived from type, it does + nothing. + + 2. otherwise it instantiates the class, using the value as an + argument. If value is None, it uses an empty arg tuple, and if + the value is a tuple, it uses just that. + +- Another new C API function: PyErr_NewException() creates a new +exception class derived from Exception; when -X is given, it creates a +new string exception. + +- core interpreter: remove the distinction between tuple and list +unpacking; allow an arbitrary sequence on the right hand side of any +unpack instruction. (UNPACK_LIST and UNPACK_TUPLE now do the same +thing, which should really be called UNPACK_SEQUENCE.) + +- classes: Allow assignments to an instance's __dict__ or __class__, +so you can change ivars (including shared ivars -- shock horror) and +change classes dynamically. Also make the check on read-only +attributes of classes less draconic -- only the specials names +__dict__, __bases__, __name__ and __{get,set,del}attr__ can't be +assigned. + +- Two new built-in functions: issubclass() and isinstance(). Both +take classes as their second arguments. The former takes a class as +the first argument and returns true iff first is second, or is a +subclass of second. The latter takes any object as the first argument +and returns true iff first is an instance of the second, or any +subclass of second. + +- configure: Added configuration tests for presence of alarm(), +pause(), and getpwent(). + +- Doc/Makefile: changed latex2html targets. + +- classes: Reverse the search order for the Don Beaudry hook so that +the first class with an applicable hook wins. Makes more sense. + +- Changed the checks made in Py_Initialize() and Py_Finalize(). It is +now legal to call these more than once. The first call to +Py_Initialize() initializes, the first call to Py_Finalize() +finalizes. There's also a new API, Py_IsInitalized() which checks +whether we are already initialized (in case you want to leave things +as they were). + +- Completely disable the declarations for malloc(), realloc() and +free(). Any 90's C compiler has these in header files, and the tests +to decide whether to suppress the declarations kept failing on some +platforms. + +- *Before* (instead of after) signalmodule.o is added, remove both +intrcheck.o and sigcheck.o. This should get rid of warnings in ar or +ld on various systems. + +- Added reop to PC/config.c + +- configure: Decided to use -Aa -D_HPUX_SOURCE on HP-UX platforms. +Removed outdated HP-UX comments from README. Added Cray T3E comments. + +- Various renames of statically defined functions that had name +conflicts on some systems, e.g. strndup (GNU libc), join (Cray), +roundup (sys/types.h). + +- urllib.py: Interpret three slashes in file: URL as local file (for +Netscape on Windows/Mac). + +- copy.py: Make sure the objects returned by __getinitargs__() are +kept alive (in the memo) to avoid a certain kind of nasty crash. (Not +easily reproducable because it requires a later call to +__getinitargs__() to return a tuple that happens to be allocated at +the same address.) + +- Added definition of AR to toplevel Makefile. Renamed @buildno temp +file to buildno1. + +- Moved Include/assert.h to Parser/assert.h, which seems to be the +only place where it's needed. + +- Tweaked the dictionary lookup code again for some more speed +(Vladimir Marangozov). + +- NT build: Changed the way python15.lib is included in the other +projects. Per Mark Hammond's suggestion, add it to the extra libs in +Settings instead of to the project's source files. + +- regrtest.py: Change default verbosity so that there are only three +levels left: -q, default and -v. In default mode, the name of each +test is now printed. -v is the same as the old -vv. -q is more quiet +than the old default mode. + +- Removed the old FAQ from the distribution. You now have to get it +from the web! + +- Removed the PC/make_nt.in file from the distribution; it is no +longer needed. + +- Changed the build sequence so that shared modules are built last. +This fixes things for AIX and doesn't hurt elsewhere. + +- Improved test for GNU MP v1 in mpzmodule.c + +- fileobject.c: ftell() on Linux discards all buffered data; changed +read() code to use lseek() instead to get the same effect + +- configure.in, configure, importdl.c: NeXT sharedlib fixes + +- tupleobject.c: PyTuple_SetItem asserts refcnt==1 + +- resource.c: Different strategy regarding whether to declare +getrusage() and getpagesize() -- #ifdef doesn't work, Linux has +conflicting decls in its headers. Choice: only declare the return +type, not the argument prototype, and not on Linux. + +- importdl.c, configure*: set sharedlib extensions properly for NeXT + +- configure*, Makefile.in, Modules/Makefile.pre.in: AIX shared libraries +fixed; moved addition of PURIFY to LINKCC to configure + +- reopmodule.c, regexmodule.c, regexpr.c, zlibmodule.c: needed casts +added to shup up various compilers. + +- _tkinter.c: removed buggy mac #ifndef + +- Doc: various Mac documentation changes, added docs for 'ic' module + +- PC/make_nt.in: deleted + +- test_time.py, test_strftime.py: tweaks to catch %Z (which may return +"") + +- test_rotor.py: print b -> print `b` + +- Tkinter.py: (tagOrId) -> (tagOrId,) + +- Tkinter.py: the Tk class now also has a configure() method and +friends (they have been moved to the Misc class to accomplish this). + +- dict.get(key[, default]) returns dict[key] if it exists, or default +if it doesn't. The default defaults to None. This is quicker for +some applications than using either has_key() or try:...except +KeyError:.... + +- Tools/webchecker/: some small changes to webchecker.py; added +websucker.py (a simple web site mirroring script). + +- Dictionary objects now have a get() method (also in UserDict.py). +dict.get(key, default) returns dict[key] if it exists and default +otherwise; default defaults to None. + +- Tools/scripts/logmerge.py: print the author, too. + +- Changes to import: support for "import a.b.c" is now built in. See +http://grail.cnri.reston.va.us/python/essays/packages.html +for more info. Most important deviations from "ni.py": __init__.py is +executed in the package's namespace instead of as a submodule; and +there's no support for "__" or "__domain__". Note that "ni.py" is not +changed to match this -- it is simply declared obsolete (while at the +same time, it is documented...:-( ). +Unfortunately, "ihooks.py" has not been upgraded (but see "knee.py" +for an example implementation of hierarchical module import written in +Python). + +- More changes to import: the site.py module is now imported by +default when Python is initialized; use -S to disable it. The site.py +module extends the path with several more directories: site-packages +inside the lib/python1.5/ directory, site-python in the lib/ +directory, and pathnames mentioned in *.pth files found in either of +those directories. See +http://grail.cnri.reston.va.us/python/essays/packages.html +for more info. + +- Changes to standard library subdirectory names: those subdirectories +that are not packages have been renamed with a hypen in their name, +e.g. lib-tk, lib-stdwin, plat-win, plat-linux2, plat-sunos5, dos-8x3. +The test suite is now a package -- to run a test, you must now use +"import test.test_foo". + +- A completely new re.py module is provided (thanks to Andrew +Kuchling, Tim Peters and Jeffrey Ollie) which uses Philip Hazel's +"pcre" re compiler and engine. For a while, the "old" re.py (which +was new in 1.5a3!) will be kept around as re1.py. The "old" regex +module and underlying parser and engine are still present -- while +regex is now officially obsolete, it will probably take several major +release cycles before it can be removed. + +- The posix module now has a strerror() function which translates an +error code to a string. + +- The emacs.py module (which was long obsolete) has been removed. + +- The universal makefile Misc/Makefile.pre.in now features an +"install" target. By default, installed shared libraries go into +$exec_prefix/lib/python$VERSION/site-packages/. + +- The install-sh script is installed with the other configuration +specific files (in the config/ subdirectory). + +- It turns out whatsound.py and sndhdr.py were identical modules. +Since there's also an imghdr.py file, I propose to make sndhdr.py the +official one. For compatibility, whatsound.py imports * from +sndhdr.py. + +- Class objects have a new attribute, __module__, giving the name of +the module in which they were declared. This is useful for pickle and +for printing the full name of a class exception. + +- Many extension modules no longer issue a fatal error when their +initialization fails; the importing code now checks whether an error +occurred during module initialization, and correctly propagates the +exception to the import statement. + +- Most extension modules now raise class-based exceptions (except when +-X is used). + +- Subtle changes to PyEval_{Save,Restore}Thread(): always swap the +thread state -- just don't manipulate the lock if it isn't there. + +- Fixed a bug in Python/getopt.c that made it do the wrong thing when +an option was a single '-'. Thanks to Andrew Kuchling. + +- New module mimetypes.py will guess a MIME type from a filename's +extension. + +- Windows: the DLL version is now settable via a resource rather than +being hardcoded. This can be used for "branding" a binary Python +distribution. + +- urllib.py is now threadsafe -- it now uses re instead of regex, and +sys.exc_info() instead of sys.exc_{type,value}. + +- Many other library modules that used to use +sys.exc_{type,value,traceback} are now more thread-safe by virtue of +using sys.exc_info(). + +- The functions in popen2 have an optional buffer size parameter. +Also, the command argument can now be either a string (passed to the +shell) or a list of arguments (passed directly to execv). + +- Alas, the thread support for _tkinter released with 1.5a3 didn't +work. It's been rewritten. The bad news is that it now requires a +modified version of a file in the standard Tcl distribution, which you +must compile with a -I option pointing to the standard Tcl source +tree. For this reason, the thread support is disabled by default. + +- The errno extension module adds two tables: errorcode maps errno +numbers to errno names (e.g. EINTR), and errorstr maps them to +message strings. (The latter is redundant because the new call +posix.strerror() now does the same, but alla...) (Marc-Andre Lemburg) + +- The readline extension module now provides some interfaces to +internal readline routines that make it possible to write a completer +in Python. An example completer, rlcompleter.py, is provided. + + When completing a simple identifier, it completes keywords, + built-ins and globals in __main__; when completing + NAME.NAME..., it evaluates (!) the expression up to the last + dot and completes its attributes. + + It's very cool to do "import string" type "string.", hit the + completion key (twice), and see the list of names defined by + the string module! + + Tip: to use the tab key as the completion key, call + + readline.parse_and_bind("tab: complete") + +- The traceback.py module has a new function tb_lineno() by Marc-Andre +Lemburg which extracts the line number from the linenumber table in +the code object. Apparently the traceback object doesn't contains the +right linenumber when -O is used. Rather than guessing whether -O is +on or off, the module itself uses tb_lineno() unconditionally. + +- Fixed Demo/tkinter/matt/canvas-moving-or-creating.py: change bind() +to tag_bind() so it works again. + +- The pystone script is now a standard library module. Example use: +"import test.pystone; test.pystone.main()". + +- The import of the readline module in interactive mode is now also +attempted when -i is specified. (Yes, I know, giving in to Marc-Andre +Lemburg, who asked for this. :-) + +- rfc822.py: Entirely rewritten parseaddr() function by Sjoerd +Mullender, to be closer to the standard. This fixes the getaddr() +method. Unfortunately, getaddrlist() is as broken as ever, since it +splits on commas without regard for RFC 822 quoting conventions. + +- pprint.py: correctly emit trailing "," in singleton tuples. + +- _tkinter.c: export names for its type objects, TkappType and +TkttType. + +- pickle.py: use __module__ when defined; fix a particularly hard to +reproduce bug that confuses the memo when temporary objects are +returned by custom pickling interfaces; and a semantic change: when +unpickling the instance variables of an instance, use +inst.__dict__.update(value) instead of a for loop with setattr() over +the value.keys(). This is more consistent (the pickling doesn't use +getattr() either but pickles inst.__dict__) and avoids problems with +instances that have a __setattr__ hook. But it *is* a semantic change +(because the setattr hook is no longer used). So beware! + +- config.h is now installed (at last) in +$exec_prefix/include/python1.5/. For most sites, this means that it +is actually in $prefix/include/python1.5/, with all the other Python +include files, since $prefix and $exec_prefix are the same by +default. + +- The imp module now supports parts of the functionality to implement +import of hierarchical module names. It now supports find_module() +and load_module() for all types of modules. Docstrings have been +added for those functions in the built-in imp module that are still +relevant (some old interfaces are obsolete). For a sample +implementation of hierarchical module import in Python, see the new +library module knee.py. + +- The % operator on string objects now allows arbitrary nested parens +in a %(...)X style format. (Brad Howes) + +- Reverse the order in which Setup and Setup.local are passed to the +makesetup script. This allows variable definitions in Setup.local to +override definitions in Setup. (But you'll still have to edit Setup +if you want to disable modules that are enabled by default, or if such +modules need non-standard options.) + +- Added PyImport_ImportModuleEx(name, globals, locals, fromlist); this +is like PyImport_ImporModule(name) but receives the globals and locals +dict and the fromlist arguments as well. (The name is a char*; the +others are PyObject*s). + +- The 'p' format in the struct extension module alloded to above is +new in 1.5a4. + +- The types.py module now uses try-except in a few places to make it +more likely that it can be imported in restricted mode. Some type +names are undefined in that case, e.g. CodeType (inaccessible), +FileType (not always accessible), and TracebackType and FrameType +(inaccessible). + +- In urllib.py: added separate administration of temporary files +created y URLopener.retrieve() so cleanup() can properly remove them. +The old code removed everything in tempcache which was a bad idea if +the user had passed a non-temp file into it. Also, in basejoin(), +interpret relative paths starting in "../". This is necessary if the +server uses symbolic links. + +- The Windows build procedure and project files are now based on +Microsoft Visual C++ 5.x. The build now takes place in the PCbuild +directory. It is much more robust, and properly builds separate Debug +and Release versions. (The installer will be added shortly.) + +- Added casts and changed some return types in regexpr.c to avoid +compiler warnings or errors on some platforms. + +- The AIX build tools for shared libraries now supports VPATH. (Donn +Cave) + +- By default, disable the "portable" multimedia modules audioop, +imageop, and rgbimg, since they don't work on 64-bit platforms. + +- Fixed a nasty bug in cStringIO.c when code was actually using the +close() method (the destructors would try to free certain fields a +second time). + +- For those who think they need it, there's a "user.py" module. This +is *not* imported by default, but can be imported to run user-specific +setup commands, ~/.pythonrc.py. + +- Various speedups suggested by Fredrik Lundh, Marc-Andre Lemburg, +Vladimir Marangozov, and others. + +- Added os.altsep; this is '/' on DOS/Windows, and None on systems +with a sane filename syntax. + +- os.py: Write out the dynamic OS choice, to avoid exec statements. +Adding support for a new OS is now a bit more work, but I bet that +'dos' or 'nt' will cover most situations... + +- The obsolete exception AccessError is now really gone. + +- Tools/faqwiz/: New installation instructions show how to maintain +multiple FAQs. Removed bootstrap script from end of faqwiz.py module. +Added instructions to bootstrap script, too. Version bumped to 0.8.1. +Added <html>...</html> feature suggested by Skip Montanaro. Added +leading text for Roulette, default to 'Hit Reload ...'. Fix typo in +default SRCDIR. + +- Documentation for the relatively new modules "keyword" and "symbol" +has been added (to the end of the section on the parser extension +module). + +- In module bisect.py, but functions have two optional argument 'lo' +and 'hi' which allow you to specify a subsequence of the array to +operate on. + +- In ftplib.py, changed most methods to return their status (even when +it is always "200 OK") rather than swallowing it. + +- main() now calls setlocale(LC_ALL, ""), if setlocale() and +<locale.h> are defined. + +- Changes to configure.in, the configure script, and both +Makefile.pre.in files, to support SGI's SGI_ABI platform selection +environment variable. + + +====================================================================== + + +From 1.4 to 1.5a3 +================= + +Security +-------- + +- If you are using the setuid script C wrapper (Misc/setuid-prog.c), +please use the new version. The old version has a huge security leak. + +Miscellaneous +------------- + +- Because of various (small) incompatible changes in the Python +bytecode interpreter, the magic number for .pyc files has changed +again. + +- The default module search path is now much saner. Both on Unix and +Windows, it is essentially derived from the path to the executable +(which can be overridden by setting the environment variable +$PYTHONHOME). The value of $PYTHONPATH on Windows is now inserted in +front of the default path, like in Unix (instead of overriding the +default path). On Windows, the directory containing the executable is +added to the end of the path. + +- A new version of python-mode.el for Emacs has been included. Also, +a new file ccpy-style.el has been added to configure Emacs cc-mode for +the preferred style in Python C sources. + +- On Unix, when using sys.argv[0] to insert the script directory in +front of sys.path, expand a symbolic link. You can now install a +program in a private directory and have a symbolic link to it in a +public bin directory, and it will put the private directory in the +module search path. Note that the symlink is expanded in sys.path[0] +but not in sys.argv[0], so you can still tell the name by which you +were invoked. + +- It is now recommended to use ``#!/usr/bin/env python'' instead of +``#!/usr/local/bin/python'' at the start of executable scripts, except +for CGI scripts. It has been determined that the use of /usr/bin/env +is more portable than that of /usr/local/bin/python -- scripts almost +never have to be edited when the Python interpreter lives in a +non-standard place. Note that this doesn't work for CGI scripts since +the python executable often doesn't live in the HTTP server's default +search path. + +- The silly -s command line option and the corresponding +PYTHONSUPPRESS environment variable (and the Py_SuppressPrint global +flag in the Python/C API) are gone. + +- Most problems on 64-bit platforms should now be fixed. Andrew +Kuchling helped. Some uncommon extension modules are still not +clean (image and audio ops?). + +- Fixed a bug where multiple anonymous tuple arguments would be mixed up +when using the debugger or profiler (reported by Just van Rossum). +The simplest example is ``def f((a,b),(c,d)): print a,b,c,d''; this +would print the wrong value when run under the debugger or profiler. + +- The hacks that the dictionary implementation used to speed up +repeated lookups of the same C string were removed; these were a +source of subtle problems and don't seem to serve much of a purpose +any longer. + +- All traces of support for the long dead access statement have been +removed from the sources. + +- Plugged the two-byte memory leak in the tokenizer when reading an +interactive EOF. + +- There's a -O option to the interpreter that removes SET_LINENO +instructions and assert statements (see below); it uses and produces +.pyo files instead of .pyc files. The speedup is only a few percent +in most cases. The line numbers are still available in the .pyo file, +as a separate table (which is also available in .pyc files). However, +the removal of the SET_LINENO instructions means that the debugger +(pdb) can't set breakpoints on lines in -O mode. The traceback module +contains a function to extract a line number from the code object +referenced in a traceback object. In the future it should be possible +to write external bytecode optimizers that create better optimized +.pyo files, and there should be more control over optimization; +consider the -O option a "teaser". Without -O, the assert statement +actually generates code that first checks __debug__; if this variable +is false, the assertion is not checked. __debug__ is a built-in +variable whose value is initialized to track the -O flag (it's true +iff -O is not specified). With -O, no code is generated for assert +statements, nor for code of the form ``if __debug__: <something>''. +Sorry, no further constant folding happens. + + +Performance +----------- + +- It's much faster (almost twice for pystone.py -- see +Tools/scripts). See the entry on string interning below. + +- Some speedup by using separate free lists for method objects (both +the C and the Python variety) and for floating point numbers. + +- Big speedup by allocating frame objects with a single malloc() call. +The Python/C API for frames is changed (you shouldn't be using this +anyway). + +- Significant speedup by inlining some common opcodes for common operand +types (e.g. i+i, i-i, and list[i]). Fredrik Lundh. + +- Small speedup by reordering the method tables of some common +objects (e.g. list.append is now first). + +- Big optimization to the read() method of file objects. A read() +without arguments now attempts to use fstat to allocate a buffer of +the right size; for pipes and sockets, it will fall back to doubling +the buffer size. While that the improvement is real on all systems, +it is most dramatic on Windows. + + +Documentation +------------- + +- Many new pieces of library documentation were contributed, mostly by +Andrew Kuchling. Even cmath is now documented! There's also a +chapter of the library manual, "libundoc.tex", which provides a +listing of all undocumented modules, plus their status (e.g. internal, +obsolete, or in need of documentation). Also contributions by Sue +Williams, Skip Montanaro, and some module authors who succumbed to +pressure to document their own contributed modules :-). Note that +printing the documentation now kills fewer trees -- the margins have +been reduced. + +- I have started documenting the Python/C API. Unfortunately this project +hasn't been completed yet. It will be complete before the final release of +Python 1.5, though. At the moment, it's better to read the LaTeX source +than to attempt to run it through LaTeX and print the resulting dvi file. + +- The posix module (and hence os.py) now has doc strings! Thanks to Neil +Schemenauer. I received a few other contributions of doc strings. In most +other places, doc strings are still wishful thinking... + + +Language changes +---------------- + +- Private variables with leading double underscore are now a permanent +feature of the language. (These were experimental in release 1.4. I have +favorable experience using them; I can't label them "experimental" +forever.) + +- There's new string literal syntax for "raw strings". Prefixing a string +literal with the letter r (or R) disables all escape processing in the +string; for example, r'\n' is a two-character string consisting of a +backslash followed by the letter n. This combines with all forms of string +quotes; it is actually useful for triple quoted doc strings which might +contain references to \n or \t. An embedded quote prefixed with a +backslash does not terminate the string, but the backslash is still +included in the string; for example, r'\'' is a two-character string +consisting of a backslash and a quote. (Raw strings are also +affectionately known as Robin strings, after their inventor, Robin +Friedrich.) + +- There's a simple assert statement, and a new exception +AssertionError. For example, ``assert foo > 0'' is equivalent to ``if +not foo > 0: raise AssertionError''. Sorry, the text of the asserted +condition is not available; it would be too complicated to generate +code for this (since the code is generated from a parse tree). +However, the text is displayed as part of the traceback! + +- The raise statement has a new feature: when using "raise SomeClass, +somevalue" where somevalue is not an instance of SomeClass, it +instantiates SomeClass(somevalue). In 1.5a4, if somevalue is an +instance of a *derived* class of SomeClass, the exception class raised +is set to somevalue.__class__, and SomeClass is ignored after that. + +- Duplicate keyword arguments are now detected at compile time; +f(a=1,a=2) is now a syntax error. + + +Changes to builtin features +--------------------------- + +- There's a new exception FloatingPointError (used only by Lee Busby's +patches to catch floating point exceptions, at the moment). + +- The obsolete exception ConflictError (presumably used by the long +obsolete access statement) has been deleted. + +- There's a new function sys.exc_info() which returns the tuple +(sys.exc_type, sys.exc_value, sys.exc_traceback) in a thread-safe way. + +- There's a new variable sys.executable, pointing to the executable file +for the Python interpreter. + +- The sort() methods for lists no longer uses the C library qsort(); I +wrote my own quicksort implementation, with lots of help (in the form +of a kind of competition) from Tim Peters. This solves a bug in +dictionary comparisons on some Solaris versions when Python is built +with threads, and makes sorting lists even faster. + +- The semantics of comparing two dictionaries have changed, to make +comparison of unequal dictionaries faster. A shorter dictionary is +always considered smaller than a larger dictionary. For dictionaries +of the same size, the smallest differing element determines the +outcome (which yields the same results as before in this case, without +explicit sorting). Thanks to Aaron Watters for suggesting something +like this. + +- The semantics of try-except have changed subtly so that calling a +function in an exception handler that itself raises and catches an +exception no longer overwrites the sys.exc_* variables. This also +alleviates the problem that objects referenced in a stack frame that +caught an exception are kept alive until another exception is caught +-- the sys.exc_* variables are restored to their previous value when +returning from a function that caught an exception. + +- There's a new "buffer" interface. Certain objects (e.g. strings and +arrays) now support the "buffer" protocol. Buffer objects are acceptable +whenever formerly a string was required for a write operation; mutable +buffer objects can be the target of a read operation using the call +f.readinto(buffer). A cool feature is that regular expression matching now +also work on array objects. Contribution by Jack Jansen. (Needs +documentation.) + +- String interning: dictionary lookups are faster when the lookup +string object is the same object as the key in the dictionary, not +just a string with the same value. This is done by having a pool of +"interned" strings. Most names generated by the interpreter are now +automatically interned, and there's a new built-in function intern(s) +that returns the interned version of a string. Interned strings are +not a different object type, and interning is totally optional, but by +interning most keys a speedup of about 15% was obtained for the +pystone benchmark. + +- Dictionary objects have several new methods; clear() and copy() have +the obvious semantics, while update(d) merges the contents of another +dictionary d into this one, overriding existing keys. The dictionary +implementation file is now called dictobject.c rather than the +confusing mappingobject.c. + +- The intrinsic function dir() is much smarter; it looks in __dict__, +__members__ and __methods__. + +- The intrinsic functions int(), long() and float() can now take a +string argument and then do the same thing as string.atoi(), +string.atol(), and string.atof(). No second 'base' argument is +allowed, and complex() does not take a string (nobody cared enough). + +- When a module is deleted, its globals are now deleted in two phases. +In the first phase, all variables whose name begins with exactly one +underscore are replaced by None; in the second phase, all variables +are deleted. This makes it possible to have global objects whose +destructors depend on other globals. The deletion order within each +phase is still random. + +- It is no longer an error for a function to be called without a +global variable __builtins__ -- an empty directory will be provided +by default. + +- Guido's corollary to the "Don Beaudry hook": it is now possible to +do metaprogramming by using an instance as a base class. Not for the +faint of heart; and undocumented as yet, but basically if a base class +is an instance, its class will be instantiated to create the new +class. Jim Fulton will love it -- it also works with instances of his +"extension classes", since it is triggered by the presence of a +__class__ attribute on the purported base class. See +Demo/metaclasses/index.html for an explanation and see that directory +for examples. + +- Another change is that the Don Beaudry hook is now invoked when +*any* base class is special. (Up to 1.5a3, the *last* special base +class is used; in 1.5a4, the more rational choice of the *first* +special base class is used.) + +- New optional parameter to the readlines() method of file objects. +This indicates the number of bytes to read (the actual number of bytes +read will be somewhat larger due to buffering reading until the end of +the line). Some optimizations have also been made to speed it up (but +not as much as read()). + +- Complex numbers no longer have the ".conj" pseudo attribute; use +z.conjugate() instead, or complex(z.real, -z.imag). Complex numbers +now *do* support the __members__ and __methods__ special attributes. + +- The complex() function now looks for a __complex__() method on class +instances before giving up. + +- Long integers now support arbitrary shift counts, so you can now +write 1L<<1000000, memory permitting. (Python 1.4 reports "outrageous +shift count for this.) + +- The hex() and oct() functions have been changed so that for regular +integers, they never emit a minus sign. For example, on a 32-bit +machine, oct(-1) now returns '037777777777' and hex(-1) returns +'0xffffffff'. While this may seem inconsistent, it is much more +useful. (For long integers, a minus sign is used as before, to fit +the result in memory :-) + +- The hash() function computes better hashes for several data types, +including strings, floating point numbers, and complex numbers. + + +New extension modules +--------------------- + +- New extension modules cStringIO.c and cPickle.c, written by Jim +Fulton and other folks at Digital Creations. These are much more +efficient than their Python counterparts StringIO.py and pickle.py, +but don't support subclassing. cPickle.c clocks up to 1000 times +faster than pickle.py; cStringIO.c's improvement is less dramatic but +still significant. + +- New extension module zlibmodule.c, interfacing to the free zlib +library (gzip compatible compression). There's also a module gzip.py +which provides a higher level interface. Written by Andrew Kuchling +and Jeremy Hylton. + +- New module readline; see the "miscellaneous" section above. + +- New Unix extension module resource.c, by Jeremy Hylton, provides +access to getrlimit(), getrusage(), setrusage(), getpagesize(), and +related symbolic constants. + +- New extension puremodule.c, by Barry Warsaw, which interfaces to the +Purify(TM) C API. See also the file Misc/PURIFY.README. It is also +possible to enable Purify by simply setting the PURIFY Makefile +variable in the Modules/Setup file. + + +Changes in extension modules +---------------------------- + +- The struct extension module has several new features to control byte +order and word size. It supports reading and writing IEEE floats even +on platforms where this is not the native format. It uses uppercase +format codes for unsigned integers of various sizes (always using +Python long ints for 'I' and 'L'), 's' with a size prefix for strings, +and 'p' for "Pascal strings" (with a leading length byte, included in +the size; blame Hannu Krosing; new in 1.5a4). A prefix '>' forces +big-endian data and '<' forces little-endian data; these also select +standard data sizes and disable automatic alignment (use pad bytes as +needed). + +- The array module supports uppercase format codes for unsigned data +formats (like the struct module). + +- The fcntl extension module now exports the needed symbolic +constants. (Formerly these were in FCNTL.py which was not available +or correct for all platforms.) + +- The extension modules dbm, gdbm and bsddb now check that the +database is still open before making any new calls. + +- The dbhash module is no more. Use bsddb instead. (There's a third +party interface for the BSD 2.x code somewhere on the web; support for +bsddb will be deprecated.) + +- The gdbm module now supports a sync() method. + +- The socket module now has some new functions: getprotobyname(), and +the set {ntoh,hton}{s,l}(). + +- Various modules now export their type object: socket.SocketType, +array.ArrayType. + +- The socket module's accept() method now returns unknown addresses as +a tuple rather than raising an exception. (This can happen in +promiscuous mode.) Theres' also a new function getprotobyname(). + +- The pthread support for the thread module now works on most platforms. + +- STDWIN is now officially obsolete. Support for it will eventually +be removed from the distribution. + +- The binascii extension module is now hopefully fully debugged. +(XXX Oops -- Fredrik Lundh promised me a uuencode fix that I never +received.) + +- audioop.c: added a ratecv() function; better handling of overflow in +add(). + +- posixmodule.c: now exports the O_* flags (O_APPEND etc.). On +Windows, also O_TEXT and O_BINARY. The 'error' variable (the +exception is raises) is renamed -- its string value is now "os.error", +so newbies don't believe they have to import posix (or nt) to catch +it when they see os.error reported as posix.error. The execve() +function now accepts any mapping object for the environment. + +- A new version of the al (audio library) module for SGI was +contributed by Sjoerd Mullender. + +- The regex module has a new function get_syntax() which retrieves the +syntax setting set by set_syntax(). The code was also sanitized, +removing worries about unclean error handling. See also below for its +successor, re.py. + +- The "new" module (which creates new objects of various types) once +again has a fully functioning new.function() method. Dangerous as +ever! Also, new.code() has several new arguments. + +- A problem has been fixed in the rotor module: on systems with signed +characters, rotor-encoded data was not portable when the key contained +8-bit characters. Also, setkey() now requires its argument rather +than having broken code to default it. + +- The sys.builtin_module_names variable is now a tuple. Another new +variables in sys is sys.executable (the full path to the Python +binary, if known). + +- The specs for time.strftime() have undergone some revisions. It +appears that not all format characters are supported in the same way +on all platforms. Rather than reimplement it, we note these +differences in the documentation, and emphasize the shared set of +features. There's also a thorough test set (that occasionally finds +problems in the C library implementation, e.g. on some Linuxes), +thanks to Skip Montanaro. + +- The nis module seems broken when used with NIS+; unfortunately +nobody knows how to fix it. It should still work with old NIS. + + +New library modules +------------------- + +- New (still experimental) Perl-style regular expression module, +re.py, which uses a new interface for matching as well as a new +syntax; the new interface avoids the thread-unsafety of the regex +interface. This comes with a helper extension reopmodule.c and vastly +rewritten regexpr.c. Most work on this was done by Jeffrey Ollie, Tim +Peters, and Andrew Kuchling. See the documentation libre.tex. In +1.5, the old regex module is still fully supported; in the future, it +will become obsolete. + +- New module gzip.py; see zlib above. + +- New module keyword.py exports knowledge about Python's built-in +keywords. (New version by Ka-Ping Yee.) + +- New module pprint.py (with documentation) which supports +pretty-printing of lists, tuples, & dictionaries recursively. By Fred +Drake. + +- New module code.py. The function code.compile_command() can +determine whether an interactively entered command is complete or not, +distinguishing incomplete from invalid input. (XXX Unfortunately, +this seems broken at this moment, and I don't have the time to fix +it. It's probably better to add an explicit interface to the parser +for this.) + +- There is now a library module xdrlib.py which can read and write the +XDR data format as used by Sun RPC, for example. It uses the struct +module. + + +Changes in library modules +-------------------------- + +- Module codehack.py is now completely obsolete. + +- The pickle.py module has been updated to make it compatible with the +new binary format that cPickle.c produces. By default it produces the +old all-ASCII format compatible with the old pickle.py, still much +faster than pickle.py; it will read both formats automatically. A few +other updates have been made. + +- A new helper module, copy_reg.py, is provided to register extensions +to the pickling code. + +- Revamped module tokenize.py is much more accurate and has an +interface that makes it a breeze to write code to colorize Python +source code. Contributed by Ka-Ping Yee. + +- In ihooks.py, ModuleLoader.load_module() now closes the file under +all circumstances. + +- The tempfile.py module has a new class, TemporaryFile, which creates +an open temporary file that will be deleted automatically when +closed. This works on Windows and MacOS as well as on Unix. (Jim +Fulton.) + +- Changes to the cgi.py module: Most imports are now done at the +top of the module, which provides a speedup when using ni (Jim +Fulton). The problem with file upload to a Windows platform is solved +by using the new tempfile.TemporaryFile class; temporary files are now +always opened in binary mode (Jim Fulton). The cgi.escape() function +now takes an optional flag argument that quotes '"' to '"'. It +is now possible to invoke cgi.py from a command line script, to test +cgi scripts more easily outside an http server. There's an optional +limit to the size of uploads to POST (Skip Montanaro). Added a +'strict_parsing' option to all parsing functions (Jim Fulton). The +function parse_qs() now uses urllib.unquote() on the name as well as +the value of fields (Clarence Gardner). The FieldStorage class now +has a __len__() method. + +- httplib.py: the socket object is no longer closed; all HTTP/1.* +responses are now accepted; and it is now thread-safe (by not using +the regex module). + +- BaseHTTPModule.py: treat all HTTP/1.* versions the same. + +- The popen2.py module is now rewritten using a class, which makes +access to the standard error stream and the process id of the +subprocess possible. + +- Added timezone support to the rfc822.py module, in the form of a +getdate_tz() method and a parsedate_tz() function; also a mktime_tz(). +Also added recognition of some non-standard date formats, by Lars +Wirzenius, and RFC 850 dates (Chris Lawrence). + +- mhlib.py: various enhancements, including almost compatible parsing +of message sequence specifiers without invoking a subprocess. Also +added a createmessage() method by Lars Wirzenius. + +- The StringIO.StringIO class now supports readline(nbytes). (Lars +Wirzenius.) (Of course, you should be using cStringIO for performance.) + +- UserDict.py supports the new dictionary methods as well. + +- Improvements for whrandom.py by Tim Peters: use 32-bit arithmetic to +speed it up, and replace 0 seed values by 1 to avoid degeneration. +A bug was fixed in the test for invalid arguments. + +- Module ftplib.py: added support for parsing a .netrc file (Fred +Drake). Also added an ntransfercmd() method to the FTP class, which +allows access to the expected size of a transfer when available, and a +parse150() function to the module which parses the corresponding 150 +response. + +- urllib.py: the ftp cache is now limited to 10 entries. Added +quote_plus() and unquote_plus() functions which are like quote() and +unquote() but also replace spaces with '+' or vice versa, for +encoding/decoding CGI form arguments. Catch all errors from the ftp +module. HTTP requests now add the Host: header line. The proxy +variable names are now mapped to lower case, for Windows. The +spliturl() function no longer erroneously throws away all data past +the first newline. The basejoin() function now intereprets "../" +correctly. I *believe* that the problems with "exception raised in +__del__" under certain circumstances have been fixed (mostly by +changes elsewher in the interpreter). + +- In urlparse.py, there is a cache for results in urlparse.urlparse(); +its size limit is set to 20. Also, new URL schemes shttp, https, and +snews are "supported". + +- shelve.py: use cPickle and cStringIO when available. Also added +a sync() method, which calls the database's sync() method if there is +one. + +- The mimetools.py module now uses the available Python modules for +decoding quoted-printable, uuencode and base64 formats, rather than +creating a subprocess. + +- The python debugger (pdb.py, and its base class bdb.py) now support +conditional breakpoints. See the docs. + +- The modules base64.py, uu.py and quopri.py can now be used as simple +command line utilities. + +- Various small fixes to the nntplib.py module that I can't bother to +document in detail. + +- Sjoerd Mullender's mimify.py module now supports base64 encoding and +includes functions to handle the funny encoding you sometimes see in mail +headers. It is now documented. + +- mailbox.py: Added BabylMailbox. Improved the way the mailbox is +gotten from the environment. + +- Many more modules now correctly open files in binary mode when this +is necessary on non-Unix platforms. + +- The copying functions in the undocumented module shutil.py are +smarter. + +- The Writer classes in the formatter.py module now have a flush() +method. + +- The sgmllib.py module accepts hyphens and periods in the middle of +attribute names. While this is against the SGML standard, there is +some HTML out there that uses this... + +- The interface for the Python bytecode disassembler module, dis.py, +has been enhanced quite a bit. There's now one main function, +dis.dis(), which takes almost any kind of object (function, module, +class, instance, method, code object) and disassembles it; without +arguments it disassembles the last frame of the last traceback. The +other functions have changed slightly, too. + +- The imghdr.py module recognizes new image types: BMP, PNG. + +- The string.py module has a new function replace(str, old, new, +[maxsplit]) which does substring replacements. It is actually +implemented in C in the strop module. The functions [r]find() an +[r]index() have an optional 4th argument indicating the end of the +substring to search, alsoo implemented by their strop counterparts. +(Remember, never import strop -- import string uses strop when +available with zero overhead.) + +- The string.join() function now accepts any sequence argument, not +just lists and tuples. + +- The string.maketrans() requires its first two arguments to be +present. The old version didn't require them, but there's not much +point without them, and the documentation suggests that they are +required, so we fixed the code to match the documentation. + +- The regsub.py module has a function clear_cache(), which clears its +internal cache of compiled regular expressions. Also, the cache now +takes the current syntax setting into account. (However, this module +is now obsolete -- use the sub() or subn() functions or methods in the +re module.) + +- The undocumented module Complex.py has been removed, now that Python +has built-in complex numbers. A similar module remains as +Demo/classes/Complex.py, as an example. + + +Changes to the build process +---------------------------- + +- The way GNU readline is configured is totally different. The +--with-readline configure option is gone. It is now an extension +module, which may be loaded dynamically. You must enable it (and +specify the correct linraries to link with) in the Modules/Setup file. +Importing the module installs some hooks which enable command line +editing. When the interpreter shell is invoked interactively, it +attempts to import the readline module; when this fails, the default +input mechanism is used. The hook variables are PyOS_InputHook and +PyOS_ReadlineFunctionPointer. (Code contributed by Lee Busby, with +ideas from William Magro.) + +- New build procedure: a single library, libpython1.5.a, is now built, +which contains absolutely everything except for a one-line main() +program (which calls Py_Main(argc, argv) to start the interpreter +shell). This makes life much simpler for applications that need to +embed Python. The serial number of the build is now included in the +version string (sys.version). + +- As far as I can tell, neither gcc -Wall nor the Microsoft compiler +emits a single warning any more when compiling Python. + +- A number of new Makefile variables have been added for special +situations, e.g. LDLAST is appended to the link command. These are +used by editing the Makefile or passing them on the make command +line. + +- A set of patches from Lee Busby has been integrated that make it +possible to catch floating point exceptions. Use the configure option +--with-fpectl to enable the patches; the extension modules fpectl and +fpetest provide control to enable/disable and test the feature, +respectively. + +- The support for shared libraries under AIX is now simpler and more +robust. Thanks to Vladimir Marangozov for revamping his own patches! + +- The Modules/makesetup script now reads a file Setup.local as well as +a file Setup. Most changes to the Setup script can be done by editing +Setup.local instead, which makes it easier to carry a particular setup +over from one release to the next. + +- The Modules/makesetup script now copies any "include" lines it +encounters verbatim into the output Makefile. It also recognizes .cxx +and .cpp as C++ source files. + +- The configure script is smarter about C compiler options; e.g. with +gcc it uses -O2 and -g when possible, and on some other platforms it +uses -Olimit 1500 to avoid a warning from the optimizer about the main +loop in ceval.c (which has more than 1000 basic blocks). + +- The configure script now detects whether malloc(0) returns a NULL +pointer or a valid block (of length zero). This avoids the nonsense +of always adding one byte to all malloc() arguments on most platforms. + +- The configure script has a new option, --with-dec-threads, to enable +DEC threads on DEC Alpha platforms. Also, --with-threads is now an +alias for --with-thread (this was the Most Common Typo in configure +arguments). + +- Many changes in Doc/Makefile; amongst others, latex2html is now used +to generate HTML from all latex documents. + + +Change to the Python/C API +-------------------------- + +- Because some interfaces have changed, the PYTHON_API macro has been +bumped. Most extensions built for the old API version will still run, +but I can't guarantee this. Python prints a warning message on +version mismatches; it dumps core when the version mismatch causes a +serious problem :-) + +- I've completed the Grand Renaming, with the help of Roger Masse and +Barry Warsaw. This makes reading or debugging the code much easier. +Many other unrelated code reorganizations have also been carried out. +The allobjects.h header file is gone; instead, you would have to +include Python.h followed by rename2.h. But you're better off running +Tools/scripts/fixcid.py -s Misc/RENAME on your source, so you can omit +the rename2.h; it will disappear in the next release. + +- Various and sundry small bugs in the "abstract" interfaces have been +fixed. Thanks to all the (involuntary) testers of the Python 1.4 +version! Some new functions have been added, e.g. PySequence_List(o), +equivalent to list(o) in Python. + +- New API functions PyLong_FromUnsignedLong() and +PyLong_AsUnsignedLong(). + +- The API functions in the file cgensupport.c are no longer +supported. This file has been moved to Modules and is only ever +compiled when the SGI specific 'gl' module is built. + +- PyObject_Compare() can now raise an exception. Check with +PyErr_Occurred(). The comparison function in an object type may also +raise an exception. + +- The slice interface uses an upper bound of INT_MAX when no explicit +upper bound is given (e.x. for a[1:]). It used to ask the object for +its length and do the calculations. + +- Support for multiple independent interpreters. See Doc/api.tex, +functions Py_NewInterpreter() and Py_EndInterpreter(). Since the +documentation is incomplete, also see the new Demo/pysvr example +(which shows how to use these in a threaded application) and the +source code. + +- There is now a Py_Finalize() function which "de-initializes" +Python. It is possible to completely restart the interpreter +repeatedly by calling Py_Finalize() followed by Py_Initialize(). A +change of functionality in Py_Initialize() means that it is now a +fatal error to call it while the interpreter is already initialized. +The old, half-hearted Py_Cleanup() routine is gone. Use of Py_Exit() +is deprecated (it is nothing more than Py_Finalize() followed by +exit()). + +- There are no known memory leaks left. While Py_Finalize() doesn't +free *all* allocated memory (some of it is hard to track down), +repeated calls to Py_Finalize() and Py_Initialize() do not create +unaccessible heap blocks. + +- There is now explicit per-thread state. (Inspired by, but not the +same as, Greg Stein's free threading patches.) + +- There is now better support for threading C applications. There are +now explicit APIs to manipulate the interpreter lock. Read the source +or the Demo/pysvr example; the new functions are +PyEval_{Acquire,Release}{Lock,Thread}(). + +- The test macro DEBUG has changed to Py_DEBUG, to avoid interference +with other libraries' DEBUG macros. Likewise for any other test +macros that didn't yet start with Py_. + +- New wrappers around malloc() and friends: Py_Malloc() etc. call +malloc() and call PyErr_NoMemory() when it fails; PyMem_Malloc() call +just malloc(). Use of these wrappers could be essential if multiple +memory allocators exist (e.g. when using certain DLL setups under +Windows). (Idea by Jim Fulton.) + +- New C API PyImport_Import() which uses whatever __import__() hook +that is installed for the current execution environment. By Jim +Fulton. + +- It is now possible for an extension module's init function to fail +non-fatally, by calling one of the PyErr_* functions and returning. + +- The PyInt_AS_LONG() and PyFloat_AS_DOUBLE() macros now cast their +argument to the proper type, like the similar PyString macros already +did. (Suggestion by Marc-Andre Lemburg.) Similar for PyList_GET_SIZE +and PyList_GET_ITEM. + +- Some of the Py_Get* function, like Py_GetVersion() (but not yet +Py_GetPath()) are now declared as returning a const char *. (More +should follow.) + +- Changed the run-time library to check for exceptions after object +comparisons. PyObject_Compare() can now return an exception; use +PyErr_Occurred() to check (there is *no* special return value). + +- PyFile_WriteString() and Py_Flushline() now return error indicators +instead of clearing exceptions. This fixes an obscure bug where using +these would clear a pending exception, discovered by Just van Rossum. + +- There's a new function, PyArg_ParseTupleAndKeywords(), which parses +an argument list including keyword arguments. Contributed by Geoff +Philbrick. + +- PyArg_GetInt() is gone. + +- It's no longer necessary to include graminit.h when calling one of +the extended parser API functions. The three public grammar start +symbols are now in Python.h as Py_single_input, Py_file_input, and +Py_eval_input. + +- The CObject interface has a new function, +PyCObject_Import(module, name). It calls PyCObject_AsVoidPtr() +on the object referenced by "module.name". + + +Tkinter +------- + +- On popular demand, _tkinter once again installs a hook for readline +that processes certain Tk events while waiting for the user to type +(using PyOS_InputHook). + +- A patch by Craig McPheeters plugs the most obnoxious memory leaks, +caused by command definitions referencing widget objects beyond their +lifetime. + +- New standard dialog modules: tkColorChooser.py, tkCommonDialog.py, +tkMessageBox.py, tkFileDialog.py, tkSimpleDialog.py These interface +with the new Tk dialog scripts, and provide more "native platform" +style file selection dialog boxes on some platforms. Contributed by +Fredrik Lundh. + +- Tkinter.py: when the first Tk object is destroyed, it sets the +hiddel global _default_root to None, so that when another Tk object is +created it becomes the new default root. Other miscellaneous +changes and fixes. + +- The Image class now has a configure method. + +- Added a bunch of new winfo options to Tkinter.py; we should now be +up to date with Tk 4.2. The new winfo options supported are: +mananger, pointerx, pointerxy, pointery, server, viewable, visualid, +visualsavailable. + +- The broken bind() method on Canvas objects defined in the Canvas.py +module has been fixed. The CanvasItem and Group classes now also have +an unbind() method. + +- The problem with Tkinter.py falling back to trying to import +"tkinter" when "_tkinter" is not found has been fixed -- it no longer +tries "tkinter", ever. This makes diagnosing the problem "_tkinter +not configured" much easier and will hopefully reduce the newsgroup +traffic on this topic. + +- The ScrolledText module once again supports the 'cnf' parameter, to +be compatible with the examples in Mark Lutz' book (I know, I know, +too late...) + +- The _tkinter.c extension module has been revamped. It now support +Tk versions 4.1 through 8.0; support for 4.0 has been dropped. It +works well under Windows and Mac (with the latest Tk ports to those +platforms). It also supports threading -- it is safe for one +(Python-created) thread to be blocked in _tkinter.mainloop() while +other threads modify widgets. To make the changes visible, those +threads must use update_idletasks()method. (The patch for threading +in 1.5a3 was broken; in 1.5a4, it is back in a different version, +which requires access to the Tcl sources to get it to work -- hence it +is disabled by default.) + +- A bug in _tkinter.c has been fixed, where Split() with a string +containing an unmatched '"' could cause an exception or core dump. + +- Unfortunately, on Windows and Mac, Tk 8.0 no longer supports +CreateFileHandler, so _tkinter.createfilehandler is not available on +those platforms when using Tk 8.0 or later. I will have to rethink +how to interface with Tcl's lower-level event mechanism, or with its +channels (which are like Python's file-like objects). Jack Jansen has +provided a fix for the Mac, so createfilehandler *is* actually +supported there; maybe I can adapt his fix for Windows. + + +Tools and Demos +--------------- + +- A new regression test suite is provided, which tests most of the +standard and built-in modules. The regression test is run by invoking +the script Lib/test/regrtest.py. Barry Warsaw wrote the test harnass; +he and Roger Masse contributed most of the new tests. + +- New tool: faqwiz -- the CGI script that is used to maintain the +Python FAQ (http://grail.cnri.reston.va.us/cgi-bin/faqw.py). In +Tools/faqwiz. + +- New tool: webchecker -- a simple extensible web robot that, when +aimed at a web server, checks that server for dead links. Available +are a command line utility as well as a Tkinter based GUI version. In +Tools/webchecker. A simplified version of this program is dissected +in my article in O'Reilly's WWW Journal, the issue on Scripting +Languages (Vol 2, No 2); Scripting the Web with Python (pp 97-120). +Includes a parser for robots.txt files by Skip Montanaro. + +- New small tools: cvsfiles.py (prints a list of all files under CVS +n a particular directory tree), treesync.py (a rather Guido-specific +script to synchronize two source trees, one on Windows NT, the other +one on Unix under CVS but accessible from the NT box), and logmerge.py +(sort a collection of RCS or CVS logs by date). In Tools/scripts. + +- The freeze script now also works under Windows (NT). Another +feature allows the -p option to be pointed at the Python source tree +instead of the installation prefix. This was loosely based on part of +xfreeze by Sam Rushing and Bill Tutt. + +- New examples (Demo/extend) that show how to use the generic +extension makefile (Misc/Makefile.pre.in). + +- Tools/scripts/h2py.py now supports C++ comments. + +- Tools/scripts/pystone.py script is upgraded to version 1.1; there +was a bug in version 1.0 (distributed with Python 1.4) that leaked +memory. Also, in 1.1, the LOOPS variable is incremented to 10000. + +- Demo/classes/Rat.py completely rewritten by Sjoerd Mullender. + + +Windows (NT and 95) +------------------- + +- New project files for Developer Studio (Visual C++) 5.0 for Windows +NT (the old VC++ 4.2 Makefile is also still supported, but will +eventually be withdrawn due to its bulkiness). + +- See the note on the new module search path in the "Miscellaneous" section +above. + +- Support for Win32s (the 32-bit Windows API under Windows 3.1) is +basically withdrawn. If it still works for you, you're lucky. + +- There's a new extension module, msvcrt.c, which provides various +low-level operations defined in the Microsoft Visual C++ Runtime Library. +These include locking(), setmode(), get_osfhandle(), set_osfhandle(), and +console I/O functions like kbhit(), getch() and putch(). + +- The -u option not only sets the standard I/O streams to unbuffered +status, but also sets them in binary mode. (This can also be done +using msvcrt.setmode(), by the way.) + +- The, sys.prefix and sys.exec_prefix variables point to the directory +where Python is installed, or to the top of the source tree, if it was run +from there. + +- The various os.path modules (posixpath, ntpath, macpath) now support +passing more than two arguments to the join() function, so +os.path.join(a, b, c) is the same as os.path.join(a, os.path.join(b, +c)). + +- The ntpath module (normally used as os.path) supports ~ to $HOME +expansion in expanduser(). + +- The freeze tool now works on Windows. + +- See also the Tkinter category for a sad note on +_tkinter.createfilehandler(). + +- The truncate() method for file objects now works on Windows. + +- Py_Initialize() is no longer called when the DLL is loaded. You +must call it yourself. + +- The time module's clock() function now has good precision through +the use of the Win32 API QueryPerformanceCounter(). + +- Mark Hammond will release Python 1.5 versions of PythonWin and his +other Windows specific code: the win32api extensions, COM/ActiveX +support, and the MFC interface. + + +Mac +--- + +- As always, the Macintosh port will be done by Jack Jansen. He will +make a separate announcement for the Mac specific source code and the +binary distribution(s) when these are ready. + + +====================================================================== + + +===================================== +==> Release 1.4 (October 25 1996) <== +===================================== + +(Starting in reverse chronological order:) + +- Changed disclaimer notice. + +- Added SHELL=/bin/sh to Misc/Makefile.pre.in -- some Make versions +default to the user's login shell. + +- In Lib/tkinter/Tkinter.py, removed bogus binding of <Delete> in Text +widget, and bogus bspace() function. + +- In Lib/cgi.py, bumped __version__ to 2.0 and restored a truncated +paragraph. + +- Fixed the NT Makefile (PC/vc40.mak) for VC 4.0 to set /MD for all +subprojects, and to remove the (broken) experimental NumPy +subprojects. + +- In Lib/py_compile.py, cast mtime to long() so it will work on Mac +(where os.stat() returns mtimes as floats.) +- Set self.rfile unbuffered (like self.wfile) in SocketServer.py, to +fix POST in CGIHTTPServer.py. + +- Version 2.83 of Misc/python-mode.el for Emacs is included. + +- In Modules/regexmodule.c, fixed symcomp() to correctly handle a new +group starting immediately after a group tag. + +- In Lib/SocketServer.py, changed the mode for rfile to unbuffered. + +- In Objects/stringobject.c, fixed the compare function to do the +first char comparison in unsigned mode, for consistency with the way +other characters are compared by memcmp(). + +- In Lib/tkinter/Tkinter.py, fixed Scale.get() to support floats. + +- In Lib/urllib.py, fix another case where openedurl wasn't set. + +(XXX Sorry, the rest is in totally random order. No time to fix it.) + +- SyntaxError exceptions detected during code generation +(e.g. assignment to an expression) now include a line number. + +- Don't leave trailing / or \ in script directory inserted in front of +sys.path. + +- Added a note to Tools/scripts/classfix.py abouts its historical +importance. + +- Added Misc/Makefile.pre.in, a universal Makefile for extensions +built outside the distribution. + +- Rewritten Misc/faq2html.py, by Ka-Ping Yee. + +- Install shared modules with mode 555 (needed for performance on some +platforms). + +- Some changes to standard library modules to avoid calling append() +with more than one argument -- while supported, this should be +outlawed, and I don't want to set a bad example. + +- bdb.py (and hence pdb.py) supports calling run() with a code object +instead of a code string. + +- Fixed an embarrassing bug cgi.py which prevented correct uploading +of binary files from Netscape (which doesn't distinguish between +binary and text files). Also added dormant logging support, which +makes it easier to debug the cgi module itself. + +- Added default writer to constructor of NullFormatter class. + +- Use binary mode for socket.makefile() calls in ftplib.py. + +- The ihooks module no longer "installs" itself upon import -- this +was an experimental feature that helped ironing out some bugs but that +slowed down code that imported it without the need to install it +(e.g. the rexec module). Also close the file in some cases and add +the __file__ attribute to loaded modules. + +- The test program for mailbox.py is now more useful. + +- Added getparamnames() to Message class in mimetools.py -- it returns +the names of parameters to the content-type header. + +- Fixed a typo in ni that broke the loop stripping "__." from names. + +- Fix sys.path[0] for scripts run via pdb.py's new main program. + +- profile.py can now also run a script, like pdb. + +- Fix a small bug in pyclbr -- don't add names starting with _ when +emulating from ... import *. + +- Fixed a series of embarrassing typos in rexec's handling of standard +I/O redirection. Added some more "safe" built-in modules: cmath, +errno, operator. + +- Fixed embarrassing typo in shelve.py. + +- Added SliceType and EllipsisType to types.py. + +- In urllib.py, added handling for error 301 (same as 302); added +geturl() method to get the URL after redirection. + +- Fixed embarrassing typo in xdrlib.py. Also fixed typo in Setup.in +for _xdrmodule.c and removed redundant #include from _xdrmodule.c. + +- Fixed bsddbmodule.c to add binary mode indicator on platforms that +have it. This should make it working on Windows NT. + +- Changed last uses of #ifdef NT to #ifdef MS_WINDOWS or MS_WIN32, +whatever applies. Also rationalized some other tests for various MS +platforms. + +- Added the sources for the NT installer script used for Python +1.4beta3. Not tested with this release, but better than nothing. + +- A compromise in pickle's defenses against Trojan horses: a +user-defined function is now okay where a class is expected. A +built-in function is not okay, to prevent pickling something that +will execute os.system("rm -f *") when unpickling. + +- dis.py will print the name of local variables referenced by local +load/store/delete instructions. + +- Improved portability of SimpleHTTPServer module to non-Unix +platform. + +- The thread.h interface adds an extra argument to down_sema(). This +only affects other C code that uses thread.c; the Python thread module +doesn't use semaphores (which aren't provided on all platforms where +Python threads are supported). Note: on NT, this change is not +implemented. + +- Fixed some typos in abstract.h; corrected signature of +PyNumber_Coerce, added PyMapping_DelItem. Also fixed a bug in +abstract.c's PyObject_CallMethod(). + +- apply(classname, (), {}) now works even if the class has no +__init__() method. + +- Implemented complex remainder and divmod() (these would dump core!). +Conversion of complex numbers to int, long int or float now raises an +exception, since there is no meaningful way to do it without losing +information. + +- Fixed bug in built-in complex() function which gave the wrong result +for two real arguments. + +- Change the hash algorithm for strings -- the multiplier is now +1000003 instead of 3, which gives better spread for short strings. + +- New default path for Windows NT, the registry structure now supports +default paths for different install packages. (Mark Hammond -- the +next PythonWin release will use this.) + +- Added more symbols to the python_nt.def file. + +- When using GNU readline, set rl_readline_name to "python". + +- The Ellipses built-in name has been renamed to Ellipsis -- this is +the correct singular form. Thanks to Ka-Ping Yee, who saved us from +eternal embarrassment. + +- Bumped the PYTHON_API_VERSION to 1006, due to the Ellipses -> +Ellipsis name change. + +- Updated the library reference manual. Added documentation of +restricted mode (rexec, Bastion) and the formatter module (for use +with the htmllib module). Fixed the documentation of htmllib +(finally). + +- The reference manual is now maintained in FrameMaker. + +- Upgraded scripts Doc/partparse.py and Doc/texi2html.py. + +- Slight improvements to Doc/Makefile. + +- Added fcntl.lockf(). This should be used for Unix file locking +instead of the posixfile module; lockf() is more portable. + +- The getopt module now supports long option names, thanks to Lars +Wizenius. + +- Plenty of changes to Tkinter and Canvas, mostly due to Fred Drake +and Nils Fischbeck. + +- Use more bits of time.time() in whrandom's default seed(). + +- Performance hack for regex module's regs attribute. + +- Don't close already closed socket in socket module. + +- Correctly handle separators containing embedded nulls in +strop.split, strop.find and strop.rfind. Also added more detail to +error message for strop.atoi and friends. + +- Moved fallback definition for hypot() to Python/hypot.c. + +- Added fallback definition for strdup, in Python/strdup.c. + +- Fixed some bugs where a function would return 0 to indicate an error +where it should return -1. + +- Test for error returned by time.localtime(), and rationalized its MS +tests. + +- Added Modules/Setup.local file, which is processed after Setup. + +- Corrected bug in toplevel Makefile.in -- execution of regen script +would not use the right PATH and PYTHONPATH. + +- Various and sundry NeXT configuration changes (sigh). + +- Support systems where libreadline needs neither termcap nor curses. + +- Improved ld_so_aix script and python.exp file (for AIX). + +- More stringent test for working <stdarg.h> in configure script. + +- Removed Demo/www subdirectory -- it was totally out of date. + +- Improved demos and docs for Fred Drake's parser module; fixed one +typo in the module itself. + + +========================================= +==> Release 1.4beta3 (August 26 1996) <== +========================================= + + +(XXX This is less readable that it should. I promise to restructure +it for the final 1.4 release.) + + +What's new in 1.4beta3 (since beta2)? +------------------------------------- + +- Name mangling to implement a simple form of class-private variables. +A name of the form "__spam" can't easily be used outside the class. +(This was added in 1.4beta3, but left out of the 1.4beta3 release +message.) + +- In urllib.urlopen(): HTTP URLs containing user:passwd@host are now +handled correctly when using a proxy server. + +- In ntpath.normpath(): don't truncate to 8+3 format. + +- In mimetools.choose_boundary(): don't die when getuid() or getpid() +aren't defined. + +- Module urllib: some optimizations to (un)quoting. + +- New module MimeWriter for writing MIME documents. + +- More changes to formatter module. + +- The freeze script works once again and is much more robust (using +sys.prefix etc.). It also supports a -o option to specify an +output directory. + +- New module whichdb recognizes dbm, gdbm and bsddb/dbhash files. + +- The Doc/Makefile targets have been reorganized somewhat to remove the +insistence on always generating PostScript. + +- The texinfo to html filter (Doc/texi2html.py) has been improved somewhat. + +- "errors.h" has been renamed to "pyerrors.h" to resolve a long-standing +name conflict on the Mac. + +- Linking a module compiled with a different setting for Py_TRACE_REFS now +generates a linker error rather than a core dump. + +- The cgi module has a new convenience function print_exception(), which +formats a python exception using HTML. It also fixes a bug in the +compatibility code and adds a dubious feature which makes it possible to +have two query strings, one in the URL and one in the POST data. + +- A subtle change in the unpickling of class instances makes it possible +to unpickle in restricted execution mode, where the __dict__ attribute is +not available (but setattr() is). + +- Documentation for os.path.splitext() (== posixpath.splitext()) has been +cleared up. It splits at the *last* dot. + +- posixfile locking is now also correctly supported on AIX. + +- The tempfile module once again honors an initial setting of tmpdir. It +now works on Windows, too. + +- The traceback module has some new functions to extract, format and print +the active stack. + +- Some translation functions in the urllib module have been made a little +less sluggish. + +- The addtag_* methods for Canvas widgets in Tkinter as well as in the +separate Canvas class have been fixed so they actually do something +meaningful. + +- A tiny _test() function has been added to Tkinter.py. + +- A generic Makefile for dynamically loaded modules is provided in the Misc +subdirectory (Misc/gMakefile). + +- A new version of python-mode.el for Emacs is provided. See +http://www.python.org/ftp/emacs/pmdetails.html for details. The +separate file pyimenu.el is no longer needed, imenu support is folded +into python-mode.el. + +- The configure script can finally correctly find the readline library in a +non-standard location. The LDFLAGS variable is passed on the Makefiles +from the configure script. + +- Shared libraries are now installed as programs (i.e. with executable +permission). This is required on HP-UX and won't hurt on other systems. + +- The objc.c module is no longer part of the distribution. Objective-C +support may become available as contributed software on the ftp site. + +- The sybase module is no longer part of the distribution. A much +improved sybase module is available as contributed software from the +ftp site. + +- _tkinter is now compatible with Tcl 7.5 / Tk 4.1 patch1 on Windows and +Mac (don't use unpatched Tcl/Tk!). The default line in the Setup.in file +now links with Tcl 7.5 / Tk 4.1 rather than 7.4/4.0. + +- In Setup, you can now write "*shared*" instead of "*noconfig*", and you +can use *.so and *.sl as shared libraries. + +- Some more fidgeting for AIX shared libraries. + +- The mpz module is now compatible with GMP 2.x. (Not tested by me.) +(Note -- a complete replacement by Niels Mo"ller, called gpmodule, is +available from the contrib directory on the ftp site.) + +- A warning is written to sys.stderr when a __del__ method raises an +exception (formerly, such exceptions were completely ignored). + +- The configure script now defines HAVE_OLD_CPP if the C preprocessor is +incapable of ANSI style token concatenation and stringification. + +- All source files (except a few platform specific modules) are once again +compatible with K&R C compilers as well as ANSI compilers. In particular, +ANSI-isms have been removed or made conditional in complexobject.c, +getargs.c and operator.c. + +- The abstract object API has three new functions, PyObject_DelItem, +PySequence_DelItem, and PySequence_DelSlice. + +- The operator module has new functions delitem and delslice, and the +functions "or" and "and" are renamed to "or_" and "and_" (since "or" and +"and" are reserved words). ("__or__" and "__and__" are unchanged.) + +- The environment module is no longer supported; putenv() is now a function +in posixmodule (also under NT). + +- Error in filter(<function>, "") has been fixed. + +- Unrecognized keyword arguments raise TypeError, not KeyError. + +- Better portability, fewer bugs and memory leaks, fewer compiler warnings, +some more documentation. + +- Bug in float power boundary case (0.0 to the negative integer power) +fixed. + +- The test of negative number to the float power has been moved from the +built-in pow() functin to floatobject.c (so complex numbers can yield the +correct result). + +- The bug introduced in beta2 where shared libraries loaded (using +dlopen()) from the current directory would fail, has been fixed. + +- Modules imported as shared libraries now also have a __file__ attribute, +giving the filename from which they were loaded. The only modules without +a __file__ attribute now are built-in modules. + +- On the Mac, dynamically loaded modules can end in either ".slb" or +".<platform>.slb" where <platform> is either "CFM68K" or "ppc". The ".slb" +extension should only be used for "fat" binaries. + +- C API addition: marshal.c now supports +PyMarshal_WriteObjectToString(object). + +- C API addition: getargs.c now supports +PyArg_ParseTupleAndKeywords(args, kwdict, format, kwnames, ...) +to parse keyword arguments. + +- The PC versioning scheme (sys.winver) has changed once again. the +version number is now "<digit>.<digit>.<digit>.<apiversion>", where the +first three <digit>s are the Python version (e.g. "1.4.0" for Python 1.4, +"1.4.1" for Python 1.4.1 -- the beta level is not included) and +<apiversion> is the four-digit PYTHON_API_VERSION (currently 1005). + +- h2py.py accepts whitespace before the # in CPP directives + +- On Solaris 2.5, it should now be possible to use either Posix threads or +Solaris threads (XXX: how do you select which is used???). (Note: the +Python pthreads interface doesn't fully support semaphores yet -- anyone +care to fix this?) + +- Thread support should now work on AIX, using either DCE threads or +pthreads. + +- New file Demo/sockets/unicast.py + +- Working Mac port, with CFM68K support, with Tk 4.1 support (though not +both) (XXX) + +- New project setup for PC port, now compatible with PythonWin, with +_tkinter and NumPy support (XXX) + +- New module site.py (XXX) + +- New module xdrlib.py and optional support module _xdrmodule.c (XXX) + +- parser module adapted to new grammar, complete w/ Doc & Demo (XXX) + +- regen script fixed (XXX) + +- new machdep subdirectories Lib/{aix3,aix4,next3_3,freebsd2,linux2} (XXX) + +- testall now also tests math module (XXX) + +- string.atoi c.s. now raise an exception for an empty input string. + +- At last, it is no longer necessary to define HAVE_CONFIG_H in order to +have config.h included at various places. + +- Unrecognized keyword arguments now raise TypeError rather than KeyError. + +- The makesetup script recognizes files with extension .so or .sl as +(shared) libraries. + +- 'access' is no longer a reserved word, and all code related to its +implementation is gone (or at least #ifdef'ed out). This should make +Python a little speedier too! + +- Performance enhancements suggested by Sjoerd Mullender. This includes +the introduction of two new optional function pointers in type object, +getattro and setattro, which are like getattr and setattr but take a +string object instead of a C string pointer. + +- New operations in string module: lstrip(s) and rstrip(s) strip whitespace +only on the left or only on the right, A new optional third argument to +split() specifies the maximum number of separators honored (so +splitfields(s, sep, n) returns a list of at most n+1 elements). (Since +1.3, splitfields(s, None) is totally equivalent to split(s).) +string.capwords() has an optional second argument specifying the +separator (which is passed to split()). + +- regsub.split() has the same addition as string.split(). regsub.splitx(s, +sep, maxsep) implements the functionality that was regsub.split(s, 1) in +1.4beta2 (return a list containing the delimiters as well as the words). + +- Final touch for AIX loading, rewritten Misc/AIX-NOTES. + +- In Modules/_tkinter.c, when using Tk 4.1 or higher, use className +argument to _tkinter.create() to set Tcl's argv0 variable, so X +resources use the right resource class again. + +- Add #undef fabs to Modules/mathmodule.c for macintosh. + +- Added some macro renames for AIX in Modules/operator.c. + +- Removed spurious 'E' from Doc/liberrno.tex. + +- Got rid of some cruft in Misc/ (dlMakefile, pyimenu.el); added new +Misc/gMakefile and new version of Misc/python-mode.el. + +- Fixed typo in Lib/ntpath.py (islink has "return false" which gives a +NameError). + +- Added missing "from types import *" to Lib/tkinter/Canvas.py. + +- Added hint about using default args for __init__ to pickle docs. + +- Corrected typo in Inclide/abstract.h: PySequence_Lenth -> +PySequence_Length. + +- Some improvements to Doc/texi2html.py. + +- In Python/import.c, Cast unsigned char * in struct _frozen to char * +in calls to rds_object(). + +- In doc/ref4.tex, added note about scope of lambda bodies. + +What's new in 1.4beta2 (since beta1)? +------------------------------------- + +- Portability bug in the md5.h header solved. + +- The PC build procedure now really works, and sets sys.platform to a +meaningful value (a few things were botched in beta 1). Lib/dos_8x3 +is now a standard part of the distribution (alas). + +- More improvements to the installation procedure. Typing "make install" +now inserts the version number in the pathnames of almost everything +installed, and creates the machine dependent modules (FCNTL.py etc.) if not +supplied by the distribution. (XXX There's still a problem with the latter +because the "regen" script requires that Python is installed. Some manual +intervention may still be required.) (This has been fixed in 1.4beta3.) + +- New modules: errno, operator (XXX). + +- Changes for use with Numerical Python: builtin function slice() and +Ellipses object, and corresponding syntax: + + x[lo:hi:stride] == x[slice(lo, hi, stride)] + x[a, ..., z] == x[(a, Ellipses, z)] + +- New documentation for errno and cgi modules. + +- The directory containing the script passed to the interpreter is +inserted in from of sys.path; "." is no longer a default path +component. + +- Optional third string argument to string.translate() specifies +characters to delete. New function string.maketrans() creates a +translation table for translate() or for regex.compile(). + +- Module posix (and hence module os under Unix) now supports putenv(). +Moreover, module os is enhanced so that if putenv() is supported, +assignments to os.environ entries make the appropriate putenv() call. +(XXX the putenv() implementation can leak a small amount of memory per +call.) + +- pdb.py can now be invoked from the command line to debug a script: +python pdb.py <script> <arg> ... + +- Much improved parseaddr() in rfc822. + +- In cgi.py, you can now pass an alternative value for environ to +nearly all functions. + +- You can now assign to instance variables whose name begins and ends +with '__'. + +- New version of Fred Drake's parser module and associates (token, +symbol, AST). + +- New PYTHON_API_VERSION value and .pyc file magic number (again!). + +- The "complex" internal structure type is now called "Py_complex" to +avoid name conflicts. + +- Numerous small bugs fixed. + +- Slight pickle speedups. + +- Some slight speedups suggested by Sjoerd (more coming in 1.4 final). + +- NeXT portability mods by Bill Bumgarner integrated. + +- Modules regexmodule.c, bsddbmodule.c and xxmodule.c have been +converted to new naming style. + + +What's new in 1.4beta1 (since 1.3)? +----------------------------------- + +- Added sys.platform and sys.exec_platform for Bill Janssen. + +- Installation has been completely overhauled. "make install" now installs +everything, not just the python binary. Installation uses the install-sh +script (borrowed from X11) to install each file. + +- New functions in the posix module: mkfifo, plock, remove (== unlink), +and ftruncate. More functions are also available under NT. + +- New function in the fcntl module: flock. + +- Shared library support for FreeBSD. + +- The --with-readline option can now be used without a DIRECTORY argument, +for systems where libreadline.* is in one of the standard places. It is +also possible for it to be a shared library. + +- The extension tkinter has been renamed to _tkinter, to avoid confusion +with Tkinter.py oncase insensitive file systems. It now supports Tk 4.1 as +well as 4.0. + +- Author's change of address from CWI in Amsterdam, The Netherlands, to +CNRI in Reston, VA, USA. + +- The math.hypot() function is now always available (if it isn't found in +the C math library, Python provides its own implementation). + +- The latex documentation is now compatible with latex2e, thanks to David +Ascher. + +- The expression x**y is now equivalent to pow(x, y). + +- The indexing expression x[a, b, c] is now equivalent to x[(a, b, c)]. + +- Complex numbers are now supported. Imaginary constants are written with +a 'j' or 'J' prefix, general complex numbers can be formed by adding a real +part to an imaginary part, like 3+4j. Complex numbers are always stored in +floating point form, so this is equivalent to 3.0+4.0j. It is also +possible to create complex numbers with the new built-in function +complex(re, [im]). For the footprint-conscious, complex number support can +be disabled by defining the symbol WITHOUT_COMPLEX. + +- New built-in function list() is the long-awaited counterpart of tuple(). + +- There's a new "cmath" module which provides the same functions as the +"math" library but with complex arguments and results. (There are very +good reasons why math.sqrt(-1) still raises an exception -- you have to use +cmath.sqrt(-1) to get 1j for an answer.) + +- The Python.h header file (which is really the same as allobjects.h except +it disables support for old style names) now includes several more files, +so you have to have fewer #include statements in the average extension. + +- The NDEBUG symbol is no longer used. Code that used to be dependent on +the presence of NDEBUG is now present on the absence of DEBUG. TRACE_REFS +and REF_DEBUG have been renamed to Py_TRACE_REFS and Py_REF_DEBUG, +respectively. At long last, the source actually compiles and links without +errors when this symbol is defined. + +- Several symbols that didn't follow the new naming scheme have been +renamed (usually by adding to rename2.h) to use a Py or _Py prefix. There +are no external symbols left without a Py or _Py prefix, not even those +defined by sources that were incorporated from elsewhere (regexpr.c, +md5c.c). (Macros are a different story...) + +- There are now typedefs for the structures defined in config.c and +frozen.c. + +- New PYTHON_API_VERSION value and .pyc file magic number. + +- New module Bastion. (XXX) + +- Improved performance of StringIO module. + +- UserList module now supports + and * operators. + +- The binhex and binascii modules now actually work. + +- The cgi module has been almost totally rewritten and documented. +It now supports file upload and a new data type to handle forms more +flexibly. + +- The formatter module (for use with htmllib) has been overhauled (again). + +- The ftplib module now supports passive mode and has doc strings. + +- In (ideally) all places where binary files are read or written, the file +is now correctly opened in binary mode ('rb' or 'wb') so the code will work +on Mac or PC. + +- Dummy versions of os.path.expandvars() and expanduser() are now provided +on non-Unix platforms. + +- Module urllib now has two new functions url2pathname and pathname2url +which turn local filenames into "file:..." URLs using the same rules as +Netscape (why be different). it also supports urlretrieve() with a +pathname parameter, and honors the proxy environment variables (http_proxy +etc.). The URL parsing has been improved somewhat, too. + +- Micro improvements to urlparse. Added urlparse.urldefrag() which +removes a trailing ``#fragment'' if any. + +- The mailbox module now supports MH style message delimiters as well. + +- The mhlib module contains some new functionality: setcontext() to set the +current folder and parsesequence() to parse a sequence as commonly passed +to MH commands (e.g. 1-10 or last:5). + +- New module mimify for conversion to and from MIME format of email +messages. + +- Module ni now automatically installs itself when first imported -- this +is against the normal rule that modules should define classes and functions +but not invoke them, but appears more useful in the case that two +different, independent modules want to use ni's features. + +- Some small performance enhancements in module pickle. + +- Small interface change to the profile.run*() family of functions -- more +sensible handling of return values. + +- The officially registered Mac creator for Python files is 'Pyth'. This +replaces 'PYTH' which was used before but never registered. + +- Added regsub.capwords(). (XXX) + +- Added string.capwords(), string.capitalize() and string.translate(). +(XXX) + +- Fixed an interface bug in the rexec module: it was impossible to pass a +hooks instance to the RExec class. rexec now also supports the dynamic +loading of modules from shared libraries. Some other interfaces have been +added too. + +- Module rfc822 now caches the headers in a dictionary for more efficient +lookup. + +- The sgmllib module now understands a limited number of SGML "shorthands" +like <A/.../ for <A>...</A>. (It's not clear that this was a good idea...) + +- The tempfile module actually tries a number of different places to find a +usable temporary directory. (This was prompted by certain Linux +installations that appear to be missing a /usr/tmp directory.) [A bug in +the implementation that would ignore a pre-existing tmpdir global has been +fixed in beta3.] + +- Much improved and enhanved FileDialog module for Tkinter. + +- Many small changes to Tkinter, to bring it more in line with Tk 4.0 (as +well as Tk 4.1). + +- New socket interfaces include ntohs(), ntohl(), htons(), htonl(), and +s.dup(). Sockets now work correctly on Windows. On Windows, the built-in +extension is called _socket and a wrapper module win/socket.py provides +"makefile()" and "dup()" functionality. On Windows, the select module +works only with socket objects. + +- Bugs in bsddb module fixed (e.g. missing default argument values). + +- The curses extension now includes <ncurses.h> when available. + +- The gdbm module now supports opening databases in "fast" mode by +specifying 'f' as the second character or the mode string. + +- new variables sys.prefix and sys.exec_prefix pass corresponding +configuration options / Makefile variables to the Python programmer. + +- The ``new'' module now supports creating new user-defined classes as well +as instances thereof. + +- The soundex module now sports get_soundex() to get the soundex value for an +arbitrary string (formerly it would only do soundex-based string +comparison) as well as doc strings. + +- New object type "cobject" to safely wrap void pointers for passing them +between various extension modules. + +- More efficient computation of float**smallint. + +- The mysterious bug whereby "x.x" (two occurrences of the same +one-character name) typed from the commandline would sometimes fail +mysteriously. + +- The initialization of the readline function can now be invoked by a C +extension through PyOS_ReadlineInit(). + +- There's now an externally visible pointer PyImport_FrozenModules which +can be changed by an embedding application. + +- The argument parsing functions now support a new format character 'D' to +specify complex numbers. + +- Various memory leaks plugged and bugs fixed. + +- Improved support for posix threads (now that real implementations are +beginning to apepar). Still no fully functioning semaphores. + +- Some various and sundry improvements and new entries in the Tools +directory. + + +===================================== +==> Release 1.3 (13 October 1995) <== +===================================== + +Major change +============ + +Two words: Keyword Arguments. See the first section of Chapter 12 of +the Tutorial. + +(The rest of this file is textually the same as the remaining sections +of that chapter.) + + +Changes to the WWW and Internet tools +===================================== + +The "htmllib" module has been rewritten in an incompatible fashion. +The new version is considerably more complete (HTML 2.0 except forms, +but including all ISO-8859-1 entity definitions), and easy to use. +Small changes to "sgmllib" have also been made, to better match the +tokenization of HTML as recognized by other web tools. + +A new module "formatter" has been added, for use with the new +"htmllib" module. + +The "urllib"and "httplib" modules have been changed somewhat to allow +overriding unknown URL types and to support authentication. They now +use "mimetools.Message" instead of "rfc822.Message" to parse headers. +The "endrequest()" method has been removed from the HTTP class since +it breaks the interaction with some servers. + +The "rfc822.Message" class has been changed to allow a flag to be +passed in that says that the file is unseekable. + +The "ftplib" module has been fixed to be (hopefully) more robust on +Linux. + +Several new operations that are optionally supported by servers have +been added to "nntplib": "xover", "xgtitle", "xpath" and "date". + +Other Language Changes +====================== + +The "raise" statement now takes an optional argument which specifies +the traceback to be used when printing the exception's stack trace. +This must be a traceback object, such as found in "sys.exc_traceback". +When omitted or given as "None", the old behavior (to generate a stack +trace entry for the current stack frame) is used. + +The tokenizer is now more tolerant of alien whitespace. Control-L in +the leading whitespace of a line resets the column number to zero, +while Control-R just before the end of the line is ignored. + +Changes to Built-in Operations +============================== + +For file objects, "f.read(0)" and "f.readline(0)" now return an empty +string rather than reading an unlimited number of bytes. For the +latter, omit the argument altogether or pass a negative value. + +A new system variable, "sys.platform", has been added. It specifies +the current platform, e.g. "sunos5" or "linux1". + +The built-in functions "input()" and "raw_input()" now use the GNU +readline library when it has been configured (formerly, only +interactive input to the interpreter itself was read using GNU +readline). The GNU readline library provides elaborate line editing +and history. The Python debugger ("pdb") is the first beneficiary of +this change. + +Two new built-in functions, "globals()" and "locals()", provide access +to dictionaries containming current global and local variables, +respectively. (These augment rather than replace "vars()", which +returns the current local variables when called without an argument, +and a module's global variables when called with an argument of type +module.) + +The built-in function "compile()" now takes a third possible value for +the kind of code to be compiled: specifying "'single'" generates code +for a single interactive statement, which prints the output of +expression statements that evaluate to something else than "None". + +Library Changes +=============== + +There are new module "ni" and "ihooks" that support importing modules +with hierarchical names such as "A.B.C". This is enabled by writing +"import ni; ni.ni()" at the very top of the main program. These +modules are amply documented in the Python source. + +The module "rexec" has been rewritten (incompatibly) to define a class +and to use "ihooks". + +The "string.split()" and "string.splitfields()" functions are now the +same function (the presence or absence of the second argument +determines which operation is invoked); similar for "string.join()" +and "string.joinfields()". + +The "Tkinter" module and its helper "Dialog" have been revamped to use +keyword arguments. Tk 4.0 is now the standard. A new module +"FileDialog" has been added which implements standard file selection +dialogs. + +The optional built-in modules "dbm" and "gdbm" are more coordinated +--- their "open()" functions now take the same values for their "flag" +argument, and the "flag" and "mode" argument have default values (to +open the database for reading only, and to create the database with +mode "0666" minuse the umask, respectively). The memory leaks have +finally been fixed. + +A new dbm-like module, "bsddb", has been added, which uses the BSD DB +package's hash method. + +A portable (though slow) dbm-clone, implemented in Python, has been +added for systems where none of the above is provided. It is aptly +dubbed "dumbdbm". + +The module "anydbm" provides a unified interface to "bsddb", "gdbm", +"dbm", and "dumbdbm", choosing the first one available. + +A new extension module, "binascii", provides a variety of operations +for conversion of text-encoded binary data. + +There are three new or rewritten companion modules implemented in +Python that can encode and decode the most common such formats: "uu" +(uuencode), "base64" and "binhex". + +A module to handle the MIME encoding quoted-printable has also been +added: "quopri". + +The parser module (which provides an interface to the Python parser's +abstract syntax trees) has been rewritten (incompatibly) by Fred +Drake. It now lets you change the parse tree and compile the result! + +The \code{syslog} module has been upgraded and documented. + +Other Changes +============= + +The dynamic module loader recognizes the fact that different filenames +point to the same shared library and loads the library only once, so +you can have a single shared library that defines multiple modules. +(SunOS / SVR4 style shared libraries only.) + +Jim Fulton's ``abstract object interface'' has been incorporated into +the run-time API. For more detailes, read the files +"Include/abstract.h" and "Objects/abstract.c". + +The Macintosh version is much more robust now. + +Numerous things I have forgotten or that are so obscure no-one will +notice them anyway :-) + + +=================================== +==> Release 1.2 (13 April 1995) <== +=================================== + +- Changes to Misc/python-mode.el: + - Wrapping and indentation within triple quote strings should work + properly now. + - `Standard' bug reporting mechanism (use C-c C-b) + - py-mark-block was moved to C-c C-m + - C-c C-v shows you the python-mode version + - a basic python-font-lock-keywords has been added for Emacs 19 + font-lock colorizations. + - proper interaction with pending-del and del-sel modes. + - New py-electric-colon (:) command for improved outdenting. Also + py-indent-line (TAB) should handle outdented lines better. + - New commands py-outdent-left (C-c C-l) and py-indent-right (C-c C-r) + +- The Library Reference has been restructured, and many new and +existing modules are now documented, in particular the debugger and +the profiler, as well as the persistency and the WWW/Internet support +modules. + +- All known bugs have been fixed. For example the pow(2,2,3L) bug on +Linux has been fixed. Also the re-entrancy problems with __del__ have +been fixed. + +- All known memory leaks have been fixed. + +- Phase 2 of the Great Renaming has been executed. The header files +now use the new names (PyObject instead of object, etc.). The linker +also sees the new names. Most source files still use the old names, +by virtue of the rename2.h header file. If you include Python.h, you +only see the new names. Dynamically linked modules have to be +recompiled. (Phase 3, fixing the rest of the sources, will be +executed gradually with the release later versions.) + +- The hooks for implementing "safe-python" (better called "restricted +execution") are in place. Specifically, the import statement is +implemented by calling the built-in function __import__, and the +built-in names used in a particular scope are taken from the +dictionary __builtins__ in that scope's global dictionary. See also +the new (unsupported, undocumented) module rexec.py. + +- The import statement now supports the syntax "import a.b.c" and +"from a.b.c import name". No officially supported implementation +exists, but one can be prototyped by replacing the built-in __import__ +function. A proposal by Ken Manheimer is provided as newimp.py. + +- All machinery used by the import statement (or the built-in +__import__ function) is now exposed through the new built-in module +"imp" (see the library reference manual). All dynamic loading +machinery is moved to the new file importdl.c. + +- Persistent storage is supported through the use of the modules +"pickle" and "shelve" (implemented in Python). There's also a "copy" +module implementing deepcopy and normal (shallow) copy operations. +See the library reference manual. + +- Documentation strings for many objects types are accessible through +the __doc__ attribute. Modules, classes and functions support special +syntax to initialize the __doc__ attribute: if the first statement +consists of just a string literal, that string literal becomes the +value of the __doc__ attribute. The default __doc__ attribute is +None. Documentation strings are also supported for built-in +functions, types and modules; however this feature hasn't been widely +used yet. See the 'new' module for an example. (Basically, the type +object's tp_doc field contains the doc string for the type, and the +4th member of the methodlist structure contains the doc string for the +method.) + +- The __coerce__ and __cmp__ methods for user-defined classes once +again work as expected. As an example, there's a new standard class +Complex in the library. + +- The functions posix.popen() and posix.fdopen() now have an optional +third argument to specify the buffer size, and default their second +(mode) argument to 'r' -- in analogy to the builtin open() function. +The same applies to posixfile.open() and the socket method makefile(). + +- The thread.exit_thread() function now raises SystemExit so that +'finally' clauses are honored and a memory leak is plugged. + +- Improved X11 and Motif support, by Sjoerd Mullender. This extension +is being maintained and distributed separately. + +- Improved support for the Apple Macintosh, in part by Jack Jansen, +e.g. interfaces to (a few) resource mananger functions, get/set file +type and creator, gestalt, sound manager, speech manager, MacTCP, comm +toolbox, and the think C console library. This is being maintained +and distributed separately. + +- Improved version for Windows NT, by Mark Hammond. This is being +maintained and distributed separately. + +- Used autoconf 2.0 to generate the configure script. Adapted +configure.in to use the new features in autoconf 2.0. + +- It now builds on the NeXT without intervention, even on the 3.3 +Sparc pre-release. + +- Characters passed to isspace() and friends are masked to nonnegative +values. + +- Correctly compute pow(-3.0, 3). + +- Fix portability problems with getopt (configure now checks for a +non-GNU getopt). + +- Don't add frozenmain.o to libPython.a. + +- Exceptions can now be classes. ALl built-in exceptions are still +string objects, but this will change in the future. + +- The socket module exports a long list of socket related symbols. +(More built-in modules will export their symbolic constants instead of +relying on a separately generated Python module.) + +- When a module object is deleted, it clears out its own dictionary. +This fixes a circularity in the references between functions and +their global dictionary. + +- Changed the error handling by [new]getargs() e.g. for "O&". + +- Dynamic loading of modules using shared libraries is supported for +several new platforms. + +- Support "O&", "[...]" and "{...}" in mkvalue(). + +- Extension to findmethod(): findmethodinchain() (where a chain is a +linked list of methodlist arrays). The calling interface for +findmethod() has changed: it now gets a pointer to the (static!) +methodlist structure rather than just to the function name -- this +saves copying flags etc. into the (short-lived) method object. + +- The callable() function is now public. + +- Object types can define a few new operations by setting function +pointers in the type object structure: tp_call defines how an object +is called, and tp_str defines how an object's str() is computed. + + +=================================== +==> Release 1.1.1 (10 Nov 1994) <== +=================================== + +This is a pure bugfix release again. See the ChangeLog file for details. + +One exception: a few new features were added to tkinter. + + +================================= +==> Release 1.1 (11 Oct 1994) <== +================================= + +This release adds several new features, improved configuration and +portability, and fixes more bugs than I can list here (including some +memory leaks). + +The source compiles and runs out of the box on more platforms than +ever -- including Windows NT. Makefiles or projects for a variety of +non-UNIX platforms are provided. + +APOLOGY: some new features are badly documented or not at all. I had +the choice -- postpone the new release indefinitely, or release it +now, with working code but some undocumented areas. The problem with +postponing the release is that people continue to suffer from existing +bugs, and send me patches based on the previous release -- which I +can't apply directly because my own source has changed. Also, some +new modules (like signal) have been ready for release for quite some +time, and people are anxiously waiting for them. In the case of +signal, the interface is simple enough to figure out without +documentation (if you're anxious enough :-). In this case it was not +simple to release the module on its own, since it relies on many small +patches elsewhere in the source. + +For most new Python modules, the source code contains comments that +explain how to use them. Documentation for the Tk interface, written +by Matt Conway, is available as tkinter-doc.tar.gz from the Python +home and mirror ftp sites (see Misc/FAQ for ftp addresses). For the +new operator overloading facilities, have a look at Demo/classes: +Complex.py and Rat.py show how to implement a numeric type without and +with __coerce__ method. Also have a look at the end of the Tutorial +document (Doc/tut.tex). If you're still confused: use the newsgroup +or mailing list. + + +New language features: + + - More flexible operator overloading for user-defined classes + (INCOMPATIBLE WITH PREVIOUS VERSIONS!) See end of tutorial. + + - Classes can define methods named __getattr__, __setattr__ and + __delattr__ to trap attribute accesses. See end of tutorial. + + - Classes can define method __call__ so instances can be called + directly. See end of tutorial. + + +New support facilities: + + - The Makefiles (for the base interpreter as well as for extensions) + now support creating dynamically loadable modules if the platform + supports shared libraries. + + - Passing the interpreter a .pyc file as script argument will execute + the code in that file. (On the Mac such files can be double-clicked!) + + - New Freeze script, to create independently distributable "binaries" + of Python programs -- look in Demo/freeze + + - Improved h2py script (in Demo/scripts) follows #includes and + supports macros with one argument + + - New module compileall generates .pyc files for all modules in a + directory (tree) without also executing them + + - Threads should work on more platforms + + +New built-in modules: + + - tkinter (support for Tcl's Tk widget set) is now part of the base + distribution + + - signal allows catching or ignoring UNIX signals (unfortunately still + undocumented -- any taker?) + + - termios provides portable access to POSIX tty settings + + - curses provides an interface to the System V curses library + + - syslog provides an interface to the (BSD?) syslog daemon + + - 'new' provides interfaces to create new built-in object types + (e.g. modules and functions) + + - sybase provides an interface to SYBASE database + + +New/obsolete built-in methods: + + - callable(x) tests whether x can be called + + - sockets now have a setblocking() method + + - sockets no longer have an allowbroadcast() method + + - socket methods send() and sendto() return byte count + + +New standard library modules: + + - types.py defines standard names for built-in types, e.g. StringType + + - urlparse.py parses URLs according to the latest Internet draft + + - uu.py does uuencode/uudecode (not the fastest in the world, but + quicker than installing uuencode on a non-UNIX machine :-) + + - New, faster and more powerful profile module.py + + - mhlib.py provides interface to MH folders and messages + + +New facilities for extension writers (unfortunately still +undocumented): + + - newgetargs() supports optional arguments and improved error messages + + - O!, O& O? formats for getargs allow more versatile type checking of + non-standard types + + - can register pending asynchronous callback, to be called the next + time the Python VM begins a new instruction (Py_AddPendingCall) + + - can register cleanup routines to be called when Python exits + (Py_AtExit) + + - makesetup script understands C++ files in Setup file (use file.C + or file.cc) + + - Make variable OPT is passed on to sub-Makefiles + + - An init<module>() routine may signal an error by not entering + the module in the module table and raising an exception instead + + - For long module names, instead of foobarbletchmodule.c you can + use foobarbletch.c + + - getintvalue() and getfloatvalue() try to convert any object + instead of requiring an "intobject" or "floatobject" + + - All the [new]getargs() formats that retrieve an integer value + will now also work if a float is passed + + - C function listtuple() converts list to tuple, fast + + - You should now call sigcheck() instead of intrcheck(); + sigcheck() also sets an exception when it returns nonzero + + +==================================== +==> Release 1.0.3 (14 July 1994) <== +==================================== + +This release consists entirely of bug fixes to the C sources; see the +head of ../ChangeLog for a complete list. Most important bugs fixed: + +- Sometimes the format operator (string%expr) would drop the last +character of the format string + +- Tokenizer looped when last line did not end in \n + +- Bug when triple-quoted string ended in quote plus newline + +- Typo in socketmodule (listen) (== instead of =) + +- typing vars() at the >>> prompt would cause recursive output + + +================================== +==> Release 1.0.2 (4 May 1994) <== +================================== + +Overview of the most visible changes. Bug fixes are not listed. See +also ChangeLog. + +Tokens +------ + +* String literals follow Standard C rules: they may be continued on +the next line using a backslash; adjacent literals are concatenated +at compile time. + +* A new kind of string literals, surrounded by triple quotes (""" or +'''), can be continued on the next line without a backslash. + +Syntax +------ + +* Function arguments may have a default value, e.g. def f(a, b=1); +defaults are evaluated at function definition time. This also applies +to lambda. + +* The try-except statement has an optional else clause, which is +executed when no exception occurs in the try clause. + +Interpreter +----------- + +* The result of a statement-level expression is no longer printed, +except_ for expressions entered interactively. Consequently, the -k +command line option is gone. + +* The result of the last printed interactive expression is assigned to +the variable '_'. + +* Access to implicit global variables has been speeded up by removing +an always-failing dictionary lookup in the dictionary of local +variables (mod suggested by Steve Makewski and Tim Peters). + +* There is a new command line option, -u, to force stdout and stderr +to be unbuffered. + +* Incorporated Steve Majewski's mods to import.c for dynamic loading +under AIX. + +* Fewer chances of dumping core when trying to reload or re-import +static built-in, dynamically loaded built-in, or frozen modules. + +* Loops over sequences now don't ask for the sequence's length when +they start, but try to access items 0, 1, 2, and so on until they hit +an IndexError. This makes it possible to create classes that generate +infinite or indefinite sequences a la Steve Majewski. This affects +for loops, the (not) in operator, and the built-in functions filter(), +map(), max(), min(), reduce(). + +Changed Built-in operations +--------------------------- + +* The '%' operator on strings (printf-style formatting) supports a new +feature (adapted from a patch by Donald Beaudry) to allow +'%(<key>)<format>' % {...} to take values from a dictionary by name +instead of from a tuple by position (see also the new function +vars()). + +* The '%s' formatting operator is changed to accept any type and +convert it to a string using str(). + +* Dictionaries with more than 20,000 entries can now be created +(thanks to Steve Kirsch). + +New Built-in Functions +---------------------- + +* vars() returns a dictionary containing the local variables; vars(m) +returns a dictionary containing the variables of module m. Note: +dir(x) is now equivalent to vars(x).keys(). + +Changed Built-in Functions +-------------------------- + +* open() has an optional third argument to specify the buffer size: 0 +for unbuffered, 1 for line buffered, >1 for explicit buffer size, <0 +for default. + +* open()'s second argument is now optional; it defaults to "r". + +* apply() now checks that its second argument is indeed a tuple. + +New Built-in Modules +-------------------- + +Changed Built-in Modules +------------------------ + +The thread module no longer supports exit_prog(). + +New Python Modules +------------------ + +* Module addpack contains a standard interface to modify sys.path to +find optional packages (groups of related modules). + +* Module urllib contains a number of functions to access +World-Wide-Web files specified by their URL. + +* Module httplib implements the client side of the HTTP protocol used +by World-Wide-Web servers. + +* Module gopherlib implements the client side of the Gopher protocol. + +* Module mailbox (by Jack Jansen) contains a parser for UNIX and MMDF +style mailbox files. + +* Module random contains various random distributions, e.g. gauss(). + +* Module lockfile locks and unlocks open files using fcntl (inspired +by a similar module by Andy Bensky). + +* Module ntpath (by Jaap Vermeulen) implements path operations for +Windows/NT. + +* Module test_thread (in Lib/test) contains a small test set for the +thread module. + +Changed Python Modules +---------------------- + +* The string module's expandvars() function is now documented and is +implemented in Python (using regular expressions) instead of forking +off a shell process. + +* Module rfc822 now supports accessing the header fields using the +mapping/dictionary interface, e.g. h['subject']. + +* Module pdb now makes it possible to set a break on a function +(syntax: break <expression>, where <expression> yields a function +object). + +Changed Demos +------------- + +* The Demo/scripts/freeze.py script is working again (thanks to Jaap +Vermeulen). + +New Demos +--------- + +* Demo/threads/Generator.py is a proposed interface for restartable +functions a la Tim Peters. + +* Demo/scripts/newslist.py, by Quentin Stafford-Fraser, generates a +directory full of HTML pages which between them contain links to all +the newsgroups available on your server. + +* Demo/dns contains a DNS (Domain Name Server) client. + +* Demo/lutz contains miscellaneous demos by Mark Lutz (e.g. psh.py, a +nice enhanced Python shell!!!). + +* Demo/turing contains a Turing machine by Amrit Prem. + +Documentation +------------- + +* Documented new language features mentioned above (but not all new +modules). + +* Added a chapter to the Tutorial describing recent additions to +Python. + +* Clarified some sentences in the reference manual, +e.g. break/continue, local/global scope, slice assignment. + +Source Structure +---------------- + +* Moved Include/tokenizer.h to Parser/tokenizer.h. + +* Added Python/getopt.c for systems that don't have it. + +Emacs mode +---------- + +* Indentation of continuated lines is done more intelligently; +consequently the variable py-continuation-offset is gone. + + +======================================== +==> Release 1.0.1 (15 February 1994) <== +======================================== + +* Many portability fixes should make it painless to build Python on +several new platforms, e.g. NeXT, SEQUENT, WATCOM, DOS, and Windows. + +* Fixed test for <stdarg.h> -- this broke on some platforms. + +* Fixed test for shared library dynalic loading -- this broke on SunOS +4.x using the GNU loader. + +* Changed order and number of SVR4 networking libraries (it is now +-lsocket -linet -lnsl, if these libraries exist). + +* Installing the build intermediate stages with "make libainstall" now +also installs config.c.in, Setup and makesetup, which are used by the +new Extensions mechanism. + +* Improved README file contains more hints and new troubleshooting +section. + +* The built-in module strop now defines fast versions of three more +functions of the standard string module: atoi(), atol() and atof(). +The strop versions of atoi() and atol() support an optional second +argument to specify the base (default 10). NOTE: you don't have to +explicitly import strop to use the faster versions -- the string +module contains code to let versions from stop override the default +versions. + +* There is now a working Lib/dospath.py for those who use Python under +DOS (or Windows). Thanks, Jaap! + +* There is now a working Modules/dosmodule.c for DOS (or Windows) +system calls. + +* Lib.os.py has been reorganized (making it ready for more operating +systems). + +* Lib/ospath.py is now obsolete (use os.path instead). + +* Many fixes to the tutorial to make it match Python 1.0. Thanks, +Tim! + +* Fixed Doc/Makefile, Doc/README and various scripts there. + +* Added missing description of fdopen to Doc/libposix.tex. + +* Made cleanup() global, for the benefit of embedded applications. + +* Added parsing of addresses and dates to Lib/rfc822.py. + +* Small fixes to Lib/aifc.py, Lib/sunau.py, Lib/tzparse.py to make +them usable at all. + +* New module Lib/wave.py reads RIFF (*.wav) audio files. + +* Module Lib/filewin.py moved to Lib/stdwin/filewin.py where it +belongs. + +* New options and comments for Modules/makesetup (used by new +Extension mechanism). + +* Misc/HYPE contains text of announcement of 1.0.0 in comp.lang.misc +and elsewhere. + +* Fixed coredump in filter(None, 'abcdefg'). + + +======================================= +==> Release 1.0.0 (26 January 1994) <== +======================================= + +As is traditional, so many things have changed that I can't pretend to +be complete in these release notes, but I'll try anyway :-) + +Note that the very last section is labeled "remaining bugs". + + +Source organization and build process +------------------------------------- + +* The sources have finally been split: instead of a single src +subdirectory there are now separate directories Include, Parser, +Grammar, Objects, Python and Modules. Other directories also start +with a capital letter: Misc, Doc, Lib, Demo. + +* A few extensions (notably Amoeba and X support) have been moved to a +separate subtree Extensions, which is no longer in the core +distribution, but separately ftp'able as extensions.tar.Z. (The +distribution contains a placeholder Ext-dummy with a description of +the Extensions subtree as well as the most recent versions of the +scripts used there.) + +* A few large specialized demos (SGI video and www) have been +moved to a separate subdirectory Demo2, which is no longer in the core +distribution, but separately ftp'able as demo2.tar.Z. + +* Parts of the standard library have been moved to subdirectories: +there are now standard subdirectories stdwin, test, sgi and sun4. + +* The configuration process has radically changed: I now use GNU +autoconf. This makes it much easier to build on new Unix flavors, as +well as fully supporting VPATH (if your Make has it). The scripts +Configure.py and Addmodule.sh are no longer needed. Many source files +have been adapted in order to work with the symbols that the configure +script generated by autoconf defines (or not); the resulting source is +much more portable to different C compilers and operating systems, +even non Unix systems (a Mac port was done in an afternoon). See the +toplevel README file for a description of the new build process. + +* GNU readline (a slightly newer version) is now a subdirectory of the +Python toplevel. It is still not automatically configured (being +totally autoconf-unaware :-). One problem has been solved: typing +Control-C to a readline prompt will now work. The distribution no +longer contains a "super-level" directory (above the python toplevel +directory), and dl, dl-dld and GNU dld are no longer part of the +Python distribution (you can still ftp them from +ftp.cwi.nl:/pub/dynload). + +* The DOS functions have been taken out of posixmodule.c and moved +into a separate file dosmodule.c. + +* There's now a separate file version.c which contains nothing but +the version number. + +* The actual main program is now contained in config.c (unless NO_MAIN +is defined); pythonmain.c now contains a function realmain() which is +called from config.c's main(). + +* All files needed to use the built-in module md5 are now contained in +the distribution. The module has been cleaned up considerably. + + +Documentation +------------- + +* The library manual has been split into many more small latex files, +so it is easier to edit Doc/lib.tex file to create a custom library +manual, describing only those modules supported on your system. (This +is not automated though.) + +* A fourth manual has been added, titled "Extending and Embedding the +Python Interpreter" (Doc/ext.tex), which collects information about +the interpreter which was previously spread over several files in the +misc subdirectory. + +* The entire documentation is now also available on-line for those who +have a WWW browser (e.g. NCSA Mosaic). Point your browser to the URL +"http://www.cwi.nl/~guido/Python.html". + + +Syntax +------ + +* Strings may now be enclosed in double quotes as well as in single +quotes. There is no difference in interpretation. The repr() of +string objects will use double quotes if the string contains a single +quote and no double quotes. Thanks to Amrit Prem for these changes! + +* There is a new keyword 'exec'. This replaces the exec() built-in +function. If a function contains an exec statement, local variable +optimization is not performed for that particular function, thus +making assignment to local variables in exec statements less +confusing. (As a consequence, os.exec and python.exec have been +renamed to execv.) + +* There is a new keyword 'lambda'. An expression of the form + + lambda <parameters> : <expression> + +yields an anonymous function. This is really only syntactic sugar; +you can just as well define a local function using + + def some_temporary_name(<parameters>): return <expression> + +Lambda expressions are particularly useful in combination with map(), +filter() and reduce(), described below. Thanks to Amrit Prem for +submitting this code (as well as map(), filter(), reduce() and +xrange())! + + +Built-in functions +------------------ + +* The built-in module containing the built-in functions is called +__builtin__ instead of builtin. + +* New built-in functions map(), filter() and reduce() perform standard +functional programming operations (though not lazily): + +- map(f, seq) returns a new sequence whose items are the items from +seq with f() applied to them. + +- filter(f, seq) returns a subsequence of seq consisting of those +items for which f() is true. + +- reduce(f, seq, initial) returns a value computed as follows: + acc = initial + for item in seq: acc = f(acc, item) + return acc + +* New function xrange() creates a "range object". Its arguments are +the same as those of range(), and when used in a for loop a range +objects also behaves identical. The advantage of xrange() over +range() is that its representation (if the range contains many +elements) is much more compact than that of range(). The disadvantage +is that the result cannot be used to initialize a list object or for +the "Python idiom" [RED, GREEN, BLUE] = range(3). On some modern +architectures, benchmarks have shown that "for i in range(...): ..." +actually executes *faster* than "for i in xrange(...): ...", but on +memory starved machines like PCs running DOS range(100000) may be just +too big to be represented at all... + +* Built-in function exec() has been replaced by the exec statement -- +see above. + + +The interpreter +--------------- + +* Syntax errors are now not printed to stderr by the parser, but +rather the offending line and other relevant information are packed up +in the SyntaxError exception argument. When the main loop catches a +SyntaxError exception it will print the error in the same format as +previously, but at the proper position in the stack traceback. + +* You can now set a maximum to the number of traceback entries +printed by assigning to sys.tracebacklimit. The default is 1000. + +* The version number in .pyc files has changed yet again. + +* It is now possible to have a .pyc file without a corresponding .py +file. (Warning: this may break existing installations if you have an +old .pyc file lingering around somewhere on your module search path +without a corresponding .py file, when there is a .py file for a +module of the same name further down the path -- the new interpreter +will find the first .pyc file and complain about it, while the old +interpreter would ignore it and use the .py file further down.) + +* The list sys.builtin_module_names is now sorted and also contains +the names of a few hardwired built-in modules (sys, __main__ and +__builtin__). + +* A module can now find its own name by accessing the global variable +__name__. Assigning to this variable essentially renames the module +(it should also be stored under a different key in sys.modules). +A neat hack follows from this: a module that wants to execute a main +program when called as a script no longer needs to compare +sys.argv[0]; it can simply do "if __name__ == '__main__': main()". + +* When an object is printed by the print statement, its implementation +of str() is used. This means that classes can define __str__(self) to +direct how their instances are printed. This is different from +__repr__(self), which should define an unambigous string +representation of the instance. (If __str__() is not defined, it +defaults to __repr__().) + +* Functions and code objects can now be compared meaningfully. + +* On systems supporting SunOS or SVR4 style shared libraries, dynamic +loading of modules using shared libraries is automatically configured. +Thanks to Bill Jansen and Denis Severson for contributing this change! + + +Built-in objects +---------------- + +* File objects have acquired a new method writelines() which is the +reverse of readlines(). (It does not actually write lines, just a +list of strings, but the symmetry makes the choice of name OK.) + + +Built-in modules +---------------- + +* Socket objects no longer support the avail() method. Use the select +module instead, or use this function to replace it: + + def avail(f): + import select + return f in select.select([f], [], [], 0)[0] + +* Initialization of stdwin is done differently. It actually modifies +sys.argv (taking out the options the X version of stdwin recognizes) +the first time it is imported. + +* A new built-in module parser provides a rudimentary interface to the +python parser. Corresponding standard library modules token and symbol +defines the numeric values of tokens and non-terminal symbols. + +* The posix module has aquired new functions setuid(), setgid(), +execve(), and exec() has been renamed to execv(). + +* The array module is extended with 8-byte object swaps, the 'i' +format character, and a reverse() method. The read() and write() +methods are renamed to fromfile() and tofile(). + +* The rotor module has freed of portability bugs. This introduces a +backward compatibility problem: strings encoded with the old rotor +module can't be decoded by the new version. + +* For select.select(), a timeout (4th) argument of None means the same +as leaving the timeout argument out. + +* Module strop (and hence standard library module string) has aquired +a new function: rindex(). Thanks to Amrit Prem! + +* Module regex defines a new function symcomp() which uses an extended +regular expression syntax: parenthesized subexpressions may be labeled +using the form "\(<labelname>...\)", and the group() method can return +sub-expressions by name. Thanks to Tracy Tims for these changes! + +* Multiple threads are now supported on Solaris 2. Thanks to Sjoerd +Mullender! + + +Standard library modules +------------------------ + +* The library is now split in several subdirectories: all stuff using +stdwin is in Lib/stdwin, all SGI specific (or SGI Indigo or GL) stuff +is in Lib/sgi, all Sun Sparc specific stuff is in Lib/sun4, and all +test modules are in Lib/test. The default module search path will +include all relevant subdirectories by default. + +* Module os now knows about trying to import dos. It defines +functions execl(), execle(), execlp() and execvp(). + +* New module dospath (should be attacked by a DOS hacker though). + +* All modules defining classes now define __init__() constructors +instead of init() methods. THIS IS AN INCOMPATIBLE CHANGE! + +* Some minor changes and bugfixes module ftplib (mostly Steve +Majewski's suggestions); the debug() method is renamed to +set_debuglevel(). + +* Some new test modules (not run automatically by testall though): +test_audioop, test_md5, test_rgbimg, test_select. + +* Module string now defines rindex() and rfind() in analogy of index() +and find(). It also defines atof() and atol() (and corresponding +exceptions) in analogy to atoi(). + +* Added help() functions to modules profile and pdb. + +* The wdb debugger (now in Lib/stdwin) now shows class or instance +variables on a double click. Thanks to Sjoerd Mullender! + +* The (undocumented) module lambda has gone -- you couldn't import it +any more, and it was basically more a demo than a library module... + + +Multimedia extensions +--------------------- + +* The optional built-in modules audioop and imageop are now standard +parts of the interpreter. Thanks to Sjoerd Mullender and Jack Jansen +for contributing this code! + +* There's a new operation in audioop: minmax(). + +* There's a new built-in module called rgbimg which supports portable +efficient reading of SGI RCG image files. Thanks also to Paul +Haeberli for the original code! (Who will contribute a GIF reader?) + +* The module aifc is gone -- you should now always use aifc, which has +received a facelift. + +* There's a new module sunau., for reading Sun (and NeXT) audio files. + +* There's a new module audiodev which provides a uniform interface to +(SGI Indigo and Sun Sparc) audio hardware. + +* There's a new module sndhdr which recognizes various sound files by +looking in their header and checking for various magic words. + + +Optimizations +------------- + +* Most optimizations below can be configured by compile-time flags. +Thanks to Sjoerd Mullender for submitting these optimizations! + +* Small integers (default -1..99) are shared -- i.e. if two different +functions compute the same value it is possible (but not +guaranteed!!!) that they return the same *object*. Python programs +can detect this but should *never* rely on it. + +* Empty tuples (which all compare equal) are shared in the same +manner. + +* Tuples of size up to 20 (default) are put in separate free lists +when deallocated. + +* There is a compile-time option to cache a string's hash function, +but this appeared to have a negligeable effect, and as it costs 4 +bytes per string it is disabled by default. + + +Embedding Python +---------------- + +* The initialization interface has been simplified somewhat. You now +only call "initall()" to initialize the interpreter. + +* The previously announced renaming of externally visible identifiers +has not been carried out. It will happen in a later release. Sorry. + + +Miscellaneous bugs that have been fixed +--------------------------------------- + +* All known portability bugs. + +* Version 0.9.9 dumped core in <listobject>.sort() which has been +fixed. Thanks to Jaap Vermeulen for fixing this and posting the fix +on the mailing list while I was away! + +* Core dump on a format string ending in '%', e.g. in the expression +'%' % None. + +* The array module yielded a bogus result for concatenation (a+b would +yield a+a). + +* Some serious memory leaks in strop.split() and strop.splitfields(). + +* Several problems with the nis module. + +* Subtle problem when copying a class method from another class +through assignment (the method could not be called). + + +Remaining bugs +-------------- + +* One problem with 64-bit machines remains -- since .pyc files are +portable and use only 4 bytes to represent an integer object, 64-bit +integer literals are silently truncated when written into a .pyc file. +Work-around: use eval('123456789101112'). + +* The freeze script doesn't work any more. A new and more portable +one can probably be cooked up using tricks from Extensions/mkext.py. + +* The dos support hasn't been tested yet. (Really Soon Now we should +have a PC with a working C compiler!) + + +=================================== +==> Release 0.9.9 (29 Jul 1993) <== +=================================== + +I *believe* these are the main user-visible changes in this release, +but there may be others. SGI users may scan the {src,lib}/ChangeLog +files for improvements of some SGI specific modules, e.g. aifc and +cl. Developers of extension modules should also read src/ChangeLog. + + +Naming of C symbols used by the Python interpreter +-------------------------------------------------- + +* This is the last release using the current naming conventions. New +naming conventions are explained in the file misc/NAMING. +Summarizing, all externally visible symbols get (at least) a "Py" +prefix, and most functions are renamed to the standard form +PyModule_FunctionName. + +* Writers of extensions are urged to start using the new naming +conventions. The next release will use the new naming conventions +throughout (it will also have a different source directory +structure). + +* As a result of the preliminary work for the great renaming, many +functions that were accidentally global have been made static. + + +BETA X11 support +---------------- + +* There are now modules interfacing to the X11 Toolkit Intrinsics, the +Athena widgets, and the Motif 1.1 widget set. These are not yet +documented except through the examples and README file in the demo/x11 +directory. It is expected that this interface will be replaced by a +more powerful and correct one in the future, which may or may not be +backward compatible. In other words, this part of the code is at most +BETA level software! (Note: the rest of Python is rock solid as ever!) + +* I understand that the above may be a bit of a disappointment, +however my current schedule does not allow me to change this situation +before putting the release out of the door. By releasing it +undocumented and buggy, at least some of the (working!) demo programs, +like itr (my Internet Talk Radio browser) become available to a larger +audience. + +* There are also modules interfacing to SGI's "Glx" widget (a GL +window wrapped in a widget) and to NCSA's "HTML" widget (which can +format HyperText Markup Language, the document format used by the +World Wide Web). + +* I've experienced some problems when building the X11 support. In +particular, the Xm and Xaw widget sets don't go together, and it +appears that using X11R5 is better than using X11R4. Also the threads +module and its link time options may spoil things. My own strategy is +to build two Python binaries: one for use with X11 and one without +it, which can contain a richer set of built-in modules. Don't even +*think* of loading the X11 modules dynamically... + + +Environmental changes +--------------------- + +* Compiled files (*.pyc files) created by this Python version are +incompatible with those created by the previous version. Both +versions detect this and silently create a correct version, but it +means that it is not a good idea to use the same library directory for +an old and a new interpreter, since they will start to "fight" over +the *.pyc files... + +* When a stack trace is printed, the exception is printed last instead +of first. This means that if the beginning of the stack trace +scrolled out of your window you can still see what exception caused +it. + +* Sometimes interrupting a Python operation does not work because it +hangs in a blocking system call. You can now kill the interpreter by +interrupting it three times. The second time you interrupt it, a +message will be printed telling you that the third interrupt will kill +the interpreter. The "sys.exitfunc" feature still makes limited +clean-up possible in this case. + + +Changes to the command line interface +------------------------------------- + +* The python usage message is now much more informative. + +* New option -i enters interactive mode after executing a script -- +useful for debugging. + +* New option -k raises an exception when an expression statement +yields a value other than None. + +* For each option there is now also a corresponding environment +variable. + + +Using Python as an embedded language +------------------------------------ + +* The distribution now contains (some) documentation on the use of +Python as an "embedded language" in other applications, as well as a +simple example. See the file misc/EMBEDDING and the directory embed/. + + +Speed improvements +------------------ + +* Function local variables are now generally stored in an array and +accessed using an integer indexing operation, instead of through a +dictionary lookup. (This compensates the somewhat slower dictionary +lookup caused by the generalization of the dictionary module.) + + +Changes to the syntax +--------------------- + +* Continuation lines can now *sometimes* be written without a +backslash: if the continuation is contained within nesting (), [] or +{} brackets the \ may be omitted. There's a much improved +python-mode.el in the misc directory which knows about this as well. + +* You can no longer use an empty set of parentheses to define a class +without base classes. That is, you no longer write this: + + class Foo(): # syntax error + ... + +You must write this instead: + + class Foo: + ... + +This was already the preferred syntax in release 0.9.8 but many +people seemed not to have picked it up. There's a Python script that +fixes old code: demo/scripts/classfix.py. + +* There's a new reserved word: "access". The syntax and semantics are +still subject of of research and debate (as well as undocumented), but +the parser knows about the keyword so you must not use it as a +variable, function, or attribute name. + + +Changes to the semantics of the language proper +----------------------------------------------- + +* The following compatibility hack is removed: if a function was +defined with two or more arguments, and called with a single argument +that was a tuple with just as many arguments, the items of this tuple +would be used as the arguments. This is no longer supported. + + +Changes to the semantics of classes and instances +------------------------------------------------- + +* Class variables are now also accessible as instance variables for +reading (assignment creates an instance variable which overrides the +class variable of the same name though). + +* If a class attribute is a user-defined function, a new kind of +object is returned: an "unbound method". This contains a pointer to +the class and can only be called with a first argument which is a +member of that class (or a derived class). + +* If a class defines a method __init__(self, arg1, ...) then this +method is called when a class instance is created by the classname() +construct. Arguments passed to classname() are passed to the +__init__() method. The __init__() methods of base classes are not +automatically called; the derived __init__() method must call these if +necessary (this was done so the derived __init__() method can choose +the call order and arguments for the base __init__() methods). + +* If a class defines a method __del__(self) then this method is called +when an instance of the class is about to be destroyed. This makes it +possible to implement clean-up of external resources attached to the +instance. As with __init__(), the __del__() methods of base classes +are not automatically called. If __del__ manages to store a reference +to the object somewhere, its destruction is postponed; when the object +is again about to be destroyed its __del__() method will be called +again. + +* Classes may define a method __hash__(self) to allow their instances +to be used as dictionary keys. This must return a 32-bit integer. + + +Minor improvements +------------------ + +* Function and class objects now know their name (the name given in +the 'def' or 'class' statement that created them). + +* Class instances now know their class name. + + +Additions to built-in operations +-------------------------------- + +* The % operator with a string left argument implements formatting +similar to sprintf() in C. The right argument is either a single +value or a tuple of values. All features of Standard C sprintf() are +supported except %p. + +* Dictionaries now support almost any key type, instead of just +strings. (The key type must be an immutable type or must be a class +instance where the class defines a method __hash__(), in order to +avoid losing track of keys whose value may change.) + +* Built-in methods are now compared properly: when comparing x.meth1 +and y.meth2, if x is equal to y and the methods are defined by the +same function, x.meth1 compares equal to y.meth2. + + +Additions to built-in functions +------------------------------- + +* str(x) returns a string version of its argument. If the argument is +a string it is returned unchanged, otherwise it returns `x`. + +* repr(x) returns the same as `x`. (Some users found it easier to +have this as a function.) + +* round(x) returns the floating point number x rounded to an whole +number, represented as a floating point number. round(x, n) returns x +rounded to n digits. + +* hasattr(x, name) returns true when x has an attribute with the given +name. + +* hash(x) returns a hash code (32-bit integer) of an arbitrary +immutable object's value. + +* id(x) returns a unique identifier (32-bit integer) of an arbitrary +object. + +* compile() compiles a string to a Python code object. + +* exec() and eval() now support execution of code objects. + + +Changes to the documented part of the library (standard modules) +---------------------------------------------------------------- + +* os.path.normpath() (a.k.a. posixpath.normpath()) has been fixed so +the border case '/foo/..' returns '/' instead of ''. + +* A new function string.find() is added with similar semantics to +string.index(); however when it does not find the given substring it +returns -1 instead of raising string.index_error. + + +Changes to built-in modules +--------------------------- + +* New optional module 'array' implements operations on sequences of +integers or floating point numbers of a particular size. This is +useful to manipulate large numerical arrays or to read and write +binary files consisting of numerical data. + +* Regular expression objects created by module regex now support a new +method named group(), which returns one or more \(...\) groups by number. +The number of groups is increased from 10 to 100. + +* Function compile() in module regex now supports an optional mapping +argument; a variable casefold is added to the module which can be used +as a standard uppercase to lowercase mapping. + +* Module time now supports many routines that are defined in the +Standard C time interface (<time.h>): gmtime(), localtime(), +asctime(), ctime(), mktime(), as well as these variables (taken from +System V): timezone, altzone, daylight and tzname. (The corresponding +functions in the undocumented module calendar have been removed; the +undocumented and unfinished module tzparse is now obsolete and will +disappear in a future release.) + +* Module strop (the fast built-in version of standard module string) +now uses C's definition of whitespace instead of fixing it to space, +tab and newline; in practice this usually means that vertical tab, +form feed and return are now also considered whitespace. It exports +the string of characters that are considered whitespace as well as the +characters that are considered lowercase or uppercase. + +* Module sys now defines the variable builtin_module_names, a list of +names of modules built into the current interpreter (including not +yet imported, but excluding two special modules that always have to be +defined -- sys and builtin). + +* Objects created by module sunaudiodev now also support flush() and +close() methods. + +* Socket objects created by module socket now support an optional +flags argument for their methods sendto() and recvfrom(). + +* Module marshal now supports dumping to and loading from strings, +through the functions dumps() and loads(). + +* Module stdwin now supports some new functionality. You may have to +ftp the latest version: ftp.cwi.nl:/pub/stdwin/stdwinforviews.tar.Z.) + + +Bugs fixed +---------- + +* Fixed comparison of negative long integers. + +* The tokenizer no longer botches input lines longer than BUFSIZ. + +* Fixed several severe memory leaks in module select. + +* Fixed memory leaks in modules socket and sv. + +* Fixed memory leak in divmod() for long integers. + +* Problems with definition of floatsleep() on Suns fixed. + +* Many portability bugs fixed (and undoubtedly new ones added :-). + + +Changes to the build procedure +------------------------------ + +* The Makefile supports some new targets: "make default" and "make +all". Both are by normally equivalent to "make python". + +* The Makefile no longer uses $> since it's not supported by all +versions of Make. + +* The header files now all contain #ifdef constructs designed to make +it safe to include the same header file twice, as well as support for +inclusion from C++ programs (automatic extern "C" { ... } added). + + +Freezing Python scripts +----------------------- + +* There is now some support for "freezing" a Python script as a +stand-alone executable binary file. See the script +demo/scripts/freeze.py. It will require some site-specific tailoring +of the script to get this working, but is quite worthwhile if you write +Python code for other who may not have built and installed Python. + + +MS-DOS +------ + +* A new MS-DOS port has been done, using MSC 6.0 (I believe). Thanks, +Marcel van der Peijl! This requires fewer compatibility hacks in +posixmodule.c. The executable is not yet available but will be soon +(check the mailing list). + +* The default PYTHONPATH has changed. + + +Changes for developers of extension modules +------------------------------------------- + +* Read src/ChangeLog for full details. + + +SGI specific changes +-------------------- + +* Read src/ChangeLog for full details. + + +================================== +==> Release 0.9.8 (9 Jan 1993) <== +================================== + +I claim no completeness here, but I've tried my best to scan the log +files throughout my source tree for interesting bits of news. A more +complete account of the changes is to be found in the various +ChangeLog files. See also "News for release 0.9.7beta" below if you're +still using release 0.9.6, and the file HISTORY if you have an even +older release. + + --Guido + + +Changes to the language proper +------------------------------ + +There's only one big change: the conformance checking for function +argument lists (of user-defined functions only) is stricter. Earlier, +you could get away with the following: + + (a) define a function of one argument and call it with any + number of arguments; if the actual argument count wasn't + one, the function would receive a tuple containing the + arguments arguments (an empty tuple if there were none). + + (b) define a function of two arguments, and call it with more + than two arguments; if there were more than two arguments, + the second argument would be passed as a tuple containing + the second and further actual arguments. + +(Note that an argument (formal or actual) that is a tuple is counted as +one; these rules don't apply inside such tuples, only at the top level +of the argument list.) + +Case (a) was needed to accommodate variable-length argument lists; +there is now an explicit "varargs" feature (precede the last argument +with a '*'). Case (b) was needed for compatibility with old class +definitions: up to release 0.9.4 a method with more than one argument +had to be declared as "def meth(self, (arg1, arg2, ...)): ...". +Version 0.9.6 provide better ways to handle both casees, bot provided +backward compatibility; version 0.9.8 retracts the compatibility hacks +since they also cause confusing behavior if a function is called with +the wrong number of arguments. + +There's a script that helps converting classes that still rely on (b), +provided their methods' first argument is called "self": +demo/scripts/methfix.py. + +If this change breaks lots of code you have developed locally, try +#defining COMPAT_HACKS in ceval.c. + +(There's a third compatibility hack, which is the reverse of (a): if a +function is defined with two or more arguments, and called with a +single argument that is a tuple with just as many arguments, the items +of this tuple will be used as the arguments. Although this can (and +should!) be done using the built-in function apply() instead, it isn't +withdrawn yet.) + + +One minor change: comparing instance methods works like expected, so +that if x is an instance of a user-defined class and has a method m, +then (x.m==x.m) yields 1. + + +The following was already present in 0.9.7beta, but not explicitly +mentioned in the NEWS file: user-defined classes can now define types +that behave in almost allrespects like numbers. See +demo/classes/Rat.py for a simple example. + + +Changes to the build process +---------------------------- + +The Configure.py script and the Makefile has been made somewhat more +bullet-proof, after reports of (minor) trouble on certain platforms. + +There is now a script to patch Makefile and config.c to add a new +optional built-in module: Addmodule.sh. Read the script before using! + +Useing Addmodule.sh, all optional modules can now be configured at +compile time using Configure.py, so there are no modules left that +require dynamic loading. + +The Makefile has been fixed to make it easier to use with the VPATH +feature of some Make versions (e.g. SunOS). + + +Changes affecting portability +----------------------------- + +Several minor portability problems have been solved, e.g. "malloc.h" +has been renamed to "mymalloc.h", "strdup.c" is no longer used, and +the system now tolerates malloc(0) returning 0. + +For dynamic loading on the SGI, Jack Jansen's dl 1.6 is now +distributed with Python. This solves several minor problems, in +particular scripts invoked using #! can now use dynamic loading. + + +Changes to the interpreter interface +------------------------------------ + +On popular demand, there's finally a "profile" feature for interactive +use of the interpreter. If the environment variable $PYTHONSTARTUP is +set to the name of an existing file, Python statements in this file +are executed when the interpreter is started in interactive mode. + +There is a new clean-up mechanism, complementing try...finally: if you +assign a function object to sys.exitfunc, it will be called when +Python exits or receives a SIGTERM or SIGHUP signal. + +The interpreter is now generally assumed to live in +/usr/local/bin/python (as opposed to /usr/local/python). The script +demo/scripts/fixps.py will update old scripts in place (you can easily +modify it to do other similar changes). + +Most I/O that uses sys.stdin/stdout/stderr will now use any object +assigned to those names as long as the object supports readline() or +write() methods. + +The parser stack has been increased to 500 to accommodate more +complicated expressions (7 levels used to be the practical maximum, +it's now about 38). + +The limit on the size of the *run-time* stack has completely been +removed -- this means that tuple or list displays can contain any +number of elements (formerly more than 50 would crash the +interpreter). + + +Changes to existing built-in functions and methods +-------------------------------------------------- + +The built-in functions int(), long(), float(), oct() and hex() now +also apply to class instalces that define corresponding methods +(__int__ etc.). + + +New built-in functions +---------------------- + +The new functions str() and repr() convert any object to a string. +The function repr(x) is in all respects equivalent to `x` -- some +people prefer a function for this. The function str(x) does the same +except if x is already a string -- then it returns x unchanged +(repr(x) adds quotes and escapes "funny" characters as octal escapes). + +The new function cmp(x, y) returns -1 if x<y, 0 if x==y, 1 if x>y. + + +Changes to general built-in modules +----------------------------------- + +The time module's functions are more general: time() returns a +floating point number and sleep() accepts one. Their accuracies +depends on the precision of the system clock. Millisleep is no longer +needed (although it still exists for now), but millitimer is still +needed since on some systems wall clock time is only available with +seconds precision, while a source of more precise time exists that +isn't synchronized with the wall clock. (On UNIX systems that support +the BSD gettimeofday() function, time.time() is as time.millitimer().) + +The string representation of a file object now includes an address: +'<file 'filename', mode 'r' at #######>' where ###### is a hex number +(the object's address) to make it unique. + +New functions added to posix: nice(), setpgrp(), and if your system +supports them: setsid(), setpgid(), tcgetpgrp(), tcsetpgrp(). + +Improvements to the socket module: socket objects have new methods +getpeername() and getsockname(), and the {get,set}sockopt methods can +now get/set any kind of option using strings built with the new struct +module. And there's a new function fromfd() which creates a socket +object given a file descriptor (useful for servers started by inetd, +which have a socket connected to stdin and stdout). + + +Changes to SGI-specific built-in modules +---------------------------------------- + +The FORMS library interface (fl) now requires FORMS 2.1a. Some new +functions have been added and some bugs have been fixed. + +Additions to al (audio library interface): added getname(), +getdefault() and getminmax(). + +The gl modules doesn't call "foreground()" when initialized (this +caused some problems) like it dit in 0.9.7beta (but not before). +There's a new gl function 'gversion() which returns a version string. + +The interface to sv (Indigo video interface) has totally changed. +(Sorry, still no documentation, but see the examples in +demo/sgi/{sv,video}.) + + +Changes to standard library modules +----------------------------------- + +Most functions in module string are now much faster: they're actually +implemented in C. The module containing the C versions is called +"strop" but you should still import "string" since strop doesn't +provide all the interfaces defined in string (and strop may be renamed +to string when it is complete in a future release). + +string.index() now accepts an optional third argument giving an index +where to start searching in the first argument, so you can find second +and further occurrences (this is similar to the regular expression +functions in regex). + +The definition of what string.splitfields(anything, '') should return +is changed for the last time: it returns a singleton list containing +its whole first argument unchanged. This is compatible with +regsub.split() which also ignores empty delimiter matches. + +posixpath, macpath: added dirname() and normpath() (and basename() to +macpath). + +The mainloop module (for use with stdwin) can now demultiplex input +from other sources, as long as they can be polled with select(). + + +New built-in modules +-------------------- + +Module struct defines functions to pack/unpack values to/from strings +representing binary values in native byte order. + +Module strop implements C versions of many functions from string (see +above). + +Optional module fcntl defines interfaces to fcntl() and ioctl() -- +UNIX only. (Not yet properly documented -- see however src/fcntl.doc.) + +Optional module mpz defines an interface to an altaernative long +integer implementation, the GNU MPZ library. + +Optional module md5 uses the GNU MPZ library to calculate MD5 +signatures of strings. + +There are also optional new modules specific to SGI machines: imageop +defines some simple operations to images represented as strings; sv +interfaces to the Indigo video board; cl interfaces to the (yet +unreleased) compression library. + + +New standard library modules +---------------------------- + +(Unfortunately the following modules are not all documented; read the +sources to find out more about them!) + +autotest: run testall without showing any output unless it differs +from the expected output + +bisect: use bisection to insert or find an item in a sorted list + +colorsys: defines conversions between various color systems (e.g. RGB +<-> YUV) + +nntplib: a client interface to NNTP servers + +pipes: utility to construct pipeline from templates, e.g. for +conversion from one file format to another using several utilities. + +regsub: contains three functions that are more or less compatible with +awk functions of the same name: sub() and gsub() do string +substitution, split() splits a string using a regular expression to +define how separators are define. + +test_types: test operations on the built-in types of Python + +toaiff: convert various audio file formats to AIFF format + +tzparse: parse the TZ environment parameter (this may be less general +than it could be, let me know if you fix it). + +(Note that the obsolete module "path" no longer exists.) + + +New SGI-specific library modules +-------------------------------- + +CL: constants for use with the built-in compression library interface (cl) + +Queue: a multi-producer, multi-consumer queue class implemented for +use with the built-in thread module + +SOCKET: constants for use with built-in module socket, e.g. to set/get +socket options. This is SGI-specific because the constants to be +passed are system-dependent. You can generate a version for your own +system by running the script demo/scripts/h2py.py with +/usr/include/sys/socket.h as input. + +cddb: interface to the database used by the CD player + +torgb: convert various image file types to rgb format (requires pbmplus) + + +New demos +--------- + +There's an experimental interface to define Sun RPC clients and +servers in demo/rpc. + +There's a collection of interfaces to WWW, WAIS and Gopher (both +Python classes and program providing a user interface) in demo/www. +This includes a program texi2html.py which converts texinfo files to +HTML files (the format used hy WWW). + +The ibrowse demo has moved from demo/stdwin/ibrowse to demo/ibrowse. + +For SGI systems, there's a whole collection of programs and classes +that make use of the Indigo video board in demo/sgi/{sv,video}. This +represents a significant amount of work that we're giving away! + +There are demos "rsa" and "md5test" that exercise the mpz and md5 +modules, respectively. The rsa demo is a complete implementation of +the RSA public-key cryptosystem! + +A bunch of games and examples submitted by Stoffel Erasmus have been +included in demo/stoffel. + +There are miscellaneous new files in some existing demo +subdirectories: classes/bitvec.py, scripts/{fixps,methfix}.py, +sgi/al/cmpaf.py, sockets/{mcast,gopher}.py. + +There are also many minor changes to existing files, but I'm too lazy +to run a diff and note the differences -- you can do this yourself if +you save the old distribution's demos. One highlight: the +stdwin/python.py demo is much improved! + + +Changes to the documentation +---------------------------- + +The LaTeX source for the library uses different macros to enable it to +be converted to texinfo, and from there to INFO or HTML format so it +can be browsed as a hypertext. The net result is that you can now +read the Python library documentation in Emacs info mode! + + +Changes to the source code that affect C extension writers +---------------------------------------------------------- + +The function strdup() no longer exists (it was used only in one places +and is somewhat of a a portability problem sice some systems have the +same function in their C library. + +The functions NEW() and RENEW() allocate one spare byte to guard +against a NULL return from malloc(0) being taken for an error, but +this should not be relied upon. + + +========================= +==> Release 0.9.7beta <== +========================= + + +Changes to the language proper +------------------------------ + +User-defined classes can now implement operations invoked through +special syntax, such as x[i] or `x` by defining methods named +__getitem__(self, i) or __repr__(self), etc. + + +Changes to the build process +---------------------------- + +Instead of extensive manual editing of the Makefile to select +compile-time options, you can now run a Configure.py script. +The Makefile as distributed builds a minimal interpreter sufficient to +run Configure.py. See also misc/BUILD + +The Makefile now includes more "utility" targets, e.g. install and +tags/TAGS + +Using the provided strtod.c and strtol.c are now separate options, as +on the Sun the provided strtod.c dumps core :-( + +The regex module is now an option chosen by the Makefile, since some +(old) C compilers choke on regexpr.c + + +Changes affecting portability +----------------------------- + +You need STDWIN version 0.9.7 (released 30 June 1992) for the stdwin +interface + +Dynamic loading is now supported for Sun (and other non-COFF systems) +throug dld-3.2.3, as well as for SGI (a new version of Jack Jansen's +DL is out, 1.4) + +The system-dependent code for the use of the select() system call is +moved to one file: myselect.h + +Thanks to Jaap Vermeulen, the code should now port cleanly to the +SEQUENT + + +Changes to the interpreter interface +------------------------------------ + +The interpretation of $PYTHONPATH in the environment is different: it +is inserted in front of the default path instead of overriding it + + +Changes to existing built-in functions and methods +-------------------------------------------------- + +List objects now support an optional argument to their sort() method, +which is a comparison function similar to qsort(3) in C + +File objects now have a method fileno(), used by the new select module +(see below) + + +New built-in function +--------------------- + +coerce(x, y): take two numbers and return a tuple containing them +both converted to a common type + + +Changes to built-in modules +--------------------------- + +sys: fixed core dumps in settrace() and setprofile() + +socket: added socket methods setsockopt() and getsockopt(); and +fileno(), used by the new select module (see below) + +stdwin: added fileno() == connectionnumber(), in support of new module +select (see below) + +posix: added get{eg,eu,g,u}id(); waitpid() is now a separate function. + +gl: added qgetfd() + +fl: added several new functions, fixed several obscure bugs, adapted +to FORMS 2.1 + + +Changes to standard modules +--------------------------- + +posixpath: changed implementation of ismount() + +string: atoi() no longer mistakes leading zero for octal number + +... + + +New built-in modules +-------------------- + +Modules marked "dynamic only" are not configured at compile time but +can be loaded dynamically. You need to turn on the DL or DLD option in +the Makefile for support dynamic loading of modules (this requires +external code). + +select: interfaces to the BSD select() system call + +dbm: interfaces to the (new) dbm library (dynamic only) + +nis: interfaces to some NIS functions (aka yellow pages) + +thread: limited form of multiple threads (sgi only) + +audioop: operations useful for audio programs, e.g. u-LAW and ADPCM +coding (dynamic only) + +cd: interface to Indigo SCSI CDROM player audio library (sgi only) + +jpeg: read files in JPEG format (dynamic only, sgi only; needs +external code) + +imgfile: read SGI image files (dynamic only, sgi only) + +sunaudiodev: interface to sun's /dev/audio (dynamic only, sun only) + +sv: interface to Indigo video library (sgi only) + +pc: a minimal set of MS-DOS interfaces (MS-DOS only) + +rotor: encryption, by Lance Ellinghouse (dynamic only) + + +New standard modules +-------------------- + +Not all these modules are documented. Read the source: +lib/<modulename>.py. Sometimes a file lib/<modulename>.doc contains +additional documentation. + +imghdr: recognizes image file headers + +sndhdr: recognizes sound file headers + +profile: print run-time statistics of Python code + +readcd, cdplayer: companion modules for built-in module cd (sgi only) + +emacs: interface to Emacs using py-connect.el (see below). + +SOCKET: symbolic constant definitions for socket options + +SUNAUDIODEV: symbolic constant definitions for sunaudiodef (sun only) + +SV: symbolic constat definitions for sv (sgi only) + +CD: symbolic constat definitions for cd (sgi only) + + +New demos +--------- + +scripts/pp.py: execute Python as a filter with a Perl-like command +line interface + +classes/: examples using the new class features + +threads/: examples using the new thread module + +sgi/cd/: examples using the new cd module + + +Changes to the documentation +---------------------------- + +The last-minute syntax changes of release 0.9.6 are now reflected +everywhere in the manuals + +The reference manual has a new section (3.2) on implementing new kinds +of numbers, sequences or mappings with user classes + +Classes are now treated extensively in the tutorial (chapter 9) + +Slightly restructured the system-dependent chapters of the library +manual + +The file misc/EXTENDING incorporates documentation for mkvalue() and +a new section on error handling + +The files misc/CLASSES and misc/ERRORS are no longer necessary + +The doc/Makefile now creates PostScript files automatically + + +Miscellaneous changes +--------------------- + +Incorporated Tim Peters' changes to python-mode.el, it's now version +1.06 + +A python/Emacs bridge (provided by Terrence M. Brannon) lets a Python +program running in an Emacs buffer execute Emacs lisp code. The +necessary Python code is in lib/emacs.py. The Emacs code is +misc/py-connect.el (it needs some external Emacs lisp code) + + +Changes to the source code that affect C extension writers +---------------------------------------------------------- + +New service function mkvalue() to construct a Python object from C +values according to a "format" string a la getargs() + +Most functions from pythonmain.c moved to new pythonrun.c which is +in libpython.a. This should make embedded versions of Python easier + +ceval.h is split in eval.h (which needs compile.h and only declares +eval_code) and ceval.h (which doesn't need compile.hand declares the +rest) + +ceval.h defines macros BGN_SAVE / END_SAVE for use with threads (to +improve the parallellism of multi-threaded programs by letting other +Python code run when a blocking system call or something similar is +made) + +In structmember.[ch], new member types BYTE, CHAR and unsigned +variants have been added + +New file xxmodule.c is a template for new extension modules. + + +================================== +==> Release 0.9.6 (6 Apr 1992) <== +================================== + +Misc news in 0.9.6: +- Restructured the misc subdirectory +- Reference manual completed, library manual much extended (with indexes!) +- the GNU Readline library is now distributed standard with Python +- the script "../demo/scripts/classfix.py" fixes Python modules using old + class syntax +- Emacs python-mode.el (was python.el) vastly improved (thanks, Tim!) +- Because of the GNU copyleft business I am not using the GNU regular + expression implementation but a free re-implementation by Tatu Ylonen + that recently appeared in comp.sources.misc (Bravo, Tatu!) + +New features in 0.9.6: +- stricter try stmt syntax: cannot mix except and finally clauses on 1 try +- New module 'os' supplants modules 'mac' and 'posix' for most cases; + module 'path' is replaced by 'os.path' +- os.path.split() return value differs from that of old path.split() +- sys.exc_type, sys.exc_value, sys.exc_traceback are set to the exception + currently being handled +- sys.last_type, sys.last_value, sys.last_traceback remember last unhandled + exception +- New function string.expandtabs() expands tabs in a string +- Added times() interface to posix (user & sys time of process & children) +- Added uname() interface to posix (returns OS type, hostname, etc.) +- New built-in function execfile() is like exec() but from a file +- Functions exec() and eval() are less picky about whitespace/newlines +- New built-in functions getattr() and setattr() access arbitrary attributes +- More generic argument handling in built-in functions (see "./EXTENDING") +- Dynamic loading of modules written in C or C++ (see "./DYNLOAD") +- Division and modulo for long and plain integers with negative operands + have changed; a/b is now floor(float(a)/float(b)) and a%b is defined + as a-(a/b)*b. So now the outcome of divmod(a,b) is the same as + (a/b, a%b) for integers. For floats, % is also changed, but of course + / is unchanged, and divmod(x,y) does not yield (x/y, x%y)... +- A function with explicit variable-length argument list can be declared + like this: def f(*args): ...; or even like this: def f(a, b, *rest): ... +- Code tracing and profiling features have been added, and two source + code debuggers are provided in the library (pdb.py, tty-oriented, + and wdb, window-oriented); you can now step through Python programs! + See sys.settrace() and sys.setprofile(), and "../lib/pdb.doc" +- '==' is now the only equality operator; "../demo/scripts/eqfix.py" is + a script that fixes old Python modules +- Plain integer right shift now uses sign extension +- Long integer shift/mask operations now simulate 2's complement + to give more useful results for negative operands +- Changed/added range checks for long/plain integer shifts +- Options found after "-c command" are now passed to the command in sys.argv + (note subtle incompatiblity with "python -c command -- -options"!) +- Module stdwin is better protected against touching objects after they've + been closed; menus can now also be closed explicitly +- Stdwin now uses its own exception (stdwin.error) + +New features in 0.9.5 (released as Macintosh application only, 2 Jan 1992): +- dictionary objects can now be compared properly; e.g., {}=={} is true +- new exception SystemExit causes termination if not caught; + it is raised by sys.exit() so that 'finally' clauses can clean up, + and it may even be caught. It does work interactively! +- new module "regex" implements GNU Emacs style regular expressions; + module "regexp" is rewritten in Python for backward compatibility +- formal parameter lists may contain trailing commas + +Bugs fixed in 0.9.6: +- assigning to or deleting a list item with a negative index dumped core +- divmod(-10L,5L) returned (-3L, 5L) instead of (-2L, 0L) + +Bugs fixed in 0.9.5: +- masking operations involving negative long integers gave wrong results + + +=================================== +==> Release 0.9.4 (24 Dec 1991) <== +=================================== + +- new function argument handling (see below) +- built-in apply(func, args) means func(args[0], args[1], ...) +- new, more refined exceptions +- new exception string values (NameError = 'NameError' etc.) +- better checking for math exceptions +- for sequences (string/tuple/list), x[-i] is now equivalent to x[len(x)-i] +- fixed list assignment bug: "a[1:1] = a" now works correctly +- new class syntax, without extraneous parentheses +- new 'global' statement to assign global variables from within a function + + +New class syntax +---------------- + +You can now declare a base class as follows: + + class B: # Was: class B(): + def some_method(self): ... + ... + +and a derived class thusly: + + class D(B): # Was: class D() = B(): + def another_method(self, arg): ... + +Multiple inheritance looks like this: + + class M(B, D): # Was: class M() = B(), D(): + def this_or_that_method(self, arg): ... + +The old syntax is still accepted by Python 0.9.4, but will disappear +in Python 1.0 (to be posted to comp.sources). + + +New 'global' statement +---------------------- + +Every now and then you have a global variable in a module that you +want to change from within a function in that module -- say, a count +of calls to a function, or an option flag, etc. Until now this was +not directly possible. While several kludges are known that +circumvent the problem, and often the need for a global variable can +be avoided by rewriting the module as a class, this does not always +lead to clearer code. + +The 'global' statement solves this dilemma. Its occurrence in a +function body means that, for the duration of that function, the +names listed there refer to global variables. For instance: + + total = 0.0 + count = 0 + + def add_to_total(amount): + global total, count + total = total + amount + count = count + 1 + +'global' must be repeated in each function where it is needed. The +names listed in a 'global' statement must not be used in the function +before the statement is reached. + +Remember that you don't need to use 'global' if you only want to *use* +a global variable in a function; nor do you need ot for assignments to +parts of global variables (e.g., list or dictionary items or +attributes of class instances). This has not changed; in fact +assignment to part of a global variable was the standard workaround. + + +New exceptions +-------------- + +Several new exceptions have been defined, to distinguish more clearly +between different types of errors. + +name meaning was + +AttributeError reference to non-existing attribute NameError +IOError unexpected I/O error RuntimeError +ImportError import of non-existing module or name NameError +IndexError invalid string, tuple or list index RuntimeError +KeyError key not in dictionary RuntimeError +OverflowError numeric overflow RuntimeError +SyntaxError invalid syntax RuntimeError +ValueError invalid argument value RuntimeError +ZeroDivisionError division by zero RuntimeError + +The string value of each exception is now its name -- this makes it +easier to experimentally find out which operations raise which +exceptions; e.g.: + + >>> KeyboardInterrupt + 'KeyboardInterrupt' + >>> + + +New argument passing semantics +------------------------------ + +Off-line discussions with Steve Majewski and Daniel LaLiberte have +convinced me that Python's parameter mechanism could be changed in a +way that made both of them happy (I hope), kept me happy, fixed a +number of outstanding problems, and, given some backward compatibility +provisions, would only break a very small amount of existing code -- +probably all mine anyway. In fact I suspect that most Python users +will hardly notice the difference. And yet it has cost me at least +one sleepless night to decide to make the change... + +Philosophically, the change is quite radical (to me, anyway): a +function is no longer called with either zero or one argument, which +is a tuple if there appear to be more arguments. Every function now +has an argument list containing 0, 1 or more arguments. This list is +always implemented as a tuple, and it is a (run-time) error if a +function is called with a different number of arguments than expected. + +What's the difference? you may ask. The answer is, very little unless +you want to write variadic functions -- functions that may be called +with a variable number of arguments. Formerly, you could write a +function that accepted one or more arguments with little trouble, but +writing a function that could be called with either 0 or 1 argument +(or more) was next to impossible. This is now a piece of cake: you +can simply declare an argument that receives the entire argument +tuple, and check its length -- it will be of size 0 if there are no +arguments. + +Another anomaly of the old system was the way multi-argument methods +(in classes) had to be declared, e.g.: + + class Point(): + def init(self, (x, y, color)): ... + def setcolor(self, color): ... + dev moveto(self, (x, y)): ... + def draw(self): ... + +Using the new scheme there is no need to enclose the method arguments +in an extra set of parentheses, so the above class could become: + + class Point: + def init(self, x, y, color): ... + def setcolor(self, color): ... + dev moveto(self, x, y): ... + def draw(self): ... + +That is, the equivalence rule between methods and functions has +changed so that now p.moveto(x,y) is equivalent to Point.moveto(p,x,y) +while formerly it was equivalent to Point.moveto(p,(x,y)). + +A special backward compatibility rule makes that the old version also +still works: whenever a function with exactly two arguments (at the top +level) is called with more than two arguments, the second and further +arguments are packed into a tuple and passed as the second argument. +This rule is invoked independently of whether the function is actually a +method, so there is a slight chance that some erroneous calls of +functions expecting two arguments with more than that number of +arguments go undetected at first -- when the function tries to use the +second argument it may find it is a tuple instead of what was expected. +Note that this rule will be removed from future versions of the +language; it is a backward compatibility provision *only*. + +Two other rules and a new built-in function handle conversion between +tuples and argument lists: + +Rule (a): when a function with more than one argument is called with a +single argument that is a tuple of the right size, the tuple's items +are used as arguments. + +Rule (b): when a function with exactly one argument receives no +arguments or more than one, that one argument will receive a tuple +containing the arguments (the tuple will be empty if there were no +arguments). + + +A new built-in function, apply(), was added to support functions that +need to call other functions with a constructed argument list. The call + + apply(function, tuple) + +is equivalent to + + function(tuple[0], tuple[1], ..., tuple[len(tuple)-1]) + + +While no new argument syntax was added in this phase, it would now be +quite sensible to add explicit syntax to Python for default argument +values (as in C++ or Modula-3), or a "rest" argument to receive the +remaining arguments of a variable-length argument list. + + +======================================================== +==> Release 0.9.3 (never made available outside CWI) <== +======================================================== + +- string sys.version shows current version (also printed on interactive entry) +- more detailed exceptions, e.g., IOError, ZeroDivisionError, etc. +- 'global' statement to declare module-global variables assigned in functions. +- new class declaration syntax: class C(Base1, Base2, ...): suite + (the old syntax is still accepted -- be sure to convert your classes now!) +- C shifting and masking operators: << >> ~ & ^ | (for ints and longs). +- C comparison operators: == != (the old = and <> remain valid). +- floating point numbers may now start with a period (e.g., .14). +- definition of integer division tightened (always truncates towards zero). +- new builtins hex(x), oct(x) return hex/octal string from (long) integer. +- new list method l.count(x) returns the number of occurrences of x in l. +- new SGI module: al (Indigo and 4D/35 audio library). +- the FORMS interface (modules fl and FL) now uses FORMS 2.0 +- module gl: added lrect{read,write}, rectzoom and pixmode; + added (non-GL) functions (un)packrect. +- new socket method: s.allowbroadcast(flag). +- many objects support __dict__, __methods__ or __members__. +- dir() lists anything that has __dict__. +- class attributes are no longer read-only. +- classes support __bases__, instances support __class__ (and __dict__). +- divmod() now also works for floats. +- fixed obscure bug in eval('1 '). + + +=================================== +==> Release 0.9.2 (Autumn 1991) <== +=================================== + +Highlights +---------- + +- tutorial now (almost) complete; library reference reorganized +- new syntax: continue statement; semicolons; dictionary constructors; + restrictions on blank lines in source files removed +- dramatically improved module load time through precompiled modules +- arbitrary precision integers: compute 2 to the power 1000 and more... +- arithmetic operators now accept mixed type operands, e.g., 3.14/4 +- more operations on list: remove, index, reverse; repetition +- improved/new file operations: readlines, seek, tell, flush, ... +- process management added to the posix module: fork/exec/wait/kill etc. +- BSD socket operations (with example servers and clients!) +- many new STDWIN features (color, fonts, polygons, ...) +- new SGI modules: font manager and FORMS library interface + + +Extended list of changes in 0.9.2 +--------------------------------- + +Here is a summary of the most important user-visible changes in 0.9.2, +in somewhat arbitrary order. Changes in later versions are listed in +the "highlights" section above. + + +1. Changes to the interpreter proper + +- Simple statements can now be separated by semicolons. + If you write "if t: s1; s2", both s1 and s2 are executed + conditionally. +- The 'continue' statement was added, with semantics as in C. +- Dictionary displays are now allowed on input: {key: value, ...}. +- Blank lines and lines bearing only a comment no longer need to + be indented properly. (A completely empty line still ends a multi- + line statement interactively.) +- Mixed arithmetic is supported, 1 compares equal to 1.0, etc. +- Option "-c command" to execute statements from the command line +- Compiled versions of modules are cached in ".pyc" files, giving a + dramatic improvement of start-up time +- Other, smaller speed improvements, e.g., extracting characters from + strings, looking up single-character keys, and looking up global + variables +- Interrupting a print operation raises KeyboardInterrupt instead of + only cancelling the print operation +- Fixed various portability problems (it now passes gcc with only + warnings -- more Standard C compatibility will be provided in later + versions) +- Source is prepared for porting to MS-DOS +- Numeric constants are now checked for overflow (this requires + standard-conforming strtol() and strtod() functions; a correct + strtol() implementation is provided, but the strtod() provided + relies on atof() for everything, including error checking + + +2. Changes to the built-in types, functions and modules + +- New module socket: interface to BSD socket primitives +- New modules pwd and grp: access the UNIX password and group databases +- (SGI only:) New module "fm" interfaces to the SGI IRIX Font Manager +- (SGI only:) New module "fl" interfaces to Mark Overmars' FORMS library +- New numeric type: long integer, for unlimited precision + - integer constants suffixed with 'L' or 'l' are long integers + - new built-in function long(x) converts int or float to long + - int() and float() now also convert from long integers +- New built-in function: + - pow(x, y) returns x to the power y +- New operation and methods for lists: + - l*n returns a new list consisting of n concatenated copies of l + - l.remove(x) removes the first occurrence of the value x from l + - l.index(x) returns the index of the first occurrence of x in l + - l.reverse() reverses l in place +- New operation for tuples: + - t*n returns a tuple consisting of n concatenated copies of t +- Improved file handling: + - f.readline() no longer restricts the line length, is faster, + and isn't confused by null bytes; same for raw_input() + - f.read() without arguments reads the entire (rest of the) file + - mixing of print and sys.stdout.write() has different effect +- New methods for files: + - f.readlines() returns a list containing the lines of the file, + as read with f.readline() + - f.flush(), f.tell(), f.seek() call their stdio counterparts + - f.isatty() tests for "tty-ness" +- New posix functions: + - _exit(), exec(), fork(), getpid(), getppid(), kill(), wait() + - popen() returns a file object connected to a pipe + - utime() replaces utimes() (the latter is not a POSIX name) +- New stdwin features, including: + - font handling + - color drawing + - scroll bars made optional + - polygons + - filled and xor shapes + - text editing objects now have a 'settext' method + + +3. Changes to the standard library + +- Name change: the functions path.cat and macpath.cat are now called + path.join and macpath.join +- Added new modules: formatter, mutex, persist, sched, mainloop +- Added some modules and functionality to the "widget set" (which is + still under development, so please bear with me): + DirList, FormSplit, TextEdit, WindowSched +- Fixed module testall to work non-interactively +- Module string: + - added functions join() and joinfields() + - fixed center() to work correct and make it "transitive" +- Obsolete modules were removed: util, minmax +- Some modules were moved to the demo directory + + +4. Changes to the demonstration programs + +- Added new useful scipts: byteyears, eptags, fact, from, lfact, + objgraph, pdeps, pi, primes, ptags, which +- Added a bunch of socket demos +- Doubled the speed of ptags +- Added new stdwin demos: microedit, miniedit +- Added a windowing interface to the Python interpreter: python (most + useful on the Mac) +- Added a browser for Emacs info files: demo/stdwin/ibrowse + (yes, I plan to put all STDWIN and Python documentation in texinfo + form in the future) + + +5. Other changes to the distribution + +- An Emacs Lisp file "python.el" is provided to facilitate editing + Python programs in GNU Emacs (slightly improved since posted to + gnu.emacs.sources) +- Some info on writing an extension in C is provided +- Some info on building Python on non-UNIX platforms is provided + + +===================================== +==> Release 0.9.1 (February 1991) <== +===================================== + +- Micro changes only +- Added file "patchlevel.h" + + +===================================== +==> Release 0.9.0 (February 1991) <== +===================================== + +Original posting to alt.sources. diff --git a/sys/src/cmd/python/Misc/NEWS b/sys/src/cmd/python/Misc/NEWS new file mode 100644 index 000000000..6475b767d --- /dev/null +++ b/sys/src/cmd/python/Misc/NEWS @@ -0,0 +1,2782 @@ ++++++++++++ +Python News ++++++++++++ + +(editors: check NEWS.help for information about editing NEWS using ReST.) + +What's New in Python 2.5.1? +============================= + +*Release date: 18-APR-2007* + +Core and builtins +----------------- + +- Revert SF #1615701: dict.update() does *not* call __getitem__() or keys() + if subclassed. This is to remain consistent with 2.5. + Also revert revision 53667 with made a similar change to set.update(). + + +What's New in Python 2.5.1c1? +============================= + +*Release date: 05-APR-2007* + +Core and builtins +----------------- + +- Patch #1682205: a TypeError while unpacking an iterable is no longer + masked by a generic one with the message "unpack non-sequence". + +- Patch #1642547: Fix an error/crash when encountering syntax errors in + complex if statements. + +- Patch #1462488: Python no longer segfaults when ``object.__reduce_ex__()`` + is called with an object that is faking its type. + +- Patch #1680015: Don't modify __slots__ tuple if it contains an unicode + name. + +- Patch #922167: Python no longer segfaults when faced with infinitely + self-recursive reload() calls (as reported by bug #742342). + +- Patch #1675981: remove unreachable code from ``type.__new__()`` method. + +- Patch #1638879: don't accept strings with embedded NUL bytes in long(). + +- Bug #1674503: close the file opened by execfile() in an error condition. + +- Patch #1674228: when assigning a slice (old-style), check for the + sq_ass_slice instead of the sq_slice slot. + +- Bug #1669182: prevent crash when trying to print an unraisable error + from a string exception. + +- The peephole optimizer left None as a global in functions with a docstring + and an explicit return value. + +- Bug #1653736: Properly discard third argument to slot_nb_inplace_power. + +- SF #151204: enumerate() now raises an Overflow error at sys.maxint items. + +- Bug #1377858: Fix the segfaulting of the interpreter when an object created + a weakref on itself during a __del__ call for new-style classes (classic + classes still have the bug). + +- Bug #1648179: set.update() did not recognize an overridden __iter__ + method in subclasses of dict. + +- Bug #1579370: Make PyTraceBack_Here use the current thread, not the + frame's thread state. + +- patch #1630975: Fix crash when replacing sys.stdout in sitecustomize.py + +- Bug #1637022: Prefix AST symbols with _Py_. + +- Prevent seg fault on shutdown which could occur if an object + raised a warning. + +- Bug #1566280: Explicitly invoke threading._shutdown from Py_Main, + to avoid relying on atexit. + +- Bug #1590891: random.randrange don't return correct value for big number + +- Bug #1456209: In some obscure cases it was possible for a class with a + custom ``__eq__()`` method to confuse set internals when class instances + were used as a set's elements and the ``__eq__()`` method mutated the set. + +- The repr for self-referential sets and fronzensets now shows "..." instead + of falling into infinite recursion. + +- Eliminated unnecessary repeated calls to hash() by set.intersection() and + set.symmetric_difference_update(). + +- Bug #1591996: Correctly forward exception in instance_contains(). + +- Bug #1588287: fix invalid assertion for `1,2` in debug builds. + +- Bug #1576657: when setting a KeyError for a tuple key, make sure that + the tuple isn't used as the "exception arguments tuple". Applied to + both sets and dictionaries. + +- Bug #1565514, SystemError not raised on too many nested blocks. + +- Bug #1576174: WindowsError now displays the windows error code + again, no longer the posix error code. + +- Patch #1549049: Support long values in structmember. + +- Bug #1542016: make sys.callstats() match its docstring and return an + 11-tuple (only relevant when Python is compiled with -DCALL_PROFILE). + +- Bug #1545497: when given an explicit base, int() did ignore NULs + embedded in the string to convert. + +- Bug #1569998: break inside a try statement (outside a loop) is now + recognized and rejected. + +- Patch #1542451: disallow continue anywhere under a finally. + +- list.pop(x) accepts any object x following the __index__ protocol. + +- Fix some leftovers from the conversion from int to Py_ssize_t + (relevant to strings and sequences of more than 2**31 items). + +- A number of places, including integer negation and absolute value, + were fixed to not rely on undefined behaviour of the C compiler + anymore. + +- Bug #1566800: make sure that EnvironmentError can be called with any + number of arguments, as was the case in Python 2.4. + +- Patch #1567691: super() and new.instancemethod() now don't accept + keyword arguments any more (previously they accepted them, but didn't + use them). + +- Fix a bug in the parser's future statement handling that led to "with" + not being recognized as a keyword after, e.g., this statement: + from __future__ import division, with_statement + +- Bug #1557232: fix seg fault with def f((((x)))) and def f(((x),)). + +- Fix %zd string formatting on Mac OS X so it prints negative numbers. + +- Allow exception instances to be directly sliced again. + + +Extension Modules +----------------- + +- Bug #1563759: struct.unpack doens't support buffer protocol objects + +- Bug #1686475: Support stat'ing open files on Windows again. + +- Bug #1647541: Array module's buffer interface can now handle empty arrays. + +- Bug #1693079: The array module can now successfully pickle empty arrays. + +- Bug #1688393: Prevent crash in socket.recvfrom if length is negative. + +- Bug #1622896: fix a rare corner case where the bz2 module raised an + error in spite of a succesful compression. + +- Patch #1654417: make operator.{get,set,del}slice use the full range + of Py_ssize_t. + +- Patch #1646728: datetime.fromtimestamp fails with negative + fractional times. With unittest. + +- Patch #1494140: Add documentation for the new struct.Struct object. + +- Patch #1657276: Make NETLINK_DNRTMSG conditional. + +- Bug #1653736: Fix signature of time_isoformat. + +- operator.count() now raises an OverflowError when the count reaches sys.maxint. + +- Bug #1575169: operator.isSequenceType() now returns False for subclasses of dict. + +- collections.defaultdict() now verifies that the factory function is callable. + +- Bug #1486663: don't reject keyword arguments for subclasses of builtin + types. + +- The version number of the ctypes package was changed to "1.0.2". + +- Bug #1664966: Fix crash in exec if Unicode filename can't be decoded. + +- Patch #1544279: Improve thread-safety of the socket module by moving + the sock_addr_t storage out of the socket object. + +- Patch #1615868: make bz2.BZFile.seek() work for offsets >2GiB. + +- Bug #1563807: _ctypes built on AIX fails with ld ffi error. + +- Bug #1598620: A ctypes Structure cannot contain itself. + +- Bug #1588217: don't parse "= " as a soft line break in binascii's + a2b_qp() function, instead leave it in the string as quopri.decode() + does. + +- Patch #838546: Make terminal become controlling in pty.fork() + +- Patch #1560695: Add .note.GNU-stack to ctypes' sysv.S so that + ctypes isn't considered as requiring executable stacks. + +- Bug #1567666: Emulate GetFileAttributesExA for Win95. + +- Bug #1548891: The cStringIO.StringIO() constructor now encodes unicode + arguments with the system default encoding just like the write() + method does, instead of converting it to a raw buffer. + +- Bug #1565150: Fix subsecond processing for os.utime on Windows. + +- Patch #1572724: fix typo ('=' instead of '==') in _msi.c. + +- Bug #1572832: fix a bug in ISO-2022 codecs which may cause segfault + when encoding non-BMP unicode characters. + +- Bug #1556784: allow format strings longer than 127 characters in + datetime's strftime function. + +- Fix itertools.count(n) to work with negative numbers again. + +- Make regex engine raise MemoryError if allocating memory fails. + +- fixed a bug with bsddb.DB.stat: the flags and txn keyword arguments + were transposed. + +- Added support for linking the bsddb module against BerkeleyDB 4.5.x. + +- Modifying an empty deque during iteration now raises RuntimeError + instead of StopIteration. + +- Bug #1552726: fix polling at the interpreter prompt when certain + versions of the readline library are in use. + +- Bug #1633621: if curses.resizeterm() or curses.resize_term() is called, + update _curses.LINES, _curses.COLS, curses.LINES and curses.COLS. + +- Fix an off-by-one bug in locale.strxfrm(). + +Library +------- + +- Patch #1685563: remove (don't add) duplicate paths in distutils.MSVCCompiler. + +- Bug #978833: Revert r50844, as it broke _socketobject.dup. + +- Bug #1675967: re patterns pickled with Python 2.4 and earlier can + now be unpickled with Python 2.5. + +- Bug #1684254: webbrowser now uses shlex to split any command lines + given to get(). It also detects when you use '&' as the last argument + and creates a BackgroundBrowser then. + +- Patch #1681153: the wave module now closes a file object it opened if + initialization failed. + +- Bug #767111: fix long-standing bug in urllib which caused an + AttributeError instead of an IOError when the server's response didn't + contain a valid HTTP status line. + +- Bug #1629369: Correctly parse multiline comment in address field. + +- Bug #1582282: Fix email.header.decode_header() to properly treat encoded + words with no delimiting whitespace as a single word. + +- Patch #1449244: Support Unicode strings in + email.message.Message.{set_charset,get_content_charset}. + +- Patch #1542681: add entries for "with", "as" and "CONTEXTMANAGERS" to + pydoc's help keywords. + +- Patch #1192590: Fix pdb's "ignore" and "condition" commands so they trap + the IndexError caused by passing in an invalid breakpoint number. + +- Bug #1531963: Make SocketServer.TCPServer's server_address always + be equal to calling getsockname() on the server's socket. Fixed by patch + #1545011. + +- Bug #1651235: When a tuple was passed to a ctypes function call, + Python would crash instead of raising an error. + +- Fix bug #1646630: ctypes.string_at(buf, 0) and ctypes.wstring_at(buf, 0) + returned string up to the first NUL character. + +- Bug #1637850: make_table in difflib did not work with unicode + +- Bugs #1676321: the empty() function in sched.py returned the wrong result + +- unittest now verifies more of its assumptions. In particular, TestCase + and TestSuite subclasses (not instances) are no longer accepted in + TestSuite.addTest(). This should cause no incompatibility since it + never made sense with ordinary subclasses -- the failure just occurred + later, with a more cumbersome exception. + +- Patch #685268: Consider a package's __path__ in imputil. + +- Patch 1463026: Support default namespace in XMLGenerator. + +- Patch 1571379: Make trace's --ignore-dir facility work in the face of + relative directory names. + +- Bug #1600860: Search for shared python library in LIBDIR, not lib/python/config, + on "linux" and "gnu" systems. + +- Bug #1124861: Automatically create pipes if GetStdHandle fails in + subprocess. + +- Patch #783050: the pty.fork() function now closes the slave fd + correctly. + +- Patch #1638243: the compiler package is now able to correctly compile + a with statement; previously, executing code containing a with statement + compiled by the compiler package crashed the interpreter. + +- Bug #1643943: Fix %U handling for time.strptime. + +- Bug #1598181: Avoid O(N**2) bottleneck in subprocess communicate(). + +- Patch #1627441: close sockets properly in urllib2. + +- Bug #1610795: ctypes.util.find_library works now on BSD systems. + +- Fix sort stability in heapq.nlargest() and nsmallest(). + +- Patch #1504073: Fix tarfile.open() for mode "r" with a fileobj argument. + +- Patch #1262036: Prevent TarFiles from being added to themselves under + certain conditions. + +- Patch #1230446: tarfile.py: fix ExFileObject so that read() and tell() + work correctly together with readline(). + +- Bug #737202: Make CGIHTTPServer work for scripts in subdirectories. + Fix by Titus Brown. + +- Patch #827559: Make SimpleHTTPServer redirect when a directory URL + is missing the trailing slash, so that relative links work correctly. + Patch by Chris Gonnerman. + +- Patch #1608267: fix a race condition in os.makedirs() is the directory + to be created is already there. + +- Patch #1610437: fix a tarfile bug with long filename headers. + +- Patch #1472877: Fix Tix subwidget name resolution. + +- Patch #1594554: Always close a tkSimpleDialog on ok(), even + if an exception occurs. + +- Patch #1538878: Don't make tkSimpleDialog dialogs transient if + the parent window is withdrawn. + +- Patch #1360200: Use unmangled_version RPM spec field to deal with + file name mangling. + +- Patch #1359217: Process 2xx response in an ftplib transfer + that precedes an 1xx response. + +- Patch #1060577: Extract list of RPM files from spec file in + bdist_rpm + +- Bug #1586613: fix zlib and bz2 codecs' incremental en/decoders. + +- Patch #1583880: fix tarfile's problems with long names and posix/ + GNU modes. + +- Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and + fix all codecs file wrappers to work correctly with the "with" + statement (bug #1586513). + +- ctypes callback functions only support 'fundamental' data types as + result type. Raise an error when something else is used. This is a + partial fix for Bug #1574584. + +- Bug #813342: Start the IDLE subprocess with -Qnew if the parent + is started with that option. + +- Bug #1446043: correctly raise a LookupError if an encoding name given + to encodings.search_function() contains a dot. + +- Bug #1545341: The 'classifier' keyword argument to the Distutils setup() + function now accepts tuples as well as lists. + +- Bug #1560617: in pyclbr, return full module name not only for classes, + but also for functions. + +- Bug #1566602: correct failure of posixpath unittest when $HOME ends + with a slash. + +- Bug #1565661: in webbrowser, split() the command for the default + GNOME browser in case it is a command with args. + +- Bug #1569790: mailbox.py: Maildir.get_folder() and MH.get_folder() + weren't passing the message factory on to newly created Maildir/MH + objects. + +- Bug #1575506: mailbox.py: Single-file mailboxes didn't re-lock + properly in their flush() method. + +- Patch #1514543: mailbox.py: In the Maildir class, report errors if there's + a filename clash instead of possibly losing a message. (Patch by David + Watson.) + +- Patch #1514544: mailbox.py: Try to ensure that messages/indexes have + been physically written to disk after calling .flush() or + .close(). (Patch by David Watson.) + +- mailbox.py: Change MH.pack() to not lock individual message files; this + wasn't consistent with existing implementations of message packing, and + was buggy on some platforms. + +- Bug #1633678: change old mailbox.UnixMailbox class to parse + 'From' lines less strictly. + +- Bug #1576241: fix functools.wraps() to work on built-in functions. + +- Patch #1574068: fix urllib/urllib2 to not insert line breaks when + HTTP authentication data was very long. + +- Patch #1617413: fix urllib's support for HTTP Basic authentication via HTTPS + (patch by Dug Song). + +- Fix a bug in traceback.format_exception_only() that led to an error + being raised when print_exc() was called without an exception set. + In version 2.4, this printed "None", restored that behavior. + +- Make webbrowser.BackgroundBrowser usable in Windows (it wasn't because + the close_fds arg to subprocess.Popen is not supported). + + +Tools/Demos +----------- + +- Patch #1552024: add decorator support to unparse.py demo script. + +- idle: Honor the "Cancel" action in the save dialog (Debian bug #299092). + + +Tests +----- + +- Cause test.test_socket_ssl:test_basic to raise + test.test_support.ResourceDenied when an HTTPS connection times out. + +- Remove passwd.adjunct.byname from list of maps + for test_nis. + + +Build +----- + +- Bug #1655392: don't add -L/usr/lib/pythonX.Y/config to the LDFLAGS + returned by python-config if Python was built with --enable-shared + because that prevented the shared library from being used. + +- Patch #1569798: fix a bug in distutils when building Python from a + directory within sys.exec_prefix. + +- Bug #1675511: Use -Kpic instead of -xcode=pic32 on Solaris/x86. + +- Disable _XOPEN_SOURCE on NetBSD 1.x. + +- Bug #1578513: Cross compilation was broken by a change to configure. + Repair so that it's back to how it was in 2.4.3. + +- Patch #1576954: Update VC6 build directory; remove redundant + files in VC7. + +- Fix build failure on kfreebsd and on the hurd. + +- Fix the build of the library reference in info format. + + +Windows +------- + +- Conditionalize definition of _CRT_SECURE_NO_DEPRECATE + and _CRT_NONSTDC_NO_DEPRECATE. + + +Documentation +------------- + +- Patch #1489771: the syntax rules in Python Reference Manual were + updated to reflect the current Python syntax. + +- Patch #1686451: Fix return type for + PySequence_{Count,Index,Fast_GET_SIZE}. + + +What's New in Python 2.5 (final) +================================ + +*Release date: 19-SEP-2006* + +No changes since release candidate 2. + + +What's New in Python 2.5 release candidate 2? +============================================= + +*Release date: 12-SEP-2006* + +Core and builtins +----------------- + +- Make _PyGILState_NoteThreadState() static, it was not used anywhere + outside of pystate.c and should not be necessary. + +- Bug #1551432: Exceptions do not define an explicit __unicode__ method. This + allows calling unicode() on exceptions classes directly to succeed. + +- Bug #1542051: Exceptions now correctly call PyObject_GC_UnTrack. + Also make sure that every exception class has __module__ set to + 'exceptions'. + +- Bug #1550983: emit better error messages for erroneous relative + imports (if not in package and if beyond toplevel package). + +- Overflow checking code in integer division ran afoul of new gcc + optimizations. Changed to be more standard-conforming. + +- Patch #1541585: fix buffer overrun when performing repr() on + a unicode string in a build with wide unicode (UCS-4) support. + +- Patch #1546288: fix seg fault in dict_equal due to ref counting bug. + +- The return tuple from str.rpartition(sep) is (tail, sep, head) where + head is the original string if sep was not found. + +- Bug #1520864: unpacking singleton tuples in list comprehensions and + generator expressions (x for x, in ... ) works again. Fixing this problem + required changing the .pyc magic number. This means that .pyc files + generated before 2.5c2 will be regenerated. + + +Library +------- + +- Reverted patch #1504333 because it introduced an infinite loop. + +- Patch #1553314: Fix the inspect.py slowdown that was hurting IPython & SAGE + by adding smarter caching in inspect.getmodule(). + +- Fix missing import of the types module in logging.config. + +- Patch #1550886: Fix decimal module context management implementation + to match the localcontext() example from PEP 343. + +- Bug #1541863: uuid.uuid1 failed to generate unique identifiers + on systems with low clock resolution. + +- Bug #1543303, patch #1543897: remove NUL padding from tarfiles. + +- Bug #1531862: Do not close standard file descriptors in subprocess. + + +Extension Modules +----------------- + +- Bug #1599782: fix segfault on bsddb.db.DB().type(). + +- Fix bugs in ctypes: + - anonymous structure fields that have a bit-width specified did not work + - cast function did not accept c_char_p or c_wchar_p instances as first arg + +- Bug #1551427: fix a wrong NULL pointer check in the win32 version + of os.urandom(). + +- Bug #1548092: fix curses.tparm seg fault on invalid input. + +- Bug #1550714: fix SystemError from itertools.tee on negative value for n. + +- Fixed a few bugs on cjkcodecs: + - gbk and gb18030 codec now handle U+30FB KATAKANA MIDDLE DOT correctly. + - iso2022_jp_2 codec now encodes into G0 for KS X 1001, GB2312 + codepoints to conform the standard. + - iso2022_jp_3 and iso2022_jp_2004 codec can encode JIS X 0213:2 + codepoints now. + +Tests +----- + +- Patch #1559413: Fix test_cmd_line if sys.executable contains a space. + +- Fix bsddb test_basics.test06_Transactions to check the version + number properly. + + +Documentation +------------- + +- Patch #1679379: add documentation for fnmatch.translate(). + +- Patch #1671450: add a section about subclassing builtin types to the + "extending and embedding" tutorial. + +- Bug #1629125: fix wrong data type (int -> Py_ssize_t) in PyDict_Next + docs. + +- Bug #1565919: document set types in the Language Reference. + +- Bug #1546052: clarify that PyString_FromString(AndSize) copies the + string pointed to by its parameter. + +- Bug #1566663: remove obsolete example from datetime docs. + +- Bug #1541682: Fix example in the "Refcount details" API docs. + Additionally, remove a faulty example showing PySequence_SetItem applied + to a newly created list object and add notes that this isn't a good idea. + + +Tools +----- + +- Bug #1546372: Fixed small bugglet in pybench that caused a missing + file not to get reported properly. + + +Build +----- + +- Bug #1568842: Fix test for uintptr_t. + +- Patch #1540470, for OpenBSD 4.0. + +- Patch #1545507: Exclude ctypes package in Win64 MSI file. + +- Fix OpenSSL debug build process. + + +C API +----- + +- Bug #1542693: remove semi-colon at end of PyImport_ImportModuleEx macro + so it can be used as an expression. + + +What's New in Python 2.5 release candidate 1? +============================================= + +*Release date: 17-AUG-2006* + +Core and builtins +----------------- + +- Fix infinite recursion when subclassing long and overriding __hash__. + +- Fix concatenation (+=) of long strings. + +- Unicode objects will no longer raise an exception when being + compared equal or unequal to a string and a UnicodeDecodeError + exception occurs, e.g. as result of a decoding failure. + + Instead, the equal (==) and unequal (!=) comparison operators will + now issue a UnicodeWarning and interpret the two objects as + unequal. The UnicodeWarning can be filtered as desired using + the warning framework, e.g. silenced completely, turned into an + exception, logged, etc. + + Note that compare operators other than equal and unequal will still + raise UnicodeDecodeError exceptions as they've always done. + +- Fix segfault when doing string formatting on subclasses of long. + +- Fix bug related to __len__ functions using values > 2**32 on 64-bit machines + with new-style classes. + +- Fix bug related to __len__ functions returning negative values with + classic classes. + +- Patch #1538606, Fix __index__() clipping. There were some problems + discovered with the API and how integers that didn't fit into Py_ssize_t + were handled. This patch attempts to provide enough alternatives + to effectively use __index__. + +- Bug #1536021: __hash__ may now return long int; the final hash + value is obtained by invoking hash on the long int. + +- Bug #1536786: buffer comparison could emit a RuntimeWarning. + +- Bug #1535165: fixed a segfault in input() and raw_input() when + sys.stdin is closed. + +- On Windows, the PyErr_Warn function is now exported from + the Python dll again. + +- Bug #1191458: tracing over for loops now produces a line event + on each iteration. Fixing this problem required changing the .pyc + magic number. This means that .pyc files generated before 2.5c1 + will be regenerated. + +- Bug #1333982: string/number constants were inappropriately stored + in the byte code and co_consts even if they were not used, ie + immediately popped off the stack. + +- Fixed a reference-counting problem in property(). + + +Library +------- + +- Fix a bug in the ``compiler`` package that caused invalid code to be + generated for generator expressions. + +- The distutils version has been changed to 2.5.0. The change to + keep it programmatically in sync with the Python version running + the code (introduced in 2.5b3) has been reverted. It will continue + to be maintained manually as static string literal. + +- If the Python part of a ctypes callback function returns None, + and this cannot be converted to the required C type, an exception is + printed with PyErr_WriteUnraisable. Before this change, the C + callback returned arbitrary values to the calling code. + +- The __repr__ method of a NULL ctypes.py_object() no longer raises + an exception. + +- uuid.UUID now has a bytes_le attribute. This returns the UUID in + little-endian byte order for Windows. In addition, uuid.py gained some + workarounds for clocks with low resolution, to stop the code yielding + duplicate UUIDs. + +- Patch #1540892: site.py Quitter() class attempts to close sys.stdin + before raising SystemExit, allowing IDLE to honor quit() and exit(). + +- Bug #1224621: make tabnanny recognize IndentationErrors raised by tokenize. + +- Patch #1536071: trace.py should now find the full module name of a + file correctly even on Windows. + +- logging's atexit hook now runs even if the rest of the module has + already been cleaned up. + +- Bug #1112549, fix DoS attack on cgi.FieldStorage. + +- Bug #1531405, format_exception no longer raises an exception if + str(exception) raised an exception. + +- Fix a bug in the ``compiler`` package that caused invalid code to be + generated for nested functions. + + +Extension Modules +----------------- + +- Ignore data that arrives before the opening start tag in C etree. + +- Patch #1511317: don't crash on invalid hostname (alias) info. + +- Patch #1535500: fix segfault in BZ2File.writelines and make sure it + raises the correct exceptions. + +- Patch # 1536908: enable building ctypes on OpenBSD/AMD64. The + '-no-stack-protector' compiler flag for OpenBSD has been removed. + +- Patch #1532975 was applied, which fixes Bug #1533481: ctypes now + uses the _as_parameter_ attribute when objects are passed to foreign + function calls. The ctypes version number was changed to 1.0.1. + +- Bug #1530559, struct.pack raises TypeError where it used to convert. + Passing float arguments to struct.pack when integers are expected + now triggers a DeprecationWarning. + + +Tests +----- + +- test_socketserver should now work on cygwin and not fail sporadically + on other platforms. + +- test_mailbox should now work on cygwin versions 2006-08-10 and later. + +- Bug #1535182: really test the xreadlines() method of bz2 objects. + +- test_threading now skips testing alternate thread stack sizes on + platforms that don't support changing thread stack size. + + +Documentation +------------- + +- Patch #1534922: unittest docs were corrected and enhanced. + + +Build +----- + +- Bug #1535502, build _hashlib on Windows, and use masm assembler + code in OpenSSL. + +- Bug #1534738, win32 debug version of _msi should be _msi_d.pyd. + +- Bug #1530448, ctypes build failure on Solaris 10 was fixed. + + +C API +----- + +- New API for Unicode rich comparisons: PyUnicode_RichCompare() + +- Bug #1069160. Internal correctness changes were made to + ``PyThreadState_SetAsyncExc()``. A test case was added, and + the documentation was changed to state that the return value + is always 1 (normal) or 0 (if the specified thread wasn't found). + + +What's New in Python 2.5 beta 3? +================================ + +*Release date: 03-AUG-2006* + +Core and builtins +----------------- + +- _PyWeakref_GetWeakrefCount() now returns a Py_ssize_t; it previously + returned a long (see PEP 353). + +- Bug #1515471: string.replace() accepts character buffers again. + +- Add PyErr_WarnEx() so C code can pass the stacklevel to warnings.warn(). + This provides the proper warning for struct.pack(). + PyErr_Warn() is now deprecated in favor of PyErr_WarnEx(). + +- Patch #1531113: Fix augmented assignment with yield expressions. + Also fix a SystemError when trying to assign to yield expressions. + +- Bug #1529871: The speed enhancement patch #921466 broke Python's compliance + with PEP 302. This was fixed by adding an ``imp.NullImporter`` type that is + used in ``sys.path_importer_cache`` to cache non-directory paths and avoid + excessive filesystem operations during imports. + +- Bug #1521947: When checking for overflow, ``PyOS_strtol()`` used some + operations on signed longs that are formally undefined by C. + Unfortunately, at least one compiler now cares about that, so complicated + the code to make that compiler happy again. + +- Bug #1524310: Properly report errors from FindNextFile in os.listdir. + +- Patch #1232023: Stop including current directory in search + path on Windows. + +- Fix some potential crashes found with failmalloc. + +- Fix warnings reported by Klocwork's static analysis tool. + +- Bug #1512814, Fix incorrect lineno's when code within a function + had more than 255 blank lines. + +- Patch #1521179: Python now accepts the standard options ``--help`` and + ``--version`` as well as ``/?`` on Windows. + +- Bug #1520864: unpacking singleton tuples in a 'for' loop (for x, in) works + again. Fixing this problem required changing the .pyc magic number. + This means that .pyc files generated before 2.5b3 will be regenerated. + +- Bug #1524317: Compiling Python ``--without-threads`` failed. + The Python core compiles again, and, in a build without threads, the + new ``sys._current_frames()`` returns a dictionary with one entry, + mapping the faux "thread id" 0 to the current frame. + +- Bug #1525447: build on MacOS X on a case-sensitive filesystem. + + +Library +------- + +- Correction of patch #1455898: In the mbcs decoder, set final=False + for stream decoder, but final=True for the decode function. + +- os.urandom no longer masks unrelated exceptions like SystemExit or + KeyboardInterrupt. + +- Bug #1525866: Don't copy directory stat times in + shutil.copytree on Windows + +- Bug #1002398: The documentation for os.path.sameopenfile now correctly + refers to file descriptors, not file objects. + +- The renaming of the xml package to xmlcore, and the import hackery done + to make it appear at both names, has been removed. Bug #1511497, + #1513611, and probably others. + +- Bug #1441397: The compiler module now recognizes module and function + docstrings correctly as it did in Python 2.4. + +- Bug #1529297: The rewrite of doctest for Python 2.4 unintentionally + lost that tests are sorted by name before being run. This rarely + matters for well-written tests, but can create baffling symptoms if + side effects from one test to the next affect outcomes. ``DocTestFinder`` + has been changed to sort the list of tests it returns. + +- The distutils version has been changed to 2.5.0, and is now kept + in sync with sys.version_info[:3]. + +- Bug #978833: Really close underlying socket in _socketobject.close. + +- Bug #1459963: urllib and urllib2 now normalize HTTP header names with + title(). + +- Patch #1525766: In pkgutil.walk_packages, correctly pass the onerror callback + to recursive calls and call it with the failing package name. + +- Bug #1525817: Don't truncate short lines in IDLE's tool tips. + +- Patch #1515343: Fix printing of deprecated string exceptions with a + value in the traceback module. + +- Resync optparse with Optik 1.5.3: minor tweaks for/to tests. + +- Patch #1524429: Use repr() instead of backticks in Tkinter again. + +- Bug #1520914: Change time.strftime() to accept a zero for any position in its + argument tuple. For arguments where zero is illegal, the value is forced to + the minimum value that is correct. This is to support an undocumented but + common way people used to fill in inconsequential information in the time + tuple pre-2.4. + +- Patch #1220874: Update the binhex module for Mach-O. + +- The email package has improved RFC 2231 support, specifically for + recognizing the difference between encoded (name*0*=<blah>) and non-encoded + (name*0=<blah>) parameter continuations. This may change the types of + values returned from email.message.Message.get_param() and friends. + Specifically in some cases where non-encoded continuations were used, + get_param() used to return a 3-tuple of (None, None, string) whereas now it + will just return the string (since non-encoded continuations don't have + charset and language parts). + + Also, whereas % values were decoded in all parameter continuations, they are + now only decoded in encoded parameter parts. + +- Bug #1517990: IDLE keybindings on MacOS X now work correctly + +- Bug #1517996: IDLE now longer shows the default Tk menu when a + path browser, class browser or debugger is the frontmost window on MacOS X + +- Patch #1520294: Support for getset and member descriptors in types.py, + inspect.py, and pydoc.py. Specifically, this allows for querying the type + of an object against these built-in types and more importantly, for getting + their docstrings printed in the interactive interpreter's help() function. + + +Extension Modules +----------------- + +- Patch #1519025 and bug #926423: If a KeyboardInterrupt occurs during + a socket operation on a socket with a timeout, the exception will be + caught correctly. Previously, the exception was not caught. + +- Patch #1529514: The _ctypes extension is now compiled on more + openbsd target platforms. + +- The ``__reduce__()`` method of the new ``collections.defaultdict`` had + a memory leak, affecting pickles and deep copies. + +- Bug #1471938: Fix curses module build problem on Solaris 8; patch by + Paul Eggert. + +- Patch #1448199: Release interpreter lock in _winreg.ConnectRegistry. + +- Patch #1521817: Index range checking on ctypes arrays containing + exactly one element enabled again. This allows iterating over these + arrays, without the need to check the array size before. + +- Bug #1521375: When the code in ctypes.util.find_library was + run with root privileges, it could overwrite or delete + /dev/null in certain cases; this is now fixed. + +- Bug #1467450: On Mac OS X 10.3, RTLD_GLOBAL is now used as the + default mode for loading shared libraries in ctypes. + +- Because of a misspelled preprocessor symbol, ctypes was always + compiled without thread support; this is now fixed. + +- pybsddb Bug #1527939: bsddb module DBEnv dbremove and dbrename + methods now allow their database parameter to be None as the + sleepycat API allows. + +- Bug #1526460: Fix socketmodule compile on NetBSD as it has a different + bluetooth API compared with Linux and FreeBSD. + +Tests +----- + +- Bug #1501330: Change test_ossaudiodev to be much more tolerant in terms of + how long the test file should take to play. Now accepts taking 2.93 secs + (exact time) +/- 10% instead of the hard-coded 3.1 sec. + +- Patch #1529686: The standard tests ``test_defaultdict``, ``test_iterlen``, + ``test_uuid`` and ``test_email_codecs`` didn't actually run any tests when + run via ``regrtest.py``. Now they do. + +Build +----- + +- Bug #1439538: Drop usage of test -e in configure as it is not portable. + +Mac +--- + +- PythonLauncher now works correctly when the path to the script contains + characters that are treated specially by the shell (such as quotes). + +- Bug #1527397: PythonLauncher now launches scripts with the working directory + set to the directory that contains the script instead of the user home + directory. That latter was an implementation accident and not what users + expect. + + +What's New in Python 2.5 beta 2? +================================ + +*Release date: 11-JUL-2006* + +Core and builtins +----------------- + +- Bug #1441486: The literal representation of -(sys.maxint - 1) + again evaluates to a int object, not a long. + +- Bug #1501934: The scope of global variables that are locally assigned + using augmented assignment is now correctly determined. + +- Bug #927248: Recursive method-wrapper objects can now safely + be released. + +- Bug #1417699: Reject locale-specific decimal point in float() + and atof(). + +- Bug #1511381: codec_getstreamcodec() in codec.c is corrected to + omit a default "error" argument for NULL pointer. This allows + the parser to take a codec from cjkcodecs again. + +- Bug #1519018: 'as' is now validated properly in import statements. + +- On 64 bit systems, int literals that use less than 64 bits are + now ints rather than longs. + +- Bug #1512814, Fix incorrect lineno's when code at module scope + started after line 256. + +- New function ``sys._current_frames()`` returns a dict mapping thread + id to topmost thread stack frame. This is for expert use, and is + especially useful for debugging application deadlocks. The functionality + was previously available in Fazal Majid's ``threadframe`` extension + module, but it wasn't possible to do this in a wholly threadsafe way from + an extension. + +Library +------- + +- Bug #1257728: Mention Cygwin in distutils error message about a missing + VS 2003. + +- Patch #1519566: Update turtle demo, make begin_fill idempotent. + +- Bug #1508010: msvccompiler now requires the DISTUTILS_USE_SDK + environment variable to be set in order to the SDK environment + for finding the compiler, include files, etc. + +- Bug #1515998: Properly generate logical ids for files in bdist_msi. + +- warnings.py now ignores ImportWarning by default + +- string.Template() now correctly handles tuple-values. Previously, + multi-value tuples would raise an exception and single-value tuples would + be treated as the value they contain, instead. + +- Bug #822974: Honor timeout in telnetlib.{expect,read_until} + even if some data are received. + +- Bug #1267547: Put proper recursive setup.py call into the + spec file generated by bdist_rpm. + +- Bug #1514693: Update turtle's heading when switching between + degrees and radians. + +- Reimplement turtle.circle using a polyline, to allow correct + filling of arcs. + +- Bug #1514703: Only setup canvas window in turtle when the canvas + is created. + +- Bug #1513223: .close() of a _socketobj now releases the underlying + socket again, which then gets closed as it becomes unreferenced. + +- Bug #1504333: Make sgmllib support angle brackets in quoted + attribute values. + +- Bug #853506: Fix IPv6 address parsing in unquoted attributes in + sgmllib ('[' and ']' were not accepted). + +- Fix a bug in the turtle module's end_fill function. + +- Bug #1510580: The 'warnings' module improperly required that a Warning + category be either a types.ClassType and a subclass of Warning. The proper + check is just that it is a subclass with Warning as the documentation states. + +- The compiler module now correctly compiles the new try-except-finally + statement (bug #1509132). + +- The wsgiref package is now installed properly on Unix. + +- A bug was fixed in logging.config.fileConfig() which caused a crash on + shutdown when fileConfig() was called multiple times. + +- The sqlite3 module did cut off data from the SQLite database at the first + null character before sending it to a custom converter. This has been fixed + now. + +Extension Modules +----------------- + +- #1494314: Fix a regression with high-numbered sockets in 2.4.3. This + means that select() on sockets > FD_SETSIZE (typically 1024) work again. + The patch makes sockets use poll() internally where available. + +- Assigning None to pointer type fields in ctypes structures possible + overwrote the wrong fields, this is fixed now. + +- Fixed a segfault in _ctypes when ctypes.wintypes were imported + on non-Windows platforms. + +- Bug #1518190: The ctypes.c_void_p constructor now accepts any + integer or long, without range checking. + +- Patch #1517790: It is now possible to use custom objects in the ctypes + foreign function argtypes sequence as long as they provide a from_param + method, no longer is it required that the object is a ctypes type. + +- The '_ctypes' extension module now works when Python is configured + with the --without-threads option. + +- Bug #1513646: os.access on Windows now correctly determines write + access, again. + +- Bug #1512695: cPickle.loads could crash if it was interrupted with + a KeyboardInterrupt. + +- Bug #1296433: parsing XML with a non-default encoding and + a CharacterDataHandler could crash the interpreter in pyexpat. + +- Patch #1516912: improve Modules support for OpenVMS. + +Build +----- + +- Automate Windows build process for the Win64 SSL module. + +- 'configure' now detects the zlib library the same way as distutils. + Previously, the slight difference could cause compilation errors of the + 'zlib' module on systems with more than one version of zlib. + +- The MSI compileall step was fixed to also support a TARGETDIR + with spaces in it. + +- Bug #1517388: sqlite3.dll is now installed on Windows independent + of Tcl/Tk. + +- Bug #1513032: 'make install' failed on FreeBSD 5.3 due to lib-old + trying to be installed even though it's empty. + +Tests +----- + +- Call os.waitpid() at the end of tests that spawn child processes in order + to minimize resources (zombies). + +Documentation +------------- + +- Cover ImportWarning, PendingDeprecationWarning and simplefilter() in the + documentation for the warnings module. + +- Patch #1509163: MS Toolkit Compiler no longer available. + +- Patch #1504046: Add documentation for xml.etree. + + +What's New in Python 2.5 beta 1? +================================ + +*Release date: 20-JUN-2006* + +Core and builtins +----------------- + +- Patch #1507676: Error messages returned by invalid abstract object operations + (such as iterating over an integer) have been improved and now include the + type of the offending object to help with debugging. + +- Bug #992017: A classic class that defined a __coerce__() method that returned + its arguments swapped would infinitely recurse and segfault the interpreter. + +- Fix the socket tests so they can be run concurrently. + +- Removed 5 integers from C frame objects (PyFrameObject). + f_nlocals, f_ncells, f_nfreevars, f_stack_size, f_restricted. + +- Bug #532646: object.__call__() will continue looking for the __call__ + attribute on objects until one without one is found. This leads to recursion + when you take a class and set its __call__ attribute to an instance of the + class. Originally fixed for classic classes, but this fix is for new-style. + Removes the infinite_rec_3 crasher. + +- The string and unicode methods startswith() and endswith() now accept + a tuple of prefixes/suffixes to look for. Implements RFE #1491485. + +- Buffer objects, at the C level, never used the char buffer + implementation even when the char buffer for the wrapped object was + explicitly requested (originally returned the read or write buffer). + Now a TypeError is raised if the char buffer is not present but is + requested. + +- Patch #1346214: Statements like "if 0: suite" are now again optimized + away like they were in Python 2.4. + +- Builtin exceptions are now full-blown new-style classes instead of + instances pretending to be classes, which speeds up exception handling + by about 80% in comparison to 2.5a2. + +- Patch #1494554: Update unicodedata.numeric and unicode.isnumeric to + Unicode 4.1. + +- Patch #921466: sys.path_importer_cache is now used to cache valid and + invalid file paths for the built-in import machinery which leads to + fewer open calls on startup. + +- Patch #1442927: ``long(str, base)`` is now up to 6x faster for non-power- + of-2 bases. The largest speedup is for inputs with about 1000 decimal + digits. Conversion from non-power-of-2 bases remains quadratic-time in + the number of input digits (it was and remains linear-time for bases + 2, 4, 8, 16 and 32). + +- Bug #1334662: ``int(string, base)`` could deliver a wrong answer + when ``base`` was not 2, 4, 8, 10, 16 or 32, and ``string`` represented + an integer close to ``sys.maxint``. This was repaired by patch + #1335972, which also gives a nice speedup. + +- Patch #1337051: reduced size of frame objects. + +- PyErr_NewException now accepts a tuple of base classes as its + "base" parameter. + +- Patch #876206: function call speedup by retaining allocated frame + objects. + +- Bug #1462152: file() now checks more thoroughly for invalid mode + strings and removes a possible "U" before passing the mode to the + C library function. + +- Patch #1488312, Fix memory alignment problem on SPARC in unicode + +- Bug #1487966: Fix SystemError with conditional expression in assignment + +- WindowsError now has two error code attributes: errno, which carries + the error values from errno.h, and winerror, which carries the error + values from winerror.h. Previous versions put the winerror.h values + (from GetLastError()) into the errno attribute. + +- Patch #1475845: Raise IndentationError for unexpected indent. + +- Patch #1479181: split open() and file() from being aliases for each other. + +- Patch #1497053 & bug #1275608: Exceptions occurring in ``__eq__()`` + methods were always silently ignored by dictionaries when comparing keys. + They are now passed through (except when using the C API function + ``PyDict_GetItem()``, whose semantics did not change). + +- Bug #1456209: In some obscure cases it was possible for a class with a + custom ``__eq__()`` method to confuse dict internals when class instances + were used as a dict's keys and the ``__eq__()`` method mutated the dict. + No, you don't have any code that did this ;-) + +Extension Modules +----------------- + +- Bug #1295808: expat symbols should be namespaced in pyexpat + +- Patch #1462338: Upgrade pyexpat to expat 2.0.0 + +- Change binascii.hexlify to accept a read-only buffer instead of only a char + buffer and actually follow its documentation. + +- Fixed a potentially invalid memory access of CJKCodecs' shift-jis decoder. + +- Patch #1478788 (modified version): The functional extension module has + been renamed to _functools and a functools Python wrapper module added. + This provides a home for additional function related utilities that are + not specifically about functional programming. See PEP 309. + +- Patch #1493701: performance enhancements for struct module. + +- Patch #1490224: time.altzone is now set correctly on Cygwin. + +- Patch #1435422: zlib's compress and decompress objects now have a + copy() method. + +- Patch #1454481: thread stack size is now tunable at runtime for thread + enabled builds on Windows and systems with Posix threads support. + +- On Win32, os.listdir now supports arbitrarily-long Unicode path names + (up to the system limit of 32K characters). + +- Use Win32 API to implement os.{access,chdir,chmod,mkdir,remove,rename,rmdir,utime}. + As a result, these functions now raise WindowsError instead of OSError. + +- ``time.clock()`` on Win64 should use the high-performance Windows + ``QueryPerformanceCounter()`` now (as was already the case on 32-bit + Windows platforms). + +- Calling Tk_Init twice is refused if the first call failed as that + may deadlock. + +- bsddb: added the DB_ARCH_REMOVE flag and fixed db.DBEnv.log_archive() to + accept it without potentially using an uninitialized pointer. + +- bsddb: added support for the DBEnv.log_stat() and DBEnv.lsn_reset() methods + assuming BerkeleyDB >= 4.0 and 4.4 respectively. [pybsddb project SF + patch numbers 1494885 and 1494902] + +- bsddb: added an interface for the BerkeleyDB >= 4.3 DBSequence class. + [pybsddb project SF patch number 1466734] + +- bsddb: fix DBCursor.pget() bug with keyword argument names when no data + parameter is supplied. [SF pybsddb bug #1477863] + +- bsddb: the __len__ method of a DB object has been fixed to return correct + results. It could previously incorrectly return 0 in some cases. + Fixes SF bug 1493322 (pybsddb bug 1184012). + +- bsddb: the bsddb.dbtables Modify method now raises the proper error and + aborts the db transaction safely when a modifier callback fails. + Fixes SF python patch/bug #1408584. + +- bsddb: multithreaded DB access using the simple bsddb module interface + now works reliably. It has been updated to use automatic BerkeleyDB + deadlock detection and the bsddb.dbutils.DeadlockWrap wrapper to retry + database calls that would previously deadlock. [SF python bug #775414] + +- Patch #1446489: add support for the ZIP64 extensions to zipfile. + +- Patch #1506645: add Python wrappers for the curses functions + is_term_resized, resize_term and resizeterm. + +Library +------- + +- Patch #815924: Restore ability to pass type= and icon= in tkMessageBox + functions. + +- Patch #812986: Update turtle output even if not tracing. + +- Patch #1494750: Destroy master after deleting children in + Tkinter.BaseWidget. + +- Patch #1096231: Add ``default`` argument to Tkinter.Wm.wm_iconbitmap. + +- Patch #763580: Add name and value arguments to Tkinter variable + classes. + +- Bug #1117556: SimpleHTTPServer now tries to find and use the system's + mime.types file for determining MIME types. + +- Bug #1339007: Shelf objects now don't raise an exception in their + __del__ method when initialization failed. + +- Patch #1455898: The MBCS codec now supports the incremental mode for + double-byte encodings. + +- ``difflib``'s ``SequenceMatcher.get_matching_blocks()`` was changed to + guarantee that adjacent triples in the return list always describe + non-adjacent blocks. Previously, a pair of matching blocks could end + up being described by multiple adjacent triples that formed a partition + of the matching pair. + +- Bug #1498146: fix optparse to handle Unicode strings in option help, + description, and epilog. + +- Bug #1366250: minor optparse documentation error. + +- Bug #1361643: fix textwrap.dedent() so it handles tabs appropriately; + clarify docs. + +- The wsgiref package has been added to the standard library. + +- The functions update_wrapper() and wraps() have been added to the functools + module. These make it easier to copy relevant metadata from the original + function when writing wrapper functions. + +- The optional ``isprivate`` argument to ``doctest.testmod()``, and the + ``doctest.is_private()`` function, both deprecated in 2.4, were removed. + +- Patch #1359618: Speed up charmap encoder by using a trie structure + for lookup. + +- The functions in the ``pprint`` module now sort dictionaries by key + before computing the display. Before 2.5, ``pprint`` sorted a dictionary + if and only if its display required more than one line, although that + wasn't documented. The new behavior increases predictability; e.g., + using ``pprint.pprint(a_dict)`` in a doctest is now reliable. + +- Patch #1497027: try HTTP digest auth before basic auth in urllib2 + (thanks for J. J. Lee). + +- Patch #1496206: improve urllib2 handling of passwords with respect to + default HTTP and HTTPS ports. + +- Patch #1080727: add "encoding" parameter to doctest.DocFileSuite. + +- Patch #1281707: speed up gzip.readline. + +- Patch #1180296: Two new functions were added to the locale module: + format_string() to get the effect of "format % items" but locale-aware, + and currency() to format a monetary number with currency sign. + +- Patch #1486962: Several bugs in the turtle Tk demo module were fixed + and several features added, such as speed and geometry control. + +- Patch #1488881: add support for external file objects in bz2 compressed + tarfiles. + +- Patch #721464: pdb.Pdb instances can now be given explicit stdin and + stdout arguments, making it possible to redirect input and output + for remote debugging. + +- Patch #1484695: Update the tarfile module to version 0.8. This fixes + a couple of issues, notably handling of long file names using the + GNU LONGNAME extension. + +- Patch #1478292. ``doctest.register_optionflag(name)`` shouldn't create a + new flag when ``name`` is already the name of an option flag. + +- Bug #1385040: don't allow "def foo(a=1, b): pass" in the compiler + package. + +- Patch #1472854: make the rlcompleter.Completer class usable on non- + UNIX platforms. + +- Patch #1470846: fix urllib2 ProxyBasicAuthHandler. + +- Bug #1472827: correctly escape newlines and tabs in attribute values in + the saxutils.XMLGenerator class. + + +Build +----- + +- Bug #1502728: Correctly link against librt library on HP-UX. + +- OpenBSD 3.9 is supported now. + +- Patch #1492356: Port to Windows CE. + +- Bug/Patch #1481770: Use .so extension for shared libraries on HP-UX for ia64. + +- Patch #1471883: Add --enable-universalsdk. + +C API +----- + +Tests +----- + +Tools +----- + +Documentation +------------- + + + +What's New in Python 2.5 alpha 2? +================================= + +*Release date: 27-APR-2006* + +Core and builtins +----------------- + +- Bug #1465834: 'bdist_wininst preinstall script support' was fixed + by converting these apis from macros into exported functions again: + + PyParser_SimpleParseFile PyParser_SimpleParseString PyRun_AnyFile + PyRun_AnyFileEx PyRun_AnyFileFlags PyRun_File PyRun_FileEx + PyRun_FileFlags PyRun_InteractiveLoop PyRun_InteractiveOne + PyRun_SimpleFile PyRun_SimpleFileEx PyRun_SimpleString + PyRun_String Py_CompileString + +- Under COUNT_ALLOCS, types are not necessarily immortal anymore. + +- All uses of PyStructSequence_InitType have been changed to initialize + the type objects only once, even if the interpreter is initialized + multiple times. + +- Bug #1454485, array.array('u') could crash the interpreter. This was + due to PyArgs_ParseTuple(args, 'u#', ...) trying to convert buffers (strings) + to unicode when it didn't make sense. 'u#' now requires a unicode string. + +- Py_UNICODE is unsigned. It was always documented as unsigned, but + due to a bug had a signed value in previous versions. + +- Patch #837242: ``id()`` of any Python object always gives a positive + number now, which might be a long integer. ``PyLong_FromVoidPtr`` and + ``PyLong_AsVoidPtr`` have been changed accordingly. Note that it has + never been correct to implement a ``__hash()__`` method that returns the + ``id()`` of an object: + + def __hash__(self): + return id(self) # WRONG + + because a hash result must be a (short) Python int but it was always + possible for ``id()`` to return a Python long. However, because ``id()`` + could return negative values before, on a 32-bit box an ``id()`` result + was always usable as a hash value before this patch. That's no longer + necessarily so. + +- Python on OS X 10.3 and above now uses dlopen() (via dynload_shlib.c) + to load extension modules and now provides the dl module. As a result, + sys.setdlopenflags() now works correctly on these systems. (SF patch + #1454844) + +- Patch #1463867: enhanced garbage collection to allow cleanup of cycles + involving generators that have paused outside of any ``try`` or ``with`` + blocks. (In 2.5a1, a paused generator that was part of a reference + cycle could not be garbage collected, regardless of whether it was + paused in a ``try`` or ``with`` block.) + +Extension Modules +----------------- + +- Patch #1191065: Fix preprocessor problems on systems where recvfrom + is a macro. + +- Bug #1467952: os.listdir() now correctly raises an error if readdir() + fails with an error condition. + +- Fixed bsddb.db.DBError derived exceptions so they can be unpickled. + +- Bug #1117761: bsddb.*open() no longer raises an exception when using + the cachesize parameter. + +- Bug #1149413: bsddb.*open() no longer raises an exception when using + a temporary db (file=None) with the 'n' flag to truncate on open. + +- Bug #1332852: bsddb module minimum BerkeleyDB version raised to 3.3 + as older versions cause excessive test failures. + +- Patch #1062014: AF_UNIX sockets under Linux have a special + abstract namespace that is now fully supported. + +Library +------- + +- Bug #1223937: subprocess.CalledProcessError reports the exit status + of the process using the returncode attribute, instead of + abusing errno. + +- Patch #1475231: ``doctest`` has a new ``SKIP`` option, which causes + a doctest to be skipped (the code is not run, and the expected output + or exception is ignored). + +- Fixed contextlib.nested to cope with exceptions being raised and + caught inside exit handlers. + +- Updated optparse module to Optik 1.5.1 (allow numeric constants in + hex, octal, or binary; add ``append_const`` action; keep going if + gettext cannot be imported; added ``OptionParser.destroy()`` method; + added ``epilog`` for better help generation). + +- Bug #1473760: ``tempfile.TemporaryFile()`` could hang on Windows, when + called from a thread spawned as a side effect of importing a module. + +- The pydoc module now supports documenting packages contained in + .zip or .egg files. + +- The pkgutil module now has several new utility functions, such + as ``walk_packages()`` to support working with packages that are either + in the filesystem or zip files. + +- The mailbox module can now modify and delete messages from + mailboxes, in addition to simply reading them. Thanks to Gregory + K. Johnson for writing the code, and to the 2005 Google Summer of + Code for funding his work. + +- The ``__del__`` method of class ``local`` in module ``_threading_local`` + returned before accomplishing any of its intended cleanup. + +- Patch #790710: Add breakpoint command lists in pdb. + +- Patch #1063914: Add Tkinter.Misc.clipboard_get(). + +- Patch #1191700: Adjust column alignment in bdb breakpoint lists. + +- SimpleXMLRPCServer relied on the fcntl module, which is unavailable on + Windows. Bug #1469163. + +- The warnings, linecache, inspect, traceback, site, and doctest modules + were updated to work correctly with modules imported from zipfiles or + via other PEP 302 __loader__ objects. + +- Patch #1467770: Reduce usage of subprocess._active to processes which + the application hasn't waited on. + +- Patch #1462222: Fix Tix.Grid. + +- Fix exception when doing glob.glob('anything*/') + +- The pstats.Stats class accepts an optional stream keyword argument to + direct output to an alternate file-like object. + +Build +----- + +- The Makefile now has a reindent target, which runs reindent.py on + the library. + +- Patch #1470875: Building Python with MS Free Compiler + +- Patch #1161914: Add a python-config script. + +- Patch #1324762:Remove ccpython.cc; replace --with-cxx with + --with-cxx-main. Link with C++ compiler only if --with-cxx-main was + specified. (Can be overridden by explicitly setting LINKCC.) Decouple + CXX from --with-cxx-main, see description in README. + +- Patch #1429775: Link extension modules with the shared libpython. + +- Fixed a libffi build problem on MIPS systems. + +- ``PyString_FromFormat``, ``PyErr_Format``, and ``PyString_FromFormatV`` + now accept formats "%u" for unsigned ints, "%lu" for unsigned longs, + and "%zu" for unsigned integers of type ``size_t``. + +Tests +----- + +- test_contextlib now checks contextlib.nested can cope with exceptions + being raised and caught inside exit handlers. + +- test_cmd_line now checks operation of the -m and -c command switches + +- The test_contextlib test in 2.5a1 wasn't actually run unless you ran + it separately and by hand. It also wasn't cleaning up its changes to + the current Decimal context. + +- regrtest.py now has a -M option to run tests that test the new limits of + containers, on 64-bit architectures. Running these tests is only sensible + on 64-bit machines with more than two gigabytes of memory. The argument + passed is the maximum amount of memory for the tests to use. + +Tools +----- + +- Added the Python benchmark suite pybench to the Tools/ directory; + contributed by Marc-Andre Lemburg. + +Documentation +------------- + +- Patch #1473132: Improve docs for ``tp_clear`` and ``tp_traverse``. + +- PEP 343: Added Context Types section to the library reference + and attempted to bring other PEP 343 related documentation into + line with the implementation and/or python-dev discussions. + +- Bug #1337990: clarified that ``doctest`` does not support examples + requiring both expected output and an exception. + + +What's New in Python 2.5 alpha 1? +================================= + +*Release date: 05-APR-2006* + +Core and builtins +----------------- + +- PEP 338: -m command line switch now delegates to runpy.run_module + allowing it to support modules in packages and zipfiles + +- On Windows, .DLL is not an accepted file name extension for + extension modules anymore; extensions are only found if they + end in .PYD. + +- Bug #1421664: sys.stderr.encoding is now set to the same value as + sys.stdout.encoding. + +- __import__ accepts keyword arguments. + +- Patch #1460496: round() now accepts keyword arguments. + +- Fixed bug #1459029 - unicode reprs were double-escaped. + +- Patch #1396919: The system scope threads are reenabled on FreeBSD + 5.4 and later versions. + +- Bug #1115379: Compiling a Unicode string with an encoding declaration + now gives a SyntaxError. + +- Previously, Python code had no easy way to access the contents of a + cell object. Now, a ``cell_contents`` attribute has been added + (closes patch #1170323). + +- Patch #1123430: Python's small-object allocator now returns an arena to + the system ``free()`` when all memory within an arena becomes unused + again. Prior to Python 2.5, arenas (256KB chunks of memory) were never + freed. Some applications will see a drop in virtual memory size now, + especially long-running applications that, from time to time, temporarily + use a large number of small objects. Note that when Python returns an + arena to the platform C's ``free()``, there's no guarantee that the + platform C library will in turn return that memory to the operating system. + The effect of the patch is to stop making that impossible, and in tests it + appears to be effective at least on Microsoft C and gcc-based systems. + Thanks to Evan Jones for hard work and patience. + +- Patch #1434038: property() now uses the getter's docstring if there is + no "doc" argument given. This makes it possible to legitimately use + property() as a decorator to produce a read-only property. + +- PEP 357, patch 1436368: add an __index__ method to int/long and a matching + nb_index slot to the PyNumberMethods struct. The slot is consulted instead + of requiring an int or long in slicing and a few other contexts, enabling + other objects (e.g. Numeric Python's integers) to be used as slice indices. + +- Fixed various bugs reported by Coverity's Prevent tool. + +- PEP 352, patch #1104669: Make exceptions new-style objects. Introduced the + new exception base class, BaseException, which has a new message attribute. + KeyboardInterrupt and SystemExit to directly inherit from BaseException now. + Raising a string exception now raises a DeprecationWarning. + +- Patch #1438387, PEP 328: relative and absolute imports. Imports can now be + explicitly relative, using 'from .module import name' to mean 'from the same + package as this module is in. Imports without dots still default to the + old relative-then-absolute, unless 'from __future__ import + absolute_import' is used. + +- Properly check if 'warnings' raises an exception (usually when a filter set + to "error" is triggered) when raising a warning for raising string + exceptions. + +- CO_GENERATOR_ALLOWED is no longer defined. This behavior is the default. + The name was removed from Include/code.h. + +- PEP 308: conditional expressions were added: (x if cond else y). + +- Patch 1433928: + - The copy module now "copies" function objects (as atomic objects). + - dict.__getitem__ now looks for a __missing__ hook before raising + KeyError. + +- PEP 343: with statement implemented. Needs ``from __future__ import + with_statement``. Use of 'with' as a variable will generate a warning. + Use of 'as' as a variable will also generate a warning (unless it's + part of an import statement). + The following objects have __context__ methods: + - The built-in file type. + - The thread.LockType type. + - The following types defined by the threading module: + Lock, RLock, Condition, Semaphore, BoundedSemaphore. + - The decimal.Context class. + +- Fix the encodings package codec search function to only search + inside its own package. Fixes problem reported in patch #1433198. + + Note: Codec packages should implement and register their own + codec search function. PEP 100 has the details. + +- PEP 353: Using ``Py_ssize_t`` as the index type. + +- ``PYMALLOC_DEBUG`` builds now add ``4*sizeof(size_t)`` bytes of debugging + info to each allocated block, since the ``Py_ssize_t`` changes (PEP 353) + now allow Python to make use of memory blocks exceeding 2**32 bytes for + some purposes on 64-bit boxes. A ``PYMALLOC_DEBUG`` build was limited + to 4-byte allocations before. + +- Patch #1400181, fix unicode string formatting to not use the locale. + This is how string objects work. u'%f' could use , instead of . + for the decimal point. Now both strings and unicode always use periods. + +- Bug #1244610, #1392915, fix build problem on OpenBSD 3.7 and 3.8. + configure would break checking curses.h. + +- Bug #959576: The pwd module is now builtin. This allows Python to be + built on UNIX platforms without $HOME set. + +- Bug #1072182, fix some potential problems if characters are signed. + +- Bug #889500, fix line number on SyntaxWarning for global declarations. + +- Bug #1378022, UTF-8 files with a leading BOM crashed the interpreter. + +- Support for converting hex strings to floats no longer works. + This was not portable. float('0x3') now raises a ValueError. + +- Patch #1382163: Expose Subversion revision number to Python. New C API + function Py_GetBuildNumber(). New attribute sys.subversion. Build number + is now displayed in interactive prompt banner. + +- Implementation of PEP 341 - Unification of try/except and try/finally. + "except" clauses can now be written together with a "finally" clause in + one try statement instead of two nested ones. Patch #1355913. + +- Bug #1379994: Builtin unicode_escape and raw_unicode_escape codec + now encodes backslash correctly. + +- Patch #1350409: Work around signal handling bug in Visual Studio 2005. + +- Bug #1281408: Py_BuildValue now works correctly even with unsigned longs + and long longs. + +- SF Bug #1350188, "setdlopenflags" leads to crash upon "import" + It was possible for dlerror() to return a NULL pointer, so + it will now use a default error message in this case. + +- Replaced most Unicode charmap codecs with new ones using the + new Unicode translate string feature in the builtin charmap + codec; the codecs were created from the mapping tables available + at ftp.unicode.org and contain a few updates (e.g. the Mac OS + encodings now include a mapping for the Apple logo) + +- Added a few more codecs for Mac OS encodings + +- Sped up some Unicode operations. + +- A new AST parser implementation was completed. The abstract + syntax tree is available for read-only (non-compile) access + to Python code; an _ast module was added. + +- SF bug #1167751: fix incorrect code being produced for generator expressions. + The following code now raises a SyntaxError: foo(a = i for i in range(10)) + +- SF Bug #976608: fix SystemError when mtime of an imported file is -1. + +- SF Bug #887946: fix segfault when redirecting stdin from a directory. + Provide a warning when a directory is passed on the command line. + +- Fix segfault with invalid coding. + +- SF bug #772896: unknown encoding results in MemoryError. + +- All iterators now have a Boolean value of True. Formerly, some iterators + supported a __len__() method which evaluated to False when the iterator + was empty. + +- On 64-bit platforms, when __len__() returns a value that cannot be + represented as a C int, raise OverflowError. + +- test__locale is skipped on OS X < 10.4 (only partial locale support is + present). + +- SF bug #893549: parsing keyword arguments was broken with a few format + codes. + +- Changes donated by Elemental Security to make it work on AIX 5.3 + with IBM's 64-bit compiler (SF patch #1284289). This also closes SF + bug #105470: test_pwd fails on 64bit system (Opteron). + +- Changes donated by Elemental Security to make it work on HP-UX 11 on + Itanium2 with HP's 64-bit compiler (SF patch #1225212). + +- Disallow keyword arguments for type constructors that don't use them + (fixes bug #1119418). + +- Forward UnicodeDecodeError into SyntaxError for source encoding errors. + +- SF bug #900092: When tracing (e.g. for hotshot), restore 'return' events for + exceptions that cause a function to exit. + +- The implementation of set() and frozenset() was revised to use its + own internal data structure. Memory consumption is reduced by 1/3 + and there are modest speed-ups as well. The API is unchanged. + +- SF bug #1238681: freed pointer is used in longobject.c:long_pow(). + +- SF bug #1229429: PyObject_CallMethod failed to decrement some + reference counts in some error exit cases. + +- SF bug #1185883: Python's small-object memory allocator took over + a block managed by the platform C library whenever a realloc specified + a small new size. However, there's no portable way to know then how + much of the address space following the pointer is valid, so there's no + portable way to copy data from the C-managed block into Python's + small-object space without risking a memory fault. Python's small-object + realloc now leaves such blocks under the control of the platform C + realloc. + +- SF bug #1232517: An overflow error was not detected properly when + attempting to convert a large float to an int in os.utime(). + +- SF bug #1224347: hex longs now print with lowercase letters just + like their int counterparts. + +- SF bug #1163563: the original fix for bug #1010677 ("thread Module + Breaks PyGILState_Ensure()") broke badly in the case of multiple + interpreter states; back out that fix and do a better job (see + http://mail.python.org/pipermail/python-dev/2005-June/054258.html + for a longer write-up of the problem). + +- SF patch #1180995: marshal now uses a binary format by default when + serializing floats. + +- SF patch #1181301: on platforms that appear to use IEEE 754 floats, + the routines that promise to produce IEEE 754 binary representations + of floats now simply copy bytes around. + +- bug #967182: disallow opening files with 'wU' or 'aU' as specified by PEP + 278. + +- patch #1109424: int, long, float, complex, and unicode now check for the + proper magic slot for type conversions when subclassed. Previously the + magic slot was ignored during conversion. Semantics now match the way + subclasses of str always behaved. int/long/float, conversion of an instance + to the base class has been moved to the proper nb_* magic slot and out of + PyNumber_*(). + Thanks Walter D�rwald. + +- Descriptors defined in C with a PyGetSetDef structure, where the setter is + NULL, now raise an AttributeError when attempting to set or delete the + attribute. Previously a TypeError was raised, but this was inconsistent + with the equivalent pure-Python implementation. + +- It is now safe to call PyGILState_Release() before + PyEval_InitThreads() (note that if there is reason to believe there + are multiple threads around you still must call PyEval_InitThreads() + before using the Python API; this fix is for extension modules that + have no way of knowing if Python is multi-threaded yet). + +- Typing Ctrl-C whilst raw_input() was waiting in a build with threads + disabled caused a crash. + +- Bug #1165306: instancemethod_new allowed the creation of a method + with im_class == im_self == NULL, which caused a crash when called. + +- Move exception finalisation later in the shutdown process - this + fixes the crash seen in bug #1165761 + +- Added two new builtins, any() and all(). + +- Defining a class with empty parentheses is now allowed + (e.g., ``class C(): pass`` is no longer a syntax error). + Patch #1176012 added support to the 'parser' module and 'compiler' package + (thanks to logistix for that added support). + +- Patch #1115086: Support PY_LONGLONG in structmember. + +- Bug #1155938: new style classes did not check that __init__() was + returning None. + +- Patch #802188: Report characters after line continuation character + ('\') with a specific error message. + +- Bug #723201: Raise a TypeError for passing bad objects to 'L' format. + +- Bug #1124295: the __name__ attribute of file objects was + inadvertently made inaccessible in restricted mode. + +- Bug #1074011: closing sys.std{out,err} now causes a flush() and + an ferror() call. + +- min() and max() now support key= arguments with the same meaning as in + list.sort(). + +- The peephole optimizer now performs simple constant folding in expressions: + (2+3) --> (5). + +- set and frozenset objects can now be marshalled. SF #1098985. + +- Bug #1077106: Poor argument checking could cause memory corruption + in calls to os.read(). + +- The parser did not complain about future statements in illegal + positions. It once again reports a syntax error if a future + statement occurs after anything other than a doc string. + +- Change the %s format specifier for str objects so that it returns a + unicode instance if the argument is not an instance of basestring and + calling __str__ on the argument returns a unicode instance. + +- Patch #1413181: changed ``PyThreadState_Delete()`` to forget about the + current thread state when the auto-GIL-state machinery knows about + it (since the thread state is being deleted, continuing to remember it + can't help, but can hurt if another thread happens to get created with + the same thread id). + +Extension Modules +----------------- + +- Patch #1380952: fix SSL objects timing out on consecutive read()s + +- Patch #1309579: wait3 and wait4 were added to the posix module. + +- Patch #1231053: The audioop module now supports encoding/decoding of alaw. + In addition, the existing ulaw code was updated. + +- RFE #567972: Socket objects' family, type and proto properties are + now exposed via new attributes. + +- Everything under lib-old was removed. This includes the following modules: + Para, addpack, cmp, cmpcache, codehack, dircmp, dump, find, fmt, grep, + lockfile, newdir, ni, packmail, poly, rand, statcache, tb, tzparse, + util, whatsound, whrandom, zmod + +- The following modules were removed: regsub, reconvert, regex, regex_syntax. + +- re and sre were swapped, so help(re) provides full help. importing sre + is deprecated. The undocumented re.engine variable no longer exists. + +- Bug #1448490: Fixed a bug that ISO-2022 codecs could not handle + SS2 (single-shift 2) escape sequences correctly. + +- The unicodedata module was updated to the 4.1 version of the Unicode + database. The 3.2 version is still available as unicodedata.db_3_2_0 + for applications that require this specific version (such as IDNA). + +- The timing module is no longer built by default. It was deprecated + in PEP 4 in Python 2.0 or earlier. + +- Patch 1433928: Added a new type, defaultdict, to the collections module. + This uses the new __missing__ hook behavior added to dict (see above). + +- Bug #854823: socketmodule now builds on Sun platforms even when + INET_ADDRSTRLEN is not defined. + +- Patch #1393157: os.startfile() now has an optional argument to specify + a "command verb" to invoke on the file. + +- Bug #876637, prevent stack corruption when socket descriptor + is larger than FD_SETSIZE. + +- Patch #1407135, bug #1424041: harmonize mmap behavior of anonymous memory. + mmap.mmap(-1, size) now returns anonymous memory in both Unix and Windows. + mmap.mmap(0, size) should not be used on Windows for anonymous memory. + +- Patch #1422385: The nis module now supports access to domains other + than the system default domain. + +- Use Win32 API to implement os.stat/fstat. As a result, subsecond timestamps + are reported, the limit on path name lengths is removed, and stat reports + WindowsError now (instead of OSError). + +- Add bsddb.db.DBEnv.set_tx_timestamp allowing time based database recovery. + +- Bug #1413192, fix seg fault in bsddb if a transaction was deleted + before the env. + +- Patch #1103116: Basic AF_NETLINK support. + +- Bug #1402308, (possible) segfault when using mmap.mmap(-1, ...) + +- Bug #1400822, _curses over{lay,write} doesn't work when passing 6 ints. + Also fix ungetmouse() which did not accept arguments properly. + The code now conforms to the documented signature. + +- Bug #1400115, Fix segfault when calling curses.panel.userptr() + without prior setting of the userptr. + +- Fix 64-bit problems in bsddb. + +- Patch #1365916: fix some unsafe 64-bit mmap methods. + +- Bug #1290333: Added a workaround for cjkcodecs' _codecs_cn build + problem on AIX. + +- Bug #869197: os.setgroups rejects long integer arguments + +- Bug #1346533, select.poll() doesn't raise an error if timeout > sys.maxint + +- Bug #1344508, Fix UNIX mmap leaking file descriptors + +- Patch #1338314, Bug #1336623: fix tarfile so it can extract + REGTYPE directories from tarfiles written by old programs. + +- Patch #1407992, fixes broken bsddb module db associate when using + BerkeleyDB 3.3, 4.0 or 4.1. + +- Get bsddb module to build with BerkeleyDB version 4.4 + +- Get bsddb module to build with BerkeleyDB version 3.2 + +- Patch #1309009, Fix segfault in pyexpat when the XML document is in latin_1, + but Python incorrectly assumes it is in UTF-8 format + +- Fix parse errors in the readline module when compiling without threads. + +- Patch #1288833: Removed thread lock from socket.getaddrinfo on + FreeBSD 5.3 and later versions which got thread-safe getaddrinfo(3). + +- Patches #1298449 and #1298499: Add some missing checks for error + returns in cStringIO.c. + +- Patch #1297028: fix segfault if call type on MultibyteCodec, + MultibyteStreamReader, or MultibyteStreamWriter + +- Fix memory leak in posix.access(). + +- Patch #1213831: Fix typo in unicodedata._getcode. + +- Bug #1007046: os.startfile() did not accept unicode strings encoded in + the file system encoding. + +- Patch #756021: Special-case socket.inet_aton('255.255.255.255') for + platforms that don't have inet_aton(). + +- Bug #1215928: Fix bz2.BZ2File.seek() for 64-bit file offsets. + +- Bug #1191043: Fix bz2.BZ2File.(x)readlines for files containing one + line without newlines. + +- Bug #728515: mmap.resize() now resizes the file on Unix as it did + on Windows. + +- Patch #1180695: Add nanosecond stat resolution, and st_gen, + st_birthtime for FreeBSD. + +- Patch #1231069: The fcntl.ioctl function now uses the 'I' code for + the request code argument, which results in more C-like behaviour + for large or negative values. + +- Bug #1234979: For the argument of thread.Lock.acquire, the Windows + implementation treated all integer values except 1 as false. + +- Bug #1194181: bz2.BZ2File didn't handle mode 'U' correctly. + +- Patch #1212117: os.stat().st_flags is now accessible as a attribute + if available on the platform. + +- Patch #1103951: Expose O_SHLOCK and O_EXLOCK in the posix module if + available on the platform. + +- Bug #1166660: The readline module could segfault if hook functions + were set in a different thread than that which called readline. + +- collections.deque objects now support a remove() method. + +- operator.itemgetter() and operator.attrgetter() now support retrieving + multiple fields. This provides direct support for sorting on multiple + keys (primary, secondary, etc). + +- os.access now supports Unicode path names on non-Win32 systems. + +- Patches #925152, #1118602: Avoid reading after the end of the buffer + in pyexpat.GetInputContext. + +- Patches #749830, #1144555: allow UNIX mmap size to default to current + file size. + +- Added functional.partial(). See PEP309. + +- Patch #1093585: raise a ValueError for negative history items in readline. + {remove_history,replace_history} + +- The spwd module has been added, allowing access to the shadow password + database. + +- stat_float_times is now True. + +- array.array objects are now picklable. + +- the cPickle module no longer accepts the deprecated None option in the + args tuple returned by __reduce__(). + +- itertools.islice() now accepts None for the start and step arguments. + This allows islice() to work more readily with slices: + islice(s.start, s.stop, s.step) + +- datetime.datetime() now has a strptime class method which can be used to + create datetime object using a string and format. + +- Patch #1117961: Replace the MD5 implementation from RSA Data Security Inc + with the implementation from http://sourceforge.net/projects/libmd5-rfc/. + +Library +------- + +- Patch #1388073: Numerous __-prefixed attributes of unittest.TestCase have + been renamed to have only a single underscore prefix. This was done to + make subclassing easier. + +- PEP 338: new module runpy defines a run_module function to support + executing modules which provide access to source code or a code object + via the PEP 302 import mechanisms. + +- The email module's parsedate_tz function now sets the daylight savings + flag to -1 (unknown) since it can't tell from the date whether it should + be set. + +- Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results + now sport attributes that provide access to the parts of the result. + +- Patch #1462498: sgmllib now handles entity and character references + in attribute values. + +- Added the sqlite3 package. This is based on pysqlite2.1.3, and provides + a DB-API interface in the standard library. You'll need sqlite 3.0.8 or + later to build this - if you have an earlier version, the C extension + module will not be built. + +- Bug #1460340: ``random.sample(dict)`` failed in various ways. Dicts + aren't officially supported here, and trying to use them will probably + raise an exception some day. But dicts have been allowed, and "mostly + worked", so support for them won't go away without warning. + +- Bug #1445068: getpass.getpass() can now be given an explicit stream + argument to specify where to write the prompt. + +- Patch #1462313, bug #1443328: the pickle modules now can handle classes + that have __private names in their __slots__. + +- Bug #1250170: mimetools now handles socket.gethostname() failures gracefully. + +- patch #1457316: "setup.py upload" now supports --identity to select the + key to be used for signing the uploaded code. + +- Queue.Queue objects now support .task_done() and .join() methods + to make it easier to monitor when daemon threads have completed + processing all enqueued tasks. Patch #1455676. + +- popen2.Popen objects now preserve the command in a .cmd attribute. + +- Added the ctypes ffi package. + +- email 4.0 package now integrated. This is largely the same as the email 3.0 + package that was included in Python 2.3, except that PEP 8 module names are + now used (e.g. mail.message instead of email.Message). The MIME classes + have been moved to a subpackage (e.g. email.mime.text instead of + email.MIMEText). The old names are still supported for now. Several + deprecated Message methods have been removed and lots of bugs have been + fixed. More details can be found in the email package documentation. + +- Patches #1436130/#1443155: codecs.lookup() now returns a CodecInfo object + (a subclass of tuple) that provides incremental decoders and encoders + (a way to use stateful codecs without the stream API). Python functions + codecs.getincrementaldecoder() and codecs.getincrementalencoder() as well + as C functions PyCodec_IncrementalEncoder() and PyCodec_IncrementalDecoder() + have been added. + +- Patch #1359365: Calling next() on a closed StringIO.String object raises + a ValueError instead of a StopIteration now (like file and cString.String do). + cStringIO.StringIO.isatty() will raise a ValueError now if close() has been + called before (like file and StringIO.StringIO do). + +- A regrtest option -w was added to re-run failed tests in verbose mode. + +- Patch #1446372: quit and exit can now be called from the interactive + interpreter to exit. + +- The function get_count() has been added to the gc module, and gc.collect() + grew an optional 'generation' argument. + +- A library msilib to generate Windows Installer files, and a distutils + command bdist_msi have been added. + +- PEP 343: new module contextlib.py defines decorator @contextmanager + and helpful context managers nested() and closing(). + +- The compiler package now supports future imports after the module docstring. + +- Bug #1413790: zipfile now sanitizes absolute archive names that are + not allowed by the specs. + +- Patch #1215184: FileInput now can be given an opening hook which can + be used to control how files are opened. + +- Patch #1212287: fileinput.input() now has a mode parameter for + specifying the file mode input files should be opened with. + +- Patch #1215184: fileinput now has a fileno() function for getting the + current file number. + +- Patch #1349274: gettext.install() now optionally installs additional + translation functions other than _() in the builtin namespace. + +- Patch #1337756: fileinput now accepts Unicode filenames. + +- Patch #1373643: The chunk module can now read chunks larger than + two gigabytes. + +- Patch #1417555: SimpleHTTPServer now returns Last-Modified headers. + +- Bug #1430298: It is now possible to send a mail with an empty + return address using smtplib. + +- Bug #1432260: The names of lambda functions are now properly displayed + in pydoc. + +- Patch #1412872: zipfile now sets the creator system to 3 (Unix) + unless the system is Win32. + +- Patch #1349118: urllib now supports user:pass@ style proxy + specifications, raises IOErrors when proxies for unsupported protocols + are defined, and uses the https proxy on https redirections. + +- Bug #902075: urllib2 now supports 'host:port' style proxy specifications. + +- Bug #1407902: Add support for sftp:// URIs to urlparse. + +- Bug #1371247: Update Windows locale identifiers in locale.py. + +- Bug #1394565: SimpleHTTPServer now doesn't choke on query parameters + any more. + +- Bug #1403410: The warnings module now doesn't get confused + when it can't find out the module name it generates a warning for. + +- Patch #1177307: Added a new codec utf_8_sig for UTF-8 with a BOM signature. + +- Patch #1157027: cookielib mishandles RFC 2109 cookies in Netscape mode + +- Patch #1117398: cookielib.LWPCookieJar and .MozillaCookieJar now raise + LoadError as documented, instead of IOError. For compatibility, + LoadError subclasses IOError. + +- Added the hashlib module. It provides secure hash functions for MD5 and + SHA1, 224, 256, 384, and 512. Note that recent developments make the + historic MD5 and SHA1 unsuitable for cryptographic-strength applications. + In <http://mail.python.org/pipermail/python-dev/2005-December/058850.html> + Ronald L. Rivest offered this advice for Python: + + "The consensus of researchers in this area (at least as + expressed at the NIST Hash Function Workshop 10/31/05), + is that SHA-256 is a good choice for the time being, but + that research should continue, and other alternatives may + arise from this research. The larger SHA's also seem OK." + +- Added a subset of Fredrik Lundh's ElementTree package. Available + modules are xml.etree.ElementTree, xml.etree.ElementPath, and + xml.etree.ElementInclude, from ElementTree 1.2.6. + +- Patch #1162825: Support non-ASCII characters in IDLE window titles. + +- Bug #1365984: urllib now opens "data:" URLs again. + +- Patch #1314396: prevent deadlock for threading.Thread.join() when an exception + is raised within the method itself on a previous call (e.g., passing in an + illegal argument) + +- Bug #1340337: change time.strptime() to always return ValueError when there + is an error in the format string. + +- Patch #754022: Greatly enhanced webbrowser.py (by Oleg Broytmann). + +- Bug #729103: pydoc.py: Fix docother() method to accept additional + "parent" argument. + +- Patch #1300515: xdrlib.py: Fix pack_fstring() to really use null bytes + for padding. + +- Bug #1296004: httplib.py: Limit maximal amount of data read from the + socket to avoid a MemoryError on Windows. + +- Patch #1166948: locale.py: Prefer LC_ALL, LC_CTYPE and LANG over LANGUAGE + to get the correct encoding. + +- Patch #1166938: locale.py: Parse LANGUAGE as a colon separated list of + languages. + +- Patch #1268314: Cache lines in StreamReader.readlines for performance. + +- Bug #1290505: Fix clearing the regex cache for time.strptime(). + +- Bug #1167128: Fix size of a symlink in a tarfile to be 0. + +- Patch #810023: Fix off-by-one bug in urllib.urlretrieve reporthook + functionality. + +- Bug #1163178: Make IDNA return an empty string when the input is empty. + +- Patch #848017: Make Cookie more RFC-compliant. Use CRLF as default output + separator and do not output trailing semicolon. + +- Patch #1062060: urllib.urlretrieve() now raises a new exception, named + ContentTooShortException, when the actually downloaded size does not + match the Content-Length header. + +- Bug #1121494: distutils.dir_utils.mkpath now accepts Unicode strings. + +- Bug #1178484: Return complete lines from codec stream readers + even if there is an exception in later lines, resulting in + correct line numbers for decoding errors in source code. + +- Bug #1192315: Disallow negative arguments to clear() in pdb. + +- Patch #827386: Support absolute source paths in msvccompiler.py. + +- Patch #1105730: Apply the new implementation of commonprefix in posixpath + to ntpath, macpath, os2emxpath and riscospath. + +- Fix a problem in Tkinter introduced by SF patch #869468: delete bogus + __hasattr__ and __delattr__ methods on class Tk that were breaking + Tkdnd. + +- Bug #1015140: disambiguated the term "article id" in nntplib docs and + docstrings to either "article number" or "message id". + +- Bug #1238170: threading.Thread.__init__ no longer has "kwargs={}" as a + parameter, but uses the usual "kwargs=None". + +- textwrap now processes text chunks at O(n) speed instead of O(n**2). + Patch #1209527 (Contributed by Connelly). + +- urllib2 has now an attribute 'httpresponses' mapping from HTTP status code + to W3C name (404 -> 'Not Found'). RFE #1216944. + +- Bug #1177468: Don't cache the /dev/urandom file descriptor for os.urandom, + as this can cause problems with apps closing all file descriptors. + +- Bug #839151: Fix an attempt to access sys.argv in the warnings module; + it can be missing in embedded interpreters + +- Bug #1155638: Fix a bug which affected HTTP 0.9 responses in httplib. + +- Bug #1100201: Cross-site scripting was possible on BaseHTTPServer via + error messages. + +- Bug #1108948: Cookie.py produced invalid JavaScript code. + +- The tokenize module now detects and reports indentation errors. + Bug #1224621. + +- The tokenize module has a new untokenize() function to support a full + roundtrip from lexed tokens back to Python source code. In addition, + the generate_tokens() function now accepts a callable argument that + terminates by raising StopIteration. + +- Bug #1196315: fix weakref.WeakValueDictionary constructor. + +- Bug #1213894: os.path.realpath didn't resolve symlinks that were the first + component of the path. + +- Patch #1120353: The xmlrpclib module provides better, more transparent, + support for datetime.{datetime,date,time} objects. With use_datetime set + to True, applications shouldn't have to fiddle with the DateTime wrapper + class at all. + +- distutils.commands.upload was added to support uploading distribution + files to PyPI. + +- distutils.commands.register now encodes the data as UTF-8 before posting + them to PyPI. + +- decimal operator and comparison methods now return NotImplemented + instead of raising a TypeError when interacting with other types. This + allows other classes to implement __radd__ style methods and have them + work as expected. + +- Bug #1163325: Decimal infinities failed to hash. Attempting to + hash a NaN raised an InvalidOperation instead of a TypeError. + +- Patch #918101: Add tarfile open mode r|* for auto-detection of the + stream compression; add, for symmetry reasons, r:* as a synonym of r. + +- Patch #1043890: Add extractall method to tarfile. + +- Patch #1075887: Don't require MSVC in distutils if there is nothing + to build. + +- Patch #1103407: Properly deal with tarfile iterators when untarring + symbolic links on Windows. + +- Patch #645894: Use getrusage for computing the time consumption in + profile.py if available. + +- Patch #1046831: Use get_python_version where appropriate in sysconfig.py. + +- Patch #1117454: Remove code to special-case cookies without values + in LWPCookieJar. + +- Patch #1117339: Add cookielib special name tests. + +- Patch #1112812: Make bsddb/__init__.py more friendly for modulefinder. + +- Patch #1110248: SYNC_FLUSH the zlib buffer for GZipFile.flush. + +- Patch #1107973: Allow to iterate over the lines of a tarfile.ExFileObject. + +- Patch #1104111: Alter setup.py --help and --help-commands. + +- Patch #1121234: Properly cleanup _exit and tkerror commands. + +- Patch #1049151: xdrlib now unpacks booleans as True or False. + +- Fixed bug in a NameError bug in cookielib. Patch #1116583. + +- Applied a security fix to SimpleXMLRPCserver (PSF-2005-001). This + disables recursive traversal through instance attributes, which can + be exploited in various ways. + +- Bug #1222790: in SimpleXMLRPCServer, set the reuse-address and close-on-exec + flags on the HTTP listening socket. + +- Bug #792570: SimpleXMLRPCServer had problems if the request grew too large. + Fixed by reading the HTTP body in chunks instead of one big socket.read(). + +- Patches #893642, #1039083: add allow_none, encoding arguments to + constructors of SimpleXMLRPCServer and CGIXMLRPCRequestHandler. + +- Bug #1110478: Revert os.environ.update to do putenv again. + +- Bug #1103844: fix distutils.install.dump_dirs() with negated options. + +- os.{SEEK_SET, SEEK_CUR, SEEK_END} have been added for convenience. + +- Enhancements to the csv module: + + + Dialects are now validated by the underlying C code, better + reflecting its capabilities, and improving its compliance with + PEP 305. + + Dialect parameter parsing has been re-implemented to improve error + reporting. + + quotechar=None and quoting=QUOTE_NONE now work the way PEP 305 + dictates. + + the parser now removes the escapechar prefix from escaped characters. + + when quoting=QUOTE_NONNUMERIC, the writer now tests for numeric + types, rather than any object that can be represented as a numeric. + + when quoting=QUOTE_NONNUMERIC, the reader now casts unquoted fields + to floats. + + reader now allows \r characters to be quoted (previously it only allowed + \n to be quoted). + + writer doublequote handling improved. + + Dialect classes passed to the module are no longer instantiated by + the module before being parsed (the former validation scheme required + this, but the mechanism was unreliable). + + The dialect registry now contains instances of the internal + C-coded dialect type, rather than references to python objects. + + the internal c-coded dialect type is now immutable. + + register_dialect now accepts the same keyword dialect specifications + as the reader and writer, allowing the user to register dialects + without first creating a dialect class. + + a configurable limit to the size of parsed fields has been added - + previously, an unmatched quote character could result in the entire + file being read into the field buffer before an error was reported. + + A new module method csv.field_size_limit() has been added that sets + the parser field size limit (returning the former limit). The initial + limit is 128kB. + + A line_num attribute has been added to the reader object, which tracks + the number of lines read from the source iterator. This is not + the same as the number of records returned, as records can span + multiple lines. + + reader and writer objects were not being registered with the cyclic-GC. + This has been fixed. + +- _DummyThread objects in the threading module now delete self.__block that is + inherited from _Thread since it uses up a lock allocated by 'thread'. The + lock primitives tend to be limited in number and thus should not be wasted on + a _DummyThread object. Fixes bug #1089632. + +- The imghdr module now detects Exif files. + +- StringIO.truncate() now correctly adjusts the size attribute. + (Bug #951915). + +- locale.py now uses an updated locale alias table (built using + Tools/i18n/makelocalealias.py, a tool to parse the X11 locale + alias file); the encoding lookup was enhanced to use Python's + encoding alias table. + +- moved deprecated modules to Lib/lib-old: whrandom, tzparse, statcache. + +- the pickle module no longer accepts the deprecated None option in the + args tuple returned by __reduce__(). + +- optparse now optionally imports gettext. This allows its use in setup.py. + +- the pickle module no longer uses the deprecated bin parameter. + +- the shelve module no longer uses the deprecated binary parameter. + +- the pstats module no longer uses the deprecated ignore() method. + +- the filecmp module no longer uses the deprecated use_statcache argument. + +- unittest.TestCase.run() and unittest.TestSuite.run() can now be successfully + extended or overridden by subclasses. Formerly, the subclassed method would + be ignored by the rest of the module. (Bug #1078905). + +- heapq.nsmallest() and heapq.nlargest() now support key= arguments with + the same meaning as in list.sort(). + +- Bug #1076985: ``codecs.StreamReader.readline()`` now calls ``read()`` only + once when a size argument is given. This prevents a buffer overflow in the + tokenizer with very long source lines. + +- Bug #1083110: ``zlib.decompress.flush()`` would segfault if called + immediately after creating the object, without any intervening + ``.decompress()`` calls. + +- The reconvert.quote function can now emit triple-quoted strings. The + reconvert module now has some simple documentation. + +- ``UserString.MutableString`` now supports negative indices in + ``__setitem__`` and ``__delitem__`` + +- Bug #1149508: ``textwrap`` now handles hyphenated numbers (eg. "2004-03-05") + correctly. + +- Partial fixes for SF bugs #1163244 and #1175396: If a chunk read by + ``codecs.StreamReader.readline()`` has a trailing "\r", read one more + character even if the user has passed a size parameter to get a proper + line ending. Remove the special handling of a "\r\n" that has been split + between two lines. + +- Bug #1251300: On UCS-4 builds the "unicode-internal" codec will now complain + about illegal code points. The codec now supports PEP 293 style error + handlers. + +- Bug #1235646: ``codecs.StreamRecoder.next()`` now reencodes the data it reads + from the input stream, so that the output is a byte string in the correct + encoding instead of a unicode string. + +- Bug #1202493: Fixing SRE parser to handle '{}' as perl does, rather than + considering it exactly like a '*'. + +- Bug #1245379: Add "unicode-1-1-utf-7" as an alias for "utf-7" to + ``encodings.aliases``. + +- ` uu.encode()`` and ``uu.decode()`` now support unicode filenames. + +- Patch #1413711: Certain patterns of differences were making difflib + touch the recursion limit. + +- Bug #947906: An object oriented interface has been added to the calendar + module. It's possible to generate HTML calendar now and the module can be + called as a script (e.g. via ``python -mcalendar``). Localized month and + weekday names can be ouput (even if an exotic encoding is used) using + special classes that use unicode. + +Build +----- + +- Fix test_float, test_long, and test_struct failures on Tru64 with gcc + by using -mieee gcc option. + +- Patch #1432345: Make python compile on DragonFly. + +- Build support for Win64-AMD64 was added. + +- Patch #1428494: Prefer linking against ncursesw over ncurses library. + +- Patch #881820: look for openpty and forkpty also in libbsd. + +- The sources of zlib are now part of the Python distribution (zlib 1.2.3). + The zlib module is now builtin on Windows. + +- Use -xcode=pic32 for CCSHARED on Solaris with SunPro. + +- Bug #1189330: configure did not correctly determine the necessary + value of LINKCC if python was built with GCC 4.0. + +- Upgrade Windows build to zlib 1.2.3 which eliminates a potential security + vulnerability in zlib 1.2.1 and 1.2.2. + +- EXTRA_CFLAGS has been introduced as an environment variable to hold compiler + flags that change binary compatibility. Changes were also made to + distutils.sysconfig to also use the environment variable when used during + compilation of the interpreter and of C extensions through distutils. + +- SF patch 1171735: Darwin 8's headers are anal about POSIX compliance, + and linking has changed (prebinding is now deprecated, and libcc_dynamic + no longer exists). This configure patch makes things right. + +- Bug #1158607: Build with --disable-unicode again. + +- spwdmodule.c is built only if either HAVE_GETSPNAM or HAVE_HAVE_GETSPENT is + defined. Discovered as a result of not being able to build on OS X. + +- setup.py now uses the directories specified in LDFLAGS using the -L option + and in CPPFLAGS using the -I option for adding library and include + directories, respectively, for compiling extension modules against. This has + led to the core being compiled using the values in CPPFLAGS. It also removes + the need for the special-casing of both DarwinPorts and Fink for darwin since + the proper directories can be specified in LDFLAGS (``-L/sw/lib`` for Fink, + ``-L/opt/local/lib`` for DarwinPorts) and CPPFLAGS (``-I/sw/include`` for + Fink, ``-I/opt/local/include`` for DarwinPorts). + +- Test in configure.in that checks for tzset no longer dependent on tm->tm_zone + to exist in the struct (not required by either ISO C nor the UNIX 2 spec). + Tests for sanity in tzname when HAVE_TZNAME defined were also defined. + Closes bug #1096244. Thanks Gregory Bond. + +C API +----- + +- ``PyMem_{Del, DEL}`` and ``PyMem_{Free, FREE}`` no longer map to + ``PyObject_{Free, FREE}``. They map to the system ``free()`` now. If memory + is obtained via the ``PyObject_`` family, it must be released via the + ``PyObject_`` family, and likewise for the ``PyMem_`` family. This has + always been officially true, but when Python's small-object allocator was + introduced, an attempt was made to cater to a few extension modules + discovered at the time that obtained memory via ``PyObject_New`` but + released it via ``PyMem_DEL``. It's years later, and if such code still + exists it will fail now (probably with segfaults, but calling wrong + low-level memory management functions can yield many symptoms). + +- Added a C API for set and frozenset objects. + +- Removed PyRange_New(). + +- Patch #1313939: PyUnicode_DecodeCharmap() accepts a unicode string as the + mapping argument now. This string is used as a mapping table. Byte values + greater than the length of the string and 0xFFFE are treated as undefined + mappings. + + +Tests +----- + +- In test_os, st_?time is now truncated before comparing it with ST_?TIME. + +- Patch #1276356: New resource "urlfetch" is implemented. This enables + even impatient people to run tests that require remote files. + + +Documentation +------------- + +- Bug #1402224: Add warning to dl docs about crashes. + +- Bug #1396471: Document that Windows' ftell() can return invalid + values for text files with UNIX-style line endings. + +- Bug #1274828: Document os.path.splitunc(). + +- Bug #1190204: Clarify which directories are searched by site.py. + +- Bug #1193849: Clarify os.path.expanduser() documentation. + +- Bug #1243192: re.UNICODE and re.LOCALE affect \d, \D, \s and \S. + +- Bug #755617: Document the effects of os.chown() on Windows. + +- Patch #1180012: The documentation for modulefinder is now in the library reference. + +- Patch #1213031: Document that os.chown() accepts argument values of -1. + +- Bug #1190563: Document os.waitpid() return value with WNOHANG flag. + +- Bug #1175022: Correct the example code for property(). + +- Document the IterableUserDict class in the UserDict module. + Closes bug #1166582. + +- Remove all latent references for "Macintosh" that referred to semantics for + Mac OS 9 and change to reflect the state for OS X. + Closes patch #1095802. Thanks Jack Jansen. + +Mac +--- + + +New platforms +------------- + +- FreeBSD 7 support is added. + + +Tools/Demos +----------- + +- Created Misc/Vim/vim_syntax.py to auto-generate a python.vim file in that + directory for syntax highlighting in Vim. Vim directory was added and placed + vimrc to it (was previous up a level). + +- Added two new files to Tools/scripts: pysource.py, which recursively + finds Python source files, and findnocoding.py, which finds Python + source files that need an encoding declaration. + Patch #784089, credits to Oleg Broytmann. + +- Bug #1072853: pindent.py used an uninitialized variable. + +- Patch #1177597: Correct Complex.__init__. + +- Fixed a display glitch in Pynche, which could cause the right arrow to + wiggle over by a pixel. + +---- + +**(For information about older versions, consult the HISTORY file.)** diff --git a/sys/src/cmd/python/Misc/NEWS.help b/sys/src/cmd/python/Misc/NEWS.help new file mode 100644 index 000000000..856785fa7 --- /dev/null +++ b/sys/src/cmd/python/Misc/NEWS.help @@ -0,0 +1,73 @@ + -*- text -*- + +If you edited Misc/NEWS before it was converted to ReST format skimming this +file should help make the transition a bit easier. For full details about +Docutils and ReST, go to the Docutils website: + + http://docutils.sourceforge.net/ + +To process Misc/NEWS using Docutils, you'll need the latest docutils +snapshot: + + http://docutils.sf.net/docutils-snapshot.tgz + +Docutils works with Python 2.2 or newer. + +To process NEWS into NEWS.html, first install Docutils, and then run +this command: + + python .../docutils/tools/rst2html.py NEWS NEWS.html + +Here ".../docutils" is the directory into which the above snapshot was +extracted. (I hope this recipe will change for the better.) + +David Goodger made a change to the allowable structure of internal +references which greatly simplified initial conversion of the file. + +The changes required fell into the following categories: + +* The top-level "What's New" section headers changed to: + + What's New in Python 2.3 alpha 1? + ================================= + + *Release date: DD-MMM-2002* + + Note that the release date line is emphasized, with a "*" at each + end. + +* Subsections are underlined with a single row of hyphens: + + Type/class unification and new-style classes + -------------------------------------------- + +* Places where "balanced" single quotes were used were changed to use + apostrophes as both the opening and closing quote (`string' -> 'string'). + +* In a few places asterisks needed to be escaped which would otherwise have + been interpreted as beginning blocks of italic or bold text, e.g.: + + - The type of tp_free has been changed from "``void (*)(PyObject *)``" + to "``void (*)(void *)``". + + Note that only the asterisks preceded by whitespace needed to be escaped. + +* One instance of a word ending with an underscore needed to be quoted + ("PyCmp_" became "``PyCmp_``"). + +* One table was converted to ReST form (search Misc/NEWS for "New codecs" + for this example). + +* A few places where chunks of code or indented text were displayed needed + to be properly introduced (preceding paragraph terminated by "::" and the + chunk of code or text indented w.r.t. the paragraph). For example: + + - Note that PyLong_AsDouble can fail! This has always been true, + but no callers checked for it. It's more likely to fail now, + because overflow errors are properly detected now. The proper way + to check:: + + double x = PyLong_AsDouble(some_long_object); + if (x == -1.0 && PyErr_Occurred()) { + /* The conversion failed. */ + } diff --git a/sys/src/cmd/python/Misc/PURIFY.README b/sys/src/cmd/python/Misc/PURIFY.README new file mode 100644 index 000000000..1e5d2ac19 --- /dev/null +++ b/sys/src/cmd/python/Misc/PURIFY.README @@ -0,0 +1,97 @@ +Purify (tm) and Quantify (tm) are commercial software quality +assurance tools available from IBM <http://www.ibm.com/software/rational/>. +Purify is essentially a memory access +verifier and leak detector; Quantify is a C level profiler. The rest +of this file assumes you generally know how to use Purify and +Quantify, and that you have installed valid licenses for these +products. If you haven't installed such licenses, you can ignore the +following since it won't help you a bit! + +You can easily build a Purify or Quantify instrumented version of the +Python interpreter by passing the PURIFY variable to the make command +at the top of the Python tree: + + make PURIFY=purify + +This assumes that the `purify' program is on your $PATH. Note that +you cannot both Purify and Quantify the Python interpreter (or any +program for that matter) at the same time. If you want to build a +Quantify'd interpreter, do this: + + make PURIFY=quantify + +Starting with Python 2.3, pymalloc is enabled by default. This +will cause many supurious warnings. Modify Objects/obmalloc.c +and enable Py_USING_MEMORY_DEBUGGER by uncommenting it. +README.valgrind has more details about why this is necessary. +See below about setting up suppressions. Some tests may not +run well with Purify due to heavy memory or CPU usage. These +tests may include: test_largefile, test_import, and test_long. + +Please report any findings (problems or no warnings) to python-dev@python.org. +It may be useful to submit a bug report for any problems. + +When running the regression test (make test), I have found it useful +to set my PURIFYOPTIONS environment variable using the following +(bash) shell function. Check out the Purify documentation for +details: + +p() { + chainlen='-chain-length=12' + ignoresigs='-ignore-signals="SIGHUP,SIGINT,SIGQUIT,SIGILL,SIGTRAP,SIGAVRT,SIGEMT,SIGFPE,SIGKILL,SIGBUS,SIGSEGV,SIGPIPE,SIGTERM,SIGUSR1,SIGUSR2,SIGPOLL,SIGXCPU,SIGXFSZ,SIGFREEZE,SIGTHAW,SIGRTMIN,SIGRTMAX"' + followchild='-follow-child-processes=yes' + threads='-max-threads=50' + export PURIFYOPTIONS="$chainlen $ignoresigs $followchild $threads" + echo $PURIFYOPTIONS +} + +Note that you may want to crank -chain-length up even further. A +value of 20 should get you the entire stack up into the Python C code +in all situations. + +With the regression test on a fatly configured interpreter +(i.e. including as many modules as possible in your Modules/Setup +file), you'll probably get a gabillion UMR errors, and a few MLK +errors. I think most of these can be safely suppressed by putting the +following in your .purify file: + + suppress umr ...; "socketmodule.c" + suppress umr ...; time_strftime + suppress umr ...; "dbmmodule.c" + suppress umr ...; "gdbmmodule.c" + suppress umr ...; "grpmodule.c" + suppress umr ...; "nismodule.c" + suppress umr ...; "pwdmodule.c" + +Note: this list is very old and may not be accurate any longer. +It's possible some of these no longer need to be suppressed. +You will also need to suppress warnings (at least umr) +from Py_ADDRESS_IN_RANGE. + +This will still leave you with just a few UMR, mostly in the readline +library, which you can safely ignore. A lot of work has gone into +Python 1.5 to plug as many leaks as possible. + +Using Purify or Quantify in this way will give you coarse grained +reports on the whole Python interpreter. You can actually get more +fine grained control over both by linking with the optional `pure' +module, which exports (most of) the Purify and Quantify C API's into +Python. To link in this module (it must be statically linked), edit +your Modules/Setup file for your site, and rebuild the interpreter. +You might want to check out the comments in the Modules/puremodule.c +file for some idiosyncrasies. + +Using this module, you can actually profile or leak test a small +section of code, instead of the whole interpreter. Using this in +conjuction with pdb.py, dbx, or the profiler.py module really gives +you quite a bit of introspective power. + +Naturally there are a couple of caveats. This has only been tested +with Purify 4.0.1 and Quantify 2.1-beta on Solaris 2.5. Purify 4.0.1 +does not work with Solaris 2.6, but Purify 4.1 which reportedly will, +is currently in beta test. There are funky problems when Purify'ing a +Python interpreter build with threads. I've had a lot of problems +getting this to work, so I generally don't build with threads when I'm +Purify'ing. If you get this to work, let us know! + +-Barry Warsaw <bwarsaw@cnri.reston.va.us> diff --git a/sys/src/cmd/python/Misc/Porting b/sys/src/cmd/python/Misc/Porting new file mode 100644 index 000000000..60ce9a824 --- /dev/null +++ b/sys/src/cmd/python/Misc/Porting @@ -0,0 +1,42 @@ +Q. I want to port Python to a new platform. How do I begin? + +A. I guess the two things to start with is to familiarize yourself +with are the development system for your target platform and the +generic build process for Python. Make sure you can compile and run a +simple hello-world program on your target platform. Make sure you can +compile and run the Python interpreter on a platform to which it has +already been ported (preferably Unix, but Mac or Windows will do, +too). + +I also would never start something like this without at least +medium-level understanding of your target platform (i.e. how it is +generally used, how to write platform specific apps etc.) and Python +(or else you'll never know how to test the results). + +The build process for Python, in particular the Makefiles in the +source distribution, will give you a hint on which files to compile +for Python. Not all source files are relevant -- some are platform +specific, others are only used in emergencies (e.g. getopt.c). The +Makefiles tell the story. + +You'll also need a pyconfig.h file tailored for your platform. You can +start with pyconfig.h.in, read the comments and turn on definitions that +apply to your platform. + +And you'll need a config.c file, which lists the built-in modules you +support. Start with Modules/config.c.in. + +Finally, you'll run into some things that aren't supported on your +target platform. Forget about the posix module for now -- simply take +it out of the config.c file. + +Bang on it until you get a >>> prompt. (You may have to disable the +importing of "site.py" and "exceptions.py" by passing -X and -S +options. + +Then bang on it until it executes very simple Python statements. + +Now bang on it some more. At some point you'll want to use the os +module; this is the time to start thinking about what to to with the +posix module. It's okay to simply #ifdef out those functions that +cause problems; the remaining ones will be quite useful. diff --git a/sys/src/cmd/python/Misc/README b/sys/src/cmd/python/Misc/README new file mode 100644 index 000000000..af6e8e8f4 --- /dev/null +++ b/sys/src/cmd/python/Misc/README @@ -0,0 +1,33 @@ +Python Misc subdirectory +======================== + +This directory contains files that wouldn't fit in elsewhere. Some +documents are only of historic importance. + +Files found here +---------------- + +ACKS Acknowledgements +AIX-NOTES Notes for building Python on AIX +BeOS-NOTES Notes for building on BeOS +BeOS-setup.py setup.py replacement for BeOS, see BeOS-NOTES +cheatsheet Quick summary of Python by Ken Manheimer +find_recursionlimit.py Script to find a value for sys.maxrecursionlimit +gdbinit Handy stuff to put in your .gdbinit file, if you use gdb +HISTORY News from previous releases -- oldest last +HPUX-NOTES Notes about dynamic loading under HP-UX +indent.pro GNU indent profile approximating my C style +NEWS News for this release (for some meaning of "this") +Porting Mini-FAQ on porting to new platforms +PURIFY.README Information for Purify users +pymemcompat.h Memory interface compatibility file. +python.man UNIX man page for the python interpreter +python-mode.el Emacs mode for editing Python programs +README The file you're reading now +README.valgrind Information for Valgrind users, see valgrind-python.supp +RFD Request For Discussion about a Python newsgroup +RPM (Old) tools to build RPMs +SpecialBuilds.txt Describes extra symbols you can set for debug builds +setuid-prog.c C helper program for set-uid Python scripts +vgrindefs Python configuration for vgrind (a generic pretty printer) +valgrind-python.supp Valgrind suppression file, see README.valgrind diff --git a/sys/src/cmd/python/Misc/README.OpenBSD b/sys/src/cmd/python/Misc/README.OpenBSD new file mode 100644 index 000000000..b417ecc76 --- /dev/null +++ b/sys/src/cmd/python/Misc/README.OpenBSD @@ -0,0 +1,38 @@ + +2005-01-08 + +If you are have a problem building on OpenBSD and see output like this +while running configure: + +checking curses.h presence... yes +configure: WARNING: curses.h: present but cannot be compiled +configure: WARNING: curses.h: check for missing prerequisite headers? +configure: WARNING: curses.h: see the Autoconf documentation +configure: WARNING: curses.h: section "Present But Cannot Be Compiled" +configure: WARNING: curses.h: proceeding with the preprocessor's result +configure: WARNING: curses.h: in the future, the compiler will take precedence + +there is likely a problem that will prevent building python. +If you see the messages above and are able to completely build python, +please tell python-dev@python.org indicating your version of OpenBSD +and any other relevant system configuration. + +The build error that occurs while making may look something like this: + + /usr/include/sys/event.h:53: error: syntax error before "u_int" + /usr/include/sys/event.h:55: error: syntax error before "u_short" + +To fix this problem, you will probably need update Python's configure +script to disable certain options. Search for a line that looks like: + + OpenBSD/2.* | OpenBSD/3.@<:@012345678@:>@) + +If your version is not in that list, e.g., 3.9, add the version +number. In this case, you would just need to add a 9 after the 8. +If you modify configure.in, you will need to regenerate configure +with autoconf. + +If your version is already in the list, this is not a known problem. +Please submit a bug report here: + + http://sourceforge.net/tracker/?group_id=5470&atid=105470 diff --git a/sys/src/cmd/python/Misc/README.coverity b/sys/src/cmd/python/Misc/README.coverity new file mode 100644 index 000000000..f5e1bf6f2 --- /dev/null +++ b/sys/src/cmd/python/Misc/README.coverity @@ -0,0 +1,22 @@ + +Coverity has a static analysis tool (Prevent) which is similar to Klocwork. +They run their tool on the Python source code (SVN head) on a daily basis. +The results are available at: + + http://scan.coverity.com/ + +About 20 people have access to the analysis reports. Other +people can be added by request. + +Prevent was first run on the Python 2.5 source code in March 2006. +There were originally about 100 defects reported. Some of these +were false positives. Over 70 issues were uncovered. + +Each warning has a unique id and comments that can be made on it. +When checking in changes due to a warning, the unique id +as reported by the tool was added to the SVN commit message. + +False positives were annotated so that the comments can +be reviewed and reversed if the analysis was incorrect. + +Contact python-dev@python.org for more information. diff --git a/sys/src/cmd/python/Misc/README.klocwork b/sys/src/cmd/python/Misc/README.klocwork new file mode 100644 index 000000000..6d2f57fc7 --- /dev/null +++ b/sys/src/cmd/python/Misc/README.klocwork @@ -0,0 +1,30 @@ + +Klocwork has a static analysis tool (K7) which is similar to Coverity. +They will run their tool on the Python source code on demand. +The results are available at: + + https://opensource.klocwork.com/ + +Currently, only Neal Norwitz has access to the analysis reports. Other +people can be added by request. + +K7 was first run on the Python 2.5 source code in mid-July 2006. +This is after Coverity had been making their results available. +There were originally 175 defects reported. Most of these +were false positives. However, there were numerous real issues +also uncovered. + +Each warning has a unique id and comments that can be made on it. +When checking in changes due to a K7 report, the unique id +as reported by the tool was added to the SVN commit message. +A comment was added to the K7 warning indicating the SVN revision +in addition to any analysis. + +False positives were also annotated so that the comments can +be reviewed and reversed if the analysis was incorrect. + +A second run was performed on 10-Aug-2006. The tool was tuned to remove +some false positives and perform some additional checks. ~150 new +warnings were produced, primarily related to dereferencing NULL pointers. + +Contact python-dev@python.org for more information. diff --git a/sys/src/cmd/python/Misc/README.valgrind b/sys/src/cmd/python/Misc/README.valgrind new file mode 100644 index 000000000..b5a9a32e1 --- /dev/null +++ b/sys/src/cmd/python/Misc/README.valgrind @@ -0,0 +1,97 @@ +This document describes some caveats about the use of Valgrind with +Python. Valgrind is used periodically by Python developers to try +to ensure there are no memory leaks or invalid memory reads/writes. + +If you don't want to read about the details of using Valgrind, there +are still two things you must do to suppress the warnings. First, +you must use a suppressions file. One is supplied in +Misc/valgrind-python.supp. Second, you must do one of the following: + + * Uncomment Py_USING_MEMORY_DEBUGGER in Objects/obmalloc.c, + then rebuild Python + * Uncomment the lines in Misc/valgrind-python.supp that + suppress the warnings for PyObject_Free and PyObject_Realloc + +If you want to use Valgrind more effectively and catch even more +memory leaks, you will need to configure python --without-pymalloc. +PyMalloc allocates a few blocks in big chunks and most object +allocations don't call malloc, they use chunks doled about by PyMalloc +from the big blocks. This means Valgrind can't detect +many allocations (and frees), except for those that are forwarded +to the system malloc. Note: configuring python --without-pymalloc +makes Python run much slower, especially when running under Valgrind. +You may need to run the tests in batches under Valgrind to keep +the memory usage down to allow the tests to complete. It seems to take +about 5 times longer to run --without-pymalloc. + +Apr 15, 2006: + test_ctypes causes Valgrind 3.1.1 to fail (crash). + test_socket_ssl should be skipped when running valgrind. + The reason is that it purposely uses uninitialized memory. + This causes many spurious warnings, so it's easier to just skip it. + + +Details: +-------- +Python uses its own small-object allocation scheme on top of malloc, +called PyMalloc. + +Valgrind may show some unexpected results when PyMalloc is used. +Starting with Python 2.3, PyMalloc is used by default. You can disable +PyMalloc when configuring python by adding the --without-pymalloc option. +If you disable PyMalloc, most of the information in this document and +the supplied suppressions file will not be useful. As discussed above, +disabling PyMalloc can catch more problems. + +If you use valgrind on a default build of Python, you will see +many errors like: + + ==6399== Use of uninitialised value of size 4 + ==6399== at 0x4A9BDE7E: PyObject_Free (obmalloc.c:711) + ==6399== by 0x4A9B8198: dictresize (dictobject.c:477) + +These are expected and not a problem. Tim Peters explains +the situation: + + PyMalloc needs to know whether an arbitrary address is one + that's managed by it, or is managed by the system malloc. + The current scheme allows this to be determined in constant + time, regardless of how many memory areas are under pymalloc's + control. + + The memory pymalloc manages itself is in one or more "arenas", + each a large contiguous memory area obtained from malloc. + The base address of each arena is saved by pymalloc + in a vector. Each arena is carved into "pools", and a field at + the start of each pool contains the index of that pool's arena's + base address in that vector. + + Given an arbitrary address, pymalloc computes the pool base + address corresponding to it, then looks at "the index" stored + near there. If the index read up is out of bounds for the + vector of arena base addresses pymalloc maintains, then + pymalloc knows for certain that this address is not under + pymalloc's control. Otherwise the index is in bounds, and + pymalloc compares + + the arena base address stored at that index in the vector + + to + + the arbitrary address pymalloc is investigating + + pymalloc controls this arbitrary address if and only if it lies + in the arena the address's pool's index claims it lies in. + + It doesn't matter whether the memory pymalloc reads up ("the + index") is initialized. If it's not initialized, then + whatever trash gets read up will lead pymalloc to conclude + (correctly) that the address isn't controlled by it, either + because the index is out of bounds, or the index is in bounds + but the arena it represents doesn't contain the address. + + This determination has to be made on every call to one of + pymalloc's free/realloc entry points, so its speed is critical + (Python allocates and frees dynamic memory at a ferocious rate + -- everything in Python, from integers to "stack frames", + lives in the heap). diff --git a/sys/src/cmd/python/Misc/RFD b/sys/src/cmd/python/Misc/RFD new file mode 100644 index 000000000..fd278c4fa --- /dev/null +++ b/sys/src/cmd/python/Misc/RFD @@ -0,0 +1,114 @@ +To: python-list +Subject: comp.lang.python RFD again +From: Guido.van.Rossum@cwi.nl + +I've followed the recent discussion and trimmed the blurb RFD down a bit +(and added the word "object-oriented" to the blurb). + +I don't think it's too early to *try* to create the newsgroup -- +whether we will succeed may depend on how many Python supporters there +are outside the mailing list. + +I'm personally not worried about moderation, and anyway I haven't +heard from any volunteers for moderation (and I won't volunteer +myself) so I suggest that we'll continue to ask for one unmoderated +newsgroup. + +My next action will be to post an updated FAQ (which will hint at the +upcoming RFD) to comp.lang.misc; then finalize the 1.0.0 release and +put it on the ftp site. I'll also try to get it into +comp.sources.unix or .misc. And all this before the end of January! + +--Guido van Rossum, CWI, Amsterdam <Guido.van.Rossum@cwi.nl> +URL: <http://www.cwi.nl/cwi/people/Guido.van.Rossum.html> + +====================================================================== + +These are the steps required (in case you don't know about the +newsgroup creation process): + +First, we need to draw up an RFD (Request For Discussion). This is a +document that tells what the purpose of the group is, and gives a case +for its creation. We post this to relevant groups (comp.lang.misc, +the mailing list, news.groups, etc.) Discussion is held on +news.groups. + +Then, after a few weeks, we run the official CFV (Call For Votes). +The votes are then collected over a period of weeks. We need 100 more +yes votes than no votes, and a 2/3 majority, to get the group. + +There are some restrictions on the vote taker: [s]he cannot actively +campaign for/against the group during the vote process. So the main +benefit to Steve instead of me running the vote is that I will be free +to campaign for its creation! + +The following is our current draft for the RFD. + +====================================================================== + +Request For Discussion: comp.lang.python + + +Purpose +------- + +The newsgroup will be for discussion on the Python computer language. +Possible topics include requests for information, general programming, +development, and bug reports. The group will be unmoderated. + + +What is Python? +--------------- + +Python is a relatively new very-high-level language developed in +Amsterdam. Python is a simple, object-oriented procedural language, +with features taken from ABC, Icon, Modula-3, and C/C++. + +Its central goal is to provide the best of both worlds: the dynamic +nature of scripting languages like Perl/TCL/REXX, but also support for +general programming found in the more traditional languages like Icon, +C, Modula,... + +Python may be FTP'd from the following sites: + + ftp.cwi.nl in directory /pub/python (its "home site", also has a FAQ) + ftp.uu.net in directory /languages/python + gatekeeper.dec.com in directory /pub/plan/python/cwi + + +Rationale +--------- + +Currently there is a mailing list with over 130 subscribers. +The activity of this list is high, and to make handling the +traffic more reasonable, a newsgroup is being proposed. We +also feel that comp.lang.misc would not be a suitable forum +for this volume of discussion on a particular language. + + +Charter +------- + +Comp.lang.python is an unmoderated newsgroup which will serve +as a forum for discussing the Python computer language. The +group will serve both those who just program in Python and +those who work on developing the language. Topics that +may be discussed include: + + - announcements of new versions of the language and + applications written in Python. + + - discussion on the internals of the Python language. + + - general information about the language. + + - discussion on programming in Python. + + +Discussion +---------- + +Any objections to this RFD will be considered and, if determined +to be appropriate, will be incorporated. The discussion period +will be for a period of 21 days after which the first CFV will be +issued. diff --git a/sys/src/cmd/python/Misc/RPM/README b/sys/src/cmd/python/Misc/RPM/README new file mode 100644 index 000000000..f3a25575f --- /dev/null +++ b/sys/src/cmd/python/Misc/RPM/README @@ -0,0 +1,16 @@ +This directory contains support file used to build RPM releases of +Python. Its contents are maintained by Sean Reifschneider +<jafo@tummy.com>. + +It is recommended that RPM builders use the python*.src.rpm file +downloaded from the "ftp.python.org:/pub/python/<version>/rpms". These +may be more up to date than the files included in the base Python +release tar-file. + +If you wish to build RPMs from the base Python release tar-file, note +that you will have to download the +"doc/<version>/html-<version>.tar.bz2" +file from python.org and place it into your "SOURCES" directory for +the build to complete. This is the same directory that you place the +Python-2.3.1 release tar-file in. You can then use the ".spec" file in +this directory to build RPMs. diff --git a/sys/src/cmd/python/Misc/RPM/python-2.5.spec b/sys/src/cmd/python/Misc/RPM/python-2.5.spec new file mode 100644 index 000000000..399907463 --- /dev/null +++ b/sys/src/cmd/python/Misc/RPM/python-2.5.spec @@ -0,0 +1,395 @@ +########################## +# User-modifiable configs +########################## + +# Is the resulting package and the installed binary named "python" or +# "python2"? +#WARNING: Commenting out doesn't work. Last line is what's used. +%define config_binsuffix none +%define config_binsuffix 2.5 + +# Build tkinter? "auto" enables it if /usr/bin/wish exists. +#WARNING: Commenting out doesn't work. Last line is what's used. +%define config_tkinter no +%define config_tkinter yes +%define config_tkinter auto + +# Use pymalloc? The last line (commented or not) determines wether +# pymalloc is used. +#WARNING: Commenting out doesn't work. Last line is what's used. +%define config_pymalloc no +%define config_pymalloc yes + +# Enable IPV6? +#WARNING: Commenting out doesn't work. Last line is what's used. +%define config_ipv6 yes +%define config_ipv6 no + +# Location of the HTML directory. +%define config_htmldir /var/www/html/python + +################################# +# End of user-modifiable configs +################################# + +%define name python +%define version 2.5.1 +%define libvers 2.5 +%define release 1pydotorg +%define __prefix /usr + +# kludge to get around rpm <percent>define weirdness +%define ipv6 %(if [ "%{config_ipv6}" = yes ]; then echo --enable-ipv6; else echo --disable-ipv6; fi) +%define pymalloc %(if [ "%{config_pymalloc}" = yes ]; then echo --with-pymalloc; else echo --without-pymalloc; fi) +%define binsuffix %(if [ "%{config_binsuffix}" = none ]; then echo ; else echo "%{config_binsuffix}"; fi) +%define include_tkinter %(if [ \\( "%{config_tkinter}" = auto -a -f /usr/bin/wish \\) -o "%{config_tkinter}" = yes ]; then echo 1; else echo 0; fi) +%define libdirname %(( uname -m | egrep -q '_64$' && [ -d /usr/lib64 ] && echo lib64 ) || echo lib) + +# detect if documentation is available +%define include_docs %(if [ -f "%{_sourcedir}/html-%{version}.tar.bz2" ]; then echo 1; else echo 0; fi) + +Summary: An interpreted, interactive, object-oriented programming language. +Name: %{name}%{binsuffix} +Version: %{version} +Release: %{release} +License: Python Software Foundation +Group: Development/Languages +Source: Python-%{version}.tar.bz2 +%if %{include_docs} +Source1: html-%{version}.tar.bz2 +%endif +BuildRoot: %{_tmppath}/%{name}-%{version}-root +BuildPrereq: expat-devel +BuildPrereq: db4-devel +BuildPrereq: gdbm-devel +BuildPrereq: sqlite-devel +Prefix: %{__prefix} +Packager: Sean Reifschneider <jafo-rpms@tummy.com> + +%description +Python is an interpreted, interactive, object-oriented programming +language. It incorporates modules, exceptions, dynamic typing, very high +level dynamic data types, and classes. Python combines remarkable power +with very clear syntax. It has interfaces to many system calls and +libraries, as well as to various window systems, and is extensible in C or +C++. It is also usable as an extension language for applications that need +a programmable interface. Finally, Python is portable: it runs on many +brands of UNIX, on PCs under Windows, MS-DOS, and OS/2, and on the +Mac. + +%package devel +Summary: The libraries and header files needed for Python extension development. +Prereq: python%{binsuffix} = %{PACKAGE_VERSION} +Group: Development/Libraries + +%description devel +The Python programming language's interpreter can be extended with +dynamically loaded extensions and can be embedded in other programs. +This package contains the header files and libraries needed to do +these types of tasks. + +Install python-devel if you want to develop Python extensions. The +python package will also need to be installed. You'll probably also +want to install the python-docs package, which contains Python +documentation. + +%if %{include_tkinter} +%package tkinter +Summary: A graphical user interface for the Python scripting language. +Group: Development/Languages +Prereq: python%{binsuffix} = %{PACKAGE_VERSION}-%{release} + +%description tkinter +The Tkinter (Tk interface) program is an graphical user interface for +the Python scripting language. + +You should install the tkinter package if you'd like to use a graphical +user interface for Python programming. +%endif + +%package tools +Summary: A collection of development tools included with Python. +Group: Development/Tools +Prereq: python%{binsuffix} = %{PACKAGE_VERSION}-%{release} + +%description tools +The Python package includes several development tools that are used +to build python programs. This package contains a selection of those +tools, including the IDLE Python IDE. + +Install python-tools if you want to use these tools to develop +Python programs. You will also need to install the python and +tkinter packages. + +%if %{include_docs} +%package docs +Summary: Python-related documentation. +Group: Development/Documentation + +%description docs +Documentation relating to the Python programming language in HTML and info +formats. +%endif + +%changelog +* Mon Dec 20 2004 Sean Reifschneider <jafo-rpms@tummy.com> [2.4-2pydotorg] +- Changing the idle wrapper so that it passes arguments to idle. + +* Tue Oct 19 2004 Sean Reifschneider <jafo-rpms@tummy.com> [2.4b1-1pydotorg] +- Updating to 2.4. + +* Thu Jul 22 2004 Sean Reifschneider <jafo-rpms@tummy.com> [2.3.4-3pydotorg] +- Paul Tiemann fixes for %{prefix}. +- Adding permission changes for directory as suggested by reimeika.ca +- Adding code to detect when it should be using lib64. +- Adding a define for the location of /var/www/html for docs. + +* Thu May 27 2004 Sean Reifschneider <jafo-rpms@tummy.com> [2.3.4-2pydotorg] +- Including changes from Ian Holsman to build under Red Hat 7.3. +- Fixing some problems with the /usr/local path change. + +* Sat Mar 27 2004 Sean Reifschneider <jafo-rpms@tummy.com> [2.3.2-3pydotorg] +- Being more agressive about finding the paths to fix for + #!/usr/local/bin/python. + +* Sat Feb 07 2004 Sean Reifschneider <jafo-rpms@tummy.com> [2.3.3-2pydotorg] +- Adding code to remove "#!/usr/local/bin/python" from particular files and + causing the RPM build to terminate if there are any unexpected files + which have that line in them. + +* Mon Oct 13 2003 Sean Reifschneider <jafo-rpms@tummy.com> [2.3.2-1pydotorg] +- Adding code to detect wether documentation is available to build. + +* Fri Sep 19 2003 Sean Reifschneider <jafo-rpms@tummy.com> [2.3.1-1pydotorg] +- Updating to the 2.3.1 release. + +* Mon Feb 24 2003 Sean Reifschneider <jafo-rpms@tummy.com> [2.3b1-1pydotorg] +- Updating to 2.3b1 release. + +* Mon Feb 17 2003 Sean Reifschneider <jafo-rpms@tummy.com> [2.3a1-1] +- Updating to 2.3 release. + +* Sun Dec 23 2001 Sean Reifschneider <jafo-rpms@tummy.com> +[Release 2.2-2] +- Added -docs package. +- Added "auto" config_tkinter setting which only enables tk if + /usr/bin/wish exists. + +* Sat Dec 22 2001 Sean Reifschneider <jafo-rpms@tummy.com> +[Release 2.2-1] +- Updated to 2.2. +- Changed the extension to "2" from "2.2". + +* Tue Nov 18 2001 Sean Reifschneider <jafo-rpms@tummy.com> +[Release 2.2c1-1] +- Updated to 2.2c1. + +* Thu Nov 1 2001 Sean Reifschneider <jafo-rpms@tummy.com> +[Release 2.2b1-3] +- Changed the way the sed for fixing the #! in pydoc works. + +* Wed Oct 24 2001 Sean Reifschneider <jafo-rpms@tummy.com> +[Release 2.2b1-2] +- Fixed missing "email" package, thanks to anonymous report on sourceforge. +- Fixed missing "compiler" package. + +* Mon Oct 22 2001 Sean Reifschneider <jafo-rpms@tummy.com> +[Release 2.2b1-1] +- Updated to 2.2b1. + +* Mon Oct 9 2001 Sean Reifschneider <jafo-rpms@tummy.com> +[Release 2.2a4-4] +- otto@balinor.mat.unimi.it mentioned that the license file is missing. + +* Sun Sep 30 2001 Sean Reifschneider <jafo-rpms@tummy.com> +[Release 2.2a4-3] +- Ignacio Vazquez-Abrams pointed out that I had a spruious double-quote in + the spec files. Thanks. + +* Wed Jul 25 2001 Sean Reifschneider <jafo-rpms@tummy.com> +[Release 2.2a1-1] +- Updated to 2.2a1 release. +- Changed idle and pydoc to use binsuffix macro + +####### +# PREP +####### +%prep +%setup -n Python-%{version} + +######## +# BUILD +######## +%build +./configure --enable-unicode=ucs4 %{ipv6} %{pymalloc} --prefix=%{__prefix} +make + +########## +# INSTALL +########## +%install +# set the install path +echo '[install_scripts]' >setup.cfg +echo 'install_dir='"${RPM_BUILD_ROOT}%{__prefix}/bin" >>setup.cfg + +[ -d "$RPM_BUILD_ROOT" -a "$RPM_BUILD_ROOT" != "/" ] && rm -rf $RPM_BUILD_ROOT +mkdir -p $RPM_BUILD_ROOT%{__prefix}/%{libdirname}/python%{libvers}/lib-dynload +make prefix=$RPM_BUILD_ROOT%{__prefix} install + +# REPLACE PATH IN PYDOC +if [ ! -z "%{binsuffix}" ] +then + for file in pydoc python-config; do + ( + cd $RPM_BUILD_ROOT%{__prefix}/bin + mv "$file" "$file".old + sed 's|#!.*|#!%{__prefix}/bin/env python'%{binsuffix}'|' \ + "$file".old >"$file" + chmod 755 "$file" + rm -f "$file".old + ) + done +fi + +# add the binsuffix +if [ ! -z "%{binsuffix}" ] +then + ( cd $RPM_BUILD_ROOT%{__prefix}/bin; rm -f python[0-9a-zA-Z]*; + mv -f python python"%{binsuffix}" ) + ( cd $RPM_BUILD_ROOT%{__prefix}/man/man1; mv python.1 python%{binsuffix}.1 ) + ( cd $RPM_BUILD_ROOT%{__prefix}/bin; mv -f smtpd.py python-smtpd ) + for file in pydoc idle python-config python-smtpd; do + ( cd $RPM_BUILD_ROOT%{__prefix}/bin; mv -f "$file" "$file""%{binsuffix}" ) + done +fi + +######## +# Tools +echo '#!%{__prefix}/bin/env python%{binsuffix}' >${RPM_BUILD_ROOT}%{__prefix}/bin/idle%{binsuffix} +echo 'import os, sys' >>${RPM_BUILD_ROOT}%{__prefix}/bin/idle%{binsuffix} +echo 'os.execvp("%{__prefix}/bin/python%{binsuffix}", ["%{__prefix}/bin/python%{binsuffix}", "%{__prefix}/lib/python%{libvers}/idlelib/idle.py"] + sys.argv[1:])' >>${RPM_BUILD_ROOT}%{__prefix}/bin/idle%{binsuffix} +echo 'print "Failed to exec Idle"' >>${RPM_BUILD_ROOT}%{__prefix}/bin/idle%{binsuffix} +echo 'sys.exit(1)' >>${RPM_BUILD_ROOT}%{__prefix}/bin/idle%{binsuffix} +chmod 755 $RPM_BUILD_ROOT%{__prefix}/bin/idle%{binsuffix} +cp -a Tools $RPM_BUILD_ROOT%{__prefix}/%{libdirname}/python%{libvers} + +# MAKE FILE LISTS +rm -f mainpkg.files +find "$RPM_BUILD_ROOT""%{__prefix}"/%{libdirname}/python%{libvers}/lib-dynload -type f | + sed "s|^${RPM_BUILD_ROOT}|/|" | + grep -v -e '_tkinter.so$' >mainpkg.files +find "$RPM_BUILD_ROOT""%{__prefix}"/bin -type f | + sed "s|^${RPM_BUILD_ROOT}|/|" | + grep -v -e '/bin/setup-config%{binsuffix}$' | + grep -v -e '/bin/idle%{binsuffix}$' >>mainpkg.files + +rm -f tools.files +find "$RPM_BUILD_ROOT""%{__prefix}"/%{libdirname}/python%{libvers}/idlelib \ + "$RPM_BUILD_ROOT""%{__prefix}"/%{libdirname}/python%{libvers}/Tools -type f | + grep -v -e '\.pyc$' -e '\.pyo$' | + sed "s|^${RPM_BUILD_ROOT}|/|" >tools.files +echo "%{__prefix}"/bin/idle%{binsuffix} >>tools.files +grep '\.py$' tools.files | sed 's/$/c/' | grep -v /idlelib/ >tools.files.tmp +grep '\.py$' tools.files | sed 's/$/o/' | grep -v /idlelib/ >>tools.files.tmp +cat tools.files.tmp >>tools.files +rm tools.files.tmp + +###### +# Docs +%if %{include_docs} +mkdir -p "$RPM_BUILD_ROOT"%{config_htmldir} +( + cd "$RPM_BUILD_ROOT"%{config_htmldir} + bunzip2 < %{SOURCE1} | tar x +) +%endif + +# fix the #! line in installed files +find "$RPM_BUILD_ROOT" -type f -print0 | + xargs -0 grep -l /usr/local/bin/python | while read file +do + FIXFILE="$file" + sed 's|^#!.*python|#!%{__prefix}/bin/env python'"%{binsuffix}"'|' \ + "$FIXFILE" >/tmp/fix-python-path.$$ + cat /tmp/fix-python-path.$$ >"$FIXFILE" + rm -f /tmp/fix-python-path.$$ +done + +# check to see if there are any straggling #! lines +find "$RPM_BUILD_ROOT" -type f | xargs egrep -n '^#! */usr/local/bin/python' \ + | grep ':1:#!' >/tmp/python-rpm-files.$$ || true +if [ -s /tmp/python-rpm-files.$$ ] +then + echo '*****************************************************' + cat /tmp/python-rpm-files.$$ + cat <<@EOF + ***************************************************** + There are still files referencing /usr/local/bin/python in the + install directory. They are listed above. Please fix the .spec + file and try again. If you are an end-user, you probably want + to report this to jafo-rpms@tummy.com as well. + ***************************************************** +@EOF + rm -f /tmp/python-rpm-files.$$ + exit 1 +fi +rm -f /tmp/python-rpm-files.$$ + +######## +# CLEAN +######## +%clean +[ -n "$RPM_BUILD_ROOT" -a "$RPM_BUILD_ROOT" != / ] && rm -rf $RPM_BUILD_ROOT +rm -f mainpkg.files tools.files + +######## +# FILES +######## +%files -f mainpkg.files +%defattr(-,root,root) +%doc Misc/README Misc/cheatsheet Misc/Porting +%doc LICENSE Misc/ACKS Misc/HISTORY Misc/NEWS +%{__prefix}/man/man1/python%{binsuffix}.1* + +%attr(755,root,root) %dir %{__prefix}/include/python%{libvers} +%attr(755,root,root) %dir %{__prefix}/%{libdirname}/python%{libvers}/ +%{__prefix}/%{libdirname}/python%{libvers}/*.txt +%{__prefix}/%{libdirname}/python%{libvers}/*.py* +%{__prefix}/%{libdirname}/python%{libvers}/pdb.doc +%{__prefix}/%{libdirname}/python%{libvers}/curses +%{__prefix}/%{libdirname}/python%{libvers}/distutils +%{__prefix}/%{libdirname}/python%{libvers}/encodings +%{__prefix}/%{libdirname}/python%{libvers}/plat-linux2 +%{__prefix}/%{libdirname}/python%{libvers}/site-packages +%{__prefix}/%{libdirname}/python%{libvers}/test +%{__prefix}/%{libdirname}/python%{libvers}/xml +%{__prefix}/%{libdirname}/python%{libvers}/email +%{__prefix}/%{libdirname}/python%{libvers}/sqlite3 +%{__prefix}/%{libdirname}/python%{libvers}/compiler +%{__prefix}/%{libdirname}/python%{libvers}/bsddb +%{__prefix}/%{libdirname}/python%{libvers}/hotshot +%{__prefix}/%{libdirname}/python%{libvers}/logging +%{__prefix}/%{libdirname}/python%{libvers}/wsgiref +%{__prefix}/%{libdirname}/python%{libvers}/ctypes +%{__prefix}/%{libdirname}/python%{libvers}/wsgiref.egg-info + +%files devel +%defattr(-,root,root) +%{__prefix}/include/python%{libvers}/*.h +%{__prefix}/%{libdirname}/python%{libvers}/config + +%files -f tools.files tools +%defattr(-,root,root) + +%if %{include_tkinter} +%files tkinter +%defattr(-,root,root) +%{__prefix}/%{libdirname}/python%{libvers}/lib-tk +%{__prefix}/%{libdirname}/python%{libvers}/lib-dynload/_tkinter.so* +%endif + +%if %{include_docs} +%files docs +%defattr(-,root,root) +%{config_htmldir}/* +%endif diff --git a/sys/src/cmd/python/Misc/SpecialBuilds.txt b/sys/src/cmd/python/Misc/SpecialBuilds.txt new file mode 100644 index 000000000..952ca42d9 --- /dev/null +++ b/sys/src/cmd/python/Misc/SpecialBuilds.txt @@ -0,0 +1,261 @@ +This file describes some special Python build types enabled via +compile-time preprocessor defines. + +It is best to define these options in the EXTRA_CFLAGS make variable; +``make EXTRA_CFLAGS="-DPy_REF_DEBUG"``. + +--------------------------------------------------------------------------- +Py_REF_DEBUG introduced in 1.4 + named REF_DEBUG before 1.4 + +Turn on aggregate reference counting. This arranges that extern +_Py_RefTotal hold a count of all references, the sum of ob_refcnt across +all objects. In a debug-mode build, this is where the "8288" comes from +in + + >>> 23 + 23 + [8288 refs] + >>> + +Note that if this count increases when you're not storing away new objects, +there's probably a leak. Remember, though, that in interactive mode the +special name "_" holds a reference to the last result displayed! + +Py_REF_DEBUG also checks after every decref to verify that the refcount +hasn't gone negative, and causes an immediate fatal error if it has. + +Special gimmicks: + +sys.gettotalrefcount() + Return current total of all refcounts. + Available under Py_REF_DEBUG in Python 2.3. + Before 2.3, Py_TRACE_REFS was required to enable this function. +--------------------------------------------------------------------------- +Py_TRACE_REFS introduced in 1.4 + named TRACE_REFS before 1.4 + +Turn on heavy reference debugging. This is major surgery. Every PyObject +grows two more pointers, to maintain a doubly-linked list of all live +heap-allocated objects. Most builtin type objects are not in this list, +as they're statically allocated. Starting in Python 2.3, if COUNT_ALLOCS +(see below) is also defined, a static type object T does appear in this +list if at least one object of type T has been created. + +Note that because the fundamental PyObject layout changes, Python modules +compiled with Py_TRACE_REFS are incompatible with modules compiled without +it. + +Py_TRACE_REFS implies Py_REF_DEBUG. + +Special gimmicks: + +sys.getobjects(max[, type]) + Return list of the (no more than) max most-recently allocated objects, + most recently allocated first in the list, least-recently allocated + last in the list. max=0 means no limit on list length. + If an optional type object is passed, the list is also restricted to + objects of that type. + The return list itself, and some temp objects created just to call + sys.getobjects(), are excluded from the return list. Note that the + list returned is just another object, though, so may appear in the + return list the next time you call getobjects(); note that every + object in the list is kept alive too, simply by virtue of being in + the list. + +envar PYTHONDUMPREFS + If this envar exists, Py_Finalize() arranges to print a list of + all still-live heap objects. This is printed twice, in different + formats, before and after Py_Finalize has cleaned up everything it + can clean up. The first output block produces the repr() of each + object so is more informative; however, a lot of stuff destined to + die is still alive then. The second output block is much harder + to work with (repr() can't be invoked anymore -- the interpreter + has been torn down too far), but doesn't list any objects that will + die. The tool script combinerefs.py can be run over this to combine + the info from both output blocks. The second output block, and + combinerefs.py, were new in Python 2.3b1. +--------------------------------------------------------------------------- +PYMALLOC_DEBUG introduced in 2.3 + +When pymalloc is enabled (WITH_PYMALLOC is defined), calls to the PyObject_ +memory routines are handled by Python's own small-object allocator, while +calls to the PyMem_ memory routines are directed to the system malloc/ +realloc/free. If PYMALLOC_DEBUG is also defined, calls to both PyObject_ +and PyMem_ memory routines are directed to a special debugging mode of +Python's small-object allocator. + +This mode fills dynamically allocated memory blocks with special, +recognizable bit patterns, and adds debugging info on each end of +dynamically allocated memory blocks. The special bit patterns are: + +#define CLEANBYTE 0xCB /* clean (newly allocated) memory */ +#define DEADBYTE 0xDB /* dead (newly freed) memory */ +#define FORBIDDENBYTE 0xFB /* fordidden -- untouchable bytes */ + +Strings of these bytes are unlikely to be valid addresses, floats, or 7-bit +ASCII strings. + +Let S = sizeof(size_t). 2*S bytes are added at each end of each block of N +bytes requested. The memory layout is like so, where p represents the +address returned by a malloc-like or realloc-like function (p[i:j] means +the slice of bytes from *(p+i) inclusive up to *(p+j) exclusive; note that +the treatment of negative indices differs from a Python slice): + +p[-2*S:-S] + Number of bytes originally asked for. This is a size_t, big-endian + (easier to read in a memory dump). +p[-S:0] + Copies of FORBIDDENBYTE. Used to catch under- writes and reads. +p[0:N] + The requested memory, filled with copies of CLEANBYTE, used to catch + reference to uninitialized memory. + When a realloc-like function is called requesting a larger memory + block, the new excess bytes are also filled with CLEANBYTE. + When a free-like function is called, these are overwritten with + DEADBYTE, to catch reference to freed memory. When a realloc- + like function is called requesting a smaller memory block, the excess + old bytes are also filled with DEADBYTE. +p[N:N+S] + Copies of FORBIDDENBYTE. Used to catch over- writes and reads. +p[N+S:N+2*S] + A serial number, incremented by 1 on each call to a malloc-like or + realloc-like function. + Big-endian size_t. + If "bad memory" is detected later, the serial number gives an + excellent way to set a breakpoint on the next run, to capture the + instant at which this block was passed out. The static function + bumpserialno() in obmalloc.c is the only place the serial number + is incremented, and exists so you can set such a breakpoint easily. + +A realloc-like or free-like function first checks that the FORBIDDENBYTEs +at each end are intact. If they've been altered, diagnostic output is +written to stderr, and the program is aborted via Py_FatalError(). The +other main failure mode is provoking a memory error when a program +reads up one of the special bit patterns and tries to use it as an address. +If you get in a debugger then and look at the object, you're likely +to see that it's entirely filled with 0xDB (meaning freed memory is +getting used) or 0xCB (meaning uninitialized memory is getting used). + +Note that PYMALLOC_DEBUG requires WITH_PYMALLOC. + +Special gimmicks: + +envar PYTHONMALLOCSTATS + If this envar exists, a report of pymalloc summary statistics is + printed to stderr whenever a new arena is allocated, and also + by Py_Finalize(). + +Changed in 2.5: The number of extra bytes allocated is 4*sizeof(size_t). +Before it was 16 on all boxes, reflecting that Python couldn't make use of +allocations >= 2**32 bytes even on 64-bit boxes before 2.5. +--------------------------------------------------------------------------- +Py_DEBUG introduced in 1.5 + named DEBUG before 1.5 + +This is what is generally meant by "a debug build" of Python. + +Py_DEBUG implies LLTRACE, Py_REF_DEBUG, Py_TRACE_REFS, and +PYMALLOC_DEBUG (if WITH_PYMALLOC is enabled). In addition, C +assert()s are enabled (via the C way: by not defining NDEBUG), and +some routines do additional sanity checks inside "#ifdef Py_DEBUG" +blocks. +--------------------------------------------------------------------------- +COUNT_ALLOCS introduced in 0.9.9 + partly broken in 2.2 and 2.2.1 + +Each type object grows three new members: + + /* Number of times an object of this type was allocated. */ + int tp_allocs; + + /* Number of times an object of this type was deallocated. */ + int tp_frees; + + /* Highwater mark: the maximum value of tp_allocs - tp_frees so + * far; or, IOW, the largest number of objects of this type alive at + * the same time. + */ + int tp_maxalloc; + +Allocation and deallocation code keeps these counts up to date. +Py_Finalize() displays a summary of the info returned by sys.getcounts() +(see below), along with assorted other special allocation counts (like +the number of tuple allocations satisfied by a tuple free-list, the number +of 1-character strings allocated, etc). + +Before Python 2.2, type objects were immortal, and the COUNT_ALLOCS +implementation relies on that. As of Python 2.2, heap-allocated type/ +class objects can go away. COUNT_ALLOCS can blow up in 2.2 and 2.2.1 +because of this; this was fixed in 2.2.2. Use of COUNT_ALLOCS makes +all heap-allocated type objects immortal, except for those for which no +object of that type is ever allocated. + +Starting with Python 2.3, If Py_TRACE_REFS is also defined, COUNT_ALLOCS +arranges to ensure that the type object for each allocated object +appears in the doubly-linked list of all objects maintained by +Py_TRACE_REFS. + +Special gimmicks: + +sys.getcounts() + Return a list of 4-tuples, one entry for each type object for which + at least one object of that type was allocated. Each tuple is of + the form: + + (tp_name, tp_allocs, tp_frees, tp_maxalloc) + + Each distinct type object gets a distinct entry in this list, even + if two or more type objects have the same tp_name (in which case + there's no way to distinguish them by looking at this list). The + list is ordered by time of first object allocation: the type object + for which the first allocation of an object of that type occurred + most recently is at the front of the list. +--------------------------------------------------------------------------- +LLTRACE introduced well before 1.0 + +Compile in support for Low Level TRACE-ing of the main interpreter loop. + +When this preprocessor symbol is defined, before PyEval_EvalFrame +(eval_frame in 2.3 and 2.2, eval_code2 before that) executes a frame's code +it checks the frame's global namespace for a variable "__lltrace__". If +such a variable is found, mounds of information about what the interpreter +is doing are sprayed to stdout, such as every opcode and opcode argument +and values pushed onto and popped off the value stack. + +Not useful very often, but very useful when needed. + +--------------------------------------------------------------------------- +CALL_PROFILE introduced for Python 2.3 + +Count the number of function calls executed. + +When this symbol is defined, the ceval mainloop and helper functions +count the number of function calls made. It keeps detailed statistics +about what kind of object was called and whether the call hit any of +the special fast paths in the code. + +--------------------------------------------------------------------------- +WITH_TSC introduced for Python 2.4 + +Super-lowlevel profiling of the interpreter. When enabled, the sys +module grows a new function: + +settscdump(bool) + If true, tell the Python interpreter to dump VM measurements to + stderr. If false, turn off dump. The measurements are based on the + processor's time-stamp counter. + +This build option requires a small amount of platform specific code. +Currently this code is present for linux/x86 and any PowerPC platform +that uses GCC (i.e. OS X and linux/ppc). + +On the PowerPC the rate at which the time base register is incremented +is not defined by the architecture specification, so you'll need to +find the manual for your specific processor. For the 750CX, 750CXe +and 750FX (all sold as the G3) we find: + + The time base counter is clocked at a frequency that is + one-fourth that of the bus clock. + +This build is enabled by the --with-tsc flag to configure. diff --git a/sys/src/cmd/python/Misc/Vim/python.vim b/sys/src/cmd/python/Misc/Vim/python.vim new file mode 100644 index 000000000..61d75e2ce --- /dev/null +++ b/sys/src/cmd/python/Misc/Vim/python.vim @@ -0,0 +1,147 @@ +" Auto-generated Vim syntax file for Python +" +" To use: copy or symlink to ~/.vim/syntax/python.vim + + +if exists("b:current_syntax") + finish +endif + +if exists("python_highlight_all") + let python_highlight_numbers = 1 + let python_highlight_builtins = 1 + let python_highlight_exceptions = 1 + let python_highlight_space_errors = 1 +endif + +syn keyword pythonStatement as assert break continue del except exec finally +syn keyword pythonStatement global lambda pass print raise return try with +syn keyword pythonStatement yield + +syn keyword pythonStatement def class nextgroup=pythonFunction skipwhite + +syn match pythonFunction "[a-zA-Z_][a-zA-Z0-9_]*" contained + +syn keyword pythonRepeat for while + +syn keyword pythonConditional if elif else + +syn keyword pythonOperator and in is not or + +syn keyword pythonPreCondit import from + +syn match pythonComment "#.*$" contains=pythonTodo + +syn keyword pythonTodo TODO FIXME XXX contained + +syn region pythonString matchgroup=Normal start=+[uU]\='+ end=+'+ skip=+\\\\\|\\'+ contains=pythonEscape +syn region pythonString matchgroup=Normal start=+[uU]\="+ end=+"+ skip=+\\\\\|\\"+ contains=pythonEscape +syn region pythonString matchgroup=Normal start=+[uU]\="""+ end=+"""+ contains=pythonEscape +syn region pythonString matchgroup=Normal start=+[uU]\='''+ end=+'''+ contains=pythonEscape +syn region pythonString matchgroup=Normal start=+[uU]\=[rR]'+ end=+'+ skip=+\\\\\|\\'+ +syn region pythonString matchgroup=Normal start=+[uU]\=[rR]"+ end=+"+ skip=+\\\\\|\\"+ +syn region pythonString matchgroup=Normal start=+[uU]\=[rR]"""+ end=+"""+ +syn region pythonString matchgroup=Normal start=+[uU]\=[rR]'''+ end=+'''+ + +syn match pythonEscape +\\[abfnrtv\'"\\]+ contained +syn match pythonEscape "\\\o\{1,3}" contained +syn match pythonEscape "\\x\x\{2}" contained +syn match pythonEscape "\(\\u\x\{4}\|\\U\x\{8}\)" contained + +syn match pythonEscape "\\$" + + +if exists("python_highlight_numbers") + syn match pythonNumber "\<0x\x\+[Ll]\=\>" + syn match pythonNumber "\<\d\+[LljJ]\=\>" + syn match pythonNumber "\.\d\+\([eE][+-]\=\d\+\)\=[jJ]\=\>" + syn match pythonNumber "\<\d\+\.\([eE][+-]\=\d\+\)\=[jJ]\=\>" + syn match pythonNumber "\<\d\+\.\d\+\([eE][+-]\=\d\+\)\=[jJ]\=\>" + +endif + + +if exists("python_highlight_builtins") + syn keyword pythonBuiltin unichr all set abs vars int __import__ unicode + syn keyword pythonBuiltin enumerate reduce coerce intern exit issubclass + syn keyword pythonBuiltin divmod file Ellipsis apply isinstance open any + syn keyword pythonBuiltin locals help filter basestring slice copyright min + syn keyword pythonBuiltin super sum tuple hex execfile long id xrange chr + syn keyword pythonBuiltin complex bool zip pow dict True oct NotImplemented + syn keyword pythonBuiltin map None float hash getattr buffer max reversed + syn keyword pythonBuiltin object quit len repr callable credits setattr + syn keyword pythonBuiltin eval frozenset sorted ord __debug__ hasattr + syn keyword pythonBuiltin delattr False input license classmethod type + syn keyword pythonBuiltin raw_input list iter compile reload range globals + syn keyword pythonBuiltin staticmethod str property round dir cmp + +endif + + +if exists("python_highlight_exceptions") + syn keyword pythonException GeneratorExit ImportError RuntimeError + syn keyword pythonException UnicodeTranslateError MemoryError StopIteration + syn keyword pythonException PendingDeprecationWarning EnvironmentError + syn keyword pythonException LookupError OSError DeprecationWarning + syn keyword pythonException UnicodeError UnicodeEncodeError + syn keyword pythonException FloatingPointError ReferenceError NameError + syn keyword pythonException IOError SyntaxError + syn keyword pythonException FutureWarning ImportWarning SystemExit + syn keyword pythonException Exception EOFError StandardError ValueError + syn keyword pythonException TabError KeyError ZeroDivisionError SystemError + syn keyword pythonException UnicodeDecodeError IndentationError + syn keyword pythonException AssertionError TypeError IndexError + syn keyword pythonException RuntimeWarning KeyboardInterrupt UserWarning + syn keyword pythonException SyntaxWarning UnboundLocalError ArithmeticError + syn keyword pythonException Warning NotImplementedError AttributeError + syn keyword pythonException OverflowError BaseException + +endif + + +if exists("python_highlight_space_errors") + syn match pythonSpaceError display excludenl "\S\s\+$"ms=s+1 + syn match pythonSpaceError display " \+\t" + syn match pythonSpaceError display "\t\+ " + +endif + + + hi def link pythonStatement Statement + hi def link pythonStatement Statement + hi def link pythonFunction Function + hi def link pythonRepeat Repeat + hi def link pythonConditional Conditional + hi def link pythonOperator Operator + hi def link pythonPreCondit PreCondit + hi def link pythonComment Comment + hi def link pythonTodo Todo + hi def link pythonString String + hi def link pythonEscape Special + hi def link pythonEscape Special + + if exists("python_highlight_numbers") + hi def link pythonNumber Number + endif + + if exists("python_highlight_builtins") + hi def link pythonBuiltin Function + endif + + if exists("python_highlight_exceptions") + hi def link pythonException Exception + endif + + if exists("python_highlight_space_errors") + hi def link pythonSpaceError Error + endif + + +" Uncomment the 'minlines' statement line and comment out the 'maxlines' +" statement line; changes behaviour to look at least 2000 lines previously for +" syntax matches instead of at most 200 lines +syn sync match pythonSync grouphere NONE "):$" +syn sync maxlines=200 +"syn sync minlines=2000 + +let b:current_syntax = "python" diff --git a/sys/src/cmd/python/Misc/Vim/syntax_test.py b/sys/src/cmd/python/Misc/Vim/syntax_test.py new file mode 100644 index 000000000..ccc7f309c --- /dev/null +++ b/sys/src/cmd/python/Misc/Vim/syntax_test.py @@ -0,0 +1,63 @@ +"""Test file for syntax highlighting of editors. + +Meant to cover a wide range of different types of statements and expressions. +Not necessarily sensical or comprehensive (assume that if one exception is +highlighted that all are, for instance). + +Highlighting extraneous whitespace at the end of the line is not represented +here as all trailing whitespace is automatically removed from .py files in the +repository. + +""" +# Comment +# OPTIONAL: XXX catch your attention + +# Statements +from __future__ import with_statement # Import +from sys import path as thing +assert True # keyword +def foo(): # function definition + return [] +class Bar(object): # Class definition + def __enter__(self): + pass + def __exit__(self, *args): + pass +foo() # UNCOLOURED: function call +while False: # 'while' + continue +for x in foo(): # 'for' + break +with Bar() as stuff: + pass +if False: pass # 'if' +elif False: pass +else: pass + +# Constants +'single-quote', u'unicode' # Strings of all kinds; prefixes not highlighted +"double-quote" +"""triple double-quote""" +'''triple single-quote''' +r'raw' +ur'unicode raw' +'escape\n' +'\04' # octal +'\xFF' # hex +'\u1111' # unicode character +1 # Integral +1L +1.0 # Float +.1 +1+2j # Complex + +# Expressions +1 and 2 or 3 # Boolean operators +2 < 3 # UNCOLOURED: comparison operators +spam = 42 # UNCOLOURED: assignment +2 + 3 # UNCOLOURED: number operators +[] # UNCOLOURED: list +{} # UNCOLOURED: dict +(1,) # UNCOLOURED: tuple +all # Built-in functions +GeneratorExit # Exceptions diff --git a/sys/src/cmd/python/Misc/Vim/vim_syntax.py b/sys/src/cmd/python/Misc/Vim/vim_syntax.py new file mode 100644 index 000000000..3f2a3d8a2 --- /dev/null +++ b/sys/src/cmd/python/Misc/Vim/vim_syntax.py @@ -0,0 +1,226 @@ +from __future__ import with_statement + +import keyword +import exceptions +import __builtin__ +from string import Template + +comment_header = """" Auto-generated Vim syntax file for Python +" +" To use: copy or symlink to ~/.vim/syntax/python.vim""" + +statement_header = """ +if exists("b:current_syntax") + finish +endif""" + +statement_footer = ''' +" Uncomment the 'minlines' statement line and comment out the 'maxlines' +" statement line; changes behaviour to look at least 2000 lines previously for +" syntax matches instead of at most 200 lines +syn sync match pythonSync grouphere NONE "):$" +syn sync maxlines=200 +"syn sync minlines=2000 + +let b:current_syntax = "python"''' + +looping = ('for', 'while') +conditionals = ('if', 'elif', 'else') +boolean_ops = ('and', 'in', 'is', 'not', 'or') +import_stmts = ('import', 'from') +object_defs = ('def', 'class') + +exception_names = frozenset(exc for exc in dir(exceptions) + if not exc.startswith('__')) + +# Need to include functions that start with '__' (e.g., __import__), but +# nothing that comes with modules (e.g., __name__), so just exclude anything in +# the 'exceptions' module since we want to ignore exceptions *and* what any +# module would have +builtin_names = frozenset(builtin for builtin in dir(__builtin__) + if builtin not in dir(exceptions)) + +escapes = (r'+\\[abfnrtv\'"\\]+', r'"\\\o\{1,3}"', r'"\\x\x\{2}"', + r'"\(\\u\x\{4}\|\\U\x\{8}\)"', r'"\\$"') + +todos = ("TODO", "FIXME", "XXX") + +# XXX codify? +numbers = (r'"\<0x\x\+[Ll]\=\>"', r'"\<\d\+[LljJ]\=\>"', + '"\.\d\+\([eE][+-]\=\d\+\)\=[jJ]\=\>"', + '"\<\d\+\.\([eE][+-]\=\d\+\)\=[jJ]\=\>"', + '"\<\d\+\.\d\+\([eE][+-]\=\d\+\)\=[jJ]\=\>"') + +contained = lambda x: "%s contained" % x + +def str_regexes(): + """Generator to yield various combinations of strings regexes""" + regex_template = Template('matchgroup=Normal ' + + 'start=+[uU]\=${raw}${sep}+ ' + + 'end=+${sep}+ ' + + '${skip} ' + + '${contains}') + skip_regex = Template(r'skip=+\\\\\|\\${sep}+') + for raw in ('', '[rR]'): + for separator in ("'", '"', '"""', "'''"): + if len(separator) == 1: + skip = skip_regex.substitute(sep=separator) + else: + skip = '' + contains = 'contains=pythonEscape' if not raw else '' + yield regex_template.substitute(raw=raw, sep=separator, skip=skip, + contains = contains) + +space_errors = (r'excludenl "\S\s\+$"ms=s+1', r'" \+\t"', r'"\t\+ "') + +statements = ( + ('', + # XXX Might need to change pythonStatement since have + # specific Repeat, Conditional, Operator, etc. for 'while', + # etc. + [("Statement", "pythonStatement", "keyword", + (kw for kw in keyword.kwlist + if kw not in (looping + conditionals + boolean_ops + + import_stmts + object_defs)) + ), + ("Statement", "pythonStatement", "keyword", + (' '.join(object_defs) + + ' nextgroup=pythonFunction skipwhite')), + ("Function","pythonFunction", "match", + contained('"[a-zA-Z_][a-zA-Z0-9_]*"')), + ("Repeat", "pythonRepeat", "keyword", looping), + ("Conditional", "pythonConditional", "keyword", + conditionals), + ("Operator", "pythonOperator", "keyword", boolean_ops), + ("PreCondit", "pythonPreCondit", "keyword", import_stmts), + ("Comment", "pythonComment", "match", + '"#.*$" contains=pythonTodo'), + ("Todo", "pythonTodo", "keyword", + contained(' '.join(todos))), + ("String", "pythonString", "region", str_regexes()), + ("Special", "pythonEscape", "match", + (contained(esc) for esc in escapes + if not '$' in esc)), + ("Special", "pythonEscape", "match", r'"\\$"'), + ] + ), + ("python_highlight_numbers", + [("Number", "pythonNumber", "match", numbers)] + ), + ("python_highlight_builtins", + [("Function", "pythonBuiltin", "keyword", builtin_names)] + ), + ("python_highlight_exceptions", + [("Exception", "pythonException", "keyword", + exception_names)] + ), + ("python_highlight_space_errors", + [("Error", "pythonSpaceError", "match", + ("display " + err for err in space_errors))] + ) + ) + +def syn_prefix(type_, kind): + return 'syn %s %s ' % (type_, kind) + +def fill_stmt(iterable, fill_len): + """Yield a string that fills at most fill_len characters with strings + returned by 'iterable' and separated by a space""" + # Deal with trailing char to handle ' '.join() calculation + fill_len += 1 + overflow = None + it = iter(iterable) + while True: + buffer_ = [] + total_len = 0 + if overflow: + buffer_.append(overflow) + total_len += len(overflow) + 1 + overflow = None + while total_len < fill_len: + try: + new_item = it.next() + buffer_.append(new_item) + total_len += len(new_item) + 1 + except StopIteration: + if buffer_: + break + if overflow: + yield overflow + return + if total_len > fill_len: + overflow = buffer_.pop() + total_len -= len(overflow) - 1 + ret = ' '.join(buffer_) + assert len(ret) <= fill_len + yield ret + +FILL = 80 + +def main(file_path): + with open(file_path, 'w') as FILE: + # Comment for file + print>>FILE, comment_header + print>>FILE, '' + # Statements at start of file + print>>FILE, statement_header + print>>FILE, '' + # Generate case for python_highlight_all + print>>FILE, 'if exists("python_highlight_all")' + for statement_var, statement_parts in statements: + if statement_var: + print>>FILE, ' let %s = 1' % statement_var + else: + print>>FILE, 'endif' + print>>FILE, '' + # Generate Python groups + for statement_var, statement_parts in statements: + if statement_var: + print>>FILE, 'if exists("%s")' % statement_var + indent = ' ' + else: + indent = '' + for colour_group, group, type_, arguments in statement_parts: + if not isinstance(arguments, basestring): + prefix = syn_prefix(type_, group) + if type_ == 'keyword': + stmt_iter = fill_stmt(arguments, + FILL - len(prefix) - len(indent)) + try: + while True: + print>>FILE, indent + prefix + stmt_iter.next() + except StopIteration: + print>>FILE, '' + else: + for argument in arguments: + print>>FILE, indent + prefix + argument + else: + print>>FILE, '' + + else: + print>>FILE, indent + syn_prefix(type_, group) + arguments + print>>FILE, '' + else: + if statement_var: + print>>FILE, 'endif' + print>>FILE, '' + print>>FILE, '' + # Associating Python group with Vim colour group + for statement_var, statement_parts in statements: + if statement_var: + print>>FILE, ' if exists("%s")' % statement_var + indent = ' ' + else: + indent = ' ' + for colour_group, group, type_, arguments in statement_parts: + print>>FILE, (indent + "hi def link %s %s" % + (group, colour_group)) + else: + if statement_var: + print>>FILE, ' endif' + print>>FILE, '' + # Statements at the end of the file + print>>FILE, statement_footer + +if __name__ == '__main__': + main("python.vim") diff --git a/sys/src/cmd/python/Misc/Vim/vimrc b/sys/src/cmd/python/Misc/Vim/vimrc new file mode 100644 index 000000000..af60614b1 --- /dev/null +++ b/sys/src/cmd/python/Misc/Vim/vimrc @@ -0,0 +1,95 @@ +" vimrc file for following the coding standards specified in PEP 7 & 8. +" +" To use this file, source it in your own personal .vimrc file (``source +" <filename>``) or, if you don't have a .vimrc file, you can just symlink to it +" (``ln -s <this file> ~/.vimrc``). All options are protected by autocmds +" (read below for an explanation of the command) so blind sourcing of this file +" is safe and will not affect your settings for non-Python or non-C files. +" +" +" All setting are protected by 'au' ('autocmd') statements. Only files ending +" in .py or .pyw will trigger the Python settings while files ending in *.c or +" *.h will trigger the C settings. This makes the file "safe" in terms of only +" adjusting settings for Python and C files. +" +" Only basic settings needed to enforce the style guidelines are set. +" Some suggested options are listed but commented out at the end of this file. + + +" Number of spaces to use for an indent. +" This will affect Ctrl-T and 'autoindent'. +" Python: 4 spaces +" C: tab (8 spaces) +au BufRead,BufNewFile *.py,*pyw set shiftwidth=4 +au BufRead,BufNewFile *.c,*.h set shiftwidth=4 + +" Number of spaces that a pre-existing tab is equal to. +" For the amount of space used for a new tab use shiftwidth. +" Python: 8 +" C: 8 +au BufRead,BufNewFile *py,*pyw,*.c,*.h set tabstop=8 + +" Replace tabs with the equivalent number of spaces. +" Also have an autocmd for Makefiles since they require hard tabs. +" Python: yes +" C: no +" Makefile: no +au BufRead,BufNewFile *.py,*.pyw set expandtab +au BufRead,BufNewFile *.c,*.h set noexpandtab +au BufRead,BufNewFile Makefile* set noexpandtab + +" Use the below highlight group when displaying bad whitespace is desired +highlight BadWhitespace ctermbg=red guibg=red + +" Display tabs at the beginning of a line in Python mode as bad +au BufRead,BufNewFile *.py,*.pyw match BadWhitespace /^\t\+/ + +" Wrap text after a certain number of characters +" Python: 79 +" C: 79 +au BufRead,BufNewFile *.py,*.pyw,*.c,*.h set textwidth=79 + +" Turn off settings in 'formatoptions' relating to comment formatting. +" - c : do not automatically insert the comment leader when wrapping based on +" 'textwidth' +" - o : do not insert the comment leader when using 'o' or 'O' from command mode +" - r : do not insert the comment leader when hitting <Enter> in insert mode +" Python: not needed +" C: prevents insertion of '*' at the beginning of every line in a comment +au BufRead,BufNewFile *.c,*.h set formatoptions-=c formatoptions-=o formatoptions-=r + +" Use UNIX (\n) line endings. +" Only used for new files so as to not force existing files to change their +" line endings. +" Python: yes +" C: yes +au BufNewFile *.py,*.pyw,*.c,*.h set fileformat=unix + + +" ---------------------------------------------------------------------------- +" The following section contains suggested settings. While in no way required +" to meet coding standards, they are helpful. + +" Set the default file encoding to UTF-8: ``set encoding=utf-8`` + +" Puts a marker at the beginning of the file to differentiate between UTF and +" UCS encoding (WARNING: can trick shells into thinking a text file is actually +" a binary file when executing the text file): ``set bomb`` + +" For full syntax highlighting: +"``let python_highlight_all=1`` +"``syntax on`` + +" Automatically indent based on file type: ``filetype indent on`` +" Keep indentation level from previous line: ``set autoindent`` + +" Folding based on indentation: ``set foldmethod=indent`` + +" Make trailing whitespace explicit (left off since this will automatically +" insert the highlight or characters *as you type*, which can get annoying): +"``match BadWhitespace /\s\+$/`` +" +" or, for a non-colored, character-based solution: +" +"``set list listchars=trail:-`` + diff --git a/sys/src/cmd/python/Misc/build.sh b/sys/src/cmd/python/Misc/build.sh new file mode 100755 index 000000000..ff46bbab0 --- /dev/null +++ b/sys/src/cmd/python/Misc/build.sh @@ -0,0 +1,227 @@ +#!/bin/sh + +## Script to build and test the latest python from svn. It basically +## does this: +## svn up ; ./configure ; make ; make test ; make install ; cd Doc ; make +## +## Logs are kept and rsync'ed to the host. If there are test failure(s), +## information about the failure(s) is mailed. +## +## This script is run on the PSF's machine as user neal via crontab. +## +## Yes, this script would probably be easier in python, but then +## there's a bootstrap problem. What if Python doesn't build? +## +## This script should be fairly clean Bourne shell, ie not too many +## bash-isms. We should try to keep it portable to other Unixes. +## Even though it will probably only run on Linux. I'm sure there are +## several GNU-isms currently (date +%s and readlink). +## +## Perhaps this script should be broken up into 2 (or more) components. +## Building doc is orthogonal to the rest of the python build/test. +## + +## FIXME: we should detect test hangs (eg, if they take more than 45 minutes) + +## FIXME: we should run valgrind +## FIXME: we should run code coverage + +## Utilities invoked in this script include: +## basename, date, dirname, expr, grep, readlink, uname +## cksum, make, mutt, rsync, svn + +## remember where did we started from +DIR=`dirname $0` +if [ "$DIR" = "" ]; then + DIR="." +fi + +## make directory absolute +DIR=`readlink -f $DIR` +FULLPATHNAME="$DIR/`basename $0`" +## we want Misc/.. +DIR=`dirname $DIR` + +## Configurable options + +FAILURE_SUBJECT="Python Regression Test Failures" +#FAILURE_MAILTO="YOUR_ACCOUNT@gmail.com" +FAILURE_MAILTO="python-checkins@python.org" + +REMOTE_SYSTEM="neal@dinsdale.python.org" +REMOTE_DIR="/data/ftp.python.org/pub/docs.python.org/dev/" +RESULT_FILE="$DIR/build/index.html" +INSTALL_DIR="/tmp/python-test/local" +RSYNC_OPTS="-aC -e ssh" + +# Always run the installed version of Python. +PYTHON=$INSTALL_DIR/bin/python + +# Python options and regression test program that should always be run. +REGRTEST_ARGS="-E -tt $INSTALL_DIR/lib/python2.5/test/regrtest.py" + +REFLOG="build/reflog.txt.out" +# These tests are not stable and falsely report leaks sometimes. +# The entire leak report will be mailed if any test not in this list leaks. +# Note: test_XXX (none currently) really leak, but are disabled +# so we don't send spam. Any test which really leaks should only +# be listed here if there are also test cases under Lib/test/leakers. +LEAKY_TESTS="test_(XXX)" # Currently no tests should report spurious leaks. + +# Skip these tests altogether when looking for leaks. These tests +# do not need to be stored above in LEAKY_TESTS too. +# test_compiler almost never finishes with the same number of refs +# since it depends on other modules, skip it. +# test_logging causes hangs, skip it. +LEAKY_SKIPS="-x test_compiler test_logging" + +# Change this flag to "yes" for old releases to only update/build the docs. +BUILD_DISABLED="no" + +## utility functions +current_time() { + date +%s +} + +update_status() { + now=`current_time` + time=`expr $now - $3` + echo "<li><a href=\"$2\">$1</a> <font size=\"-1\">($time seconds)</font></li>" >> $RESULT_FILE +} + +mail_on_failure() { + if [ "$NUM_FAILURES" != "0" ]; then + mutt -s "$FAILURE_SUBJECT $1 ($NUM_FAILURES)" $FAILURE_MAILTO < $2 + fi +} + +## setup +cd $DIR +mkdir -p build +rm -f $RESULT_FILE build/*.out +rm -rf $INSTALL_DIR + +## create results file +TITLE="Automated Python Build Results" +echo "<html>" >> $RESULT_FILE +echo " <head>" >> $RESULT_FILE +echo " <title>$TITLE</title>" >> $RESULT_FILE +echo " <meta http-equiv=\"refresh\" content=\"43200\">" >> $RESULT_FILE +echo " </head>" >> $RESULT_FILE +echo "<body>" >> $RESULT_FILE +echo "<h2>Automated Python Build Results</h2>" >> $RESULT_FILE +echo "<table>" >> $RESULT_FILE +echo " <tr>" >> $RESULT_FILE +echo " <td>Built on:</td><td>`date`</td>" >> $RESULT_FILE +echo " </tr><tr>" >> $RESULT_FILE +echo " <td>Hostname:</td><td>`uname -n`</td>" >> $RESULT_FILE +echo " </tr><tr>" >> $RESULT_FILE +echo " <td>Platform:</td><td>`uname -srmpo`</td>" >> $RESULT_FILE +echo " </tr>" >> $RESULT_FILE +echo "</table>" >> $RESULT_FILE +echo "<ul>" >> $RESULT_FILE + +## update, build, and test +ORIG_CHECKSUM=`cksum $FULLPATHNAME` +F=svn-update.out +start=`current_time` +svn update >& build/$F +err=$? +update_status "Updating" "$F" $start +if [ $err = 0 -a "$BUILD_DISABLED" != "yes" ]; then + ## FIXME: we should check if this file has changed. + ## If it has changed, we should re-run the script to pick up changes. + if [ "$ORIG_CHECKSUM" != "$ORIG_CHECKSUM" ]; then + exec $FULLPATHNAME $@ + fi + + F=svn-stat.out + start=`current_time` + svn stat >& build/$F + ## ignore some of the diffs + NUM_DIFFS=`egrep -vc '^. (@test|db_home|Lib/test/(regrtest\.py|db_home))$' build/$F` + update_status "svn stat ($NUM_DIFFS possibly important diffs)" "$F" $start + + F=configure.out + start=`current_time` + ./configure --prefix=$INSTALL_DIR --with-pydebug >& build/$F + err=$? + update_status "Configuring" "$F" $start + if [ $err = 0 ]; then + F=make.out + start=`current_time` + make >& build/$F + err=$? + warnings=`grep warning build/$F | egrep -vc "te?mpnam(_r|)' is dangerous,"` + update_status "Building ($warnings warnings)" "$F" $start + if [ $err = 0 ]; then + ## make install + F=make-install.out + start=`current_time` + make install >& build/$F + update_status "Installing" "$F" $start + + if [ ! -x $PYTHON ]; then + ln -s ${PYTHON}2.* $PYTHON + fi + + ## make and run basic tests + F=make-test.out + start=`current_time` + $PYTHON $REGRTEST_ARGS >& build/$F + NUM_FAILURES=`grep -ic " failed:" build/$F` + update_status "Testing basics ($NUM_FAILURES failures)" "$F" $start + mail_on_failure "basics" build/$F + + F=make-test-opt.out + start=`current_time` + $PYTHON -O $REGRTEST_ARGS >& build/$F + NUM_FAILURES=`grep -ic " failed:" build/$F` + update_status "Testing opt ($NUM_FAILURES failures)" "$F" $start + mail_on_failure "opt" build/$F + + ## run the tests looking for leaks + F=make-test-refleak.out + start=`current_time` + ## ensure that the reflog exists so the grep doesn't fail + touch $REFLOG + $PYTHON $REGRTEST_ARGS -R 4:3:$REFLOG -u network $LEAKY_SKIPS >& build/$F + NUM_FAILURES=`egrep -vc "$LEAKY_TESTS" $REFLOG` + update_status "Testing refleaks ($NUM_FAILURES failures)" "$F" $start + mail_on_failure "refleak" $REFLOG + + ## now try to run all the tests + F=make-testall.out + start=`current_time` + ## skip curses when running from cron since there's no terminal + ## skip sound since it's not setup on the PSF box (/dev/dsp) + $PYTHON $REGRTEST_ARGS -uall -x test_curses test_linuxaudiodev test_ossaudiodev >& build/$F + NUM_FAILURES=`grep -ic " failed:" build/$F` + update_status "Testing all except curses and sound ($NUM_FAILURES failures)" "$F" $start + mail_on_failure "all" build/$F + fi + fi +fi + + +## make doc +cd $DIR/Doc +F="make-doc.out" +start=`current_time` +make >& ../build/$F +err=$? +update_status "Making doc" "$F" $start +if [ $err != 0 ]; then + NUM_FAILURES=1 + mail_on_failure "doc" ../build/$F +fi + +echo "</ul>" >> $RESULT_FILE +echo "</body>" >> $RESULT_FILE +echo "</html>" >> $RESULT_FILE + +## copy results +rsync $RSYNC_OPTS html/* $REMOTE_SYSTEM:$REMOTE_DIR +cd ../build +rsync $RSYNC_OPTS index.html *.out $REMOTE_SYSTEM:$REMOTE_DIR/results/ + diff --git a/sys/src/cmd/python/Misc/cheatsheet b/sys/src/cmd/python/Misc/cheatsheet new file mode 100644 index 000000000..4b145ea57 --- /dev/null +++ b/sys/src/cmd/python/Misc/cheatsheet @@ -0,0 +1,2279 @@ + Python 2.3 Quick Reference + + + 25 Jan 2003 upgraded by Raymond Hettinger for Python 2.3 + 16 May 2001 upgraded by Richard Gruet and Simon Brunning for Python 2.0 + 2000/07/18 upgraded by Richard Gruet, rgruet@intraware.com for Python 1.5.2 +from V1.3 ref +1995/10/30, by Chris Hoffmann, choffman@vicorp.com + +Based on: + Python Bestiary, Author: Ken Manheimer, ken.manheimer@nist.gov + Python manuals, Authors: Guido van Rossum and Fred Drake + What's new in Python 2.0, Authors: A.M. Kuchling and Moshe Zadka + python-mode.el, Author: Tim Peters, tim_one@email.msn.com + + and the readers of comp.lang.python + +Python's nest: http://www.python.org Developement: http:// +python.sourceforge.net/ ActivePython : http://www.ActiveState.com/ASPN/ +Python/ +newsgroup: comp.lang.python Help desk: help@python.org +Resources: http://starship.python.net/ + http://www.vex.net/parnassus/ + http://aspn.activestate.com/ASPN/Cookbook/Python +FAQ: http://www.python.org/cgi-bin/faqw.py +Full documentation: http://www.python.org/doc/ +Excellent reference books: + Python Essential Reference by David Beazley (New Riders) + Python Pocket Reference by Mark Lutz (O'Reilly) + + +Invocation Options + +python [-diOStuUvxX?] [-c command | script | - ] [args] + + Invocation Options +Option Effect +-c cmd program passed in as string (terminates option list) +-d Outputs parser debugging information (also PYTHONDEBUG=x) +-E ignore environment variables (such as PYTHONPATH) +-h print this help message and exit +-i Inspect interactively after running script (also PYTHONINSPECT=x) and + force prompts, even if stdin appears not to be a terminal +-O optimize generated bytecode (a tad; also PYTHONOPTIMIZE=x) +-OO remove doc-strings in addition to the -O optimizations +-Q arg division options: -Qold (default), -Qwarn, -Qwarnall, -Qnew +-S Don't perform 'import site' on initialization +-t Issue warnings about inconsistent tab usage (-tt: issue errors) +-u Unbuffered binary stdout and stderr (also PYTHONUNBUFFERED=x). +-v Verbose (trace import statements) (also PYTHONVERBOSE=x) +-W arg : warning control (arg is action:message:category:module:lineno) +-x Skip first line of source, allowing use of non-unix Forms of #!cmd +-? Help! +-c Specify the command to execute (see next section). This terminates the +command option list (following options are passed as arguments to the command). + the name of a python file (.py) to execute read from stdin. +script Anything afterward is passed as options to python script or command, + not interpreted as an option to interpreter itself. +args passed to script or command (in sys.argv[1:]) + If no script or command, Python enters interactive mode. + + * Available IDEs in std distrib: IDLE (tkinter based, portable), Pythonwin + (Windows). + + + +Environment variables + + Environment variables + Variable Effect +PYTHONHOME Alternate prefix directory (or prefix;exec_prefix). The + default module search path uses prefix/lib + Augments the default search path for module files. The format + is the same as the shell's $PATH: one or more directory + pathnames separated by ':' or ';' without spaces around + (semi-)colons! +PYTHONPATH On Windows first search for Registry key HKEY_LOCAL_MACHINE\ + Software\Python\PythonCore\x.y\PythonPath (default value). You + may also define a key named after your application with a + default string value giving the root directory path of your + app. + If this is the name of a readable file, the Python commands in +PYTHONSTARTUP that file are executed before the first prompt is displayed in + interactive mode (no default). +PYTHONDEBUG If non-empty, same as -d option +PYTHONINSPECT If non-empty, same as -i option +PYTHONSUPPRESS If non-empty, same as -s option +PYTHONUNBUFFERED If non-empty, same as -u option +PYTHONVERBOSE If non-empty, same as -v option +PYTHONCASEOK If non-empty, ignore case in file/module names (imports) + + + + +Notable lexical entities + +Keywords + + and del for is raise + assert elif from lambda return + break else global not try + class except if or while + continue exec import pass yield + def finally in print + + * (list of keywords in std module: keyword) + * Illegitimate Tokens (only valid in strings): @ $ ? + * A statement must all be on a single line. To break a statement over + multiple lines use "\", as with the C preprocessor. + Exception: can always break when inside any (), [], or {} pair, or in + triple-quoted strings. + * More than one statement can appear on a line if they are separated with + semicolons (";"). + * Comments start with "#" and continue to end of line. + +Identifiers + + (letter | "_") (letter | digit | "_")* + + * Python identifiers keywords, attributes, etc. are case-sensitive. + * Special forms: _ident (not imported by 'from module import *'); __ident__ + (system defined name); + __ident (class-private name mangling) + +Strings + + "a string enclosed by double quotes" + 'another string delimited by single quotes and with a " inside' + '''a string containing embedded newlines and quote (') marks, can be + delimited with triple quotes.''' + """ may also use 3- double quotes as delimiters """ + u'a unicode string' U"Another unicode string" + r'a raw string where \ are kept (literalized): handy for regular + expressions and windows paths!' + R"another raw string" -- raw strings cannot end with a \ + ur'a unicode raw string' UR"another raw unicode" + + Use \ at end of line to continue a string on next line. + adjacent strings are concatened, e.g. 'Monty' ' Python' is the same as + 'Monty Python'. + u'hello' + ' world' --> u'hello world' (coerced to unicode) + + String Literal Escapes + + \newline Ignored (escape newline) + \\ Backslash (\) \e Escape (ESC) \v Vertical Tab (VT) + \' Single quote (') \f Formfeed (FF) \OOO char with octal value OOO + \" Double quote (") \n Linefeed (LF) + \a Bell (BEL) \r Carriage Return (CR) \xHH char with hex value HH + \b Backspace (BS) \t Horizontal Tab (TAB) + \uHHHH unicode char with hex value HHHH, can only be used in unicode string + \UHHHHHHHH unicode char with hex value HHHHHHHH, can only be used in unicode string + \AnyOtherChar is left as-is + + * NUL byte (\000) is NOT an end-of-string marker; NULs may be embedded in + strings. + * Strings (and tuples) are immutable: they cannot be modified. + +Numbers + + Decimal integer: 1234, 1234567890546378940L (or l) + Octal integer: 0177, 0177777777777777777 (begin with a 0) + Hex integer: 0xFF, 0XFFFFffffFFFFFFFFFF (begin with 0x or 0X) + Long integer (unlimited precision): 1234567890123456 + Float (double precision): 3.14e-10, .001, 10., 1E3 + Complex: 1J, 2+3J, 4+5j (ends with J or j, + separates (float) real and + imaginary parts) + +Sequences + + * String of length 0, 1, 2 (see above) + '', '1', "12", 'hello\n' + * Tuple of length 0, 1, 2, etc: + () (1,) (1,2) # parentheses are optional if len > 0 + * List of length 0, 1, 2, etc: + [] [1] [1,2] + +Indexing is 0-based. Negative indices (usually) mean count backwards from end +of sequence. + +Sequence slicing [starting-at-index : but-less-than-index]. Start defaults to +'0'; End defaults to 'sequence-length'. + +a = (0,1,2,3,4,5,6,7) + a[3] ==> 3 + a[-1] ==> 7 + a[2:4] ==> (2, 3) + a[1:] ==> (1, 2, 3, 4, 5, 6, 7) + a[:3] ==> (0, 1, 2) + a[:] ==> (0,1,2,3,4,5,6,7) # makes a copy of the sequence. + +Dictionaries (Mappings) + + {} # Zero length empty dictionary + {1 : 'first'} # Dictionary with one (key, value) pair + {1 : 'first', 'next': 'second'} + dict([('one',1),('two',2)]) # Construct a dict from an item list + dict('one'=1, 'two'=2) # Construct a dict using keyword args + dict.fromkeys(['one', 'keys']) # Construct a dict from a sequence + +Operators and their evaluation order + + Operators and their evaluation order +Highest Operator Comment + (...) [...] {...} `...` Tuple, list & dict. creation; string + conv. + s[i] s[i:j] s.attr f(...) indexing & slicing; attributes, fct + calls + +x, -x, ~x Unary operators + x**y Power + x*y x/y x%y x//y mult, division, modulo, floor division + x+y x-y addition, subtraction + x<<y x>>y Bit shifting + x&y Bitwise and + x^y Bitwise exclusive or + x|y Bitwise or + x<y x<=y x>y x>=y x==y x!=y Comparison, + x<>y identity, + x is y x is not y membership + x in s x not in s + not x boolean negation + x and y boolean and + x or y boolean or +Lowest lambda args: expr anonymous function + +Alternate names are defined in module operator (e.g. __add__ and add for +) +Most operators are overridable. + +Many binary operators also support augmented assignment: + x += 1 # Same as x = x + 1 + + +Basic Types and Their Operations + +Comparisons (defined between *any* types) + + Comparisons +Comparison Meaning Notes +< strictly less than (1) +<= less than or equal to +> strictly greater than +>= greater than or equal to +== equal to +!= or <> not equal to +is object identity (2) +is not negated object identity (2) + +Notes : + Comparison behavior can be overridden for a given class by defining special +method __cmp__. + The above comparisons return True or False which are of type bool +(a subclass of int) and behave exactly as 1 or 0 except for their type and +that they print as True or False instead of 1 or 0. + (1) X < Y < Z < W has expected meaning, unlike C + (2) Compare object identities (i.e. id(object)), not object values. + +Boolean values and operators + + Boolean values and operators + Value or Operator Returns Notes +None, numeric zeros, empty sequences and False +mappings +all other values True +not x True if x is False, else + True +x or y if x is False then y, else (1) + x +x and y if x is False then x, else (1) + y + +Notes : + Truth testing behavior can be overridden for a given class by defining +special method __nonzero__. + (1) Evaluate second arg only if necessary to determine outcome. + +None + + None is used as default return value on functions. Built-in single object + with type NoneType. + Input that evaluates to None does not print when running Python + interactively. + +Numeric types + +Floats, integers and long integers. + + Floats are implemented with C doubles. + Integers are implemented with C longs. + Long integers have unlimited size (only limit is system resources) + +Operators on all numeric types + + Operators on all numeric types + Operation Result +abs(x) the absolute value of x +int(x) x converted to integer +long(x) x converted to long integer +float(x) x converted to floating point +-x x negated ++x x unchanged +x + y the sum of x and y +x - y difference of x and y +x * y product of x and y +x / y quotient of x and y +x % y remainder of x / y +divmod(x, y) the tuple (x/y, x%y) +x ** y x to the power y (the same as pow(x, y)) + +Bit operators on integers and long integers + + Bit operators +Operation >Result +~x the bits of x inverted +x ^ y bitwise exclusive or of x and y +x & y bitwise and of x and y +x | y bitwise or of x and y +x << n x shifted left by n bits +x >> n x shifted right by n bits + +Complex Numbers + + * represented as a pair of machine-level double precision floating point + numbers. + * The real and imaginary value of a complex number z can be retrieved through + the attributes z.real and z.imag. + +Numeric exceptions + +TypeError + raised on application of arithmetic operation to non-number +OverflowError + numeric bounds exceeded +ZeroDivisionError + raised when zero second argument of div or modulo op +FloatingPointError + raised when a floating point operation fails + +Operations on all sequence types (lists, tuples, strings) + + Operations on all sequence types +Operation Result Notes +x in s True if an item of s is equal to x, else False +x not in s False if an item of s is equal to x, else True +for x in s: loops over the sequence +s + t the concatenation of s and t +s * n, n*s n copies of s concatenated +s[i] i'th item of s, origin 0 (1) +s[i:j] slice of s from i (included) to j (excluded) (1), (2) +len(s) length of s +min(s) smallest item of s +max(s) largest item of (s) +iter(s) returns an iterator over s. iterators define __iter__ and next() + +Notes : + (1) if i or j is negative, the index is relative to the end of the string, +ie len(s)+ i or len(s)+j is + substituted. But note that -0 is still 0. + (2) The slice of s from i to j is defined as the sequence of items with +index k such that i <= k < j. + If i or j is greater than len(s), use len(s). If i is omitted, use +len(s). If i is greater than or + equal to j, the slice is empty. + +Operations on mutable (=modifiable) sequences (lists) + + Operations on mutable sequences + Operation Result Notes +s[i] =x item i of s is replaced by x +s[i:j] = t slice of s from i to j is replaced by t +del s[i:j] same as s[i:j] = [] +s.append(x) same as s[len(s) : len(s)] = [x] +s.count(x) return number of i's for which s[i] == x +s.extend(x) same as s[len(s):len(s)]= x +s.index(x) return smallest i such that s[i] == x (1) +s.insert(i, x) same as s[i:i] = [x] if i >= 0 +s.pop([i]) same as x = s[i]; del s[i]; return x (4) +s.remove(x) same as del s[s.index(x)] (1) +s.reverse() reverse the items of s in place (3) +s.sort([cmpFct]) sort the items of s in place (2), (3) + +Notes : + (1) raise a ValueError exception when x is not found in s (i.e. out of +range). + (2) The sort() method takes an optional argument specifying a comparison +fct of 2 arguments (list items) which should + return -1, 0, or 1 depending on whether the 1st argument is +considered smaller than, equal to, or larger than the 2nd + argument. Note that this slows the sorting process down considerably. + (3) The sort() and reverse() methods modify the list in place for economy +of space when sorting or reversing a large list. + They don't return the sorted or reversed list to remind you of this +side effect. + (4) [New 1.5.2] The optional argument i defaults to -1, so that by default the last +item is removed and returned. + + + +Operations on mappings (dictionaries) + + Operations on mappings + Operation Result Notes +len(d) the number of items in d +d[k] the item of d with key k (1) +d[k] = x set d[k] to x +del d[k] remove d[k] from d (1) +d.clear() remove all items from d +d.copy() a shallow copy of d +d.get(k,defaultval) the item of d with key k (4) +d.has_key(k) True if d has key k, else False +d.items() a copy of d's list of (key, item) pairs (2) +d.iteritems() an iterator over (key, value) pairs (7) +d.iterkeys() an iterator over the keys of d (7) +d.itervalues() an iterator over the values of d (7) +d.keys() a copy of d's list of keys (2) +d1.update(d2) for k, v in d2.items(): d1[k] = v (3) +d.values() a copy of d's list of values (2) +d.pop(k) remove d[k] and return its value +d.popitem() remove and return an arbitrary (6) + (key, item) pair +d.setdefault(k,defaultval) the item of d with key k (5) + + Notes : + TypeError is raised if key is not acceptable + (1) KeyError is raised if key k is not in the map + (2) Keys and values are listed in random order + (3) d2 must be of the same type as d1 + (4) Never raises an exception if k is not in the map, instead it returns + defaultVal. + defaultVal is optional, when not provided and k is not in the map, + None is returned. + (5) Never raises an exception if k is not in the map, instead it returns + defaultVal, and adds k to map with value defaultVal. defaultVal is + optional. When not provided and k is not in the map, None is returned and + added to map. + (6) Raises a KeyError if the dictionary is emtpy. + (7) While iterating over a dictionary, the values may be updated but + the keys cannot be changed. + +Operations on strings + +Note that these string methods largely (but not completely) supersede the +functions available in the string module. + + + Operations on strings + Operation Result Notes +s.capitalize() return a copy of s with only its first character + capitalized. +s.center(width) return a copy of s centered in a string of length width (1) + . +s.count(sub[ return the number of occurrences of substring sub in (2) +,start[,end]]) string s. +s.decode(([ return a decoded version of s. (3) + encoding + [,errors]]) +s.encode([ return an encoded version of s. Default encoding is the + encoding current default string encoding. (3) + [,errors]]) +s.endswith(suffix return true if s ends with the specified suffix, (2) + [,start[,end]]) otherwise return False. +s.expandtabs([ return a copy of s where all tab characters are (4) +tabsize]) expanded using spaces. +s.find(sub[,start return the lowest index in s where substring sub is (2) +[,end]]) found. Return -1 if sub is not found. +s.index(sub[ like find(), but raise ValueError when the substring is (2) +,start[,end]]) not found. +s.isalnum() return True if all characters in s are alphanumeric, (5) + False otherwise. +s.isalpha() return True if all characters in s are alphabetic, (5) + False otherwise. +s.isdigit() return True if all characters in s are digit (5) + characters, False otherwise. +s.islower() return True if all characters in s are lowercase, False (6) + otherwise. +s.isspace() return True if all characters in s are whitespace (5) + characters, False otherwise. +s.istitle() return True if string s is a titlecased string, False (7) + otherwise. +s.isupper() return True if all characters in s are uppercase, False (6) + otherwise. +s.join(seq) return a concatenation of the strings in the sequence + seq, seperated by 's's. +s.ljust(width) return s left justified in a string of length width. (1), + (8) +s.lower() return a copy of s converted to lowercase. +s.lstrip() return a copy of s with leading whitespace removed. +s.replace(old, return a copy of s with all occurrences of substring (9) +new[, maxsplit]) old replaced by new. +s.rfind(sub[ return the highest index in s where substring sub is (2) +,start[,end]]) found. Return -1 if sub is not found. +s.rindex(sub[ like rfind(), but raise ValueError when the substring (2) +,start[,end]]) is not found. +s.rjust(width) return s right justified in a string of length width. (1), + (8) +s.rstrip() return a copy of s with trailing whitespace removed. +s.split([sep[ return a list of the words in s, using sep as the (10) +,maxsplit]]) delimiter string. +s.splitlines([ return a list of the lines in s, breaking at line (11) +keepends]) boundaries. +s.startswith return true if s starts with the specified prefix, +(prefix[,start[ otherwise return false. (2) +,end]]) +s.strip() return a copy of s with leading and trailing whitespace + removed. +s.swapcase() return a copy of s with uppercase characters converted + to lowercase and vice versa. + return a titlecased copy of s, i.e. words start with +s.title() uppercase characters, all remaining cased characters + are lowercase. +s.translate(table return a copy of s mapped through translation table (12) +[,deletechars]) table. +s.upper() return a copy of s converted to uppercase. +s.zfill(width) return a string padded with zeroes on the left side and + sliding a minus sign left if necessary. never truncates. + +Notes : + (1) Padding is done using spaces. + (2) If optional argument start is supplied, substring s[start:] is +processed. If optional arguments start and end are supplied, substring s[start: +end] is processed. + (3) Optional argument errors may be given to set a different error handling +scheme. The default for errors is 'strict', meaning that encoding errors raise +a ValueError. Other possible values are 'ignore' and 'replace'. + (4) If optional argument tabsize is not given, a tab size of 8 characters +is assumed. + (5) Returns false if string s does not contain at least one character. + (6) Returns false if string s does not contain at least one cased +character. + (7) A titlecased string is a string in which uppercase characters may only +follow uncased characters and lowercase characters only cased ones. + (8) s is returned if width is less than len(s). + (9) If the optional argument maxsplit is given, only the first maxsplit +occurrences are replaced. + (10) If sep is not specified or None, any whitespace string is a separator. +If maxsplit is given, at most maxsplit splits are done. + (11) Line breaks are not included in the resulting list unless keepends is +given and true. + (12) table must be a string of length 256. All characters occurring in the +optional argument deletechars are removed prior to translation. + +String formatting with the % operator + +formatString % args--> evaluates to a string + + * formatString uses C printf format codes : %, c, s, i, d, u, o, x, X, e, E, + f, g, G, r (details below). + * Width and precision may be a * to specify that an integer argument gives + the actual width or precision. + * The flag characters -, +, blank, # and 0 are understood. (details below) + * %s will convert any type argument to string (uses str() function) + * args may be a single arg or a tuple of args + + '%s has %03d quote types.' % ('Python', 2) # => 'Python has 002 quote types.' + + * Right-hand-side can also be a mapping: + + a = '%(lang)s has %(c)03d quote types.' % {'c':2, 'lang':'Python} +(vars() function very handy to use on right-hand-side.) + + Format codes +Conversion Meaning +d Signed integer decimal. +i Signed integer decimal. +o Unsigned octal. +u Unsigned decimal. +x Unsigned hexidecimal (lowercase). +X Unsigned hexidecimal (uppercase). +e Floating point exponential format (lowercase). +E Floating point exponential format (uppercase). +f Floating point decimal format. +F Floating point decimal format. +g Same as "e" if exponent is greater than -4 or less than precision, + "f" otherwise. +G Same as "E" if exponent is greater than -4 or less than precision, + "F" otherwise. +c Single character (accepts integer or single character string). +r String (converts any python object using repr()). +s String (converts any python object using str()). +% No argument is converted, results in a "%" character in the result. + (The complete specification is %%.) + + Conversion flag characters +Flag Meaning +# The value conversion will use the ``alternate form''. +0 The conversion will be zero padded. +- The converted value is left adjusted (overrides "-"). + (a space) A blank should be left before a positive number (or empty + string) produced by a signed conversion. ++ A sign character ("+" or "-") will precede the conversion (overrides a + "space" flag). + +File Objects + +Created with built-in function open; may be created by other modules' functions +as well. + +Operators on file objects + + File operations + Operation Result +f.close() Close file f. +f.fileno() Get fileno (fd) for file f. +f.flush() Flush file f's internal buffer. +f.isatty() True if file f is connected to a tty-like dev, else False. +f.read([size]) Read at most size bytes from file f and return as a string + object. If size omitted, read to EOF. +f.readline() Read one entire line from file f. +f.readlines() Read until EOF with readline() and return list of lines read. + Set file f's position, like "stdio's fseek()". +f.seek(offset[, whence == 0 then use absolute indexing. +whence=0]) whence == 1 then offset relative to current pos. + whence == 2 then offset relative to file end. +f.tell() Return file f's current position (byte offset). +f.write(str) Write string to file f. +f.writelines(list Write list of strings to file f. +) + +File Exceptions + + EOFError + End-of-file hit when reading (may be raised many times, e.g. if f is a + tty). + IOError + Other I/O-related I/O operation failure. + OSError + OS system call failed. + + + Advanced Types + + -See manuals for more details - + + Module objects + + Class objects + + Class instance objects + + Type objects (see module: types) + + File objects (see above) + + Slice objects + + XRange objects + + Callable types: + o User-defined (written in Python): + # User-defined Function objects + # User-defined Method objects + o Built-in (written in C): + # Built-in Function objects + # Built-in Method objects + + Internal Types: + o Code objects (byte-compile executable Python code: bytecode) + o Frame objects (execution frames) + o Traceback objects (stack trace of an exception) + + + Statements + + pass -- Null statement + del name[,name]* -- Unbind name(s) from object. Object will be indirectly + (and automatically) deleted only if no longer referenced. + print [>> fileobject,] [s1 [, s2 ]* [,] + -- Writes to sys.stdout, or to fileobject if supplied. + Puts spaces between arguments. Puts newline at end + unless statement ends with comma. + Print is not required when running interactively, + simply typing an expression will print its value, + unless the value is None. + exec x [in globals [,locals]] + -- Executes x in namespaces provided. Defaults + to current namespaces. x can be a string, file + object or a function object. + callable(value,... [id=value], [*args], [**kw]) + -- Call function callable with parameters. Parameters can + be passed by name or be omitted if function + defines default values. E.g. if callable is defined as + "def callable(p1=1, p2=2)" + "callable()" <=> "callable(1, 2)" + "callable(10)" <=> "callable(10, 2)" + "callable(p2=99)" <=> "callable(1, 99)" + *args is a tuple of positional arguments. + **kw is a dictionary of keyword arguments. + + Assignment operators + + Caption + Operator Result Notes + a = b Basic assignment - assign object b to label a (1) + a += b Roughly equivalent to a = a + b (2) + a -= b Roughly equivalent to a = a - b (2) + a *= b Roughly equivalent to a = a * b (2) + a /= b Roughly equivalent to a = a / b (2) + a %= b Roughly equivalent to a = a % b (2) + a **= b Roughly equivalent to a = a ** b (2) + a &= b Roughly equivalent to a = a & b (2) + a |= b Roughly equivalent to a = a | b (2) + a ^= b Roughly equivalent to a = a ^ b (2) + a >>= b Roughly equivalent to a = a >> b (2) + a <<= b Roughly equivalent to a = a << b (2) + + Notes : + (1) Can unpack tuples, lists, and strings. + first, second = a[0:2]; [f, s] = range(2); c1,c2,c3='abc' + Tip: x,y = y,x swaps x and y. + (2) Not exactly equivalent - a is evaluated only once. Also, where + possible, operation performed in-place - a is modified rather than + replaced. + + Control Flow + + if condition: suite + [elif condition: suite]* + [else: suite] -- usual if/else_if/else statement + while condition: suite + [else: suite] + -- usual while statement. "else" suite is executed + after loop exits, unless the loop is exited with + "break" + for element in sequence: suite + [else: suite] + -- iterates over sequence, assigning each element to element. + Use built-in range function to iterate a number of times. + "else" suite executed at end unless loop exited + with "break" + break -- immediately exits "for" or "while" loop + continue -- immediately does next iteration of "for" or "while" loop + return [result] -- Exits from function (or method) and returns result (use a tuple to + return more than one value). If no result given, then returns None. + yield result -- Freezes the execution frame of a generator and returns the result + to the iterator's .next() method. Upon the next call to next(), + resumes execution at the frozen point with all of the local variables + still intact. + + Exception Statements + + assert expr[, message] + -- expr is evaluated. if false, raises exception AssertionError + with message. Inhibited if __debug__ is 0. + try: suite1 + [except [exception [, value]: suite2]+ + [else: suite3] + -- statements in suite1 are executed. If an exception occurs, look + in "except" clauses for matching <exception>. If matches or bare + "except" execute suite of that clause. If no exception happens + suite in "else" clause is executed after suite1. + If exception has a value, it is put in value. + exception can also be tuple of exceptions, e.g. + "except (KeyError, NameError), val: print val" + try: suite1 + finally: suite2 + -- statements in suite1 are executed. If no + exception, execute suite2 (even if suite1 is + exited with a "return", "break" or "continue" + statement). If exception did occur, executes + suite2 and then immediately reraises exception. + raise exception [,value [, traceback]] + -- raises exception with optional value + value. Arg traceback specifies a traceback object to + use when printing the exception's backtrace. + raise -- a raise statement without arguments re-raises + the last exception raised in the current function +An exception is either a string (object) or a class instance. + Can create a new one simply by creating a new string: + + my_exception = 'You did something wrong' + try: + if bad: + raise my_exception, bad + except my_exception, value: + print 'Oops', value + +Exception classes must be derived from the predefined class: Exception, e.g.: + class text_exception(Exception): pass + try: + if bad: + raise text_exception() + # This is a shorthand for the form + # "raise <class>, <instance>" + except Exception: + print 'Oops' + # This will be printed because + # text_exception is a subclass of Exception +When an error message is printed for an unhandled exception which is a +class, the class name is printed, then a colon and a space, and +finally the instance converted to a string using the built-in function +str(). +All built-in exception classes derives from StandardError, itself +derived from Exception. + +Name Space Statements + +[1.51: On Mac & Windows, the case of module file names must now match the case +as used + in the import statement] +Packages (>1.5): a package is a name space which maps to a directory including + module(s) and the special initialization module '__init__.py' + (possibly empty). Packages/dirs can be nested. You address a + module's symbol via '[package.[package...]module.symbol's. +import module1 [as name1] [, module2]* + -- imports modules. Members of module must be + referred to by qualifying with [package.]module name: + "import sys; print sys.argv:" + "import package1.subpackage.module; package1.subpackage.module.foo()" + module1 renamed as name1, if supplied. +from module import name1 [as othername1] [, name2]* + -- imports names from module module in current namespace. + "from sys import argv; print argv" + "from package1 import module; module.foo()" + "from package1.module import foo; foo()" + name1 renamed as othername1, if supplied. +from module import * + -- imports all names in module, except those starting with "_"; + *to be used sparsely, beware of name clashes* : + "from sys import *; print argv" + "from package.module import *; print x' + NB: "from package import *" only imports the symbols defined + in the package's __init__.py file, not those in the + template modules! +global name1 [, name2]* + -- names are from global scope (usually meaning from module) + rather than local (usually meaning only in function). + -- E.g. in fct without "global" statements, assuming + "a" is name that hasn't been used in fct or module + so far: + -Try to read from "a" -> NameError + -Try to write to "a" -> creates "a" local to fcn + -If "a" not defined in fct, but is in module, then + -Try to read from "a", gets value from module + -Try to write to "a", creates "a" local to fct + But note "a[0]=3" starts with search for "a", + will use to global "a" if no local "a". + +Function Definition + +def func_id ([param_list]): suite + -- Creates a function object & binds it to name func_id. + + param_list ::= [id [, id]*] + id ::= value | id = value | *id | **id + [Args are passed by value.Thus only args representing a mutable object + can be modified (are inout parameters). Use a tuple to return more than + one value] + +Example: + def test (p1, p2 = 1+1, *rest, **keywords): + -- Parameters with "=" have default value (v is + evaluated when function defined). + If list has "*id" then id is assigned a tuple of + all remaining args passed to function (like C vararg) + If list has "**id" then id is assigned a dictionary of + all extra arguments passed as keywords. + +Class Definition + +class <class_id> [(<super_class1> [,<super_class2>]*)]: <suite> + -- Creates a class object and assigns it name <class_id> + <suite> may contain local "defs" of class methods and + assignments to class attributes. +Example: + class my_class (class1, class_list[3]): ... + Creates a class object inheriting from both "class1" and whatever + class object "class_list[3]" evaluates to. Assigns new + class object to name "my_class". + - First arg to class methods is always instance object, called 'self' + by convention. + - Special method __init__() is called when instance is created. + - Special method __del__() called when no more reference to object. + - Create instance by "calling" class object, possibly with arg + (thus instance=apply(aClassObject, args...) creates an instance!) + - In current implementation, can't subclass off built-in + classes. But can "wrap" them, see UserDict & UserList modules, + and see __getattr__() below. +Example: + class c (c_parent): + def __init__(self, name): self.name = name + def print_name(self): print "I'm", self.name + def call_parent(self): c_parent.print_name(self) + instance = c('tom') + print instance.name + 'tom' + instance.print_name() + "I'm tom" + Call parent's super class by accessing parent's method + directly and passing "self" explicitly (see "call_parent" + in example above). + Many other special methods available for implementing + arithmetic operators, sequence, mapping indexing, etc. + +Documentation Strings + +Modules, classes and functions may be documented by placing a string literal by +itself as the first statement in the suite. The documentation can be retrieved +by getting the '__doc__' attribute from the module, class or function. +Example: + class C: + "A description of C" + def __init__(self): + "A description of the constructor" + # etc. +Then c.__doc__ == "A description of C". +Then c.__init__.__doc__ == "A description of the constructor". + +Others + +lambda [param_list]: returnedExpr + -- Creates an anonymous function. returnedExpr must be + an expression, not a statement (e.g., not "if xx:...", + "print xxx", etc.) and thus can't contain newlines. + Used mostly for filter(), map(), reduce() functions, and GUI callbacks.. +List comprehensions +result = [expression for item1 in sequence1 [if condition1] + [for item2 in sequence2 ... for itemN in sequenceN] + ] +is equivalent to: +result = [] +for item1 in sequence1: + for item2 in sequence2: + ... + for itemN in sequenceN: + if (condition1) and furthur conditions: + result.append(expression) + + + +Built-In Functions + + Built-In Functions + Function Result +__import__(name[, Imports module within the given context (see lib ref for +globals[, locals[, more details) +fromlist]]]) +abs(x) Return the absolute value of number x. +apply(f, args[, Calls func/method f with arguments args and optional +keywords]) keywords. +bool(x) Returns True when the argument x is true and False otherwise. +buffer(obj) Creates a buffer reference to an object. +callable(x) Returns True if x callable, else False. +chr(i) Returns one-character string whose ASCII code isinteger i +classmethod(f) Converts a function f, into a method with the class as the + first argument. Useful for creating alternative constructors. +cmp(x,y) Returns negative, 0, positive if x <, ==, > to y +coerce(x,y) Returns a tuple of the two numeric arguments converted to a + common type. + Compiles string into a code object.filename is used in + error message, can be any string. It isusually the file +compile(string, from which the code was read, or eg. '<string>'if not read +filename, kind) from file.kind can be 'eval' if string is a single stmt, or + 'single' which prints the output of expression statements + thatevaluate to something else than None, or be 'exec'. +complex(real[, Builds a complex object (can also be done using J or j +image]) suffix,e.g. 1+3J) +delattr(obj, name) deletes attribute named name of object obj <=> del obj.name + If no args, returns the list of names in current +dict([items]) Create a new dictionary from the specified item list. +dir([object]) localsymbol table. With a module, class or class + instanceobject as arg, returns list of names in its attr. + dict. +divmod(a,b) Returns tuple of (a/b, a%b) +enumerate(seq) Return a iterator giving: (0, seq[0]), (1, seq[1]), ... +eval(s[, globals[, Eval string s in (optional) globals, locals contexts.s must +locals]]) have no NUL's or newlines. s can also be acode object. + Example: x = 1; incr_x = eval('x + 1') +execfile(file[, Executes a file without creating a new module, unlike +globals[, locals]]) import. +file() Synonym for open(). +filter(function, Constructs a list from those elements of sequence for which +sequence) function returns true. function takes one parameter. +float(x) Converts a number or a string to floating point. +getattr(object, [<default> arg added in 1.5.2]Gets attribute called name +name[, default])) from object,e.g. getattr(x, 'f') <=> x.f). If not found, + raisesAttributeError or returns default if specified. +globals() Returns a dictionary containing current global variables. +hasattr(object, Returns true if object has attr called name. +name) +hash(object) Returns the hash value of the object (if it has one) +help(f) Display documentation on object f. +hex(x) Converts a number x to a hexadecimal string. +id(object) Returns a unique 'identity' integer for an object. +input([prompt]) Prints prompt if given. Reads input and evaluates it. + Converts a number or a string to a plain integer. Optional +int(x[, base]) base paramenter specifies base from which to convert string + values. +intern(aString) Enters aString in the table of "interned strings" + andreturns the string. Interned strings are 'immortals'. +isinstance(obj, returns true if obj is an instance of class. Ifissubclass +class) (A,B) then isinstance(x,A) => isinstance(x,B) +issubclass(class1, returns true if class1 is derived from class2 +class2) + Returns the length (the number of items) of an object +iter(collection) Returns an iterator over the collection. +len(obj) (sequence, dictionary, or instance of class implementing + __len__). +list(sequence) Converts sequence into a list. If already a list,returns a + copy of it. +locals() Returns a dictionary containing current local variables. + Converts a number or a string to a long integer. Optional +long(x[, base]) base paramenter specifies base from which to convert string + values. + Applies function to every item of list and returns a listof +map(function, list, the results. If additional arguments are passed,function +...) must take that many arguments and it is givento function on + each call. +max(seq) Returns the largest item of the non-empty sequence seq. +min(seq) Returns the smallest item of a non-empty sequence seq. +oct(x) Converts a number to an octal string. +open(filename [, Returns a new file object. First two args are same asthose +mode='r', [bufsize= for C's "stdio open" function. bufsize is 0for unbuffered, +implementation 1 for line-buffered, negative forsys-default, all else, of +dependent]]) (about) given size. +ord(c) Returns integer ASCII value of c (a string of len 1). Works + with Unicode char. +object() Create a base type. Used as a superclass for new-style objects. +open(name Open a file. + [, mode + [, buffering]]) +pow(x, y [, z]) Returns x to power y [modulo z]. See also ** operator. +property() Created a property with access controlled by functions. +range(start [,end Returns list of ints from >= start and < end.With 1 arg, +[, step]]) list from 0..arg-1With 2 args, list from start..end-1With 3 + args, list from start up to end by step +raw_input([prompt]) Prints prompt if given, then reads string from stdinput (no + trailing \n). See also input(). +reduce(f, list [, Applies the binary function f to the items oflist so as to +init]) reduce the list to a single value.If init given, it is + "prepended" to list. + Re-parses and re-initializes an already imported module. + Useful in interactive mode, if you want to reload amodule +reload(module) after fixing it. If module was syntacticallycorrect but had + an error in initialization, mustimport it one more time + before calling reload(). + Returns a string containing a printable and if possible +repr(object) evaluable representation of an object. <=> `object` + (usingbackquotes). Class redefinissable (__repr__). See + also str() +round(x, n=0) Returns the floating point value x rounded to n digitsafter + the decimal point. +setattr(object, This is the counterpart of getattr().setattr(o, 'foobar', +name, value) 3) <=> o.foobar = 3Creates attribute if it doesn't exist! +slice([start,] stop Returns a slice object representing a range, with R/ +[, step]) Oattributes: start, stop, step. + Returns a string containing a nicely +staticmethod() Convert a function to method with no self or class + argument. Useful for methods associated with a class that + do not need access to an object's internal state. +str(object) printablerepresentation of an object. Class overridable + (__str__).See also repr(). +super(type) Create an unbound super object. Used to call cooperative + superclass methods. +sum(sequence, Add the values in the sequence and return the sum. + [start]) +tuple(sequence) Creates a tuple with same elements as sequence. If already + a tuple, return itself (not a copy). + Returns a type object [see module types] representing + thetype of obj. Example: import typesif type(x) == +type(obj) types.StringType: print 'It is a string'NB: it is + recommanded to use the following form:if isinstance(x, + types.StringType): etc... +unichr(code) code. +unicode(string[, Creates a Unicode string from a 8-bit string, using +encoding[, error thegiven encoding name and error treatment ('strict', +]]]) 'ignore',or 'replace'}. + Without arguments, returns a dictionary correspondingto the + current local symbol table. With a module,class or class +vars([object]) instance object as argumentreturns a dictionary + corresponding to the object'ssymbol table. Useful with "%" + formatting operator. +xrange(start [, end Like range(), but doesn't actually store entire listall at +[, step]]) once. Good to use in "for" loops when there is abig range + and little memory. +zip(seq1[, seq2, Returns a list of tuples where each tuple contains the nth +...]) element of each of the argument sequences. + + + + +Built-In Exceptions + +Exception> + Root class for all exceptions + SystemExit + On 'sys.exit()' + StopIteration + Signal the end from iterator.next() + StandardError + Base class for all built-in exceptions; derived from Exception + root class. + ArithmeticError + Base class for OverflowError, ZeroDivisionError, + FloatingPointError + FloatingPointError + When a floating point operation fails. + OverflowError + On excessively large arithmetic operation + ZeroDivisionError + On division or modulo operation with 0 as 2nd arg + AssertionError + When an assert statement fails. + AttributeError + On attribute reference or assignment failure + EnvironmentError [new in 1.5.2] + On error outside Python; error arg tuple is (errno, errMsg...) + IOError [changed in 1.5.2] + I/O-related operation failure + OSError [new in 1.5.2] + used by the os module's os.error exception. + EOFError + Immediate end-of-file hit by input() or raw_input() + ImportError + On failure of `import' to find module or name + KeyboardInterrupt + On user entry of the interrupt key (often `Control-C') + LookupError + base class for IndexError, KeyError + IndexError + On out-of-range sequence subscript + KeyError + On reference to a non-existent mapping (dict) key + MemoryError + On recoverable memory exhaustion + NameError + On failure to find a local or global (unqualified) name + RuntimeError + Obsolete catch-all; define a suitable error instead + NotImplementedError [new in 1.5.2] + On method not implemented + SyntaxError + On parser encountering a syntax error + IndentationError + On parser encountering an indentation syntax error + TabError + On parser encountering an indentation syntax error + SystemError + On non-fatal interpreter error - bug - report it + TypeError + On passing inappropriate type to built-in op or func + ValueError + On arg error not covered by TypeError or more precise + Warning + UserWarning + DeprecationWarning + PendingDeprecationWarning + SyntaxWarning + RuntimeWarning + FutureWarning + + + +Standard methods & operators redefinition in classes + +Standard methods & operators map to special '__methods__' and thus may be + redefined (mostly in in user-defined classes), e.g.: + class x: + def __init__(self, v): self.value = v + def __add__(self, r): return self.value + r + a = x(3) # sort of like calling x.__init__(a, 3) + a + 4 # is equivalent to a.__add__(4) + +Special methods for any class + +(s: self, o: other) + __init__(s, args) instance initialization (on construction) + __del__(s) called on object demise (refcount becomes 0) + __repr__(s) repr() and `...` conversions + __str__(s) str() and 'print' statement + __cmp__(s, o) Compares s to o and returns <0, 0, or >0. + Implements >, <, == etc... + __hash__(s) Compute a 32 bit hash code; hash() and dictionary ops + __nonzero__(s) Returns False or True for truth value testing + __getattr__(s, name) called when attr lookup doesn't find <name> + __setattr__(s, name, val) called when setting an attr + (inside, don't use "self.name = value" + use "self.__dict__[name] = val") + __delattr__(s, name) called to delete attr <name> + __call__(self, *args) called when an instance is called as function. + +Operators + + See list in the operator module. Operator function names are provided with + 2 variants, with or without + ading & trailing '__' (eg. __add__ or add). + + Numeric operations special methods + (s: self, o: other) + + s+o = __add__(s,o) s-o = __sub__(s,o) + s*o = __mul__(s,o) s/o = __div__(s,o) + s%o = __mod__(s,o) divmod(s,o) = __divmod__(s,o) + s**o = __pow__(s,o) + s&o = __and__(s,o) + s^o = __xor__(s,o) s|o = __or__(s,o) + s<<o = __lshift__(s,o) s>>o = __rshift__(s,o) + nonzero(s) = __nonzero__(s) (used in boolean testing) + -s = __neg__(s) +s = __pos__(s) + abs(s) = __abs__(s) ~s = __invert__(s) (bitwise) + s+=o = __iadd__(s,o) s-=o = __isub__(s,o) + s*=o = __imul__(s,o) s/=o = __idiv__(s,o) + s%=o = __imod__(s,o) + s**=o = __ipow__(s,o) + s&=o = __iand__(s,o) + s^=o = __ixor__(s,o) s|=o = __ior__(s,o) + s<<=o = __ilshift__(s,o) s>>=o = __irshift__(s,o) + Conversions + int(s) = __int__(s) long(s) = __long__(s) + float(s) = __float__(s) complex(s) = __complex__(s) + oct(s) = __oct__(s) hex(s) = __hex__(s) + coerce(s,o) = __coerce__(s,o) + Right-hand-side equivalents for all binary operators exist; + are called when class instance is on r-h-s of operator: + a + 3 calls __add__(a, 3) + 3 + a calls __radd__(a, 3) + + All seqs and maps, general operations plus: + (s: self, i: index or key) + + len(s) = __len__(s) length of object, >= 0. Length 0 == false + s[i] = __getitem__(s,i) Element at index/key i, origin 0 + + Sequences, general methods, plus: + s[i]=v = __setitem__(s,i,v) + del s[i] = __delitem__(s,i) + s[i:j] = __getslice__(s,i,j) + s[i:j]=seq = __setslice__(s,i,j,seq) + del s[i:j] = __delslice__(s,i,j) == s[i:j] = [] + seq * n = __repeat__(seq, n) + s1 + s2 = __concat__(s1, s2) + i in s = __contains__(s, i) + Mappings, general methods, plus + hash(s) = __hash__(s) - hash value for dictionary references + s[k]=v = __setitem__(s,k,v) + del s[k] = __delitem__(s,k) + +Special informative state attributes for some types: + + Modules: + __doc__ (string/None, R/O): doc string (<=> __dict__['__doc__']) + __name__(string, R/O): module name (also in __dict__['__name__']) + __dict__ (dict, R/O): module's name space + __file__(string/undefined, R/O): pathname of .pyc, .pyo or .pyd (undef for + modules statically linked to the interpreter) + + Classes: [in bold: writable since 1.5.2] + __doc__ (string/None, R/W): doc string (<=> __dict__['__doc__']) + __module__ is the module name in which the class was defined + __name__(string, R/W): class name (also in __dict__['__name__']) + __bases__ (tuple, R/W): parent classes + __dict__ (dict, R/W): attributes (class name space) + + Instances: + __class__ (class, R/W): instance's class + __dict__ (dict, R/W): attributes + + User-defined functions: [bold: writable since 1.5.2] + __doc__ (string/None, R/W): doc string + __name__(string, R/O): function name + func_doc (R/W): same as __doc__ + func_name (R/O): same as __name__ + func_defaults (tuple/None, R/W): default args values if any + func_code (code, R/W): code object representing the compiled function body + func_globals (dict, R/O): ref to dictionary of func global variables + func_dict (dict, R/W): same as __dict__ contains the namespace supporting + arbitrary function attributes + func_closure (R/O): None or a tuple of cells that contain bindings + for the function's free variables. + + + User-defined Methods: + __doc__ (string/None, R/O): doc string + __name__(string, R/O): method name (same as im_func.__name__) + im_class (class, R/O): class defining the method (may be a base class) + im_self (instance/None, R/O): target instance object (None if unbound) + im_func (function, R/O): function object + + Built-in Functions & methods: + __doc__ (string/None, R/O): doc string + __name__ (string, R/O): function name + __self__ : [methods only] target object + + Codes: + co_name (string, R/O): function name + co_argcount (int, R/0): number of positional args + co_nlocals (int, R/O): number of local vars (including args) + co_varnames (tuple, R/O): names of local vars (starting with args) + co_cellvars (tuple, R/O)) the names of local variables referenced by + nested functions + co_freevars (tuple, R/O)) names of free variables + co_code (string, R/O): sequence of bytecode instructions + co_consts (tuple, R/O): litterals used by the bytecode, 1st one is + fct doc (or None) + co_names (tuple, R/O): names used by the bytecode + co_filename (string, R/O): filename from which the code was compiled + co_firstlineno (int, R/O): first line number of the function + co_lnotab (string, R/O): string encoding bytecode offsets to line numbers. + co_stacksize (int, R/O): required stack size (including local vars) + co_flags (int, R/O): flags for the interpreter + bit 2 set if fct uses "*arg" syntax + bit 3 set if fct uses '**keywords' syntax + Frames: + f_back (frame/None, R/O): previous stack frame (toward the caller) + f_code (code, R/O): code object being executed in this frame + f_locals (dict, R/O): local vars + f_globals (dict, R/O): global vars + f_builtins (dict, R/O): built-in (intrinsic) names + f_restricted (int, R/O): flag indicating whether fct is executed in + restricted mode + f_lineno (int, R/O): current line number + f_lasti (int, R/O): precise instruction (index into bytecode) + f_trace (function/None, R/W): debug hook called at start of each source line + f_exc_type (Type/None, R/W): Most recent exception type + f_exc_value (any, R/W): Most recent exception value + f_exc_traceback (traceback/None, R/W): Most recent exception traceback + Tracebacks: + tb_next (frame/None, R/O): next level in stack trace (toward the frame where + the exception occurred) + tb_frame (frame, R/O): execution frame of the current level + tb_lineno (int, R/O): line number where the exception occurred + tb_lasti (int, R/O): precise instruction (index into bytecode) + + Slices: + start (any/None, R/O): lowerbound + stop (any/None, R/O): upperbound + step (any/None, R/O): step value + + Complex numbers: + real (float, R/O): real part + imag (float, R/O): imaginary part + + +Important Modules + + sys + + Some sys variables + Variable Content +argv The list of command line arguments passed to aPython + script. sys.argv[0] is the script name. +builtin_module_names A list of strings giving the names of all moduleswritten + in C that are linked into this interpreter. +check_interval How often to check for thread switches or signals(measured + in number of virtual machine instructions) +exc_type, exc_value, Deprecated since release 1.5. Use exc_info() instead. +exc_traceback +exitfunc User can set to a parameterless fcn. It will getcalled + before interpreter exits. +last_type, Set only when an exception not handled andinterpreter +last_value, prints an error. Used by debuggers. +last_traceback +maxint maximum positive value for integers +modules Dictionary of modules that have already been loaded. +path Search path for external modules. Can be modifiedby + program. sys.path[0] == dir of script executing +platform The current platform, e.g. "sunos5", "win32" +ps1, ps2 prompts to use in interactive mode. + File objects used for I/O. One can redirect byassigning a +stdin, stdout, new file object to them (or any object:.with a method +stderr write(string) for stdout/stderr,.with a method readline() + for stdin) +version string containing version info about Python interpreter. + (and also: copyright, dllhandle, exec_prefix, prefix) +version_info tuple containing Python version info - (major, minor, + micro, level, serial). + + Some sys functions + Function Result +exit(n) Exits with status n. Raises SystemExit exception.(Hence can + be caught and ignored by program) +getrefcount(object Returns the reference count of the object. Generally one +) higher than you might expect, because of object arg temp + reference. +setcheckinterval( Sets the interpreter's thread switching interval (in number +interval) of virtual code instructions, default:100). +settrace(func) Sets a trace function: called before each line ofcode is + exited. +setprofile(func) Sets a profile function for performance profiling. + Info on exception currently being handled; this is atuple + (exc_type, exc_value, exc_traceback).Warning: assigning the +exc_info() traceback return value to a loca variable in a + function handling an exception will cause a circular + reference. +setdefaultencoding Change default Unicode encoding - defaults to 7-bit ASCII. +(encoding) +getrecursionlimit Retrieve maximum recursion depth. +() +setrecursionlimit Set maximum recursion depth. (Defaults to 1000.) +() + + + + os +"synonym" for whatever O/S-specific module is proper for current environment. +this module uses posix whenever possible. +(see also M.A. Lemburg's utility http://www.lemburg.com/files/python/ +platform.py) + + Some os variables + Variable Meaning +name name of O/S-specific module (e.g. "posix", "mac", "nt") +path O/S-specific module for path manipulations. + On Unix, os.path.split() <=> posixpath.split() +curdir string used to represent current directory ('.') +pardir string used to represent parent directory ('..') +sep string used to separate directories ('/' or '\'). Tip: use + os.path.join() to build portable paths. +altsep Alternate sep +if applicable (None +otherwise) +pathsep character used to separate search path components (as in + $PATH), eg. ';' for windows. +linesep line separator as used in binary files, ie '\n' on Unix, '\ + r\n' on Dos/Win, '\r' + + Some os functions + Function Result +makedirs(path[, Recursive directory creation (create required intermediary +mode=0777]) dirs); os.error if fails. +removedirs(path) Recursive directory delete (delete intermediary empty + dirs); if fails. +renames(old, new) Recursive directory or file renaming; os.error if fails. + + + + posix +don't import this module directly, import os instead ! +(see also module: shutil for file copy & remove fcts) + + posix Variables +Variable Meaning +environ dictionary of environment variables, e.g.posix.environ['HOME']. +error exception raised on POSIX-related error. + Corresponding value is tuple of errno code and perror() string. + + Some posix functions + Function Result +chdir(path) Changes current directory to path. +chmod(path, Changes the mode of path to the numeric mode +mode) +close(fd) Closes file descriptor fd opened with posix.open. +_exit(n) Immediate exit, with no cleanups, no SystemExit,etc. Should use + this to exit a child process. +execv(p, args) "Become" executable p with args args +getcwd() Returns a string representing the current working directory +getpid() Returns the current process id +fork() Like C's fork(). Returns 0 to child, child pid to parent.[Not + on Windows] +kill(pid, Like C's kill [Not on Windows] +signal) +listdir(path) Lists (base)names of entries in directory path, excluding '.' + and '..' +lseek(fd, pos, Sets current position in file fd to position pos, expressedas +how) an offset relative to beginning of file (how=0), tocurrent + position (how=1), or to end of file (how=2) +mkdir(path[, Creates a directory named path with numeric mode (default 0777) +mode]) +open(file, Like C's open(). Returns file descriptor. Use file object +flags, mode) fctsrather than this low level ones. +pipe() Creates a pipe. Returns pair of file descriptors (r, w) [Not on + Windows]. +popen(command, Opens a pipe to or from command. Result is a file object to +mode='r', read to orwrite from, as indicated by mode being 'r' or 'w'. +bufSize=0) Use it to catch acommand output ('r' mode) or to feed it ('w' + mode). +remove(path) See unlink. +rename(src, dst Renames/moves the file or directory src to dst. [error iftarget +) name already exists] +rmdir(path) Removes the empty directory path +read(fd, n) Reads n bytes from file descriptor fd and return as string. + Returns st_mode, st_ino, st_dev, st_nlink, st_uid,st_gid, +stat(path) st_size, st_atime, st_mtime, st_ctime.[st_ino, st_uid, st_gid + are dummy on Windows] +system(command) Executes string command in a subshell. Returns exitstatus of + subshell (usually 0 means OK). + Returns accumulated CPU times in sec (user, system, children's +times() user,children's sys, elapsed real time). [3 last not on + Windows] +unlink(path) Unlinks ("deletes") the file (not dir!) path. same as: remove +utime(path, ( Sets the access & modified time of the file to the given tuple +aTime, mTime)) of values. +wait() Waits for child process completion. Returns tuple ofpid, + exit_status [Not on Windows] +waitpid(pid, Waits for process pid to complete. Returns tuple ofpid, +options) exit_status [Not on Windows] +write(fd, str) Writes str to file fd. Returns nb of bytes written. + + + + posixpath +Do not import this module directly, import os instead and refer to this module +as os.path. (e.g. os.path.exists(p)) ! + + Some posixpath functions + Function Result +abspath(p) Returns absolute path for path p, taking current working dir in + account. +dirname/ +basename(p directory and name parts of the path p. See also split. +) +exists(p) True if string p is an existing path (file or directory) +expanduser Returns string that is (a copy of) p with "~" expansion done. +(p) +expandvars Returns string that is (a copy of) p with environment vars expanded. +(p) [Windows: case significant; must use Unix: $var notation, not %var%] +getsize( return the size in bytes of filename. raise os.error. +filename) +getmtime( return last modification time of filename (integer nb of seconds +filename) since epoch). +getatime( return last access time of filename (integer nb of seconds since +filename) epoch). +isabs(p) True if string p is an absolute path. +isdir(p) True if string p is a directory. +islink(p) True if string p is a symbolic link. +ismount(p) True if string p is a mount point [true for all dirs on Windows]. +join(p[,q Joins one or more path components intelligently. +[,...]]) + Splits p into (head, tail) where tail is lastpathname component and +split(p) <head> is everything leadingup to that. <=> (dirname(p), basename + (p)) +splitdrive Splits path p in a pair ('drive:', tail) [Windows] +(p) +splitext(p Splits into (root, ext) where last comp of root contains no periods +) and ext is empty or startswith a period. + Calls the function visit with arguments(arg, dirname, names) for + each directory recursively inthe directory tree rooted at p +walk(p, (including p itself if it's a dir)The argument dirname specifies the +visit, arg visited directory, the argumentnames lists the files in the +) directory. The visit function maymodify names to influence the set + of directories visited belowdirname, e.g., to avoid visiting certain + parts of the tree. + + + + shutil +high-level file operations (copying, deleting). + + Main shutil functions + Function Result +copy(src, dst) Copies the contents of file src to file dst, retaining file + permissions. +copytree(src, dst Recursively copies an entire directory tree rooted at src +[, symlinks]) into dst (which should not already exist). If symlinks is + true, links insrc are kept as such in dst. +rmtree(path[, Deletes an entire directory tree, ignoring errors if +ignore_errors[, ignore_errors true,or calling onerror(func, path, +onerror]]) sys.exc_info()) if supplied with + +(and also: copyfile, copymode, copystat, copy2) + +time + + Variables +Variable Meaning +altzone signed offset of local DST timezone in sec west of the 0th meridian. +daylight nonzero if a DST timezone is specified + + Functions + Function Result +time() return a float representing UTC time in seconds since the epoch. +gmtime(secs), return a tuple representing time : (year aaaa, month(1-12),day +localtime( (1-31), hour(0-23), minute(0-59), second(0-59), weekday(0-6, 0 is +secs) monday), Julian day(1-366), daylight flag(-1,0 or 1)) +asctime( +timeTuple), +strftime( +format, return a formated string representing time. +timeTuple) +mktime(tuple) inverse of localtime(). Return a float. +strptime( parse a formated string representing time, return tuple as in +string[, gmtime(). +format]) +sleep(secs) Suspend execution for <secs> seconds. <secs> can be a float. + +and also: clock, ctime. + + string + +As of Python 2.0, much (though not all) of the functionality provided by the +string module have been superseded by built-in string methods - see Operations +on strings for details. + + Some string variables + Variable Meaning +digits The string '0123456789' +hexdigits, octdigits legal hexadecimal & octal digits +letters, uppercase, lowercase, Strings containing the appropriate +whitespace characters +index_error Exception raised by index() if substr not + found. + + Some string functions + Function Result +expandtabs(s, returns a copy of string <s> with tabs expanded. +tabSize) +find/rfind(s, sub Return the lowest/highest index in <s> where the substring +[, start=0[, end= <sub> is found such that <sub> is wholly contained ins +0]) [start:end]. Return -1 if <sub> not found. +ljust/rjust/center Return a copy of string <s> left/right justified/centerd in +(s, width) afield of given width, padded with spaces. <s> is + nevertruncated. +lower/upper(s) Return a string that is (a copy of) <s> in lowercase/ + uppercase +split(s[, sep= Return a list containing the words of the string <s>,using +whitespace[, the string <sep> as a separator. +maxsplit=0]]) +join(words[, sep=' Concatenate a list or tuple of words with +']) interveningseparators; inverse of split. +replace(s, old, Returns a copy of string <s> with all occurrences of +new[, maxsplit=0] substring<old> replaced by <new>. Limits to <maxsplit> + firstsubstitutions if specified. +strip(s) Return a string that is (a copy of) <s> without leadingand + trailing whitespace. see also lstrip, rstrip. + + + + re (sre) + +Handles Unicode strings. Implemented in new module sre, re now a mere front-end +for compatibility. +Patterns are specified as strings. Tip: Use raw strings (e.g. r'\w*') to +litteralize backslashes. + + + Regular expression syntax + Form Description +. matches any character (including newline if DOTALL flag specified) +^ matches start of the string (of every line in MULTILINE mode) +$ matches end of the string (of every line in MULTILINE mode) +* 0 or more of preceding regular expression (as many as possible) ++ 1 or more of preceding regular expression (as many as possible) +? 0 or 1 occurrence of preceding regular expression +*?, +?, ?? Same as *, + and ? but matches as few characters as possible +{m,n} matches from m to n repetitions of preceding RE +{m,n}? idem, attempting to match as few repetitions as possible +[ ] defines character set: e.g. '[a-zA-Z]' to match all letters(see also + \w \S) +[^ ] defines complemented character set: matches if char is NOT in set + escapes special chars '*?+&$|()' and introduces special sequences +\ (see below). Due to Python string rules, write as '\\' orr'\' in the + pattern string. +\\ matches a litteral '\'; due to Python string rules, write as '\\\\ + 'in pattern string, or better using raw string: r'\\'. +| specifies alternative: 'foo|bar' matches 'foo' or 'bar' +(...) matches any RE inside (), and delimits a group. +(?:...) idem but doesn't delimit a group. + matches if ... matches next, but doesn't consume any of the string +(?=...) e.g. 'Isaac (?=Asimov)' matches 'Isaac' only if followed by + 'Asimov'. +(?!...) matches if ... doesn't match next. Negative of (?=...) +(?P<name matches any RE inside (), and delimits a named group. (e.g. r'(?P +>...) <id>[a-zA-Z_]\w*)' defines a group named id) +(?P=name) matches whatever text was matched by the earlier group named name. +(?#...) A comment; ignored. +(?letter) letter is one of 'i','L', 'm', 's', 'x'. Set the corresponding flags + (re.I, re.L, re.M, re.S, re.X) for the entire RE. + + Special sequences +Sequence Description +number matches content of the group of the same number; groups are numbered + starting from 1 +\A matches only at the start of the string +\b empty str at beg or end of word: '\bis\b' matches 'is', but not 'his' +\B empty str NOT at beginning or end of word +\d any decimal digit (<=> [0-9]) +\D any non-decimal digit char (<=> [^O-9]) +\s any whitespace char (<=> [ \t\n\r\f\v]) +\S any non-whitespace char (<=> [^ \t\n\r\f\v]) +\w any alphaNumeric char (depends on LOCALE flag) +\W any non-alphaNumeric char (depends on LOCALE flag) +\Z matches only at the end of the string + + Variables +Variable Meaning +error Exception when pattern string isn't a valid regexp. + + Functions + Function Result + Compile a RE pattern string into a regular expression object. + Flags (combinable by |): + + I or IGNORECASE or (?i) + case insensitive matching +compile( L or LOCALE or (?L) +pattern[, make \w, \W, \b, \B dependent on thecurrent locale +flags=0]) M or MULTILINE or (?m) + matches every new line and not onlystart/end of the whole + string + S or DOTALL or (?s) + '.' matches ALL chars, including newline + X or VERBOSE or (?x) + Ignores whitespace outside character sets +escape(string) return (a copy of) string with all non-alphanumerics + backslashed. +match(pattern, if 0 or more chars at beginning of <string> match the RE pattern +string[, flags string,return a corresponding MatchObject instance, or None if +]) no match. +search(pattern scan thru <string> for a location matching <pattern>, return +, string[, acorresponding MatchObject instance, or None if no match. +flags]) +split(pattern, split <string> by occurrences of <pattern>. If capturing () are +string[, used inpattern, then occurrences of patterns or subpatterns are +maxsplit=0]) also returned. +findall( return a list of non-overlapping matches in <pattern>, either a +pattern, list ofgroups or a list of tuples if the pattern has more than 1 +string) group. + return string obtained by replacing the (<count> first) lefmost +sub(pattern, non-overlapping occurrences of <pattern> (a string or a RE +repl, string[, object) in <string>by <repl>; <repl> can be a string or a fct +count=0]) called with a single MatchObj arg, which must return the + replacement string. +subn(pattern, +repl, string[, same as sub(), but returns a tuple (newString, numberOfSubsMade) +count=0]) + +Regular Expression Objects + + +(RE objects are returned by the compile fct) + + re object attributes +Attribute Descrition +flags flags arg used when RE obj was compiled, or 0 if none provided +groupindex dictionary of {group name: group number} in pattern +pattern pattern string from which RE obj was compiled + + re object methods + Method Result + If zero or more characters at the beginning of string match this + regular expression, return a corresponding MatchObject instance. + Return None if the string does not match the pattern; note that + this is different from a zero-length match. + The optional second parameter pos gives an index in the string +match( where the search is to start; it defaults to 0. This is not +string[, completely equivalent to slicing the string; the '' pattern +pos][, character matches at the real beginning of the string and at +endpos]) positions just after a newline, but not necessarily at the index + where the search is to start. + The optional parameter endpos limits how far the string will be + searched; it will be as if the string is endpos characters long, so + only the characters from pos to endpos will be searched for a + match. + Scan through string looking for a location where this regular +search( expression produces a match, and return a corresponding MatchObject +string[, instance. Return None if no position in the string matches the +pos][, pattern; note that this is different from finding a zero-length +endpos]) match at some point in the string. + The optional pos and endpos parameters have the same meaning as for + the match() method. +split( +string[, Identical to the split() function, using the compiled pattern. +maxsplit= +0]) +findall( Identical to the findall() function, using the compiled pattern. +string) +sub(repl, +string[, Identical to the sub() function, using the compiled pattern. +count=0]) +subn(repl, +string[, Identical to the subn() function, using the compiled pattern. +count=0]) + +Match Objects + + +(Match objects are returned by the match & search functions) + + Match object attributes +Attribute Description +pos value of pos passed to search or match functions; index intostring at + which RE engine started search. +endpos value of endpos passed to search or match functions; index intostring + beyond which RE engine won't go. +re RE object whose match or search fct produced this MatchObj instance +string string passed to match() or search() + + Match object functions +Function Result + returns one or more groups of the match. If one arg, result is a +group([g1 string;if multiple args, result is a tuple with one item per arg. If +, g2, gi is 0,return value is entire matching string; if 1 <= gi <= 99, +...]) returnstring matching group #gi (or None if no such group); gi may + also bea group name. + returns a tuple of all groups of the match; groups not +groups() participatingto the match have a value of None. Returns a string + instead of tupleif len(tuple)=1 +start( +group), returns indices of start & end of substring matched by group (or +end(group Noneif group exists but doesn't contribute to the match) +) +span( returns the 2-tuple (start(group), end(group)); can be (None, None)if +group) group didn't contibute to the match. + + + + math + +Variables: +pi +e +Functions (see ordinary C man pages for info): +acos(x) +asin(x) +atan(x) +atan2(x, y) +ceil(x) +cos(x) +cosh(x) +degrees(x) +exp(x) +fabs(x) +floor(x) +fmod(x, y) +frexp(x) -- Unlike C: (float, int) = frexp(float) +ldexp(x, y) +log(x [,base]) +log10(x) +modf(x) -- Unlike C: (float, float) = modf(float) +pow(x, y) +radians(x) +sin(x) +sinh(x) +sqrt(x) +tan(x) +tanh(x) + + getopt + +Functions: +getopt(list, optstr) -- Similar to C. <optstr> is option + letters to look for. Put ':' after letter + if option takes arg. E.g. + # invocation was "python test.py -c hi -a arg1 arg2" + opts, args = getopt.getopt(sys.argv[1:], 'ab:c:') + # opts would be + [('-c', 'hi'), ('-a', '')] + # args would be + ['arg1', 'arg2'] + + +List of modules and packages in base distribution + +(built-ins and content of python Lib directory) +(Python NT distribution, may be slightly different in other distributions) + + Standard library modules + Operation Result +aifc Stuff to parse AIFF-C and AIFF files. +anydbm Generic interface to all dbm clones. (dbhash, gdbm, + dbm,dumbdbm) +asynchat Support for 'chat' style protocols +asyncore Asynchronous File I/O (in select style) +atexit Register functions to be called at exit of Python interpreter. +audiodev Audio support for a few platforms. +base64 Conversions to/from base64 RFC-MIME transport encoding . +BaseHTTPServer Base class forhttp services. +Bastion "Bastionification" utility (control access to instance vars) +bdb A generic Python debugger base class. +binhex Macintosh binhex compression/decompression. +bisect List bisection algorithms. +bz2 Support for bz2 compression/decompression. +calendar Calendar printing functions. +cgi Wraps the WWW Forms Common Gateway Interface (CGI). +cgitb Utility for handling CGI tracebacks. +CGIHTTPServer CGI http services. +cmd A generic class to build line-oriented command interpreters. +datetime Basic date and time types. +code Utilities needed to emulate Python's interactive interpreter +codecs Lookup existing Unicode encodings and register new ones. +colorsys Conversion functions between RGB and other color systems. +commands Tools for executing UNIX commands . +compileall Force "compilation" of all .py files in a directory. +ConfigParser Configuration file parser (much like windows .ini files) +copy Generic shallow and deep copying operations. +copy_reg Helper to provide extensibility for pickle/cPickle. +csv Read and write files with comma separated values. +dbhash (g)dbm-compatible interface to bsdhash.hashopen. +dircache Sorted list of files in a dir, using a cache. +[DEL:dircmp:DEL] [DEL:Defines a class to build directory diff tools on.:DEL] +difflib Tool for creating delta between sequences. +dis Bytecode disassembler. +distutils Package installation system. +doctest Tool for running and verifying tests inside doc strings. +dospath Common operations on DOS pathnames. +dumbdbm A dumb and slow but simple dbm clone. +[DEL:dump:DEL] [DEL:Print python code that reconstructs a variable.:DEL] +email Comprehensive support for internet email. +exceptions Class based built-in exception hierarchy. +filecmp File comparison. +fileinput Helper class to quickly write a loop over all standard input + files. +[DEL:find:DEL] [DEL:Find files directory hierarchy matching a pattern.:DEL] +fnmatch Filename matching with shell patterns. +formatter A test formatter. +fpformat General floating point formatting functions. +ftplib An FTP client class. Based on RFC 959. +gc Perform garbacge collection, obtain GC debug stats, and tune + GC parameters. +getopt Standard command line processing. See also ftp:// + www.pauahtun.org/pub/getargspy.zip +getpass Utilities to get a password and/or the current user name. +glob filename globbing. +gopherlib Gopher protocol client interface. +[DEL:grep:DEL] [DEL:'grep' utilities.:DEL] +gzip Read & write gzipped files. +heapq Priority queue implemented using lists organized as heaps. +HMAC Keyed-Hashing for Message Authentication -- RFC 2104. +htmlentitydefs Proposed entity definitions for HTML. +htmllib HTML parsing utilities. +HTMLParser A parser for HTML and XHTML. +httplib HTTP client class. +ihooks Hooks into the "import" mechanism. +imaplib IMAP4 client.Based on RFC 2060. +imghdr Recognizing image files based on their first few bytes. +imputil Privides a way of writing customised import hooks. +inspect Tool for probing live Python objects. +keyword List of Python keywords. +knee A Python re-implementation of hierarchical module import. +linecache Cache lines from files. +linuxaudiodev Lunix /dev/audio support. +locale Support for number formatting using the current locale + settings. +logging Python logging facility. +macpath Pathname (or related) operations for the Macintosh. +macurl2path Mac specific module for conversion between pathnames and URLs. +mailbox A class to handle a unix-style or mmdf-style mailbox. +mailcap Mailcap file handling (RFC 1524). +mhlib MH (mailbox) interface. +mimetools Various tools used by MIME-reading or MIME-writing programs. +mimetypes Guess the MIME type of a file. +MimeWriter Generic MIME writer. +mimify Mimification and unmimification of mail messages. +mmap Interface to memory-mapped files - they behave like mutable + strings./font> +multifile Class to make multi-file messages easier to handle. +mutex Mutual exclusion -- for use with module sched. +netrc +nntplib An NNTP client class. Based on RFC 977. +ntpath Common operations on DOS pathnames. +nturl2path Mac specific module for conversion between pathnames and URLs. +optparse A comprehensive tool for processing command line options. +os Either mac, dos or posix depending system. +[DEL:packmail: [DEL:Create a self-unpacking shell archive.:DEL] +DEL] +pdb A Python debugger. +pickle Pickling (save and restore) of Python objects (a faster + Cimplementation exists in built-in module: cPickle). +pipes Conversion pipeline templates. +pkgunil Utilities for working with Python packages. +popen2 variations on pipe open. +poplib A POP3 client class. Based on the J. Myers POP3 draft. +posixfile Extended (posix) file operations. +posixpath Common operations on POSIX pathnames. +pprint Support to pretty-print lists, tuples, & dictionaries + recursively. +profile Class for profiling python code. +pstats Class for printing reports on profiled python code. +pydoc Utility for generating documentation from source files. +pty Pseudo terminal utilities. +pyexpat Interface to the Expay XML parser. +py_compile Routine to "compile" a .py file to a .pyc file. +pyclbr Parse a Python file and retrieve classes and methods. +Queue A multi-producer, multi-consumer queue. +quopri Conversions to/from quoted-printable transport encoding. +rand Don't use unless you want compatibility with C's rand(). +random Random variable generators +re Regular Expressions. +repr Redo repr() but with limits on most sizes. +rexec Restricted execution facilities ("safe" exec, eval, etc). +rfc822 RFC-822 message manipulation class. +rlcompleter Word completion for GNU readline 2.0. +robotparser Parse robots.txt files, useful for web spiders. +sched A generally useful event scheduler class. +sets Module for a set datatype. +sgmllib A parser for SGML. +shelve Manage shelves of pickled objects. +shlex Lexical analyzer class for simple shell-like syntaxes. +shutil Utility functions usable in a shell-like program. +SimpleHTTPServer Simple extension to base http class +site Append module search paths for third-party packages to + sys.path. +smtplib SMTP Client class (RFC 821) +sndhdr Several routines that help recognizing sound. +SocketServer Generic socket server classes. +stat Constants and functions for interpreting stat/lstat struct. +statcache Maintain a cache of file stats. +statvfs Constants for interpreting statvfs struct as returned by + os.statvfs()and os.fstatvfs() (if they exist). +string A collection of string operations. +StringIO File-like objects that read/write a string buffer (a fasterC + implementation exists in built-in module: cStringIO). +sunau Stuff to parse Sun and NeXT audio files. +sunaudio Interpret sun audio headers. +symbol Non-terminal symbols of Python grammar (from "graminit.h"). +tabnanny,/font> Check Python source for ambiguous indentation. +tarfile Facility for reading and writing to the *nix tarfile format. +telnetlib TELNET client class. Based on RFC 854. +tempfile Temporary file name allocation. +textwrap Object for wrapping and filling text. +threading Proposed new higher-level threading interfaces +threading_api (doc of the threading module) +toaiff Convert "arbitrary" sound files to AIFF files . +token Tokens (from "token.h"). +tokenize Compiles a regular expression that recognizes Python tokens. +traceback Format and print Python stack traces. +tty Terminal utilities. +turtle LogoMation-like turtle graphics +types Define names for all type symbols in the std interpreter. +tzparse Parse a timezone specification. +unicodedata Interface to unicode properties. +urllib Open an arbitrary URL. +urlparse Parse URLs according to latest draft of standard. +user Hook to allow user-specified customization code to run. +UserDict A wrapper to allow subclassing of built-in dict class. +UserList A wrapper to allow subclassing of built-in list class. +UserString A wrapper to allow subclassing of built-in string class. +[DEL:util:DEL] [DEL:some useful functions that don't fit elsewhere !!:DEL] +uu UUencode/UUdecode. +unittest Utilities for implementing unit testing. +wave Stuff to parse WAVE files. +weakref Tools for creating and managing weakly referenced objects. +webbrowser Platform independent URL launcher. +[DEL:whatsound: [DEL:Several routines that help recognizing sound files.:DEL] +DEL] +whichdb Guess which db package to use to open a db file. +xdrlib Implements (a subset of) Sun XDR (eXternal Data + Representation) +xmllib A parser for XML, using the derived class as static DTD. +xml.dom Classes for processing XML using the Document Object Model. +xml.sax Classes for processing XML using the SAX API. +xmlrpclib Support for remote procedure calls using XML. +zipfile Read & write PK zipped files. +[DEL:zmod:DEL] [DEL:Demonstration of abstruse mathematical concepts.:DEL] + + + +* Built-ins * + + sys Interpreter state vars and functions + __built-in__ Access to all built-in python identifiers + __main__ Scope of the interpreters main program, script or stdin + array Obj efficiently representing arrays of basic values + math Math functions of C standard + time Time-related functions (also the newer datetime module) + marshal Read and write some python values in binary format + struct Convert between python values and C structs + +* Standard * + + getopt Parse cmd line args in sys.argv. A la UNIX 'getopt'. + os A more portable interface to OS dependent functionality + re Functions useful for working with regular expressions + string Useful string and characters functions and exceptions + random Mersenne Twister pseudo-random number generator + thread Low-level primitives for working with process threads + threading idem, new recommanded interface. + +* Unix/Posix * + + dbm Interface to Unix ndbm database library + grp Interface to Unix group database + posix OS functionality standardized by C and POSIX standards + posixpath POSIX pathname functions + pwd Access to the Unix password database + select Access to Unix select multiplex file synchronization + socket Access to BSD socket interface + +* Tk User-interface Toolkit * + + tkinter Main interface to Tk + +* Multimedia * + + audioop Useful operations on sound fragments + imageop Useful operations on images + jpeg Access to jpeg image compressor and decompressor + rgbimg Access SGI imglib image files + +* Cryptographic Extensions * + + md5 Interface to RSA's MD5 message digest algorithm + sha Interface to the SHA message digest algorithm + HMAC Keyed-Hashing for Message Authentication -- RFC 2104. + +* Stdwin * Standard Window System + + stdwin Standard Window System interface + stdwinevents Stdwin event, command, and selection constants + rect Rectangle manipulation operations + +* SGI IRIX * (4 & 5) + + al SGI audio facilities + AL al constants + fl Interface to FORMS library + FL fl constants + flp Functions for form designer + fm Access to font manager library + gl Access to graphics library + GL Constants for gl + DEVICE More constants for gl + imgfile Imglib image file interface + +* Suns * + + sunaudiodev Access to sun audio interface + + +Workspace exploration and idiom hints + + dir(<module>) list functions, variables in <module> + dir() get object keys, defaults to local name space + if __name__ == '__main__': main() invoke main if running as script + map(None, lst1, lst2, ...) merge lists + b = a[:] create copy of seq structure + _ in interactive mode, is last value printed + + + + + + + +Python Mode for Emacs + +(Not revised, possibly not up to date) +Type C-c ? when in python-mode for extensive help. +INDENTATION +Primarily for entering new code: + TAB indent line appropriately + LFD insert newline, then indent + DEL reduce indentation, or delete single character +Primarily for reindenting existing code: + C-c : guess py-indent-offset from file content; change locally + C-u C-c : ditto, but change globally + C-c TAB reindent region to match its context + C-c < shift region left by py-indent-offset + C-c > shift region right by py-indent-offset +MARKING & MANIPULATING REGIONS OF CODE +C-c C-b mark block of lines +M-C-h mark smallest enclosing def +C-u M-C-h mark smallest enclosing class +C-c # comment out region of code +C-u C-c # uncomment region of code +MOVING POINT +C-c C-p move to statement preceding point +C-c C-n move to statement following point +C-c C-u move up to start of current block +M-C-a move to start of def +C-u M-C-a move to start of class +M-C-e move to end of def +C-u M-C-e move to end of class +EXECUTING PYTHON CODE +C-c C-c sends the entire buffer to the Python interpreter +C-c | sends the current region +C-c ! starts a Python interpreter window; this will be used by + subsequent C-c C-c or C-c | commands +C-c C-w runs PyChecker + +VARIABLES +py-indent-offset indentation increment +py-block-comment-prefix comment string used by py-comment-region +py-python-command shell command to invoke Python interpreter +py-scroll-process-buffer t means always scroll Python process buffer +py-temp-directory directory used for temp files (if needed) +py-beep-if-tab-change ring the bell if tab-width is changed + + +The Python Debugger + +(Not revised, possibly not up to date, see 1.5.2 Library Ref section 9.1; in 1.5.2, you may also use debugger integrated in IDLE) + +Accessing + +import pdb (it's a module written in Python) + -- defines functions : + run(statement[,globals[, locals]]) + -- execute statement string under debugger control, with optional + global & local environment. + runeval(expression[,globals[, locals]]) + -- same as run, but evaluate expression and return value. + runcall(function[, argument, ...]) + -- run function object with given arg(s) + pm() -- run postmortem on last exception (like debugging a core file) + post_mortem(t) + -- run postmortem on traceback object <t> + + -- defines class Pdb : + use Pdb to create reusable debugger objects. Object + preserves state (i.e. break points) between calls. + + runs until a breakpoint hit, exception, or end of program + If exception, variable '__exception__' holds (exception,value). + +Commands + +h, help + brief reminder of commands +b, break [<arg>] + if <arg> numeric, break at line <arg> in current file + if <arg> is function object, break on entry to fcn <arg> + if no arg, list breakpoints +cl, clear [<arg>] + if <arg> numeric, clear breakpoint at <arg> in current file + if no arg, clear all breakpoints after confirmation +w, where + print current call stack +u, up + move up one stack frame (to top-level caller) +d, down + move down one stack frame +s, step + advance one line in the program, stepping into calls +n, next + advance one line, stepping over calls +r, return + continue execution until current function returns + (return value is saved in variable "__return__", which + can be printed or manipulated from debugger) +c, continue + continue until next breakpoint +j, jump lineno + Set the next line that will be executed +a, args + print args to current function +rv, retval + prints return value from last function that returned +p, print <arg> + prints value of <arg> in current stack frame +l, list [<first> [, <last>]] + List source code for the current file. + Without arguments, list 11 lines around the current line + or continue the previous listing. + With one argument, list 11 lines starting at that line. + With two arguments, list the given range; + if the second argument is less than the first, it is a count. +whatis <arg> + prints type of <arg> +! + executes rest of line as a Python statement in the current stack frame +q quit + immediately stop execution and leave debugger +<return> + executes last command again +Any input debugger doesn't recognize as a command is assumed to be a +Python statement to execute in the current stack frame, the same way +the exclamation mark ("!") command does. + +Example + +(1394) python +Python 1.0.3 (Sep 26 1994) +Copyright 1991-1994 Stichting Mathematisch Centrum, Amsterdam +>>> import rm +>>> rm.run() +Traceback (innermost last): + File "<stdin>", line 1 + File "./rm.py", line 7 + x = div(3) + File "./rm.py", line 2 + return a / r +ZeroDivisionError: integer division or modulo +>>> import pdb +>>> pdb.pm() +> ./rm.py(2)div: return a / r +(Pdb) list + 1 def div(a): + 2 -> return a / r + 3 + 4 def run(): + 5 global r + 6 r = 0 + 7 x = div(3) + 8 print x +[EOF] +(Pdb) print r +0 +(Pdb) q +>>> pdb.runcall(rm.run) +etc. + +Quirks + +Breakpoints are stored as filename, line number tuples. If a module is reloaded +after editing, any remembered breakpoints are likely to be wrong. + +Always single-steps through top-most stack frame. That is, "c" acts like "n". + + diff --git a/sys/src/cmd/python/Misc/developers.txt b/sys/src/cmd/python/Misc/developers.txt new file mode 100644 index 000000000..c08590815 --- /dev/null +++ b/sys/src/cmd/python/Misc/developers.txt @@ -0,0 +1,141 @@ +Developer Log +============= + +This file is a running log of developers given permissions on SourceForge. + +The purpose is to provide some institutional memory of who was given access +and why. + +The first entry starts in April 2005. In keeping with the style of +Misc/NEWS, newer entries should be added to the top. Any markup should +be in the form of ReST. Entries should include the initials of the +project admin who made the change or granted access. Feel free to revise +the format to accommodate documentation needs as they arise. + + + +Permissions History +------------------- + +- 2006 Summer of Code entries: SoC developers are expected to work + primarily in nondist/sandbox or on a branch of their own, and will + have their work reviewed before changes are accepted into the trunk. + + - Matt Fleming was added on 25 May 2006 by AMK; he'll be working on + enhancing the Python debugger. + + - Jackilyn Hoxworth was added on 25 May 2006 by AMK; she'll be adding logging + to the standard library. + + - Mateusz Rukowicz was added on 30 May 2006 by AMK; he'll be + translating the decimal module into C. + +- SVN access granted to the "Need for Speed" Iceland sprint attendees, + between May 17 and 21, 2006, by Tim Peters. All work is to be done + in new sandbox projects or on new branches, with merging to the + trunk as approved: + + Andrew Dalke + Christian Tismer + Jack Diederich + John Benediktsson + Kristján V. Jónsson + Martin Blais + Richard Emslie + Richard Jones + Runar Petursson + Steve Holden + Richard M. Tew + +- Steven Bethard was given SVN access on 27 Apr 2006 by DJG, for PEP + update access. + +- Talin was given SVN access on 27 Apr 2006 by DJG, for PEP update + access. + +- George Yoshida (SF name "quiver") added to the SourceForge Python + project 14 Apr 2006, by Tim Peters, as a tracker admin. See + contemporaneous python-checkins thread with the unlikely Subject: + + r45329 - python/trunk/Doc/whatsnew/whatsnew25.tex + +- Ronald Oussoren was given SVN access on 3 Mar 2006 by NCN, for Mac + related work. + +- Bob Ippolito was given SVN access on 2 Mar 2006 by NCN, for Mac + related work. + +- Nick Coghlan requested CVS access so he could update his PEP directly. + Granted by GvR on 16 Oct 2005. + +- Added two new developers for the Summer of Code project. 8 July 2005 + by RDH. Andrew Kuchling will be mentoring Gregory K Johnson for a + project to enhance mailbox. Brett Cannon requested access for Flovis + Bruynooghe (sirolf) to work on pstats, profile, and hotshot. Both users + are expected to work primarily in nondist/sandbox and have their work + reviewed before making updates to active code. + +- Georg Brandl was given SF tracker permissions on 28 May 2005 + by RDH. Since the beginning of 2005, he has been active in discussions + on python-dev and has submitted a dozen patch reviews. The permissions + add the ability to change tracker status and to attach patches. On + 3 June 2005, this was expanded by RDH to include checkin permissions. + +- Terry Reedy was given SF tracker permissions on 7 Apr 2005 by RDH. + +- Nick Coghlan was given SF tracker permissions on 5 Apr 2005 by RDH. + For several months, he has been active in reviewing and contributing + patches. The added permissions give him greater flexibility in + working with the tracker. + +- Eric Price was made a developer on 2 May 2003 by TGP. This was + specifically to work on the new ``decimal`` package, which lived in + ``nondist/sandbox/decimal/`` at the time. + +- Eric S. Raymond was made a developer on 2 Jul 2000 by TGP, for general + library work. His request is archived here: + + http://mail.python.org/pipermail/python-dev/2000-July/005314.html + + +Permissions Dropped on Request +------------------------------ + +- Per note from Andrew Kuchling, the permissions for Gregory K Johnson + and the Summer Of Code project are no longer needed. AMK will make + any future checkins directly. 16 Oct 2005 RDH + +- Johannes Gijsbers sent a drop request. 27 July 2005 RDH + +- Flovis Bruynooghe sent a drop request. 14 July 2005 RDH + +- Paul Prescod sent a drop request. 30 Apr 2005 RDH + +- Finn Bock sent a drop request. 13 Apr 2005 RDH + +- Eric Price sent a drop request. 10 Apr 2005 RDH + +- Irmen de Jong requested dropping CVS access while keeping tracker + access. 10 Apr 2005 RDH + +- Moshe Zadka and Ken Manheimer sent drop requests. 8 Apr 2005 by RDH + +- Steve Holden, Gerhard Haring, and David Cole sent email stating that + they no longer use their access. 7 Apr 2005 RDH + + +Permissions Dropped after Loss of Contact +----------------------------------------- + +- Several unsuccessful efforts were made to contact Charles G Waldman. + Removed on 8 Apr 2005 by RDH. + + +Initials of Project Admins +-------------------------- + +GvR: Guido van Rossum +NCN: Neal Norwitz +RDH: Raymond Hettinger +TGP: Tim Peters +DJG: David Goodger diff --git a/sys/src/cmd/python/Misc/find_recursionlimit.py b/sys/src/cmd/python/Misc/find_recursionlimit.py new file mode 100644 index 000000000..e6454c9c3 --- /dev/null +++ b/sys/src/cmd/python/Misc/find_recursionlimit.py @@ -0,0 +1,87 @@ +#! /usr/bin/env python +"""Find the maximum recursion limit that prevents core dumps + +This script finds the maximum safe recursion limit on a particular +platform. If you need to change the recursion limit on your system, +this script will tell you a safe upper bound. To use the new limit, +call sys.setrecursionlimit. + +This module implements several ways to create infinite recursion in +Python. Different implementations end up pushing different numbers of +C stack frames, depending on how many calls through Python's abstract +C API occur. + +After each round of tests, it prints a message +Limit of NNNN is fine. + +It ends when Python causes a segmentation fault because the limit is +too high. On platforms like Mac and Windows, it should exit with a +MemoryError. + +NB: A program that does not use __methods__ can set a higher limit. +""" + +import sys + +class RecursiveBlowup1: + def __init__(self): + self.__init__() + +def test_init(): + return RecursiveBlowup1() + +class RecursiveBlowup2: + def __repr__(self): + return repr(self) + +def test_repr(): + return repr(RecursiveBlowup2()) + +class RecursiveBlowup4: + def __add__(self, x): + return x + self + +def test_add(): + return RecursiveBlowup4() + RecursiveBlowup4() + +class RecursiveBlowup5: + def __getattr__(self, attr): + return getattr(self, attr) + +def test_getattr(): + return RecursiveBlowup5().attr + +class RecursiveBlowup6: + def __getitem__(self, item): + return self[item - 2] + self[item - 1] + +def test_getitem(): + return RecursiveBlowup6()[5] + +def test_recurse(): + return test_recurse() + +def check_limit(n, test_func_name): + sys.setrecursionlimit(n) + if test_func_name.startswith("test_"): + print test_func_name[5:] + else: + print test_func_name + test_func = globals()[test_func_name] + try: + test_func() + except RuntimeError: + pass + else: + print "Yikes!" + +limit = 1000 +while 1: + check_limit(limit, "test_recurse") + check_limit(limit, "test_add") + check_limit(limit, "test_repr") + check_limit(limit, "test_init") + check_limit(limit, "test_getattr") + check_limit(limit, "test_getitem") + print "Limit of %d is fine" % limit + limit = limit + 100 diff --git a/sys/src/cmd/python/Misc/gdbinit b/sys/src/cmd/python/Misc/gdbinit new file mode 100644 index 000000000..f3cb2ead0 --- /dev/null +++ b/sys/src/cmd/python/Misc/gdbinit @@ -0,0 +1,140 @@ +# -*- ksh -*- +# +# If you use the GNU debugger gdb to debug the Python C runtime, you +# might find some of the following commands useful. Copy this to your +# ~/.gdbinit file and it'll get loaded into gdb automatically when you +# start it up. Then, at the gdb prompt you can do things like: +# +# (gdb) pyo apyobjectptr +# <module 'foobar' (built-in)> +# refcounts: 1 +# address : 84a7a2c +# $1 = void +# (gdb) + +# Prints a representation of the object to stderr, along with the +# number of reference counts it current has and the hex address the +# object is allocated at. The argument must be a PyObject* +define pyo +print _PyObject_Dump($arg0) +end + +# Prints a representation of the object to stderr, along with the +# number of reference counts it current has and the hex address the +# object is allocated at. The argument must be a PyGC_Head* +define pyg +print _PyGC_Dump($arg0) +end + +# print the local variables of the current frame +define pylocals + set $_i = 0 + while $_i < f->f_nlocals + if f->f_localsplus + $_i != 0 + set $_names = co->co_varnames + set $_name = PyString_AsString(PyTuple_GetItem($_names, $_i)) + printf "%s:\n", $_name + # side effect of calling _PyObject_Dump is to dump the object's + # info - assigning just prevents gdb from printing the + # NULL return value + set $_val = _PyObject_Dump(f->f_localsplus[$_i]) + end + set $_i = $_i + 1 + end +end + +# A rewrite of the Python interpreter's line number calculator in GDB's +# command language +define lineno + set $__continue = 1 + set $__co = f->f_code + set $__lasti = f->f_lasti + set $__sz = ((PyStringObject *)$__co->co_lnotab)->ob_size/2 + set $__p = (unsigned char *)((PyStringObject *)$__co->co_lnotab)->ob_sval + set $__li = $__co->co_firstlineno + set $__ad = 0 + while ($__sz-1 >= 0 && $__continue) + set $__sz = $__sz - 1 + set $__ad = $__ad + *$__p + set $__p = $__p + 1 + if ($__ad > $__lasti) + set $__continue = 0 + end + set $__li = $__li + *$__p + set $__p = $__p + 1 + end + printf "%d", $__li +end + +# print the current frame - verbose +define pyframev + pyframe + pylocals +end + +define pyframe + set $__fn = (char *)((PyStringObject *)co->co_filename)->ob_sval + set $__n = (char *)((PyStringObject *)co->co_name)->ob_sval + printf "%s (", $__fn + lineno + printf "): %s\n", $__n +### Uncomment these lines when using from within Emacs/XEmacs so it will +### automatically track/display the current Python source line +# printf "%c%c%s:", 032, 032, $__fn +# lineno +# printf ":1\n" +end + +### Use these at your own risk. It appears that a bug in gdb causes it +### to crash in certain circumstances. + +#define up +# up-silently 1 +# printframe +#end + +#define down +# down-silently 1 +# printframe +#end + +define printframe + if $pc > PyEval_EvalFrameEx && $pc < PyEval_EvalCodeEx + pyframe + else + frame + end +end + +# Here's a somewhat fragile way to print the entire Python stack from gdb. +# It's fragile because the tests for the value of $pc depend on the layout +# of specific functions in the C source code. + +# Explanation of while and if tests: We want to pop up the stack until we +# land in Py_Main (this is probably an incorrect assumption in an embedded +# interpreter, but the test can be extended by an interested party). If +# Py_Main <= $pc <= Py_GetArgcArv is true, $pc is in Py_Main(), so the while +# tests succeeds as long as it's not true. In a similar fashion the if +# statement tests to see if we are in PyEval_EvalFrame(). + +# print the entire Python call stack +define pystack + while $pc < Py_Main || $pc > Py_GetArgcArgv + if $pc > PyEval_EvalFrame && $pc < PyEval_EvalCodeEx + pyframe + end + up-silently 1 + end + select-frame 0 +end + +# print the entire Python call stack - verbose mode +define pystackv + while $pc < Py_Main || $pc > Py_GetArgcArgv + if $pc > PyEval_EvalFrame && $pc < PyEval_EvalCodeEx + pyframev + end + up-silently 1 + end + select-frame 0 +end diff --git a/sys/src/cmd/python/Misc/indent.pro b/sys/src/cmd/python/Misc/indent.pro new file mode 100644 index 000000000..3efac89b7 --- /dev/null +++ b/sys/src/cmd/python/Misc/indent.pro @@ -0,0 +1,15 @@ +-sob +-nbad +-bap +-br +-nce +-ncs +-npcs +-i8 +-ip8 +-c25 +-T PyObject + + + + diff --git a/sys/src/cmd/python/Misc/pymemcompat.h b/sys/src/cmd/python/Misc/pymemcompat.h new file mode 100644 index 000000000..2757e3acd --- /dev/null +++ b/sys/src/cmd/python/Misc/pymemcompat.h @@ -0,0 +1,85 @@ +/* The idea of this file is that you bundle it with your extension, + #include it, program to Python 2.3's memory API and have your + extension build with any version of Python from 1.5.2 through to + 2.3 (and hopefully beyond). */ + +#ifndef Py_PYMEMCOMPAT_H +#define Py_PYMEMCOMPAT_H + +#include "Python.h" + +/* There are three "families" of memory API: the "raw memory", "object + memory" and "object" families. (This is ignoring the matter of the + cycle collector, about which more is said below). + + Raw Memory: + + PyMem_Malloc, PyMem_Realloc, PyMem_Free + + Object Memory: + + PyObject_Malloc, PyObject_Realloc, PyObject_Free + + Object: + + PyObject_New, PyObject_NewVar, PyObject_Del + + The raw memory and object memory allocators both mimic the + malloc/realloc/free interface from ANSI C, but the object memory + allocator can (and, since 2.3, does by default) use a different + allocation strategy biased towards lots of "small" allocations. + + The object family is used for allocating Python objects, and the + initializers take care of some basic initialization (setting the + refcount to 1 and filling out the ob_type field) as well as having + a somewhat different interface. + + Do not mix the families! E.g. do not allocate memory with + PyMem_Malloc and free it with PyObject_Free. You may get away with + it quite a lot of the time, but there *are* scenarios where this + will break. You Have Been Warned. + + Also, in many versions of Python there are an insane amount of + memory interfaces to choose from. Use the ones described above. */ + +#if PY_VERSION_HEX < 0x01060000 +/* raw memory interface already present */ + +/* there is no object memory interface in 1.5.2 */ +#define PyObject_Malloc PyMem_Malloc +#define PyObject_Realloc PyMem_Realloc +#define PyObject_Free PyMem_Free + +/* the object interface is there, but the names have changed */ +#define PyObject_New PyObject_NEW +#define PyObject_NewVar PyObject_NEW_VAR +#define PyObject_Del PyMem_Free +#endif + +/* If your object is a container you probably want to support the + cycle collector, which was new in Python 2.0. + + Unfortunately, the interface to the collector that was present in + Python 2.0 and 2.1 proved to be tricky to use, and so changed in + 2.2 -- in a way that can't easily be papered over with macros. + + This file contains macros that let you program to the 2.2 GC API. + Your module will compile against any Python since version 1.5.2, + but the type will only participate in the GC in versions 2.2 and + up. Some work is still necessary on your part to only fill out the + tp_traverse and tp_clear fields when they exist and set tp_flags + appropriately. + + It is possible to support both the 2.0 and 2.2 GC APIs, but it's + not pretty and this comment block is too narrow to contain a + desciption of what's required... */ + +#if PY_VERSION_HEX < 0x020200B1 +#define PyObject_GC_New PyObject_New +#define PyObject_GC_NewVar PyObject_NewVar +#define PyObject_GC_Del PyObject_Del +#define PyObject_GC_Track(op) +#define PyObject_GC_UnTrack(op) +#endif + +#endif /* !Py_PYMEMCOMPAT_H */ diff --git a/sys/src/cmd/python/Misc/python-config.in b/sys/src/cmd/python/Misc/python-config.in new file mode 100644 index 000000000..9ac44146d --- /dev/null +++ b/sys/src/cmd/python/Misc/python-config.in @@ -0,0 +1,53 @@ +#!@EXENAME@ + +import sys +import os +import getopt +from distutils import sysconfig + +valid_opts = ['prefix', 'exec-prefix', 'includes', 'libs', 'cflags', + 'ldflags', 'help'] + +def exit_with_usage(code=1): + print >>sys.stderr, "Usage: %s [%s]" % (sys.argv[0], + '|'.join('--'+opt for opt in valid_opts)) + sys.exit(code) + +try: + opts, args = getopt.getopt(sys.argv[1:], '', valid_opts) +except getopt.error: + exit_with_usage() + +if not opts: + exit_with_usage() + +opt = opts[0][0] + +pyver = sysconfig.get_config_var('VERSION') +getvar = sysconfig.get_config_var + +if opt == '--help': + exit_with_usage(0) + +elif opt == '--prefix': + print sysconfig.PREFIX + +elif opt == '--exec-prefix': + print sysconfig.EXEC_PREFIX + +elif opt in ('--includes', '--cflags'): + flags = ['-I' + sysconfig.get_python_inc(), + '-I' + sysconfig.get_python_inc(plat_specific=True)] + if opt == '--cflags': + flags.extend(getvar('CFLAGS').split()) + print ' '.join(flags) + +elif opt in ('--libs', '--ldflags'): + libs = getvar('LIBS').split() + getvar('SYSLIBS').split() + libs.append('-lpython'+pyver) + # add the prefix/lib/pythonX.Y/config dir, but only if there is no + # shared library in prefix/lib/. + if opt == '--ldflags' and not getvar('Py_ENABLE_SHARED'): + libs.insert(0, '-L' + getvar('LIBPL')) + print ' '.join(libs) + diff --git a/sys/src/cmd/python/Misc/python-mode.el b/sys/src/cmd/python/Misc/python-mode.el new file mode 100644 index 000000000..995d40d2b --- /dev/null +++ b/sys/src/cmd/python/Misc/python-mode.el @@ -0,0 +1,3768 @@ +;;; python-mode.el --- Major mode for editing Python programs + +;; Copyright (C) 1992,1993,1994 Tim Peters + +;; Author: 1995-2002 Barry A. Warsaw +;; 1992-1994 Tim Peters +;; Maintainer: python-mode@python.org +;; Created: Feb 1992 +;; Keywords: python languages oop + +(defconst py-version "$Revision: 34960 $" + "`python-mode' version number.") + +;; This software is provided as-is, without express or implied +;; warranty. Permission to use, copy, modify, distribute or sell this +;; software, without fee, for any purpose and by any individual or +;; organization, is hereby granted, provided that the above copyright +;; notice and this paragraph appear in all copies. + +;;; Commentary: + +;; This is a major mode for editing Python programs. It was developed +;; by Tim Peters after an original idea by Michael A. Guravage. Tim +;; subsequently left the net; in 1995, Barry Warsaw inherited the mode +;; and is the current maintainer. Tim's now back but disavows all +;; responsibility for the mode. Smart Tim :-) + +;; pdbtrack support contributed by Ken Manheimer, April 2001. + +;; Please use the SourceForge Python project to submit bugs or +;; patches: +;; +;; http://sourceforge.net/projects/python + +;; FOR MORE INFORMATION: + +;; There is some information on python-mode.el at + +;; http://www.python.org/emacs/python-mode/ +;; +;; It does contain links to other packages that you might find useful, +;; such as pdb interfaces, OO-Browser links, etc. + +;; BUG REPORTING: + +;; As mentioned above, please use the SourceForge Python project for +;; submitting bug reports or patches. The old recommendation, to use +;; C-c C-b will still work, but those reports have a higher chance of +;; getting buried in my mailbox. Please include a complete, but +;; concise code sample and a recipe for reproducing the bug. Send +;; suggestions and other comments to python-mode@python.org. + +;; When in a Python mode buffer, do a C-h m for more help. It's +;; doubtful that a texinfo manual would be very useful, but if you +;; want to contribute one, I'll certainly accept it! + +;;; Code: + +(require 'comint) +(require 'custom) +(require 'cl) +(require 'compile) + + +;; user definable variables +;; vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv + +(defgroup python nil + "Support for the Python programming language, <http://www.python.org/>" + :group 'languages + :prefix "py-") + +(defcustom py-python-command "python" + "*Shell command used to start Python interpreter." + :type 'string + :group 'python) + +(defcustom py-jpython-command "jpython" + "*Shell command used to start the JPython interpreter." + :type 'string + :group 'python + :tag "JPython Command") + +(defcustom py-default-interpreter 'cpython + "*Which Python interpreter is used by default. +The value for this variable can be either `cpython' or `jpython'. + +When the value is `cpython', the variables `py-python-command' and +`py-python-command-args' are consulted to determine the interpreter +and arguments to use. + +When the value is `jpython', the variables `py-jpython-command' and +`py-jpython-command-args' are consulted to determine the interpreter +and arguments to use. + +Note that this variable is consulted only the first time that a Python +mode buffer is visited during an Emacs session. After that, use +\\[py-toggle-shells] to change the interpreter shell." + :type '(choice (const :tag "Python (a.k.a. CPython)" cpython) + (const :tag "JPython" jpython)) + :group 'python) + +(defcustom py-python-command-args '("-i") + "*List of string arguments to be used when starting a Python shell." + :type '(repeat string) + :group 'python) + +(defcustom py-jpython-command-args '("-i") + "*List of string arguments to be used when starting a JPython shell." + :type '(repeat string) + :group 'python + :tag "JPython Command Args") + +(defcustom py-indent-offset 4 + "*Amount of offset per level of indentation. +`\\[py-guess-indent-offset]' can usually guess a good value when +you're editing someone else's Python code." + :type 'integer + :group 'python) + +(defcustom py-continuation-offset 4 + "*Additional amount of offset to give for some continuation lines. +Continuation lines are those that immediately follow a backslash +terminated line. Only those continuation lines for a block opening +statement are given this extra offset." + :type 'integer + :group 'python) + +(defcustom py-smart-indentation t + "*Should `python-mode' try to automagically set some indentation variables? +When this variable is non-nil, two things happen when a buffer is set +to `python-mode': + + 1. `py-indent-offset' is guessed from existing code in the buffer. + Only guessed values between 2 and 8 are considered. If a valid + guess can't be made (perhaps because you are visiting a new + file), then the value in `py-indent-offset' is used. + + 2. `indent-tabs-mode' is turned off if `py-indent-offset' does not + equal `tab-width' (`indent-tabs-mode' is never turned on by + Python mode). This means that for newly written code, tabs are + only inserted in indentation if one tab is one indentation + level, otherwise only spaces are used. + +Note that both these settings occur *after* `python-mode-hook' is run, +so if you want to defeat the automagic configuration, you must also +set `py-smart-indentation' to nil in your `python-mode-hook'." + :type 'boolean + :group 'python) + +(defcustom py-align-multiline-strings-p t + "*Flag describing how multi-line triple quoted strings are aligned. +When this flag is non-nil, continuation lines are lined up under the +preceding line's indentation. When this flag is nil, continuation +lines are aligned to column zero." + :type '(choice (const :tag "Align under preceding line" t) + (const :tag "Align to column zero" nil)) + :group 'python) + +(defcustom py-block-comment-prefix "##" + "*String used by \\[comment-region] to comment out a block of code. +This should follow the convention for non-indenting comment lines so +that the indentation commands won't get confused (i.e., the string +should be of the form `#x...' where `x' is not a blank or a tab, and +`...' is arbitrary). However, this string should not end in whitespace." + :type 'string + :group 'python) + +(defcustom py-honor-comment-indentation t + "*Controls how comment lines influence subsequent indentation. + +When nil, all comment lines are skipped for indentation purposes, and +if possible, a faster algorithm is used (i.e. X/Emacs 19 and beyond). + +When t, lines that begin with a single `#' are a hint to subsequent +line indentation. If the previous line is such a comment line (as +opposed to one that starts with `py-block-comment-prefix'), then its +indentation is used as a hint for this line's indentation. Lines that +begin with `py-block-comment-prefix' are ignored for indentation +purposes. + +When not nil or t, comment lines that begin with a single `#' are used +as indentation hints, unless the comment character is in column zero." + :type '(choice + (const :tag "Skip all comment lines (fast)" nil) + (const :tag "Single # `sets' indentation for next line" t) + (const :tag "Single # `sets' indentation except at column zero" + other) + ) + :group 'python) + +(defcustom py-temp-directory + (let ((ok '(lambda (x) + (and x + (setq x (expand-file-name x)) ; always true + (file-directory-p x) + (file-writable-p x) + x)))) + (or (funcall ok (getenv "TMPDIR")) + (funcall ok "/usr/tmp") + (funcall ok "/tmp") + (funcall ok "/var/tmp") + (funcall ok ".") + (error + "Couldn't find a usable temp directory -- set `py-temp-directory'"))) + "*Directory used for temporary files created by a *Python* process. +By default, the first directory from this list that exists and that you +can write into: the value (if any) of the environment variable TMPDIR, +/usr/tmp, /tmp, /var/tmp, or the current directory." + :type 'string + :group 'python) + +(defcustom py-beep-if-tab-change t + "*Ring the bell if `tab-width' is changed. +If a comment of the form + + \t# vi:set tabsize=<number>: + +is found before the first code line when the file is entered, and the +current value of (the general Emacs variable) `tab-width' does not +equal <number>, `tab-width' is set to <number>, a message saying so is +displayed in the echo area, and if `py-beep-if-tab-change' is non-nil +the Emacs bell is also rung as a warning." + :type 'boolean + :group 'python) + +(defcustom py-jump-on-exception t + "*Jump to innermost exception frame in *Python Output* buffer. +When this variable is non-nil and an exception occurs when running +Python code synchronously in a subprocess, jump immediately to the +source code of the innermost traceback frame." + :type 'boolean + :group 'python) + +(defcustom py-ask-about-save t + "If not nil, ask about which buffers to save before executing some code. +Otherwise, all modified buffers are saved without asking." + :type 'boolean + :group 'python) + +(defcustom py-backspace-function 'backward-delete-char-untabify + "*Function called by `py-electric-backspace' when deleting backwards." + :type 'function + :group 'python) + +(defcustom py-delete-function 'delete-char + "*Function called by `py-electric-delete' when deleting forwards." + :type 'function + :group 'python) + +(defcustom py-imenu-show-method-args-p nil + "*Controls echoing of arguments of functions & methods in the Imenu buffer. +When non-nil, arguments are printed." + :type 'boolean + :group 'python) +(make-variable-buffer-local 'py-indent-offset) + +(defcustom py-pdbtrack-do-tracking-p t + "*Controls whether the pdbtrack feature is enabled or not. +When non-nil, pdbtrack is enabled in all comint-based buffers, +e.g. shell buffers and the *Python* buffer. When using pdb to debug a +Python program, pdbtrack notices the pdb prompt and displays the +source file and line that the program is stopped at, much the same way +as gud-mode does for debugging C programs with gdb." + :type 'boolean + :group 'python) +(make-variable-buffer-local 'py-pdbtrack-do-tracking-p) + +(defcustom py-pdbtrack-minor-mode-string " PDB" + "*String to use in the minor mode list when pdbtrack is enabled." + :type 'string + :group 'python) + +(defcustom py-import-check-point-max + 20000 + "Maximum number of characters to search for a Java-ish import statement. +When `python-mode' tries to calculate the shell to use (either a +CPython or a JPython shell), it looks at the so-called `shebang' line +-- i.e. #! line. If that's not available, it looks at some of the +file heading imports to see if they look Java-like." + :type 'integer + :group 'python + ) + +(defcustom py-jpython-packages + '("java" "javax" "org" "com") + "Imported packages that imply `jpython-mode'." + :type '(repeat string) + :group 'python) + +;; Not customizable +(defvar py-master-file nil + "If non-nil, execute the named file instead of the buffer's file. +The intent is to allow you to set this variable in the file's local +variable section, e.g.: + + # Local Variables: + # py-master-file: \"master.py\" + # End: + +so that typing \\[py-execute-buffer] in that buffer executes the named +master file instead of the buffer's file. If the file name has a +relative path, the value of variable `default-directory' for the +buffer is prepended to come up with a file name.") +(make-variable-buffer-local 'py-master-file) + +(defcustom py-pychecker-command "pychecker" + "*Shell command used to run Pychecker." + :type 'string + :group 'python + :tag "Pychecker Command") + +(defcustom py-pychecker-command-args '("--stdlib") + "*List of string arguments to be passed to pychecker." + :type '(repeat string) + :group 'python + :tag "Pychecker Command Args") + +(defvar py-shell-alist + '(("jpython" . 'jpython) + ("jython" . 'jpython) + ("python" . 'cpython)) + "*Alist of interpreters and python shells. Used by `py-choose-shell' +to select the appropriate python interpreter mode for a file.") + + +;; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +;; NO USER DEFINABLE VARIABLES BEYOND THIS POINT + +(defconst py-emacs-features + (let (features) + features) + "A list of features extant in the Emacs you are using. +There are many flavors of Emacs out there, with different levels of +support for features needed by `python-mode'.") + +;; Face for None, True, False, self, and Ellipsis +(defvar py-pseudo-keyword-face 'py-pseudo-keyword-face + "Face for pseudo keywords in Python mode, like self, True, False, Ellipsis.") +(make-face 'py-pseudo-keyword-face) + +(defun py-font-lock-mode-hook () + (or (face-differs-from-default-p 'py-pseudo-keyword-face) + (copy-face 'font-lock-keyword-face 'py-pseudo-keyword-face))) +(add-hook 'font-lock-mode-hook 'py-font-lock-mode-hook) + +(defvar python-font-lock-keywords + (let ((kw1 (mapconcat 'identity + '("and" "assert" "break" "class" + "continue" "def" "del" "elif" + "else" "except" "exec" "for" + "from" "global" "if" "import" + "in" "is" "lambda" "not" + "or" "pass" "print" "raise" + "return" "while" "yield" + ) + "\\|")) + (kw2 (mapconcat 'identity + '("else:" "except:" "finally:" "try:") + "\\|")) + (kw3 (mapconcat 'identity + '("ArithmeticError" "AssertionError" + "AttributeError" "DeprecationWarning" "EOFError" + "Ellipsis" "EnvironmentError" "Exception" "False" + "FloatingPointError" "FutureWarning" "IOError" + "ImportError" "IndentationError" "IndexError" + "KeyError" "KeyboardInterrupt" "LookupError" + "MemoryError" "NameError" "None" "NotImplemented" + "NotImplementedError" "OSError" "OverflowError" + "OverflowWarning" "PendingDeprecationWarning" + "ReferenceError" "RuntimeError" "RuntimeWarning" + "StandardError" "StopIteration" "SyntaxError" + "SyntaxWarning" "SystemError" "SystemExit" + "TabError" "True" "TypeError" "UnboundLocalError" + "UnicodeDecodeError" "UnicodeEncodeError" + "UnicodeError" "UnicodeTranslateError" + "UserWarning" "ValueError" "Warning" + "ZeroDivisionError" "__debug__" + "__import__" "__name__" "abs" "apply" "basestring" + "bool" "buffer" "callable" "chr" "classmethod" + "cmp" "coerce" "compile" "complex" "copyright" + "delattr" "dict" "dir" "divmod" + "enumerate" "eval" "execfile" "exit" "file" + "filter" "float" "getattr" "globals" "hasattr" + "hash" "hex" "id" "input" "int" "intern" + "isinstance" "issubclass" "iter" "len" "license" + "list" "locals" "long" "map" "max" "min" "object" + "oct" "open" "ord" "pow" "property" "range" + "raw_input" "reduce" "reload" "repr" "round" + "setattr" "slice" "staticmethod" "str" "sum" + "super" "tuple" "type" "unichr" "unicode" "vars" + "xrange" "zip") + "\\|")) + ) + (list + ;; keywords + (cons (concat "\\b\\(" kw1 "\\)\\b[ \n\t(]") 1) + ;; builtins when they don't appear as object attributes + (cons (concat "\\(\\b\\|[.]\\)\\(" kw3 "\\)\\b[ \n\t(]") 2) + ;; block introducing keywords with immediately following colons. + ;; Yes "except" is in both lists. + (cons (concat "\\b\\(" kw2 "\\)[ \n\t(]") 1) + ;; `as' but only in "import foo as bar" + '("[ \t]*\\(\\bfrom\\b.*\\)?\\bimport\\b.*\\b\\(as\\)\\b" . 2) + ;; classes + '("\\bclass[ \t]+\\([a-zA-Z_]+[a-zA-Z0-9_]*\\)" + 1 font-lock-type-face) + ;; functions + '("\\bdef[ \t]+\\([a-zA-Z_]+[a-zA-Z0-9_]*\\)" + 1 font-lock-function-name-face) + ;; pseudo-keywords + '("\\b\\(self\\|None\\|True\\|False\\|Ellipsis\\)\\b" + 1 py-pseudo-keyword-face) + )) + "Additional expressions to highlight in Python mode.") +(put 'python-mode 'font-lock-defaults '(python-font-lock-keywords)) + +;; have to bind py-file-queue before installing the kill-emacs-hook +(defvar py-file-queue nil + "Queue of Python temp files awaiting execution. +Currently-active file is at the head of the list.") + +(defvar py-pdbtrack-is-tracking-p nil) +(defvar py-pdbtrack-last-grubbed-buffer nil + "Record of the last buffer used when the source path was invalid. + +This buffer is consulted before the buffer-list history for satisfying +`py-pdbtrack-grub-for-buffer', since it's the most often the likely +prospect as debugging continues.") +(make-variable-buffer-local 'py-pdbtrack-last-grubbed-buffer) +(defvar py-pychecker-history nil) + + + +;; Constants + +(defconst py-stringlit-re + (concat + ;; These fail if backslash-quote ends the string (not worth + ;; fixing?). They precede the short versions so that the first two + ;; quotes don't look like an empty short string. + ;; + ;; (maybe raw), long single quoted triple quoted strings (SQTQ), + ;; with potential embedded single quotes + "[rR]?'''[^']*\\(\\('[^']\\|''[^']\\)[^']*\\)*'''" + "\\|" + ;; (maybe raw), long double quoted triple quoted strings (DQTQ), + ;; with potential embedded double quotes + "[rR]?\"\"\"[^\"]*\\(\\(\"[^\"]\\|\"\"[^\"]\\)[^\"]*\\)*\"\"\"" + "\\|" + "[rR]?'\\([^'\n\\]\\|\\\\.\\)*'" ; single-quoted + "\\|" ; or + "[rR]?\"\\([^\"\n\\]\\|\\\\.\\)*\"" ; double-quoted + ) + "Regular expression matching a Python string literal.") + +(defconst py-continued-re + ;; This is tricky because a trailing backslash does not mean + ;; continuation if it's in a comment + (concat + "\\(" "[^#'\"\n\\]" "\\|" py-stringlit-re "\\)*" + "\\\\$") + "Regular expression matching Python backslash continuation lines.") + +(defconst py-blank-or-comment-re "[ \t]*\\($\\|#\\)" + "Regular expression matching a blank or comment line.") + +(defconst py-outdent-re + (concat "\\(" (mapconcat 'identity + '("else:" + "except\\(\\s +.*\\)?:" + "finally:" + "elif\\s +.*:") + "\\|") + "\\)") + "Regular expression matching statements to be dedented one level.") + +(defconst py-block-closing-keywords-re + "\\(return\\|raise\\|break\\|continue\\|pass\\)" + "Regular expression matching keywords which typically close a block.") + +(defconst py-no-outdent-re + (concat + "\\(" + (mapconcat 'identity + (list "try:" + "except\\(\\s +.*\\)?:" + "while\\s +.*:" + "for\\s +.*:" + "if\\s +.*:" + "elif\\s +.*:" + (concat py-block-closing-keywords-re "[ \t\n]") + ) + "\\|") + "\\)") + "Regular expression matching lines not to dedent after.") + +(defconst py-defun-start-re + "^\\([ \t]*\\)def[ \t]+\\([a-zA-Z_0-9]+\\)\\|\\(^[a-zA-Z_0-9]+\\)[ \t]*=" + ;; If you change this, you probably have to change py-current-defun + ;; as well. This is only used by py-current-defun to find the name + ;; for add-log.el. + "Regular expression matching a function, method, or variable assignment.") + +(defconst py-class-start-re "^class[ \t]*\\([a-zA-Z_0-9]+\\)" + ;; If you change this, you probably have to change py-current-defun + ;; as well. This is only used by py-current-defun to find the name + ;; for add-log.el. + "Regular expression for finding a class name.") + +(defconst py-traceback-line-re + "[ \t]+File \"\\([^\"]+\\)\", line \\([0-9]+\\)" + "Regular expression that describes tracebacks.") + +;; pdbtrack contants +(defconst py-pdbtrack-stack-entry-regexp +; "^> \\([^(]+\\)(\\([0-9]+\\))\\([?a-zA-Z0-9_]+\\)()" + "^> \\(.*\\)(\\([0-9]+\\))\\([?a-zA-Z0-9_]+\\)()" + "Regular expression pdbtrack uses to find a stack trace entry.") + +(defconst py-pdbtrack-input-prompt "\n[(<]*pdb[>)]+ " + "Regular expression pdbtrack uses to recognize a pdb prompt.") + +(defconst py-pdbtrack-track-range 10000 + "Max number of characters from end of buffer to search for stack entry.") + + + +;; Major mode boilerplate + +;; define a mode-specific abbrev table for those who use such things +(defvar python-mode-abbrev-table nil + "Abbrev table in use in `python-mode' buffers.") +(define-abbrev-table 'python-mode-abbrev-table nil) + +(defvar python-mode-hook nil + "*Hook called by `python-mode'.") + +(defvar jpython-mode-hook nil + "*Hook called by `jpython-mode'. `jpython-mode' also calls +`python-mode-hook'.") + +(defvar py-shell-hook nil + "*Hook called by `py-shell'.") + +;; In previous version of python-mode.el, the hook was incorrectly +;; called py-mode-hook, and was not defvar'd. Deprecate its use. +(and (fboundp 'make-obsolete-variable) + (make-obsolete-variable 'py-mode-hook 'python-mode-hook)) + +(defvar py-mode-map () + "Keymap used in `python-mode' buffers.") +(if py-mode-map + nil + (setq py-mode-map (make-sparse-keymap)) + ;; electric keys + (define-key py-mode-map ":" 'py-electric-colon) + ;; indentation level modifiers + (define-key py-mode-map "\C-c\C-l" 'py-shift-region-left) + (define-key py-mode-map "\C-c\C-r" 'py-shift-region-right) + (define-key py-mode-map "\C-c<" 'py-shift-region-left) + (define-key py-mode-map "\C-c>" 'py-shift-region-right) + ;; paragraph and string filling + (define-key py-mode-map "\eq" 'py-fill-paragraph) + ;; subprocess commands + (define-key py-mode-map "\C-c\C-c" 'py-execute-buffer) + (define-key py-mode-map "\C-c\C-m" 'py-execute-import-or-reload) + (define-key py-mode-map "\C-c\C-s" 'py-execute-string) + (define-key py-mode-map "\C-c|" 'py-execute-region) + (define-key py-mode-map "\e\C-x" 'py-execute-def-or-class) + (define-key py-mode-map "\C-c!" 'py-shell) + (define-key py-mode-map "\C-c\C-t" 'py-toggle-shells) + ;; Caution! Enter here at your own risk. We are trying to support + ;; several behaviors and it gets disgusting. :-( This logic ripped + ;; largely from CC Mode. + ;; + ;; In XEmacs 19, Emacs 19, and Emacs 20, we use this to bind + ;; backwards deletion behavior to DEL, which both Delete and + ;; Backspace get translated to. There's no way to separate this + ;; behavior in a clean way, so deal with it! Besides, it's been + ;; this way since the dawn of time. + (if (not (boundp 'delete-key-deletes-forward)) + (define-key py-mode-map "\177" 'py-electric-backspace) + ;; However, XEmacs 20 actually achieved enlightenment. It is + ;; possible to sanely define both backward and forward deletion + ;; behavior under X separately (TTYs are forever beyond hope, but + ;; who cares? XEmacs 20 does the right thing with these too). + (define-key py-mode-map [delete] 'py-electric-delete) + (define-key py-mode-map [backspace] 'py-electric-backspace)) + ;; Separate M-BS from C-M-h. The former should remain + ;; backward-kill-word. + (define-key py-mode-map [(control meta h)] 'py-mark-def-or-class) + (define-key py-mode-map "\C-c\C-k" 'py-mark-block) + ;; Miscellaneous + (define-key py-mode-map "\C-c:" 'py-guess-indent-offset) + (define-key py-mode-map "\C-c\t" 'py-indent-region) + (define-key py-mode-map "\C-c\C-d" 'py-pdbtrack-toggle-stack-tracking) + (define-key py-mode-map "\C-c\C-n" 'py-next-statement) + (define-key py-mode-map "\C-c\C-p" 'py-previous-statement) + (define-key py-mode-map "\C-c\C-u" 'py-goto-block-up) + (define-key py-mode-map "\C-c#" 'py-comment-region) + (define-key py-mode-map "\C-c?" 'py-describe-mode) + (define-key py-mode-map "\C-c\C-h" 'py-help-at-point) + (define-key py-mode-map "\e\C-a" 'py-beginning-of-def-or-class) + (define-key py-mode-map "\e\C-e" 'py-end-of-def-or-class) + (define-key py-mode-map "\C-c-" 'py-up-exception) + (define-key py-mode-map "\C-c=" 'py-down-exception) + ;; stuff that is `standard' but doesn't interface well with + ;; python-mode, which forces us to rebind to special commands + (define-key py-mode-map "\C-xnd" 'py-narrow-to-defun) + ;; information + (define-key py-mode-map "\C-c\C-b" 'py-submit-bug-report) + (define-key py-mode-map "\C-c\C-v" 'py-version) + (define-key py-mode-map "\C-c\C-w" 'py-pychecker-run) + ;; shadow global bindings for newline-and-indent w/ the py- version. + ;; BAW - this is extremely bad form, but I'm not going to change it + ;; for now. + (mapcar #'(lambda (key) + (define-key py-mode-map key 'py-newline-and-indent)) + (where-is-internal 'newline-and-indent)) + ;; Force RET to be py-newline-and-indent even if it didn't get + ;; mapped by the above code. motivation: Emacs' default binding for + ;; RET is `newline' and C-j is `newline-and-indent'. Most Pythoneers + ;; expect RET to do a `py-newline-and-indent' and any Emacsers who + ;; dislike this are probably knowledgeable enough to do a rebind. + ;; However, we do *not* change C-j since many Emacsers have already + ;; swapped RET and C-j and they don't want C-j bound to `newline' to + ;; change. + (define-key py-mode-map "\C-m" 'py-newline-and-indent) + ) + +(defvar py-mode-output-map nil + "Keymap used in *Python Output* buffers.") +(if py-mode-output-map + nil + (setq py-mode-output-map (make-sparse-keymap)) + (define-key py-mode-output-map [button2] 'py-mouseto-exception) + (define-key py-mode-output-map "\C-c\C-c" 'py-goto-exception) + ;; TBD: Disable all self-inserting keys. This is bogus, we should + ;; really implement this as *Python Output* buffer being read-only + (mapcar #' (lambda (key) + (define-key py-mode-output-map key + #'(lambda () (interactive) (beep)))) + (where-is-internal 'self-insert-command)) + ) + +(defvar py-shell-map nil + "Keymap used in *Python* shell buffers.") +(if py-shell-map + nil + (setq py-shell-map (copy-keymap comint-mode-map)) + (define-key py-shell-map [tab] 'tab-to-tab-stop) + (define-key py-shell-map "\C-c-" 'py-up-exception) + (define-key py-shell-map "\C-c=" 'py-down-exception) + ) + +(defvar py-mode-syntax-table nil + "Syntax table used in `python-mode' buffers.") +(when (not py-mode-syntax-table) + (setq py-mode-syntax-table (make-syntax-table)) + (modify-syntax-entry ?\( "()" py-mode-syntax-table) + (modify-syntax-entry ?\) ")(" py-mode-syntax-table) + (modify-syntax-entry ?\[ "(]" py-mode-syntax-table) + (modify-syntax-entry ?\] ")[" py-mode-syntax-table) + (modify-syntax-entry ?\{ "(}" py-mode-syntax-table) + (modify-syntax-entry ?\} "){" py-mode-syntax-table) + ;; Add operator symbols misassigned in the std table + (modify-syntax-entry ?\$ "." py-mode-syntax-table) + (modify-syntax-entry ?\% "." py-mode-syntax-table) + (modify-syntax-entry ?\& "." py-mode-syntax-table) + (modify-syntax-entry ?\* "." py-mode-syntax-table) + (modify-syntax-entry ?\+ "." py-mode-syntax-table) + (modify-syntax-entry ?\- "." py-mode-syntax-table) + (modify-syntax-entry ?\/ "." py-mode-syntax-table) + (modify-syntax-entry ?\< "." py-mode-syntax-table) + (modify-syntax-entry ?\= "." py-mode-syntax-table) + (modify-syntax-entry ?\> "." py-mode-syntax-table) + (modify-syntax-entry ?\| "." py-mode-syntax-table) + ;; For historical reasons, underscore is word class instead of + ;; symbol class. GNU conventions say it should be symbol class, but + ;; there's a natural conflict between what major mode authors want + ;; and what users expect from `forward-word' and `backward-word'. + ;; Guido and I have hashed this out and have decided to keep + ;; underscore in word class. If you're tempted to change it, try + ;; binding M-f and M-b to py-forward-into-nomenclature and + ;; py-backward-into-nomenclature instead. This doesn't help in all + ;; situations where you'd want the different behavior + ;; (e.g. backward-kill-word). + (modify-syntax-entry ?\_ "w" py-mode-syntax-table) + ;; Both single quote and double quote are string delimiters + (modify-syntax-entry ?\' "\"" py-mode-syntax-table) + (modify-syntax-entry ?\" "\"" py-mode-syntax-table) + ;; backquote is open and close paren + (modify-syntax-entry ?\` "$" py-mode-syntax-table) + ;; comment delimiters + (modify-syntax-entry ?\# "<" py-mode-syntax-table) + (modify-syntax-entry ?\n ">" py-mode-syntax-table) + ) + +;; An auxiliary syntax table which places underscore and dot in the +;; symbol class for simplicity +(defvar py-dotted-expression-syntax-table nil + "Syntax table used to identify Python dotted expressions.") +(when (not py-dotted-expression-syntax-table) + (setq py-dotted-expression-syntax-table + (copy-syntax-table py-mode-syntax-table)) + (modify-syntax-entry ?_ "_" py-dotted-expression-syntax-table) + (modify-syntax-entry ?. "_" py-dotted-expression-syntax-table)) + + + +;; Utilities +(defmacro py-safe (&rest body) + "Safely execute BODY, return nil if an error occurred." + (` (condition-case nil + (progn (,@ body)) + (error nil)))) + +(defsubst py-keep-region-active () + "Keep the region active in XEmacs." + ;; Ignore byte-compiler warnings you might see. Also note that + ;; FSF's Emacs 19 does it differently; its policy doesn't require us + ;; to take explicit action. + (and (boundp 'zmacs-region-stays) + (setq zmacs-region-stays t))) + +(defsubst py-point (position) + "Returns the value of point at certain commonly referenced POSITIONs. +POSITION can be one of the following symbols: + + bol -- beginning of line + eol -- end of line + bod -- beginning of def or class + eod -- end of def or class + bob -- beginning of buffer + eob -- end of buffer + boi -- back to indentation + bos -- beginning of statement + +This function does not modify point or mark." + (let ((here (point))) + (cond + ((eq position 'bol) (beginning-of-line)) + ((eq position 'eol) (end-of-line)) + ((eq position 'bod) (py-beginning-of-def-or-class)) + ((eq position 'eod) (py-end-of-def-or-class)) + ;; Kind of funny, I know, but useful for py-up-exception. + ((eq position 'bob) (beginning-of-buffer)) + ((eq position 'eob) (end-of-buffer)) + ((eq position 'boi) (back-to-indentation)) + ((eq position 'bos) (py-goto-initial-line)) + (t (error "Unknown buffer position requested: %s" position)) + ) + (prog1 + (point) + (goto-char here)))) + +(defsubst py-highlight-line (from to file line) + (cond + ((fboundp 'make-extent) + ;; XEmacs + (let ((e (make-extent from to))) + (set-extent-property e 'mouse-face 'highlight) + (set-extent-property e 'py-exc-info (cons file line)) + (set-extent-property e 'keymap py-mode-output-map))) + (t + ;; Emacs -- Please port this! + ) + )) + +(defun py-in-literal (&optional lim) + "Return non-nil if point is in a Python literal (a comment or string). +Optional argument LIM indicates the beginning of the containing form, +i.e. the limit on how far back to scan." + ;; This is the version used for non-XEmacs, which has a nicer + ;; interface. + ;; + ;; WARNING: Watch out for infinite recursion. + (let* ((lim (or lim (py-point 'bod))) + (state (parse-partial-sexp lim (point)))) + (cond + ((nth 3 state) 'string) + ((nth 4 state) 'comment) + (t nil)))) + +;; XEmacs has a built-in function that should make this much quicker. +;; In this case, lim is ignored +(defun py-fast-in-literal (&optional lim) + "Fast version of `py-in-literal', used only by XEmacs. +Optional LIM is ignored." + ;; don't have to worry about context == 'block-comment + (buffer-syntactic-context)) + +(if (fboundp 'buffer-syntactic-context) + (defalias 'py-in-literal 'py-fast-in-literal)) + + + +;; Menu definitions, only relevent if you have the easymenu.el package +;; (standard in the latest Emacs 19 and XEmacs 19 distributions). +(defvar py-menu nil + "Menu for Python Mode. +This menu will get created automatically if you have the `easymenu' +package. Note that the latest X/Emacs releases contain this package.") + +(and (py-safe (require 'easymenu) t) + (easy-menu-define + py-menu py-mode-map "Python Mode menu" + '("Python" + ["Comment Out Region" py-comment-region (mark)] + ["Uncomment Region" (py-comment-region (point) (mark) '(4)) (mark)] + "-" + ["Mark current block" py-mark-block t] + ["Mark current def" py-mark-def-or-class t] + ["Mark current class" (py-mark-def-or-class t) t] + "-" + ["Shift region left" py-shift-region-left (mark)] + ["Shift region right" py-shift-region-right (mark)] + "-" + ["Import/reload file" py-execute-import-or-reload t] + ["Execute buffer" py-execute-buffer t] + ["Execute region" py-execute-region (mark)] + ["Execute def or class" py-execute-def-or-class (mark)] + ["Execute string" py-execute-string t] + ["Start interpreter..." py-shell t] + "-" + ["Go to start of block" py-goto-block-up t] + ["Go to start of class" (py-beginning-of-def-or-class t) t] + ["Move to end of class" (py-end-of-def-or-class t) t] + ["Move to start of def" py-beginning-of-def-or-class t] + ["Move to end of def" py-end-of-def-or-class t] + "-" + ["Describe mode" py-describe-mode t] + ))) + + + +;; Imenu definitions +(defvar py-imenu-class-regexp + (concat ; <<classes>> + "\\(" ; + "^[ \t]*" ; newline and maybe whitespace + "\\(class[ \t]+[a-zA-Z0-9_]+\\)" ; class name + ; possibly multiple superclasses + "\\([ \t]*\\((\\([a-zA-Z0-9_,. \t\n]\\)*)\\)?\\)" + "[ \t]*:" ; and the final : + "\\)" ; >>classes<< + ) + "Regexp for Python classes for use with the Imenu package." + ) + +(defvar py-imenu-method-regexp + (concat ; <<methods and functions>> + "\\(" ; + "^[ \t]*" ; new line and maybe whitespace + "\\(def[ \t]+" ; function definitions start with def + "\\([a-zA-Z0-9_]+\\)" ; name is here + ; function arguments... +;; "[ \t]*(\\([-+/a-zA-Z0-9_=,\* \t\n.()\"'#]*\\))" + "[ \t]*(\\([^:#]*\\))" + "\\)" ; end of def + "[ \t]*:" ; and then the : + "\\)" ; >>methods and functions<< + ) + "Regexp for Python methods/functions for use with the Imenu package." + ) + +(defvar py-imenu-method-no-arg-parens '(2 8) + "Indices into groups of the Python regexp for use with Imenu. + +Using these values will result in smaller Imenu lists, as arguments to +functions are not listed. + +See the variable `py-imenu-show-method-args-p' for more +information.") + +(defvar py-imenu-method-arg-parens '(2 7) + "Indices into groups of the Python regexp for use with imenu. +Using these values will result in large Imenu lists, as arguments to +functions are listed. + +See the variable `py-imenu-show-method-args-p' for more +information.") + +;; Note that in this format, this variable can still be used with the +;; imenu--generic-function. Otherwise, there is no real reason to have +;; it. +(defvar py-imenu-generic-expression + (cons + (concat + py-imenu-class-regexp + "\\|" ; or... + py-imenu-method-regexp + ) + py-imenu-method-no-arg-parens) + "Generic Python expression which may be used directly with Imenu. +Used by setting the variable `imenu-generic-expression' to this value. +Also, see the function \\[py-imenu-create-index] for a better +alternative for finding the index.") + +;; These next two variables are used when searching for the Python +;; class/definitions. Just saving some time in accessing the +;; generic-python-expression, really. +(defvar py-imenu-generic-regexp nil) +(defvar py-imenu-generic-parens nil) + + +(defun py-imenu-create-index-function () + "Python interface function for the Imenu package. +Finds all Python classes and functions/methods. Calls function +\\[py-imenu-create-index-engine]. See that function for the details +of how this works." + (setq py-imenu-generic-regexp (car py-imenu-generic-expression) + py-imenu-generic-parens (if py-imenu-show-method-args-p + py-imenu-method-arg-parens + py-imenu-method-no-arg-parens)) + (goto-char (point-min)) + ;; Warning: When the buffer has no classes or functions, this will + ;; return nil, which seems proper according to the Imenu API, but + ;; causes an error in the XEmacs port of Imenu. Sigh. + (py-imenu-create-index-engine nil)) + +(defun py-imenu-create-index-engine (&optional start-indent) + "Function for finding Imenu definitions in Python. + +Finds all definitions (classes, methods, or functions) in a Python +file for the Imenu package. + +Returns a possibly nested alist of the form + + (INDEX-NAME . INDEX-POSITION) + +The second element of the alist may be an alist, producing a nested +list as in + + (INDEX-NAME . INDEX-ALIST) + +This function should not be called directly, as it calls itself +recursively and requires some setup. Rather this is the engine for +the function \\[py-imenu-create-index-function]. + +It works recursively by looking for all definitions at the current +indention level. When it finds one, it adds it to the alist. If it +finds a definition at a greater indentation level, it removes the +previous definition from the alist. In its place it adds all +definitions found at the next indentation level. When it finds a +definition that is less indented then the current level, it returns +the alist it has created thus far. + +The optional argument START-INDENT indicates the starting indentation +at which to continue looking for Python classes, methods, or +functions. If this is not supplied, the function uses the indentation +of the first definition found." + (let (index-alist + sub-method-alist + looking-p + def-name prev-name + cur-indent def-pos + (class-paren (first py-imenu-generic-parens)) + (def-paren (second py-imenu-generic-parens))) + (setq looking-p + (re-search-forward py-imenu-generic-regexp (point-max) t)) + (while looking-p + (save-excursion + ;; used to set def-name to this value but generic-extract-name + ;; is new to imenu-1.14. this way it still works with + ;; imenu-1.11 + ;;(imenu--generic-extract-name py-imenu-generic-parens)) + (let ((cur-paren (if (match-beginning class-paren) + class-paren def-paren))) + (setq def-name + (buffer-substring-no-properties (match-beginning cur-paren) + (match-end cur-paren)))) + (save-match-data + (py-beginning-of-def-or-class 'either)) + (beginning-of-line) + (setq cur-indent (current-indentation))) + ;; HACK: want to go to the next correct definition location. We + ;; explicitly list them here but it would be better to have them + ;; in a list. + (setq def-pos + (or (match-beginning class-paren) + (match-beginning def-paren))) + ;; if we don't have a starting indent level, take this one + (or start-indent + (setq start-indent cur-indent)) + ;; if we don't have class name yet, take this one + (or prev-name + (setq prev-name def-name)) + ;; what level is the next definition on? must be same, deeper + ;; or shallower indentation + (cond + ;; Skip code in comments and strings + ((py-in-literal)) + ;; at the same indent level, add it to the list... + ((= start-indent cur-indent) + (push (cons def-name def-pos) index-alist)) + ;; deeper indented expression, recurse + ((< start-indent cur-indent) + ;; the point is currently on the expression we're supposed to + ;; start on, so go back to the last expression. The recursive + ;; call will find this place again and add it to the correct + ;; list + (re-search-backward py-imenu-generic-regexp (point-min) 'move) + (setq sub-method-alist (py-imenu-create-index-engine cur-indent)) + (if sub-method-alist + ;; we put the last element on the index-alist on the start + ;; of the submethod alist so the user can still get to it. + (let ((save-elmt (pop index-alist))) + (push (cons prev-name + (cons save-elmt sub-method-alist)) + index-alist)))) + ;; found less indented expression, we're done. + (t + (setq looking-p nil) + (re-search-backward py-imenu-generic-regexp (point-min) t))) + ;; end-cond + (setq prev-name def-name) + (and looking-p + (setq looking-p + (re-search-forward py-imenu-generic-regexp + (point-max) 'move)))) + (nreverse index-alist))) + + + +(defun py-choose-shell-by-shebang () + "Choose CPython or JPython mode by looking at #! on the first line. +Returns the appropriate mode function. +Used by `py-choose-shell', and similar to but distinct from +`set-auto-mode', though it uses `auto-mode-interpreter-regexp' (if available)." + ;; look for an interpreter specified in the first line + ;; similar to set-auto-mode (files.el) + (let* ((re (if (boundp 'auto-mode-interpreter-regexp) + auto-mode-interpreter-regexp + ;; stolen from Emacs 21.2 + "#![ \t]?\\([^ \t\n]*/bin/env[ \t]\\)?\\([^ \t\n]+\\)")) + (interpreter (save-excursion + (goto-char (point-min)) + (if (looking-at re) + (match-string 2) + ""))) + elt) + ;; Map interpreter name to a mode. + (setq elt (assoc (file-name-nondirectory interpreter) + py-shell-alist)) + (and elt (caddr elt)))) + + + +(defun py-choose-shell-by-import () + "Choose CPython or JPython mode based imports. +If a file imports any packages in `py-jpython-packages', within +`py-import-check-point-max' characters from the start of the file, +return `jpython', otherwise return nil." + (let (mode) + (save-excursion + (goto-char (point-min)) + (while (and (not mode) + (search-forward-regexp + "^\\(\\(from\\)\\|\\(import\\)\\) \\([^ \t\n.]+\\)" + py-import-check-point-max t)) + (setq mode (and (member (match-string 4) py-jpython-packages) + 'jpython + )))) + mode)) + + +(defun py-choose-shell () + "Choose CPython or JPython mode. Returns the appropriate mode function. +This does the following: + - look for an interpreter with `py-choose-shell-by-shebang' + - examine imports using `py-choose-shell-by-import' + - default to the variable `py-default-interpreter'" + (interactive) + (or (py-choose-shell-by-shebang) + (py-choose-shell-by-import) + py-default-interpreter +; 'cpython ;; don't use to py-default-interpreter, because default +; ;; is only way to choose CPython + )) + + +;;;###autoload +(defun python-mode () + "Major mode for editing Python files. +To submit a problem report, enter `\\[py-submit-bug-report]' from a +`python-mode' buffer. Do `\\[py-describe-mode]' for detailed +documentation. To see what version of `python-mode' you are running, +enter `\\[py-version]'. + +This mode knows about Python indentation, tokens, comments and +continuation lines. Paragraphs are separated by blank lines only. + +COMMANDS +\\{py-mode-map} +VARIABLES + +py-indent-offset\t\tindentation increment +py-block-comment-prefix\t\tcomment string used by `comment-region' +py-python-command\t\tshell command to invoke Python interpreter +py-temp-directory\t\tdirectory used for temp files (if needed) +py-beep-if-tab-change\t\tring the bell if `tab-width' is changed" + (interactive) + ;; set up local variables + (kill-all-local-variables) + (make-local-variable 'font-lock-defaults) + (make-local-variable 'paragraph-separate) + (make-local-variable 'paragraph-start) + (make-local-variable 'require-final-newline) + (make-local-variable 'comment-start) + (make-local-variable 'comment-end) + (make-local-variable 'comment-start-skip) + (make-local-variable 'comment-column) + (make-local-variable 'comment-indent-function) + (make-local-variable 'indent-region-function) + (make-local-variable 'indent-line-function) + (make-local-variable 'add-log-current-defun-function) + ;; + (set-syntax-table py-mode-syntax-table) + (setq major-mode 'python-mode + mode-name "Python" + local-abbrev-table python-mode-abbrev-table + font-lock-defaults '(python-font-lock-keywords) + paragraph-separate "^[ \t]*$" + paragraph-start "^[ \t]*$" + require-final-newline t + comment-start "# " + comment-end "" + comment-start-skip "# *" + comment-column 40 + comment-indent-function 'py-comment-indent-function + indent-region-function 'py-indent-region + indent-line-function 'py-indent-line + ;; tell add-log.el how to find the current function/method/variable + add-log-current-defun-function 'py-current-defun + ) + (use-local-map py-mode-map) + ;; add the menu + (if py-menu + (easy-menu-add py-menu)) + ;; Emacs 19 requires this + (if (boundp 'comment-multi-line) + (setq comment-multi-line nil)) + ;; Install Imenu if available + (when (py-safe (require 'imenu)) + (setq imenu-create-index-function #'py-imenu-create-index-function) + (setq imenu-generic-expression py-imenu-generic-expression) + (if (fboundp 'imenu-add-to-menubar) + (imenu-add-to-menubar (format "%s-%s" "IM" mode-name))) + ) + ;; Run the mode hook. Note that py-mode-hook is deprecated. + (if python-mode-hook + (run-hooks 'python-mode-hook) + (run-hooks 'py-mode-hook)) + ;; Now do the automagical guessing + (if py-smart-indentation + (let ((offset py-indent-offset)) + ;; It's okay if this fails to guess a good value + (if (and (py-safe (py-guess-indent-offset)) + (<= py-indent-offset 8) + (>= py-indent-offset 2)) + (setq offset py-indent-offset)) + (setq py-indent-offset offset) + ;; Only turn indent-tabs-mode off if tab-width != + ;; py-indent-offset. Never turn it on, because the user must + ;; have explicitly turned it off. + (if (/= tab-width py-indent-offset) + (setq indent-tabs-mode nil)) + )) + ;; Set the default shell if not already set + (when (null py-which-shell) + (py-toggle-shells (py-choose-shell)))) + + +(defun jpython-mode () + "Major mode for editing JPython/Jython files. +This is a simple wrapper around `python-mode'. +It runs `jpython-mode-hook' then calls `python-mode.' +It is added to `interpreter-mode-alist' and `py-choose-shell'. +" + (interactive) + (python-mode) + (py-toggle-shells 'jpython) + (when jpython-mode-hook + (run-hooks 'jpython-mode-hook))) + + +;; It's handy to add recognition of Python files to the +;; interpreter-mode-alist and to auto-mode-alist. With the former, we +;; can specify different `derived-modes' based on the #! line, but +;; with the latter, we can't. So we just won't add them if they're +;; already added. +(let ((modes '(("jpython" . jpython-mode) + ("jython" . jpython-mode) + ("python" . python-mode)))) + (while modes + (when (not (assoc (car modes) interpreter-mode-alist)) + (push (car modes) interpreter-mode-alist)) + (setq modes (cdr modes)))) + +(when (not (or (rassq 'python-mode auto-mode-alist) + (rassq 'jpython-mode auto-mode-alist))) + (push '("\\.py$" . python-mode) auto-mode-alist)) + + + +;; electric characters +(defun py-outdent-p () + "Returns non-nil if the current line should dedent one level." + (save-excursion + (and (progn (back-to-indentation) + (looking-at py-outdent-re)) + ;; short circuit infloop on illegal construct + (not (bobp)) + (progn (forward-line -1) + (py-goto-initial-line) + (back-to-indentation) + (while (or (looking-at py-blank-or-comment-re) + (bobp)) + (backward-to-indentation 1)) + (not (looking-at py-no-outdent-re))) + ))) + +(defun py-electric-colon (arg) + "Insert a colon. +In certain cases the line is dedented appropriately. If a numeric +argument ARG is provided, that many colons are inserted +non-electrically. Electric behavior is inhibited inside a string or +comment." + (interactive "*P") + (self-insert-command (prefix-numeric-value arg)) + ;; are we in a string or comment? + (if (save-excursion + (let ((pps (parse-partial-sexp (save-excursion + (py-beginning-of-def-or-class) + (point)) + (point)))) + (not (or (nth 3 pps) (nth 4 pps))))) + (save-excursion + (let ((here (point)) + (outdent 0) + (indent (py-compute-indentation t))) + (if (and (not arg) + (py-outdent-p) + (= indent (save-excursion + (py-next-statement -1) + (py-compute-indentation t))) + ) + (setq outdent py-indent-offset)) + ;; Don't indent, only dedent. This assumes that any lines + ;; that are already dedented relative to + ;; py-compute-indentation were put there on purpose. It's + ;; highly annoying to have `:' indent for you. Use TAB, C-c + ;; C-l or C-c C-r to adjust. TBD: Is there a better way to + ;; determine this??? + (if (< (current-indentation) indent) nil + (goto-char here) + (beginning-of-line) + (delete-horizontal-space) + (indent-to (- indent outdent)) + ))))) + + +;; Python subprocess utilities and filters +(defun py-execute-file (proc filename) + "Send to Python interpreter process PROC \"execfile('FILENAME')\". +Make that process's buffer visible and force display. Also make +comint believe the user typed this string so that +`kill-output-from-shell' does The Right Thing." + (let ((curbuf (current-buffer)) + (procbuf (process-buffer proc)) +; (comint-scroll-to-bottom-on-output t) + (msg (format "## working on region in file %s...\n" filename)) + (cmd (format "execfile(r'%s')\n" filename))) + (unwind-protect + (save-excursion + (set-buffer procbuf) + (goto-char (point-max)) + (move-marker (process-mark proc) (point)) + (funcall (process-filter proc) proc msg)) + (set-buffer curbuf)) + (process-send-string proc cmd))) + +(defun py-comint-output-filter-function (string) + "Watch output for Python prompt and exec next file waiting in queue. +This function is appropriate for `comint-output-filter-functions'." + ;; TBD: this should probably use split-string + (when (and (or (string-equal string ">>> ") + (and (>= (length string) 5) + (string-equal (substring string -5) "\n>>> "))) + py-file-queue) + (pop-to-buffer (current-buffer)) + (py-safe (delete-file (car py-file-queue))) + (setq py-file-queue (cdr py-file-queue)) + (if py-file-queue + (let ((pyproc (get-buffer-process (current-buffer)))) + (py-execute-file pyproc (car py-file-queue)))) + )) + +(defun py-pdbtrack-overlay-arrow (activation) + "Activate or de arrow at beginning-of-line in current buffer." + ;; This was derived/simplified from edebug-overlay-arrow + (cond (activation + (setq overlay-arrow-position (make-marker)) + (setq overlay-arrow-string "=>") + (set-marker overlay-arrow-position (py-point 'bol) (current-buffer)) + (setq py-pdbtrack-is-tracking-p t)) + (overlay-arrow-position + (setq overlay-arrow-position nil) + (setq py-pdbtrack-is-tracking-p nil)) + )) + +(defun py-pdbtrack-track-stack-file (text) + "Show the file indicated by the pdb stack entry line, in a separate window. + +Activity is disabled if the buffer-local variable +`py-pdbtrack-do-tracking-p' is nil. + +We depend on the pdb input prompt matching `py-pdbtrack-input-prompt' +at the beginning of the line. + +If the traceback target file path is invalid, we look for the most +recently visited python-mode buffer which either has the name of the +current function \(or class) or which defines the function \(or +class). This is to provide for remote scripts, eg, Zope's 'Script +(Python)' - put a _copy_ of the script in a buffer named for the +script, and set to python-mode, and pdbtrack will find it.)" + ;; Instead of trying to piece things together from partial text + ;; (which can be almost useless depending on Emacs version), we + ;; monitor to the point where we have the next pdb prompt, and then + ;; check all text from comint-last-input-end to process-mark. + ;; + ;; Also, we're very conservative about clearing the overlay arrow, + ;; to minimize residue. This means, for instance, that executing + ;; other pdb commands wipe out the highlight. You can always do a + ;; 'where' (aka 'w') command to reveal the overlay arrow. + (let* ((origbuf (current-buffer)) + (currproc (get-buffer-process origbuf))) + + (if (not (and currproc py-pdbtrack-do-tracking-p)) + (py-pdbtrack-overlay-arrow nil) + + (let* ((procmark (process-mark currproc)) + (block (buffer-substring (max comint-last-input-end + (- procmark + py-pdbtrack-track-range)) + procmark)) + target target_fname target_lineno) + + (if (not (string-match (concat py-pdbtrack-input-prompt "$") block)) + (py-pdbtrack-overlay-arrow nil) + + (setq target (py-pdbtrack-get-source-buffer block)) + + (if (stringp target) + (message "pdbtrack: %s" target) + + (setq target_lineno (car target)) + (setq target_buffer (cadr target)) + (setq target_fname (buffer-file-name target_buffer)) + (switch-to-buffer-other-window target_buffer) + (goto-line target_lineno) + (message "pdbtrack: line %s, file %s" target_lineno target_fname) + (py-pdbtrack-overlay-arrow t) + (pop-to-buffer origbuf t) + + ))))) + ) + +(defun py-pdbtrack-get-source-buffer (block) + "Return line number and buffer of code indicated by block's traceback text. + +We look first to visit the file indicated in the trace. + +Failing that, we look for the most recently visited python-mode buffer +with the same name or having +having the named function. + +If we're unable find the source code we return a string describing the +problem as best as we can determine." + + (if (not (string-match py-pdbtrack-stack-entry-regexp block)) + + "Traceback cue not found" + + (let* ((filename (match-string 1 block)) + (lineno (string-to-int (match-string 2 block))) + (funcname (match-string 3 block)) + funcbuffer) + + (cond ((file-exists-p filename) + (list lineno (find-file-noselect filename))) + + ((setq funcbuffer (py-pdbtrack-grub-for-buffer funcname lineno)) + (if (string-match "/Script (Python)$" filename) + ;; Add in number of lines for leading '##' comments: + (setq lineno + (+ lineno + (save-excursion + (set-buffer funcbuffer) + (count-lines + (point-min) + (max (point-min) + (string-match "^\\([^#]\\|#[^#]\\|#$\\)" + (buffer-substring (point-min) + (point-max))) + )))))) + (list lineno funcbuffer)) + + ((= (elt filename 0) ?\<) + (format "(Non-file source: '%s')" filename)) + + (t (format "Not found: %s(), %s" funcname filename))) + ) + ) + ) + +(defun py-pdbtrack-grub-for-buffer (funcname lineno) + "Find most recent buffer itself named or having function funcname. + +We first check the last buffer this function found, if any, then walk +throught the buffer-list history for python-mode buffers that are +named for funcname or define a function funcname." + (let ((buffers (buffer-list)) + curbuf + got) + (while (and buffers (not got)) + (setq buf (car buffers) + buffers (cdr buffers)) + (if (and (save-excursion (set-buffer buf) + (string= major-mode "python-mode")) + (or (string-match funcname (buffer-name buf)) + (string-match (concat "^\\s-*\\(def\\|class\\)\\s-+" + funcname "\\s-*(") + (save-excursion + (set-buffer buf) + (buffer-substring (point-min) + (point-max)))))) + (setq got buf))) + (setq py-pdbtrack-last-grubbed-buffer got))) + +(defun py-postprocess-output-buffer (buf) + "Highlight exceptions found in BUF. +If an exception occurred return t, otherwise return nil. BUF must exist." + (let (line file bol err-p) + (save-excursion + (set-buffer buf) + (beginning-of-buffer) + (while (re-search-forward py-traceback-line-re nil t) + (setq file (match-string 1) + line (string-to-int (match-string 2)) + bol (py-point 'bol)) + (py-highlight-line bol (py-point 'eol) file line))) + (when (and py-jump-on-exception line) + (beep) + (py-jump-to-exception file line) + (setq err-p t)) + err-p)) + + + +;;; Subprocess commands + +;; only used when (memq 'broken-temp-names py-emacs-features) +(defvar py-serial-number 0) +(defvar py-exception-buffer nil) +(defconst py-output-buffer "*Python Output*") +(make-variable-buffer-local 'py-output-buffer) + +;; for toggling between CPython and JPython +(defvar py-which-shell nil) +(defvar py-which-args py-python-command-args) +(defvar py-which-bufname "Python") +(make-variable-buffer-local 'py-which-shell) +(make-variable-buffer-local 'py-which-args) +(make-variable-buffer-local 'py-which-bufname) + +(defun py-toggle-shells (arg) + "Toggles between the CPython and JPython shells. + +With positive argument ARG (interactively \\[universal-argument]), +uses the CPython shell, with negative ARG uses the JPython shell, and +with a zero argument, toggles the shell. + +Programmatically, ARG can also be one of the symbols `cpython' or +`jpython', equivalent to positive arg and negative arg respectively." + (interactive "P") + ;; default is to toggle + (if (null arg) + (setq arg 0)) + ;; preprocess arg + (cond + ((equal arg 0) + ;; toggle + (if (string-equal py-which-bufname "Python") + (setq arg -1) + (setq arg 1))) + ((equal arg 'cpython) (setq arg 1)) + ((equal arg 'jpython) (setq arg -1))) + (let (msg) + (cond + ((< 0 arg) + ;; set to CPython + (setq py-which-shell py-python-command + py-which-args py-python-command-args + py-which-bufname "Python" + msg "CPython" + mode-name "Python")) + ((> 0 arg) + (setq py-which-shell py-jpython-command + py-which-args py-jpython-command-args + py-which-bufname "JPython" + msg "JPython" + mode-name "JPython")) + ) + (message "Using the %s shell" msg) + (setq py-output-buffer (format "*%s Output*" py-which-bufname)))) + +;;;###autoload +(defun py-shell (&optional argprompt) + "Start an interactive Python interpreter in another window. +This is like Shell mode, except that Python is running in the window +instead of a shell. See the `Interactive Shell' and `Shell Mode' +sections of the Emacs manual for details, especially for the key +bindings active in the `*Python*' buffer. + +With optional \\[universal-argument], the user is prompted for the +flags to pass to the Python interpreter. This has no effect when this +command is used to switch to an existing process, only when a new +process is started. If you use this, you will probably want to ensure +that the current arguments are retained (they will be included in the +prompt). This argument is ignored when this function is called +programmatically, or when running in Emacs 19.34 or older. + +Note: You can toggle between using the CPython interpreter and the +JPython interpreter by hitting \\[py-toggle-shells]. This toggles +buffer local variables which control whether all your subshell +interactions happen to the `*JPython*' or `*Python*' buffers (the +latter is the name used for the CPython buffer). + +Warning: Don't use an interactive Python if you change sys.ps1 or +sys.ps2 from their default values, or if you're running code that +prints `>>> ' or `... ' at the start of a line. `python-mode' can't +distinguish your output from Python's output, and assumes that `>>> ' +at the start of a line is a prompt from Python. Similarly, the Emacs +Shell mode code assumes that both `>>> ' and `... ' at the start of a +line are Python prompts. Bad things can happen if you fool either +mode. + +Warning: If you do any editing *in* the process buffer *while* the +buffer is accepting output from Python, do NOT attempt to `undo' the +changes. Some of the output (nowhere near the parts you changed!) may +be lost if you do. This appears to be an Emacs bug, an unfortunate +interaction between undo and process filters; the same problem exists in +non-Python process buffers using the default (Emacs-supplied) process +filter." + (interactive "P") + ;; Set the default shell if not already set + (when (null py-which-shell) + (py-toggle-shells py-default-interpreter)) + (let ((args py-which-args)) + (when (and argprompt + (interactive-p) + (fboundp 'split-string)) + ;; TBD: Perhaps force "-i" in the final list? + (setq args (split-string + (read-string (concat py-which-bufname + " arguments: ") + (concat + (mapconcat 'identity py-which-args " ") " ") + )))) + (switch-to-buffer-other-window + (apply 'make-comint py-which-bufname py-which-shell nil args)) + (make-local-variable 'comint-prompt-regexp) + (setq comint-prompt-regexp "^>>> \\|^[.][.][.] \\|^(pdb) ") + (add-hook 'comint-output-filter-functions + 'py-comint-output-filter-function) + ;; pdbtrack + (add-hook 'comint-output-filter-functions 'py-pdbtrack-track-stack-file) + (setq py-pdbtrack-do-tracking-p t) + (set-syntax-table py-mode-syntax-table) + (use-local-map py-shell-map) + (run-hooks 'py-shell-hook) + )) + +(defun py-clear-queue () + "Clear the queue of temporary files waiting to execute." + (interactive) + (let ((n (length py-file-queue))) + (mapcar 'delete-file py-file-queue) + (setq py-file-queue nil) + (message "%d pending files de-queued." n))) + + +(defun py-execute-region (start end &optional async) + "Execute the region in a Python interpreter. + +The region is first copied into a temporary file (in the directory +`py-temp-directory'). If there is no Python interpreter shell +running, this file is executed synchronously using +`shell-command-on-region'. If the program is long running, use +\\[universal-argument] to run the command asynchronously in its own +buffer. + +When this function is used programmatically, arguments START and END +specify the region to execute, and optional third argument ASYNC, if +non-nil, specifies to run the command asynchronously in its own +buffer. + +If the Python interpreter shell is running, the region is execfile()'d +in that shell. If you try to execute regions too quickly, +`python-mode' will queue them up and execute them one at a time when +it sees a `>>> ' prompt from Python. Each time this happens, the +process buffer is popped into a window (if it's not already in some +window) so you can see it, and a comment of the form + + \t## working on region in file <name>... + +is inserted at the end. See also the command `py-clear-queue'." + (interactive "r\nP") + ;; Skip ahead to the first non-blank line + (let* ((proc (get-process py-which-bufname)) + (temp (if (memq 'broken-temp-names py-emacs-features) + (let + ((sn py-serial-number) + (pid (and (fboundp 'emacs-pid) (emacs-pid)))) + (setq py-serial-number (1+ py-serial-number)) + (if pid + (format "python-%d-%d" sn pid) + (format "python-%d" sn))) + (make-temp-name "python-"))) + (file (concat (expand-file-name temp py-temp-directory) ".py")) + (cur (current-buffer)) + (buf (get-buffer-create file)) + shell) + ;; Write the contents of the buffer, watching out for indented regions. + (save-excursion + (goto-char start) + (beginning-of-line) + (while (and (looking-at "\\s *$") + (< (point) end)) + (forward-line 1)) + (setq start (point)) + (or (< start end) + (error "Region is empty")) + (let ((needs-if (/= (py-point 'bol) (py-point 'boi)))) + (set-buffer buf) + (python-mode) + (when needs-if + (insert "if 1:\n")) + (insert-buffer-substring cur start end) + ;; Set the shell either to the #! line command, or to the + ;; py-which-shell buffer local variable. + (setq shell (or (py-choose-shell-by-shebang) + (py-choose-shell-by-import) + py-which-shell)))) + (cond + ;; always run the code in its own asynchronous subprocess + (async + ;; User explicitly wants this to run in its own async subprocess + (save-excursion + (set-buffer buf) + (write-region (point-min) (point-max) file nil 'nomsg)) + (let* ((buf (generate-new-buffer-name py-output-buffer)) + ;; TBD: a horrible hack, but why create new Custom variables? + (arg (if (string-equal py-which-bufname "Python") + "-u" ""))) + (start-process py-which-bufname buf shell arg file) + (pop-to-buffer buf) + (py-postprocess-output-buffer buf) + ;; TBD: clean up the temporary file! + )) + ;; if the Python interpreter shell is running, queue it up for + ;; execution there. + (proc + ;; use the existing python shell + (save-excursion + (set-buffer buf) + (write-region (point-min) (point-max) file nil 'nomsg)) + (if (not py-file-queue) + (py-execute-file proc file) + (message "File %s queued for execution" file)) + (setq py-file-queue (append py-file-queue (list file))) + (setq py-exception-buffer (cons file (current-buffer)))) + (t + ;; TBD: a horrible hack, but why create new Custom variables? + (let ((cmd (concat shell (if (string-equal py-which-bufname "JPython") + " -" "")))) + ;; otherwise either run it synchronously in a subprocess + (save-excursion + (set-buffer buf) + (shell-command-on-region (point-min) (point-max) + cmd py-output-buffer)) + ;; shell-command-on-region kills the output buffer if it never + ;; existed and there's no output from the command + (if (not (get-buffer py-output-buffer)) + (message "No output.") + (setq py-exception-buffer (current-buffer)) + (let ((err-p (py-postprocess-output-buffer py-output-buffer))) + (pop-to-buffer py-output-buffer) + (if err-p + (pop-to-buffer py-exception-buffer))) + )) + )) + ;; Clean up after ourselves. + (kill-buffer buf))) + + +;; Code execution commands +(defun py-execute-buffer (&optional async) + "Send the contents of the buffer to a Python interpreter. +If the file local variable `py-master-file' is non-nil, execute the +named file instead of the buffer's file. + +If there is a *Python* process buffer it is used. If a clipping +restriction is in effect, only the accessible portion of the buffer is +sent. A trailing newline will be supplied if needed. + +See the `\\[py-execute-region]' docs for an account of some +subtleties, including the use of the optional ASYNC argument." + (interactive "P") + (if py-master-file + (let* ((filename (expand-file-name py-master-file)) + (buffer (or (get-file-buffer filename) + (find-file-noselect filename)))) + (set-buffer buffer))) + (py-execute-region (point-min) (point-max) async)) + +(defun py-execute-import-or-reload (&optional async) + "Import the current buffer's file in a Python interpreter. + +If the file has already been imported, then do reload instead to get +the latest version. + +If the file's name does not end in \".py\", then do execfile instead. + +If the current buffer is not visiting a file, do `py-execute-buffer' +instead. + +If the file local variable `py-master-file' is non-nil, import or +reload the named file instead of the buffer's file. The file may be +saved based on the value of `py-execute-import-or-reload-save-p'. + +See the `\\[py-execute-region]' docs for an account of some +subtleties, including the use of the optional ASYNC argument. + +This may be preferable to `\\[py-execute-buffer]' because: + + - Definitions stay in their module rather than appearing at top + level, where they would clutter the global namespace and not affect + uses of qualified names (MODULE.NAME). + + - The Python debugger gets line number information about the functions." + (interactive "P") + ;; Check file local variable py-master-file + (if py-master-file + (let* ((filename (expand-file-name py-master-file)) + (buffer (or (get-file-buffer filename) + (find-file-noselect filename)))) + (set-buffer buffer))) + (let ((file (buffer-file-name (current-buffer)))) + (if file + (progn + ;; Maybe save some buffers + (save-some-buffers (not py-ask-about-save) nil) + (py-execute-string + (if (string-match "\\.py$" file) + (let ((f (file-name-sans-extension + (file-name-nondirectory file)))) + (format "if globals().has_key('%s'):\n reload(%s)\nelse:\n import %s\n" + f f f)) + (format "execfile(r'%s')\n" file)) + async)) + ;; else + (py-execute-buffer async)))) + + +(defun py-execute-def-or-class (&optional async) + "Send the current function or class definition to a Python interpreter. + +If there is a *Python* process buffer it is used. + +See the `\\[py-execute-region]' docs for an account of some +subtleties, including the use of the optional ASYNC argument." + (interactive "P") + (save-excursion + (py-mark-def-or-class) + ;; mark is before point + (py-execute-region (mark) (point) async))) + + +(defun py-execute-string (string &optional async) + "Send the argument STRING to a Python interpreter. + +If there is a *Python* process buffer it is used. + +See the `\\[py-execute-region]' docs for an account of some +subtleties, including the use of the optional ASYNC argument." + (interactive "sExecute Python command: ") + (save-excursion + (set-buffer (get-buffer-create + (generate-new-buffer-name " *Python Command*"))) + (insert string) + (py-execute-region (point-min) (point-max) async))) + + + +(defun py-jump-to-exception (file line) + "Jump to the Python code in FILE at LINE." + (let ((buffer (cond ((string-equal file "<stdin>") + (if (consp py-exception-buffer) + (cdr py-exception-buffer) + py-exception-buffer)) + ((and (consp py-exception-buffer) + (string-equal file (car py-exception-buffer))) + (cdr py-exception-buffer)) + ((py-safe (find-file-noselect file))) + ;; could not figure out what file the exception + ;; is pointing to, so prompt for it + (t (find-file (read-file-name "Exception file: " + nil + file t)))))) + (pop-to-buffer buffer) + ;; Force Python mode + (if (not (eq major-mode 'python-mode)) + (python-mode)) + (goto-line line) + (message "Jumping to exception in file %s on line %d" file line))) + +(defun py-mouseto-exception (event) + "Jump to the code which caused the Python exception at EVENT. +EVENT is usually a mouse click." + (interactive "e") + (cond + ((fboundp 'event-point) + ;; XEmacs + (let* ((point (event-point event)) + (buffer (event-buffer event)) + (e (and point buffer (extent-at point buffer 'py-exc-info))) + (info (and e (extent-property e 'py-exc-info)))) + (message "Event point: %d, info: %s" point info) + (and info + (py-jump-to-exception (car info) (cdr info))) + )) + ;; Emacs -- Please port this! + )) + +(defun py-goto-exception () + "Go to the line indicated by the traceback." + (interactive) + (let (file line) + (save-excursion + (beginning-of-line) + (if (looking-at py-traceback-line-re) + (setq file (match-string 1) + line (string-to-int (match-string 2))))) + (if (not file) + (error "Not on a traceback line")) + (py-jump-to-exception file line))) + +(defun py-find-next-exception (start buffer searchdir errwhere) + "Find the next Python exception and jump to the code that caused it. +START is the buffer position in BUFFER from which to begin searching +for an exception. SEARCHDIR is a function, either +`re-search-backward' or `re-search-forward' indicating the direction +to search. ERRWHERE is used in an error message if the limit (top or +bottom) of the trackback stack is encountered." + (let (file line) + (save-excursion + (set-buffer buffer) + (goto-char (py-point start)) + (if (funcall searchdir py-traceback-line-re nil t) + (setq file (match-string 1) + line (string-to-int (match-string 2))))) + (if (and file line) + (py-jump-to-exception file line) + (error "%s of traceback" errwhere)))) + +(defun py-down-exception (&optional bottom) + "Go to the next line down in the traceback. +With \\[univeral-argument] (programmatically, optional argument +BOTTOM), jump to the bottom (innermost) exception in the exception +stack." + (interactive "P") + (let* ((proc (get-process "Python")) + (buffer (if proc "*Python*" py-output-buffer))) + (if bottom + (py-find-next-exception 'eob buffer 're-search-backward "Bottom") + (py-find-next-exception 'eol buffer 're-search-forward "Bottom")))) + +(defun py-up-exception (&optional top) + "Go to the previous line up in the traceback. +With \\[universal-argument] (programmatically, optional argument TOP) +jump to the top (outermost) exception in the exception stack." + (interactive "P") + (let* ((proc (get-process "Python")) + (buffer (if proc "*Python*" py-output-buffer))) + (if top + (py-find-next-exception 'bob buffer 're-search-forward "Top") + (py-find-next-exception 'bol buffer 're-search-backward "Top")))) + + +;; Electric deletion +(defun py-electric-backspace (arg) + "Delete preceding character or levels of indentation. +Deletion is performed by calling the function in `py-backspace-function' +with a single argument (the number of characters to delete). + +If point is at the leftmost column, delete the preceding newline. + +Otherwise, if point is at the leftmost non-whitespace character of a +line that is neither a continuation line nor a non-indenting comment +line, or if point is at the end of a blank line, this command reduces +the indentation to match that of the line that opened the current +block of code. The line that opened the block is displayed in the +echo area to help you keep track of where you are. With +\\[universal-argument] dedents that many blocks (but not past column +zero). + +Otherwise the preceding character is deleted, converting a tab to +spaces if needed so that only a single column position is deleted. +\\[universal-argument] specifies how many characters to delete; +default is 1. + +When used programmatically, argument ARG specifies the number of +blocks to dedent, or the number of characters to delete, as indicated +above." + (interactive "*p") + (if (or (/= (current-indentation) (current-column)) + (bolp) + (py-continuation-line-p) +; (not py-honor-comment-indentation) +; (looking-at "#[^ \t\n]") ; non-indenting # + ) + (funcall py-backspace-function arg) + ;; else indent the same as the colon line that opened the block + ;; force non-blank so py-goto-block-up doesn't ignore it + (insert-char ?* 1) + (backward-char) + (let ((base-indent 0) ; indentation of base line + (base-text "") ; and text of base line + (base-found-p nil)) + (save-excursion + (while (< 0 arg) + (condition-case nil ; in case no enclosing block + (progn + (py-goto-block-up 'no-mark) + (setq base-indent (current-indentation) + base-text (py-suck-up-leading-text) + base-found-p t)) + (error nil)) + (setq arg (1- arg)))) + (delete-char 1) ; toss the dummy character + (delete-horizontal-space) + (indent-to base-indent) + (if base-found-p + (message "Closes block: %s" base-text))))) + + +(defun py-electric-delete (arg) + "Delete preceding or following character or levels of whitespace. + +The behavior of this function depends on the variable +`delete-key-deletes-forward'. If this variable is nil (or does not +exist, as in older Emacsen and non-XEmacs versions), then this +function behaves identically to \\[c-electric-backspace]. + +If `delete-key-deletes-forward' is non-nil and is supported in your +Emacs, then deletion occurs in the forward direction, by calling the +function in `py-delete-function'. + +\\[universal-argument] (programmatically, argument ARG) specifies the +number of characters to delete (default is 1)." + (interactive "*p") + (if (or (and (fboundp 'delete-forward-p) ;XEmacs 21 + (delete-forward-p)) + (and (boundp 'delete-key-deletes-forward) ;XEmacs 20 + delete-key-deletes-forward)) + (funcall py-delete-function arg) + (py-electric-backspace arg))) + +;; required for pending-del and delsel modes +(put 'py-electric-colon 'delete-selection t) ;delsel +(put 'py-electric-colon 'pending-delete t) ;pending-del +(put 'py-electric-backspace 'delete-selection 'supersede) ;delsel +(put 'py-electric-backspace 'pending-delete 'supersede) ;pending-del +(put 'py-electric-delete 'delete-selection 'supersede) ;delsel +(put 'py-electric-delete 'pending-delete 'supersede) ;pending-del + + + +(defun py-indent-line (&optional arg) + "Fix the indentation of the current line according to Python rules. +With \\[universal-argument] (programmatically, the optional argument +ARG non-nil), ignore dedenting rules for block closing statements +(e.g. return, raise, break, continue, pass) + +This function is normally bound to `indent-line-function' so +\\[indent-for-tab-command] will call it." + (interactive "P") + (let* ((ci (current-indentation)) + (move-to-indentation-p (<= (current-column) ci)) + (need (py-compute-indentation (not arg)))) + ;; see if we need to dedent + (if (py-outdent-p) + (setq need (- need py-indent-offset))) + (if (/= ci need) + (save-excursion + (beginning-of-line) + (delete-horizontal-space) + (indent-to need))) + (if move-to-indentation-p (back-to-indentation)))) + +(defun py-newline-and-indent () + "Strives to act like the Emacs `newline-and-indent'. +This is just `strives to' because correct indentation can't be computed +from scratch for Python code. In general, deletes the whitespace before +point, inserts a newline, and takes an educated guess as to how you want +the new line indented." + (interactive) + (let ((ci (current-indentation))) + (if (< ci (current-column)) ; if point beyond indentation + (newline-and-indent) + ;; else try to act like newline-and-indent "normally" acts + (beginning-of-line) + (insert-char ?\n 1) + (move-to-column ci)))) + +(defun py-compute-indentation (honor-block-close-p) + "Compute Python indentation. +When HONOR-BLOCK-CLOSE-P is non-nil, statements such as `return', +`raise', `break', `continue', and `pass' force one level of +dedenting." + (save-excursion + (beginning-of-line) + (let* ((bod (py-point 'bod)) + (pps (parse-partial-sexp bod (point))) + (boipps (parse-partial-sexp bod (py-point 'boi))) + placeholder) + (cond + ;; are we inside a multi-line string or comment? + ((or (and (nth 3 pps) (nth 3 boipps)) + (and (nth 4 pps) (nth 4 boipps))) + (save-excursion + (if (not py-align-multiline-strings-p) 0 + ;; skip back over blank & non-indenting comment lines + ;; note: will skip a blank or non-indenting comment line + ;; that happens to be a continuation line too + (re-search-backward "^[ \t]*\\([^ \t\n#]\\|#[ \t\n]\\)" nil 'move) + (back-to-indentation) + (current-column)))) + ;; are we on a continuation line? + ((py-continuation-line-p) + (let ((startpos (point)) + (open-bracket-pos (py-nesting-level)) + endpos searching found state) + (if open-bracket-pos + (progn + ;; align with first item in list; else a normal + ;; indent beyond the line with the open bracket + (goto-char (1+ open-bracket-pos)) ; just beyond bracket + ;; is the first list item on the same line? + (skip-chars-forward " \t") + (if (null (memq (following-char) '(?\n ?# ?\\))) + ; yes, so line up with it + (current-column) + ;; first list item on another line, or doesn't exist yet + (forward-line 1) + (while (and (< (point) startpos) + (looking-at "[ \t]*[#\n\\\\]")) ; skip noise + (forward-line 1)) + (if (and (< (point) startpos) + (/= startpos + (save-excursion + (goto-char (1+ open-bracket-pos)) + (forward-comment (point-max)) + (point)))) + ;; again mimic the first list item + (current-indentation) + ;; else they're about to enter the first item + (goto-char open-bracket-pos) + (setq placeholder (point)) + (py-goto-initial-line) + (py-goto-beginning-of-tqs + (save-excursion (nth 3 (parse-partial-sexp + placeholder (point))))) + (+ (current-indentation) py-indent-offset)))) + + ;; else on backslash continuation line + (forward-line -1) + (if (py-continuation-line-p) ; on at least 3rd line in block + (current-indentation) ; so just continue the pattern + ;; else started on 2nd line in block, so indent more. + ;; if base line is an assignment with a start on a RHS, + ;; indent to 2 beyond the leftmost "="; else skip first + ;; chunk of non-whitespace characters on base line, + 1 more + ;; column + (end-of-line) + (setq endpos (point) + searching t) + (back-to-indentation) + (setq startpos (point)) + ;; look at all "=" from left to right, stopping at first + ;; one not nested in a list or string + (while searching + (skip-chars-forward "^=" endpos) + (if (= (point) endpos) + (setq searching nil) + (forward-char 1) + (setq state (parse-partial-sexp startpos (point))) + (if (and (zerop (car state)) ; not in a bracket + (null (nth 3 state))) ; & not in a string + (progn + (setq searching nil) ; done searching in any case + (setq found + (not (or + (eq (following-char) ?=) + (memq (char-after (- (point) 2)) + '(?< ?> ?!))))))))) + (if (or (not found) ; not an assignment + (looking-at "[ \t]*\\\\")) ; <=><spaces><backslash> + (progn + (goto-char startpos) + (skip-chars-forward "^ \t\n"))) + ;; if this is a continuation for a block opening + ;; statement, add some extra offset. + (+ (current-column) (if (py-statement-opens-block-p) + py-continuation-offset 0) + 1) + )))) + + ;; not on a continuation line + ((bobp) (current-indentation)) + + ;; Dfn: "Indenting comment line". A line containing only a + ;; comment, but which is treated like a statement for + ;; indentation calculation purposes. Such lines are only + ;; treated specially by the mode; they are not treated + ;; specially by the Python interpreter. + + ;; The rules for indenting comment lines are a line where: + ;; - the first non-whitespace character is `#', and + ;; - the character following the `#' is whitespace, and + ;; - the line is dedented with respect to (i.e. to the left + ;; of) the indentation of the preceding non-blank line. + + ;; The first non-blank line following an indenting comment + ;; line is given the same amount of indentation as the + ;; indenting comment line. + + ;; All other comment-only lines are ignored for indentation + ;; purposes. + + ;; Are we looking at a comment-only line which is *not* an + ;; indenting comment line? If so, we assume that it's been + ;; placed at the desired indentation, so leave it alone. + ;; Indenting comment lines are aligned as statements down + ;; below. + ((and (looking-at "[ \t]*#[^ \t\n]") + ;; NOTE: this test will not be performed in older Emacsen + (fboundp 'forward-comment) + (<= (current-indentation) + (save-excursion + (forward-comment (- (point-max))) + (current-indentation)))) + (current-indentation)) + + ;; else indentation based on that of the statement that + ;; precedes us; use the first line of that statement to + ;; establish the base, in case the user forced a non-std + ;; indentation for the continuation lines (if any) + (t + ;; skip back over blank & non-indenting comment lines note: + ;; will skip a blank or non-indenting comment line that + ;; happens to be a continuation line too. use fast Emacs 19 + ;; function if it's there. + (if (and (eq py-honor-comment-indentation nil) + (fboundp 'forward-comment)) + (forward-comment (- (point-max))) + (let ((prefix-re (concat py-block-comment-prefix "[ \t]*")) + done) + (while (not done) + (re-search-backward "^[ \t]*\\([^ \t\n#]\\|#\\)" nil 'move) + (setq done (or (bobp) + (and (eq py-honor-comment-indentation t) + (save-excursion + (back-to-indentation) + (not (looking-at prefix-re)) + )) + (and (not (eq py-honor-comment-indentation t)) + (save-excursion + (back-to-indentation) + (and (not (looking-at prefix-re)) + (or (looking-at "[^#]") + (not (zerop (current-column))) + )) + )) + )) + ))) + ;; if we landed inside a string, go to the beginning of that + ;; string. this handles triple quoted, multi-line spanning + ;; strings. + (py-goto-beginning-of-tqs (nth 3 (parse-partial-sexp bod (point)))) + ;; now skip backward over continued lines + (setq placeholder (point)) + (py-goto-initial-line) + ;; we may *now* have landed in a TQS, so find the beginning of + ;; this string. + (py-goto-beginning-of-tqs + (save-excursion (nth 3 (parse-partial-sexp + placeholder (point))))) + (+ (current-indentation) + (if (py-statement-opens-block-p) + py-indent-offset + (if (and honor-block-close-p (py-statement-closes-block-p)) + (- py-indent-offset) + 0))) + ))))) + +(defun py-guess-indent-offset (&optional global) + "Guess a good value for, and change, `py-indent-offset'. + +By default, make a buffer-local copy of `py-indent-offset' with the +new value, so that other Python buffers are not affected. With +\\[universal-argument] (programmatically, optional argument GLOBAL), +change the global value of `py-indent-offset'. This affects all +Python buffers (that don't have their own buffer-local copy), both +those currently existing and those created later in the Emacs session. + +Some people use a different value for `py-indent-offset' than you use. +There's no excuse for such foolishness, but sometimes you have to deal +with their ugly code anyway. This function examines the file and sets +`py-indent-offset' to what it thinks it was when they created the +mess. + +Specifically, it searches forward from the statement containing point, +looking for a line that opens a block of code. `py-indent-offset' is +set to the difference in indentation between that line and the Python +statement following it. If the search doesn't succeed going forward, +it's tried again going backward." + (interactive "P") ; raw prefix arg + (let (new-value + (start (point)) + (restart (point)) + (found nil) + colon-indent) + (py-goto-initial-line) + (while (not (or found (eobp))) + (when (and (re-search-forward ":[ \t]*\\($\\|[#\\]\\)" nil 'move) + (not (py-in-literal restart))) + (setq restart (point)) + (py-goto-initial-line) + (if (py-statement-opens-block-p) + (setq found t) + (goto-char restart)))) + (unless found + (goto-char start) + (py-goto-initial-line) + (while (not (or found (bobp))) + (setq found (and + (re-search-backward ":[ \t]*\\($\\|[#\\]\\)" nil 'move) + (or (py-goto-initial-line) t) ; always true -- side effect + (py-statement-opens-block-p))))) + (setq colon-indent (current-indentation) + found (and found (zerop (py-next-statement 1))) + new-value (- (current-indentation) colon-indent)) + (goto-char start) + (if (not found) + (error "Sorry, couldn't guess a value for py-indent-offset") + (funcall (if global 'kill-local-variable 'make-local-variable) + 'py-indent-offset) + (setq py-indent-offset new-value) + (or noninteractive + (message "%s value of py-indent-offset set to %d" + (if global "Global" "Local") + py-indent-offset))) + )) + +(defun py-comment-indent-function () + "Python version of `comment-indent-function'." + ;; This is required when filladapt is turned off. Without it, when + ;; filladapt is not used, comments which start in column zero + ;; cascade one character to the right + (save-excursion + (beginning-of-line) + (let ((eol (py-point 'eol))) + (and comment-start-skip + (re-search-forward comment-start-skip eol t) + (setq eol (match-beginning 0))) + (goto-char eol) + (skip-chars-backward " \t") + (max comment-column (+ (current-column) (if (bolp) 0 1))) + ))) + +(defun py-narrow-to-defun (&optional class) + "Make text outside current defun invisible. +The defun visible is the one that contains point or follows point. +Optional CLASS is passed directly to `py-beginning-of-def-or-class'." + (interactive "P") + (save-excursion + (widen) + (py-end-of-def-or-class class) + (let ((end (point))) + (py-beginning-of-def-or-class class) + (narrow-to-region (point) end)))) + + +(defun py-shift-region (start end count) + "Indent lines from START to END by COUNT spaces." + (save-excursion + (goto-char end) + (beginning-of-line) + (setq end (point)) + (goto-char start) + (beginning-of-line) + (setq start (point)) + (indent-rigidly start end count))) + +(defun py-shift-region-left (start end &optional count) + "Shift region of Python code to the left. +The lines from the line containing the start of the current region up +to (but not including) the line containing the end of the region are +shifted to the left, by `py-indent-offset' columns. + +If a prefix argument is given, the region is instead shifted by that +many columns. With no active region, dedent only the current line. +You cannot dedent the region if any line is already at column zero." + (interactive + (let ((p (point)) + (m (mark)) + (arg current-prefix-arg)) + (if m + (list (min p m) (max p m) arg) + (list p (save-excursion (forward-line 1) (point)) arg)))) + ;; if any line is at column zero, don't shift the region + (save-excursion + (goto-char start) + (while (< (point) end) + (back-to-indentation) + (if (and (zerop (current-column)) + (not (looking-at "\\s *$"))) + (error "Region is at left edge")) + (forward-line 1))) + (py-shift-region start end (- (prefix-numeric-value + (or count py-indent-offset)))) + (py-keep-region-active)) + +(defun py-shift-region-right (start end &optional count) + "Shift region of Python code to the right. +The lines from the line containing the start of the current region up +to (but not including) the line containing the end of the region are +shifted to the right, by `py-indent-offset' columns. + +If a prefix argument is given, the region is instead shifted by that +many columns. With no active region, indent only the current line." + (interactive + (let ((p (point)) + (m (mark)) + (arg current-prefix-arg)) + (if m + (list (min p m) (max p m) arg) + (list p (save-excursion (forward-line 1) (point)) arg)))) + (py-shift-region start end (prefix-numeric-value + (or count py-indent-offset))) + (py-keep-region-active)) + +(defun py-indent-region (start end &optional indent-offset) + "Reindent a region of Python code. + +The lines from the line containing the start of the current region up +to (but not including) the line containing the end of the region are +reindented. If the first line of the region has a non-whitespace +character in the first column, the first line is left alone and the +rest of the region is reindented with respect to it. Else the entire +region is reindented with respect to the (closest code or indenting +comment) statement immediately preceding the region. + +This is useful when code blocks are moved or yanked, when enclosing +control structures are introduced or removed, or to reformat code +using a new value for the indentation offset. + +If a numeric prefix argument is given, it will be used as the value of +the indentation offset. Else the value of `py-indent-offset' will be +used. + +Warning: The region must be consistently indented before this function +is called! This function does not compute proper indentation from +scratch (that's impossible in Python), it merely adjusts the existing +indentation to be correct in context. + +Warning: This function really has no idea what to do with +non-indenting comment lines, and shifts them as if they were indenting +comment lines. Fixing this appears to require telepathy. + +Special cases: whitespace is deleted from blank lines; continuation +lines are shifted by the same amount their initial line was shifted, +in order to preserve their relative indentation with respect to their +initial line; and comment lines beginning in column 1 are ignored." + (interactive "*r\nP") ; region; raw prefix arg + (save-excursion + (goto-char end) (beginning-of-line) (setq end (point-marker)) + (goto-char start) (beginning-of-line) + (let ((py-indent-offset (prefix-numeric-value + (or indent-offset py-indent-offset))) + (indents '(-1)) ; stack of active indent levels + (target-column 0) ; column to which to indent + (base-shifted-by 0) ; amount last base line was shifted + (indent-base (if (looking-at "[ \t\n]") + (py-compute-indentation t) + 0)) + ci) + (while (< (point) end) + (setq ci (current-indentation)) + ;; figure out appropriate target column + (cond + ((or (eq (following-char) ?#) ; comment in column 1 + (looking-at "[ \t]*$")) ; entirely blank + (setq target-column 0)) + ((py-continuation-line-p) ; shift relative to base line + (setq target-column (+ ci base-shifted-by))) + (t ; new base line + (if (> ci (car indents)) ; going deeper; push it + (setq indents (cons ci indents)) + ;; else we should have seen this indent before + (setq indents (memq ci indents)) ; pop deeper indents + (if (null indents) + (error "Bad indentation in region, at line %d" + (save-restriction + (widen) + (1+ (count-lines 1 (point))))))) + (setq target-column (+ indent-base + (* py-indent-offset + (- (length indents) 2)))) + (setq base-shifted-by (- target-column ci)))) + ;; shift as needed + (if (/= ci target-column) + (progn + (delete-horizontal-space) + (indent-to target-column))) + (forward-line 1)))) + (set-marker end nil)) + +(defun py-comment-region (beg end &optional arg) + "Like `comment-region' but uses double hash (`#') comment starter." + (interactive "r\nP") + (let ((comment-start py-block-comment-prefix)) + (comment-region beg end arg))) + + +;; Functions for moving point +(defun py-previous-statement (count) + "Go to the start of the COUNTth preceding Python statement. +By default, goes to the previous statement. If there is no such +statement, goes to the first statement. Return count of statements +left to move. `Statements' do not include blank, comment, or +continuation lines." + (interactive "p") ; numeric prefix arg + (if (< count 0) (py-next-statement (- count)) + (py-goto-initial-line) + (let (start) + (while (and + (setq start (point)) ; always true -- side effect + (> count 0) + (zerop (forward-line -1)) + (py-goto-statement-at-or-above)) + (setq count (1- count))) + (if (> count 0) (goto-char start))) + count)) + +(defun py-next-statement (count) + "Go to the start of next Python statement. +If the statement at point is the i'th Python statement, goes to the +start of statement i+COUNT. If there is no such statement, goes to the +last statement. Returns count of statements left to move. `Statements' +do not include blank, comment, or continuation lines." + (interactive "p") ; numeric prefix arg + (if (< count 0) (py-previous-statement (- count)) + (beginning-of-line) + (let (start) + (while (and + (setq start (point)) ; always true -- side effect + (> count 0) + (py-goto-statement-below)) + (setq count (1- count))) + (if (> count 0) (goto-char start))) + count)) + +(defun py-goto-block-up (&optional nomark) + "Move up to start of current block. +Go to the statement that starts the smallest enclosing block; roughly +speaking, this will be the closest preceding statement that ends with a +colon and is indented less than the statement you started on. If +successful, also sets the mark to the starting point. + +`\\[py-mark-block]' can be used afterward to mark the whole code +block, if desired. + +If called from a program, the mark will not be set if optional argument +NOMARK is not nil." + (interactive) + (let ((start (point)) + (found nil) + initial-indent) + (py-goto-initial-line) + ;; if on blank or non-indenting comment line, use the preceding stmt + (if (looking-at "[ \t]*\\($\\|#[^ \t\n]\\)") + (progn + (py-goto-statement-at-or-above) + (setq found (py-statement-opens-block-p)))) + ;; search back for colon line indented less + (setq initial-indent (current-indentation)) + (if (zerop initial-indent) + ;; force fast exit + (goto-char (point-min))) + (while (not (or found (bobp))) + (setq found + (and + (re-search-backward ":[ \t]*\\($\\|[#\\]\\)" nil 'move) + (or (py-goto-initial-line) t) ; always true -- side effect + (< (current-indentation) initial-indent) + (py-statement-opens-block-p)))) + (if found + (progn + (or nomark (push-mark start)) + (back-to-indentation)) + (goto-char start) + (error "Enclosing block not found")))) + +(defun py-beginning-of-def-or-class (&optional class count) + "Move point to start of `def' or `class'. + +Searches back for the closest preceding `def'. If you supply a prefix +arg, looks for a `class' instead. The docs below assume the `def' +case; just substitute `class' for `def' for the other case. +Programmatically, if CLASS is `either', then moves to either `class' +or `def'. + +When second optional argument is given programmatically, move to the +COUNTth start of `def'. + +If point is in a `def' statement already, and after the `d', simply +moves point to the start of the statement. + +Otherwise (i.e. when point is not in a `def' statement, or at or +before the `d' of a `def' statement), searches for the closest +preceding `def' statement, and leaves point at its start. If no such +statement can be found, leaves point at the start of the buffer. + +Returns t iff a `def' statement is found by these rules. + +Note that doing this command repeatedly will take you closer to the +start of the buffer each time. + +To mark the current `def', see `\\[py-mark-def-or-class]'." + (interactive "P") ; raw prefix arg + (setq count (or count 1)) + (let ((at-or-before-p (<= (current-column) (current-indentation))) + (start-of-line (goto-char (py-point 'bol))) + (start-of-stmt (goto-char (py-point 'bos))) + (start-re (cond ((eq class 'either) "^[ \t]*\\(class\\|def\\)\\>") + (class "^[ \t]*class\\>") + (t "^[ \t]*def\\>"))) + ) + ;; searching backward + (if (and (< 0 count) + (or (/= start-of-stmt start-of-line) + (not at-or-before-p))) + (end-of-line)) + ;; search forward + (if (and (> 0 count) + (zerop (current-column)) + (looking-at start-re)) + (end-of-line)) + (if (re-search-backward start-re nil 'move count) + (goto-char (match-beginning 0))))) + +;; Backwards compatibility +(defalias 'beginning-of-python-def-or-class 'py-beginning-of-def-or-class) + +(defun py-end-of-def-or-class (&optional class count) + "Move point beyond end of `def' or `class' body. + +By default, looks for an appropriate `def'. If you supply a prefix +arg, looks for a `class' instead. The docs below assume the `def' +case; just substitute `class' for `def' for the other case. +Programmatically, if CLASS is `either', then moves to either `class' +or `def'. + +When second optional argument is given programmatically, move to the +COUNTth end of `def'. + +If point is in a `def' statement already, this is the `def' we use. + +Else, if the `def' found by `\\[py-beginning-of-def-or-class]' +contains the statement you started on, that's the `def' we use. + +Otherwise, we search forward for the closest following `def', and use that. + +If a `def' can be found by these rules, point is moved to the start of +the line immediately following the `def' block, and the position of the +start of the `def' is returned. + +Else point is moved to the end of the buffer, and nil is returned. + +Note that doing this command repeatedly will take you closer to the +end of the buffer each time. + +To mark the current `def', see `\\[py-mark-def-or-class]'." + (interactive "P") ; raw prefix arg + (if (and count (/= count 1)) + (py-beginning-of-def-or-class (- 1 count))) + (let ((start (progn (py-goto-initial-line) (point))) + (which (cond ((eq class 'either) "\\(class\\|def\\)") + (class "class") + (t "def"))) + (state 'not-found)) + ;; move point to start of appropriate def/class + (if (looking-at (concat "[ \t]*" which "\\>")) ; already on one + (setq state 'at-beginning) + ;; else see if py-beginning-of-def-or-class hits container + (if (and (py-beginning-of-def-or-class class) + (progn (py-goto-beyond-block) + (> (point) start))) + (setq state 'at-end) + ;; else search forward + (goto-char start) + (if (re-search-forward (concat "^[ \t]*" which "\\>") nil 'move) + (progn (setq state 'at-beginning) + (beginning-of-line))))) + (cond + ((eq state 'at-beginning) (py-goto-beyond-block) t) + ((eq state 'at-end) t) + ((eq state 'not-found) nil) + (t (error "Internal error in `py-end-of-def-or-class'"))))) + +;; Backwards compabitility +(defalias 'end-of-python-def-or-class 'py-end-of-def-or-class) + + +;; Functions for marking regions +(defun py-mark-block (&optional extend just-move) + "Mark following block of lines. With prefix arg, mark structure. +Easier to use than explain. It sets the region to an `interesting' +block of succeeding lines. If point is on a blank line, it goes down to +the next non-blank line. That will be the start of the region. The end +of the region depends on the kind of line at the start: + + - If a comment, the region will include all succeeding comment lines up + to (but not including) the next non-comment line (if any). + + - Else if a prefix arg is given, and the line begins one of these + structures: + + if elif else try except finally for while def class + + the region will be set to the body of the structure, including + following blocks that `belong' to it, but excluding trailing blank + and comment lines. E.g., if on a `try' statement, the `try' block + and all (if any) of the following `except' and `finally' blocks + that belong to the `try' structure will be in the region. Ditto + for if/elif/else, for/else and while/else structures, and (a bit + degenerate, since they're always one-block structures) def and + class blocks. + + - Else if no prefix argument is given, and the line begins a Python + block (see list above), and the block is not a `one-liner' (i.e., + the statement ends with a colon, not with code), the region will + include all succeeding lines up to (but not including) the next + code statement (if any) that's indented no more than the starting + line, except that trailing blank and comment lines are excluded. + E.g., if the starting line begins a multi-statement `def' + structure, the region will be set to the full function definition, + but without any trailing `noise' lines. + + - Else the region will include all succeeding lines up to (but not + including) the next blank line, or code or indenting-comment line + indented strictly less than the starting line. Trailing indenting + comment lines are included in this case, but not trailing blank + lines. + +A msg identifying the location of the mark is displayed in the echo +area; or do `\\[exchange-point-and-mark]' to flip down to the end. + +If called from a program, optional argument EXTEND plays the role of +the prefix arg, and if optional argument JUST-MOVE is not nil, just +moves to the end of the block (& does not set mark or display a msg)." + (interactive "P") ; raw prefix arg + (py-goto-initial-line) + ;; skip over blank lines + (while (and + (looking-at "[ \t]*$") ; while blank line + (not (eobp))) ; & somewhere to go + (forward-line 1)) + (if (eobp) + (error "Hit end of buffer without finding a non-blank stmt")) + (let ((initial-pos (point)) + (initial-indent (current-indentation)) + last-pos ; position of last stmt in region + (followers + '((if elif else) (elif elif else) (else) + (try except finally) (except except) (finally) + (for else) (while else) + (def) (class) ) ) + first-symbol next-symbol) + + (cond + ;; if comment line, suck up the following comment lines + ((looking-at "[ \t]*#") + (re-search-forward "^[ \t]*[^ \t#]" nil 'move) ; look for non-comment + (re-search-backward "^[ \t]*#") ; and back to last comment in block + (setq last-pos (point))) + + ;; else if line is a block line and EXTEND given, suck up + ;; the whole structure + ((and extend + (setq first-symbol (py-suck-up-first-keyword) ) + (assq first-symbol followers)) + (while (and + (or (py-goto-beyond-block) t) ; side effect + (forward-line -1) ; side effect + (setq last-pos (point)) ; side effect + (py-goto-statement-below) + (= (current-indentation) initial-indent) + (setq next-symbol (py-suck-up-first-keyword)) + (memq next-symbol (cdr (assq first-symbol followers)))) + (setq first-symbol next-symbol))) + + ;; else if line *opens* a block, search for next stmt indented <= + ((py-statement-opens-block-p) + (while (and + (setq last-pos (point)) ; always true -- side effect + (py-goto-statement-below) + (> (current-indentation) initial-indent) + ))) + + ;; else plain code line; stop at next blank line, or stmt or + ;; indenting comment line indented < + (t + (while (and + (setq last-pos (point)) ; always true -- side effect + (or (py-goto-beyond-final-line) t) + (not (looking-at "[ \t]*$")) ; stop at blank line + (or + (>= (current-indentation) initial-indent) + (looking-at "[ \t]*#[^ \t\n]"))) ; ignore non-indenting # + nil))) + + ;; skip to end of last stmt + (goto-char last-pos) + (py-goto-beyond-final-line) + + ;; set mark & display + (if just-move + () ; just return + (push-mark (point) 'no-msg) + (forward-line -1) + (message "Mark set after: %s" (py-suck-up-leading-text)) + (goto-char initial-pos)))) + +(defun py-mark-def-or-class (&optional class) + "Set region to body of def (or class, with prefix arg) enclosing point. +Pushes the current mark, then point, on the mark ring (all language +modes do this, but although it's handy it's never documented ...). + +In most Emacs language modes, this function bears at least a +hallucinogenic resemblance to `\\[py-end-of-def-or-class]' and +`\\[py-beginning-of-def-or-class]'. + +And in earlier versions of Python mode, all 3 were tightly connected. +Turned out that was more confusing than useful: the `goto start' and +`goto end' commands are usually used to search through a file, and +people expect them to act a lot like `search backward' and `search +forward' string-search commands. But because Python `def' and `class' +can nest to arbitrary levels, finding the smallest def containing +point cannot be done via a simple backward search: the def containing +point may not be the closest preceding def, or even the closest +preceding def that's indented less. The fancy algorithm required is +appropriate for the usual uses of this `mark' command, but not for the +`goto' variations. + +So the def marked by this command may not be the one either of the +`goto' commands find: If point is on a blank or non-indenting comment +line, moves back to start of the closest preceding code statement or +indenting comment line. If this is a `def' statement, that's the def +we use. Else searches for the smallest enclosing `def' block and uses +that. Else signals an error. + +When an enclosing def is found: The mark is left immediately beyond +the last line of the def block. Point is left at the start of the +def, except that: if the def is preceded by a number of comment lines +followed by (at most) one optional blank line, point is left at the +start of the comments; else if the def is preceded by a blank line, +point is left at its start. + +The intent is to mark the containing def/class and its associated +documentation, to make moving and duplicating functions and classes +pleasant." + (interactive "P") ; raw prefix arg + (let ((start (point)) + (which (cond ((eq class 'either) "\\(class\\|def\\)") + (class "class") + (t "def")))) + (push-mark start) + (if (not (py-go-up-tree-to-keyword which)) + (progn (goto-char start) + (error "Enclosing %s not found" + (if (eq class 'either) + "def or class" + which))) + ;; else enclosing def/class found + (setq start (point)) + (py-goto-beyond-block) + (push-mark (point)) + (goto-char start) + (if (zerop (forward-line -1)) ; if there is a preceding line + (progn + (if (looking-at "[ \t]*$") ; it's blank + (setq start (point)) ; so reset start point + (goto-char start)) ; else try again + (if (zerop (forward-line -1)) + (if (looking-at "[ \t]*#") ; a comment + ;; look back for non-comment line + ;; tricky: note that the regexp matches a blank + ;; line, cuz \n is in the 2nd character class + (and + (re-search-backward "^[ \t]*[^ \t#]" nil 'move) + (forward-line 1)) + ;; no comment, so go back + (goto-char start))))))) + (exchange-point-and-mark) + (py-keep-region-active)) + +;; ripped from cc-mode +(defun py-forward-into-nomenclature (&optional arg) + "Move forward to end of a nomenclature section or word. +With \\[universal-argument] (programmatically, optional argument ARG), +do it that many times. + +A `nomenclature' is a fancy way of saying AWordWithMixedCaseNotUnderscores." + (interactive "p") + (let ((case-fold-search nil)) + (if (> arg 0) + (re-search-forward + "\\(\\W\\|[_]\\)*\\([A-Z]*[a-z0-9]*\\)" + (point-max) t arg) + (while (and (< arg 0) + (re-search-backward + "\\(\\W\\|[a-z0-9]\\)[A-Z]+\\|\\(\\W\\|[_]\\)\\w+" + (point-min) 0)) + (forward-char 1) + (setq arg (1+ arg))))) + (py-keep-region-active)) + +(defun py-backward-into-nomenclature (&optional arg) + "Move backward to beginning of a nomenclature section or word. +With optional ARG, move that many times. If ARG is negative, move +forward. + +A `nomenclature' is a fancy way of saying AWordWithMixedCaseNotUnderscores." + (interactive "p") + (py-forward-into-nomenclature (- arg)) + (py-keep-region-active)) + + + +;; pdbtrack functions +(defun py-pdbtrack-toggle-stack-tracking (arg) + (interactive "P") + (if (not (get-buffer-process (current-buffer))) + (error "No process associated with buffer '%s'" (current-buffer))) + ;; missing or 0 is toggle, >0 turn on, <0 turn off + (if (or (not arg) + (zerop (setq arg (prefix-numeric-value arg)))) + (setq py-pdbtrack-do-tracking-p (not py-pdbtrack-do-tracking-p)) + (setq py-pdbtrack-do-tracking-p (> arg 0))) + (message "%sabled Python's pdbtrack" + (if py-pdbtrack-do-tracking-p "En" "Dis"))) + +(defun turn-on-pdbtrack () + (interactive) + (py-pdbtrack-toggle-stack-tracking 1)) + +(defun turn-off-pdbtrack () + (interactive) + (py-pdbtrack-toggle-stack-tracking 0)) + + + +;; Pychecker +(defun py-pychecker-run (command) + "*Run pychecker (default on the file currently visited)." + (interactive + (let ((default + (format "%s %s %s" py-pychecker-command + (mapconcat 'identity py-pychecker-command-args " ") + (buffer-file-name))) + (last (when py-pychecker-history + (let* ((lastcmd (car py-pychecker-history)) + (cmd (cdr (reverse (split-string lastcmd)))) + (newcmd (reverse (cons (buffer-file-name) cmd)))) + (mapconcat 'identity newcmd " "))))) + + (list + (if (fboundp 'read-shell-command) + (read-shell-command "Run pychecker like this: " + (if last + last + default) + 'py-pychecker-history) + (read-string "Run pychecker like this: " + (if last + last + default) + 'py-pychecker-history)) + ))) + (save-some-buffers (not py-ask-about-save) nil) + (compile-internal command "No more errors")) + + + +;; pydoc commands. The guts of this function is stolen from XEmacs's +;; symbol-near-point, but without the useless regexp-quote call on the +;; results, nor the interactive bit. Also, we've added the temporary +;; syntax table setting, which Skip originally had broken out into a +;; separate function. Note that Emacs doesn't have the original +;; function. +(defun py-symbol-near-point () + "Return the first textual item to the nearest point." + ;; alg stolen from etag.el + (save-excursion + (with-syntax-table py-dotted-expression-syntax-table + (if (or (bobp) (not (memq (char-syntax (char-before)) '(?w ?_)))) + (while (not (looking-at "\\sw\\|\\s_\\|\\'")) + (forward-char 1))) + (while (looking-at "\\sw\\|\\s_") + (forward-char 1)) + (if (re-search-backward "\\sw\\|\\s_" nil t) + (progn (forward-char 1) + (buffer-substring (point) + (progn (forward-sexp -1) + (while (looking-at "\\s'") + (forward-char 1)) + (point)))) + nil)))) + +(defun py-help-at-point () + "Get help from Python based on the symbol nearest point." + (interactive) + (let* ((sym (py-symbol-near-point)) + (base (substring sym 0 (or (search "." sym :from-end t) 0))) + cmd) + (if (not (equal base "")) + (setq cmd (concat "import " base "\n"))) + (setq cmd (concat "import pydoc\n" + cmd + "try: pydoc.help('" sym "')\n" + "except: print 'No help available on:', \"" sym "\"")) + (message cmd) + (py-execute-string cmd) + (set-buffer "*Python Output*") + ;; BAW: Should we really be leaving the output buffer in help-mode? + (help-mode))) + + + +;; Documentation functions + +;; dump the long form of the mode blurb; does the usual doc escapes, +;; plus lines of the form ^[vc]:name$ to suck variable & command docs +;; out of the right places, along with the keys they're on & current +;; values +(defun py-dump-help-string (str) + (with-output-to-temp-buffer "*Help*" + (let ((locals (buffer-local-variables)) + funckind funcname func funcdoc + (start 0) mstart end + keys ) + (while (string-match "^%\\([vc]\\):\\(.+\\)\n" str start) + (setq mstart (match-beginning 0) end (match-end 0) + funckind (substring str (match-beginning 1) (match-end 1)) + funcname (substring str (match-beginning 2) (match-end 2)) + func (intern funcname)) + (princ (substitute-command-keys (substring str start mstart))) + (cond + ((equal funckind "c") ; command + (setq funcdoc (documentation func) + keys (concat + "Key(s): " + (mapconcat 'key-description + (where-is-internal func py-mode-map) + ", ")))) + ((equal funckind "v") ; variable + (setq funcdoc (documentation-property func 'variable-documentation) + keys (if (assq func locals) + (concat + "Local/Global values: " + (prin1-to-string (symbol-value func)) + " / " + (prin1-to-string (default-value func))) + (concat + "Value: " + (prin1-to-string (symbol-value func)))))) + (t ; unexpected + (error "Error in py-dump-help-string, tag `%s'" funckind))) + (princ (format "\n-> %s:\t%s\t%s\n\n" + (if (equal funckind "c") "Command" "Variable") + funcname keys)) + (princ funcdoc) + (terpri) + (setq start end)) + (princ (substitute-command-keys (substring str start)))) + (print-help-return-message))) + +(defun py-describe-mode () + "Dump long form of Python-mode docs." + (interactive) + (py-dump-help-string "Major mode for editing Python files. +Knows about Python indentation, tokens, comments and continuation lines. +Paragraphs are separated by blank lines only. + +Major sections below begin with the string `@'; specific function and +variable docs begin with `->'. + +@EXECUTING PYTHON CODE + +\\[py-execute-import-or-reload]\timports or reloads the file in the Python interpreter +\\[py-execute-buffer]\tsends the entire buffer to the Python interpreter +\\[py-execute-region]\tsends the current region +\\[py-execute-def-or-class]\tsends the current function or class definition +\\[py-execute-string]\tsends an arbitrary string +\\[py-shell]\tstarts a Python interpreter window; this will be used by +\tsubsequent Python execution commands +%c:py-execute-import-or-reload +%c:py-execute-buffer +%c:py-execute-region +%c:py-execute-def-or-class +%c:py-execute-string +%c:py-shell + +@VARIABLES + +py-indent-offset\tindentation increment +py-block-comment-prefix\tcomment string used by comment-region + +py-python-command\tshell command to invoke Python interpreter +py-temp-directory\tdirectory used for temp files (if needed) + +py-beep-if-tab-change\tring the bell if tab-width is changed +%v:py-indent-offset +%v:py-block-comment-prefix +%v:py-python-command +%v:py-temp-directory +%v:py-beep-if-tab-change + +@KINDS OF LINES + +Each physical line in the file is either a `continuation line' (the +preceding line ends with a backslash that's not part of a comment, or +the paren/bracket/brace nesting level at the start of the line is +non-zero, or both) or an `initial line' (everything else). + +An initial line is in turn a `blank line' (contains nothing except +possibly blanks or tabs), a `comment line' (leftmost non-blank +character is `#'), or a `code line' (everything else). + +Comment Lines + +Although all comment lines are treated alike by Python, Python mode +recognizes two kinds that act differently with respect to indentation. + +An `indenting comment line' is a comment line with a blank, tab or +nothing after the initial `#'. The indentation commands (see below) +treat these exactly as if they were code lines: a line following an +indenting comment line will be indented like the comment line. All +other comment lines (those with a non-whitespace character immediately +following the initial `#') are `non-indenting comment lines', and +their indentation is ignored by the indentation commands. + +Indenting comment lines are by far the usual case, and should be used +whenever possible. Non-indenting comment lines are useful in cases +like these: + +\ta = b # a very wordy single-line comment that ends up being +\t #... continued onto another line + +\tif a == b: +##\t\tprint 'panic!' # old code we've `commented out' +\t\treturn a + +Since the `#...' and `##' comment lines have a non-whitespace +character following the initial `#', Python mode ignores them when +computing the proper indentation for the next line. + +Continuation Lines and Statements + +The Python-mode commands generally work on statements instead of on +individual lines, where a `statement' is a comment or blank line, or a +code line and all of its following continuation lines (if any) +considered as a single logical unit. The commands in this mode +generally (when it makes sense) automatically move to the start of the +statement containing point, even if point happens to be in the middle +of some continuation line. + + +@INDENTATION + +Primarily for entering new code: +\t\\[indent-for-tab-command]\t indent line appropriately +\t\\[py-newline-and-indent]\t insert newline, then indent +\t\\[py-electric-backspace]\t reduce indentation, or delete single character + +Primarily for reindenting existing code: +\t\\[py-guess-indent-offset]\t guess py-indent-offset from file content; change locally +\t\\[universal-argument] \\[py-guess-indent-offset]\t ditto, but change globally + +\t\\[py-indent-region]\t reindent region to match its context +\t\\[py-shift-region-left]\t shift region left by py-indent-offset +\t\\[py-shift-region-right]\t shift region right by py-indent-offset + +Unlike most programming languages, Python uses indentation, and only +indentation, to specify block structure. Hence the indentation supplied +automatically by Python-mode is just an educated guess: only you know +the block structure you intend, so only you can supply correct +indentation. + +The \\[indent-for-tab-command] and \\[py-newline-and-indent] keys try to suggest plausible indentation, based on +the indentation of preceding statements. E.g., assuming +py-indent-offset is 4, after you enter +\tif a > 0: \\[py-newline-and-indent] +the cursor will be moved to the position of the `_' (_ is not a +character in the file, it's just used here to indicate the location of +the cursor): +\tif a > 0: +\t _ +If you then enter `c = d' \\[py-newline-and-indent], the cursor will move +to +\tif a > 0: +\t c = d +\t _ +Python-mode cannot know whether that's what you intended, or whether +\tif a > 0: +\t c = d +\t_ +was your intent. In general, Python-mode either reproduces the +indentation of the (closest code or indenting-comment) preceding +statement, or adds an extra py-indent-offset blanks if the preceding +statement has `:' as its last significant (non-whitespace and non- +comment) character. If the suggested indentation is too much, use +\\[py-electric-backspace] to reduce it. + +Continuation lines are given extra indentation. If you don't like the +suggested indentation, change it to something you do like, and Python- +mode will strive to indent later lines of the statement in the same way. + +If a line is a continuation line by virtue of being in an unclosed +paren/bracket/brace structure (`list', for short), the suggested +indentation depends on whether the current line contains the first item +in the list. If it does, it's indented py-indent-offset columns beyond +the indentation of the line containing the open bracket. If you don't +like that, change it by hand. The remaining items in the list will mimic +whatever indentation you give to the first item. + +If a line is a continuation line because the line preceding it ends with +a backslash, the third and following lines of the statement inherit their +indentation from the line preceding them. The indentation of the second +line in the statement depends on the form of the first (base) line: if +the base line is an assignment statement with anything more interesting +than the backslash following the leftmost assigning `=', the second line +is indented two columns beyond that `='. Else it's indented to two +columns beyond the leftmost solid chunk of non-whitespace characters on +the base line. + +Warning: indent-region should not normally be used! It calls \\[indent-for-tab-command] +repeatedly, and as explained above, \\[indent-for-tab-command] can't guess the block +structure you intend. +%c:indent-for-tab-command +%c:py-newline-and-indent +%c:py-electric-backspace + + +The next function may be handy when editing code you didn't write: +%c:py-guess-indent-offset + + +The remaining `indent' functions apply to a region of Python code. They +assume the block structure (equals indentation, in Python) of the region +is correct, and alter the indentation in various ways while preserving +the block structure: +%c:py-indent-region +%c:py-shift-region-left +%c:py-shift-region-right + +@MARKING & MANIPULATING REGIONS OF CODE + +\\[py-mark-block]\t mark block of lines +\\[py-mark-def-or-class]\t mark smallest enclosing def +\\[universal-argument] \\[py-mark-def-or-class]\t mark smallest enclosing class +\\[comment-region]\t comment out region of code +\\[universal-argument] \\[comment-region]\t uncomment region of code +%c:py-mark-block +%c:py-mark-def-or-class +%c:comment-region + +@MOVING POINT + +\\[py-previous-statement]\t move to statement preceding point +\\[py-next-statement]\t move to statement following point +\\[py-goto-block-up]\t move up to start of current block +\\[py-beginning-of-def-or-class]\t move to start of def +\\[universal-argument] \\[py-beginning-of-def-or-class]\t move to start of class +\\[py-end-of-def-or-class]\t move to end of def +\\[universal-argument] \\[py-end-of-def-or-class]\t move to end of class + +The first two move to one statement beyond the statement that contains +point. A numeric prefix argument tells them to move that many +statements instead. Blank lines, comment lines, and continuation lines +do not count as `statements' for these commands. So, e.g., you can go +to the first code statement in a file by entering +\t\\[beginning-of-buffer]\t to move to the top of the file +\t\\[py-next-statement]\t to skip over initial comments and blank lines +Or do `\\[py-previous-statement]' with a huge prefix argument. +%c:py-previous-statement +%c:py-next-statement +%c:py-goto-block-up +%c:py-beginning-of-def-or-class +%c:py-end-of-def-or-class + +@LITTLE-KNOWN EMACS COMMANDS PARTICULARLY USEFUL IN PYTHON MODE + +`\\[indent-new-comment-line]' is handy for entering a multi-line comment. + +`\\[set-selective-display]' with a `small' prefix arg is ideally suited for viewing the +overall class and def structure of a module. + +`\\[back-to-indentation]' moves point to a line's first non-blank character. + +`\\[indent-relative]' is handy for creating odd indentation. + +@OTHER EMACS HINTS + +If you don't like the default value of a variable, change its value to +whatever you do like by putting a `setq' line in your .emacs file. +E.g., to set the indentation increment to 4, put this line in your +.emacs: +\t(setq py-indent-offset 4) +To see the value of a variable, do `\\[describe-variable]' and enter the variable +name at the prompt. + +When entering a key sequence like `C-c C-n', it is not necessary to +release the CONTROL key after doing the `C-c' part -- it suffices to +press the CONTROL key, press and release `c' (while still holding down +CONTROL), press and release `n' (while still holding down CONTROL), & +then release CONTROL. + +Entering Python mode calls with no arguments the value of the variable +`python-mode-hook', if that value exists and is not nil; for backward +compatibility it also tries `py-mode-hook'; see the `Hooks' section of +the Elisp manual for details. + +Obscure: When python-mode is first loaded, it looks for all bindings +to newline-and-indent in the global keymap, and shadows them with +local bindings to py-newline-and-indent.")) + +(require 'info-look) +;; The info-look package does not always provide this function (it +;; appears this is the case with XEmacs 21.1) +(when (fboundp 'info-lookup-maybe-add-help) + (info-lookup-maybe-add-help + :mode 'python-mode + :regexp "[a-zA-Z0-9_]+" + :doc-spec '(("(python-lib)Module Index") + ("(python-lib)Class-Exception-Object Index") + ("(python-lib)Function-Method-Variable Index") + ("(python-lib)Miscellaneous Index"))) + ) + + +;; Helper functions +(defvar py-parse-state-re + (concat + "^[ \t]*\\(elif\\|else\\|while\\|def\\|class\\)\\>" + "\\|" + "^[^ #\t\n]")) + +(defun py-parse-state () + "Return the parse state at point (see `parse-partial-sexp' docs)." + (save-excursion + (let ((here (point)) + pps done) + (while (not done) + ;; back up to the first preceding line (if any; else start of + ;; buffer) that begins with a popular Python keyword, or a + ;; non- whitespace and non-comment character. These are good + ;; places to start parsing to see whether where we started is + ;; at a non-zero nesting level. It may be slow for people who + ;; write huge code blocks or huge lists ... tough beans. + (re-search-backward py-parse-state-re nil 'move) + (beginning-of-line) + ;; In XEmacs, we have a much better way to test for whether + ;; we're in a triple-quoted string or not. Emacs does not + ;; have this built-in function, which is its loss because + ;; without scanning from the beginning of the buffer, there's + ;; no accurate way to determine this otherwise. + (save-excursion (setq pps (parse-partial-sexp (point) here))) + ;; make sure we don't land inside a triple-quoted string + (setq done (or (not (nth 3 pps)) + (bobp))) + ;; Just go ahead and short circuit the test back to the + ;; beginning of the buffer. This will be slow, but not + ;; nearly as slow as looping through many + ;; re-search-backwards. + (if (not done) + (goto-char (point-min)))) + pps))) + +(defun py-nesting-level () + "Return the buffer position of the last unclosed enclosing list. +If nesting level is zero, return nil." + (let ((status (py-parse-state))) + (if (zerop (car status)) + nil ; not in a nest + (car (cdr status))))) ; char# of open bracket + +(defun py-backslash-continuation-line-p () + "Return t iff preceding line ends with backslash that is not in a comment." + (save-excursion + (beginning-of-line) + (and + ;; use a cheap test first to avoid the regexp if possible + ;; use 'eq' because char-after may return nil + (eq (char-after (- (point) 2)) ?\\ ) + ;; make sure; since eq test passed, there is a preceding line + (forward-line -1) ; always true -- side effect + (looking-at py-continued-re)))) + +(defun py-continuation-line-p () + "Return t iff current line is a continuation line." + (save-excursion + (beginning-of-line) + (or (py-backslash-continuation-line-p) + (py-nesting-level)))) + +(defun py-goto-beginning-of-tqs (delim) + "Go to the beginning of the triple quoted string we find ourselves in. +DELIM is the TQS string delimiter character we're searching backwards +for." + (let ((skip (and delim (make-string 1 delim))) + (continue t)) + (when skip + (save-excursion + (while continue + (py-safe (search-backward skip)) + (setq continue (and (not (bobp)) + (= (char-before) ?\\)))) + (if (and (= (char-before) delim) + (= (char-before (1- (point))) delim)) + (setq skip (make-string 3 delim)))) + ;; we're looking at a triple-quoted string + (py-safe (search-backward skip))))) + +(defun py-goto-initial-line () + "Go to the initial line of the current statement. +Usually this is the line we're on, but if we're on the 2nd or +following lines of a continuation block, we need to go up to the first +line of the block." + ;; Tricky: We want to avoid quadratic-time behavior for long + ;; continued blocks, whether of the backslash or open-bracket + ;; varieties, or a mix of the two. The following manages to do that + ;; in the usual cases. + ;; + ;; Also, if we're sitting inside a triple quoted string, this will + ;; drop us at the line that begins the string. + (let (open-bracket-pos) + (while (py-continuation-line-p) + (beginning-of-line) + (if (py-backslash-continuation-line-p) + (while (py-backslash-continuation-line-p) + (forward-line -1)) + ;; else zip out of nested brackets/braces/parens + (while (setq open-bracket-pos (py-nesting-level)) + (goto-char open-bracket-pos))))) + (beginning-of-line)) + +(defun py-goto-beyond-final-line () + "Go to the point just beyond the fine line of the current statement. +Usually this is the start of the next line, but if this is a +multi-line statement we need to skip over the continuation lines." + ;; Tricky: Again we need to be clever to avoid quadratic time + ;; behavior. + ;; + ;; XXX: Not quite the right solution, but deals with multi-line doc + ;; strings + (if (looking-at (concat "[ \t]*\\(" py-stringlit-re "\\)")) + (goto-char (match-end 0))) + ;; + (forward-line 1) + (let (state) + (while (and (py-continuation-line-p) + (not (eobp))) + ;; skip over the backslash flavor + (while (and (py-backslash-continuation-line-p) + (not (eobp))) + (forward-line 1)) + ;; if in nest, zip to the end of the nest + (setq state (py-parse-state)) + (if (and (not (zerop (car state))) + (not (eobp))) + (progn + (parse-partial-sexp (point) (point-max) 0 nil state) + (forward-line 1)))))) + +(defun py-statement-opens-block-p () + "Return t iff the current statement opens a block. +I.e., iff it ends with a colon that is not in a comment. Point should +be at the start of a statement." + (save-excursion + (let ((start (point)) + (finish (progn (py-goto-beyond-final-line) (1- (point)))) + (searching t) + (answer nil) + state) + (goto-char start) + (while searching + ;; look for a colon with nothing after it except whitespace, and + ;; maybe a comment + (if (re-search-forward ":\\([ \t]\\|\\\\\n\\)*\\(#.*\\)?$" + finish t) + (if (eq (point) finish) ; note: no `else' clause; just + ; keep searching if we're not at + ; the end yet + ;; sure looks like it opens a block -- but it might + ;; be in a comment + (progn + (setq searching nil) ; search is done either way + (setq state (parse-partial-sexp start + (match-beginning 0))) + (setq answer (not (nth 4 state))))) + ;; search failed: couldn't find another interesting colon + (setq searching nil))) + answer))) + +(defun py-statement-closes-block-p () + "Return t iff the current statement closes a block. +I.e., if the line starts with `return', `raise', `break', `continue', +and `pass'. This doesn't catch embedded statements." + (let ((here (point))) + (py-goto-initial-line) + (back-to-indentation) + (prog1 + (looking-at (concat py-block-closing-keywords-re "\\>")) + (goto-char here)))) + +(defun py-goto-beyond-block () + "Go to point just beyond the final line of block begun by the current line. +This is the same as where `py-goto-beyond-final-line' goes unless +we're on colon line, in which case we go to the end of the block. +Assumes point is at the beginning of the line." + (if (py-statement-opens-block-p) + (py-mark-block nil 'just-move) + (py-goto-beyond-final-line))) + +(defun py-goto-statement-at-or-above () + "Go to the start of the first statement at or preceding point. +Return t if there is such a statement, otherwise nil. `Statement' +does not include blank lines, comments, or continuation lines." + (py-goto-initial-line) + (if (looking-at py-blank-or-comment-re) + ;; skip back over blank & comment lines + ;; note: will skip a blank or comment line that happens to be + ;; a continuation line too + (if (re-search-backward "^[ \t]*[^ \t#\n]" nil t) + (progn (py-goto-initial-line) t) + nil) + t)) + +(defun py-goto-statement-below () + "Go to start of the first statement following the statement containing point. +Return t if there is such a statement, otherwise nil. `Statement' +does not include blank lines, comments, or continuation lines." + (beginning-of-line) + (let ((start (point))) + (py-goto-beyond-final-line) + (while (and + (or (looking-at py-blank-or-comment-re) + (py-in-literal)) + (not (eobp))) + (forward-line 1)) + (if (eobp) + (progn (goto-char start) nil) + t))) + +(defun py-go-up-tree-to-keyword (key) + "Go to begining of statement starting with KEY, at or preceding point. + +KEY is a regular expression describing a Python keyword. Skip blank +lines and non-indenting comments. If the statement found starts with +KEY, then stop, otherwise go back to first enclosing block starting +with KEY. If successful, leave point at the start of the KEY line and +return t. Otherwise, leav point at an undefined place and return nil." + ;; skip blanks and non-indenting # + (py-goto-initial-line) + (while (and + (looking-at "[ \t]*\\($\\|#[^ \t\n]\\)") + (zerop (forward-line -1))) ; go back + nil) + (py-goto-initial-line) + (let* ((re (concat "[ \t]*" key "\\b")) + (case-fold-search nil) ; let* so looking-at sees this + (found (looking-at re)) + (dead nil)) + (while (not (or found dead)) + (condition-case nil ; in case no enclosing block + (py-goto-block-up 'no-mark) + (error (setq dead t))) + (or dead (setq found (looking-at re)))) + (beginning-of-line) + found)) + +(defun py-suck-up-leading-text () + "Return string in buffer from start of indentation to end of line. +Prefix with \"...\" if leading whitespace was skipped." + (save-excursion + (back-to-indentation) + (concat + (if (bolp) "" "...") + (buffer-substring (point) (progn (end-of-line) (point)))))) + +(defun py-suck-up-first-keyword () + "Return first keyword on the line as a Lisp symbol. +`Keyword' is defined (essentially) as the regular expression +([a-z]+). Returns nil if none was found." + (let ((case-fold-search nil)) + (if (looking-at "[ \t]*\\([a-z]+\\)\\b") + (intern (buffer-substring (match-beginning 1) (match-end 1))) + nil))) + +(defun py-current-defun () + "Python value for `add-log-current-defun-function'. +This tells add-log.el how to find the current function/method/variable." + (save-excursion + (if (re-search-backward py-defun-start-re nil t) + (or (match-string 3) + (let ((method (match-string 2))) + (if (and (not (zerop (length (match-string 1)))) + (re-search-backward py-class-start-re nil t)) + (concat (match-string 1) "." method) + method))) + nil))) + + +(defconst py-help-address "python-mode@python.org" + "Address accepting submission of bug reports.") + +(defun py-version () + "Echo the current version of `python-mode' in the minibuffer." + (interactive) + (message "Using `python-mode' version %s" py-version) + (py-keep-region-active)) + +;; only works under Emacs 19 +;(eval-when-compile +; (require 'reporter)) + +(defun py-submit-bug-report (enhancement-p) + "Submit via mail a bug report on `python-mode'. +With \\[universal-argument] (programmatically, argument ENHANCEMENT-P +non-nil) just submit an enhancement request." + (interactive + (list (not (y-or-n-p + "Is this a bug report (hit `n' to send other comments)? ")))) + (let ((reporter-prompt-for-summary-p (if enhancement-p + "(Very) brief summary: " + t))) + (require 'reporter) + (reporter-submit-bug-report + py-help-address ;address + (concat "python-mode " py-version) ;pkgname + ;; varlist + (if enhancement-p nil + '(py-python-command + py-indent-offset + py-block-comment-prefix + py-temp-directory + py-beep-if-tab-change)) + nil ;pre-hooks + nil ;post-hooks + "Dear Barry,") ;salutation + (if enhancement-p nil + (set-mark (point)) + (insert +"Please replace this text with a sufficiently large code sample\n\ +and an exact recipe so that I can reproduce your problem. Failure\n\ +to do so may mean a greater delay in fixing your bug.\n\n") + (exchange-point-and-mark) + (py-keep-region-active)))) + + +(defun py-kill-emacs-hook () + "Delete files in `py-file-queue'. +These are Python temporary files awaiting execution." + (mapcar #'(lambda (filename) + (py-safe (delete-file filename))) + py-file-queue)) + +;; arrange to kill temp files when Emacs exists +(add-hook 'kill-emacs-hook 'py-kill-emacs-hook) +(add-hook 'comint-output-filter-functions 'py-pdbtrack-track-stack-file) + +;; Add a designator to the minor mode strings +(or (assq 'py-pdbtrack-minor-mode-string minor-mode-alist) + (push '(py-pdbtrack-is-tracking-p py-pdbtrack-minor-mode-string) + minor-mode-alist)) + + + +;;; paragraph and string filling code from Bernhard Herzog +;;; see http://mail.python.org/pipermail/python-list/2002-May/103189.html + +(defun py-fill-comment (&optional justify) + "Fill the comment paragraph around point" + (let (;; Non-nil if the current line contains a comment. + has-comment + + ;; If has-comment, the appropriate fill-prefix for the comment. + comment-fill-prefix) + + ;; Figure out what kind of comment we are looking at. + (save-excursion + (beginning-of-line) + (cond + ;; A line with nothing but a comment on it? + ((looking-at "[ \t]*#[# \t]*") + (setq has-comment t + comment-fill-prefix (buffer-substring (match-beginning 0) + (match-end 0)))) + + ;; A line with some code, followed by a comment? Remember that the hash + ;; which starts the comment shouldn't be part of a string or character. + ((progn + (while (not (looking-at "#\\|$")) + (skip-chars-forward "^#\n\"'\\") + (cond + ((eq (char-after (point)) ?\\) (forward-char 2)) + ((memq (char-after (point)) '(?\" ?')) (forward-sexp 1)))) + (looking-at "#+[\t ]*")) + (setq has-comment t) + (setq comment-fill-prefix + (concat (make-string (current-column) ? ) + (buffer-substring (match-beginning 0) (match-end 0))))))) + + (if (not has-comment) + (fill-paragraph justify) + + ;; Narrow to include only the comment, and then fill the region. + (save-restriction + (narrow-to-region + + ;; Find the first line we should include in the region to fill. + (save-excursion + (while (and (zerop (forward-line -1)) + (looking-at "^[ \t]*#"))) + + ;; We may have gone to far. Go forward again. + (or (looking-at "^[ \t]*#") + (forward-line 1)) + (point)) + + ;; Find the beginning of the first line past the region to fill. + (save-excursion + (while (progn (forward-line 1) + (looking-at "^[ \t]*#"))) + (point))) + + ;; Lines with only hashes on them can be paragraph boundaries. + (let ((paragraph-start (concat paragraph-start "\\|[ \t#]*$")) + (paragraph-separate (concat paragraph-separate "\\|[ \t#]*$")) + (fill-prefix comment-fill-prefix)) + ;;(message "paragraph-start %S paragraph-separate %S" + ;;paragraph-start paragraph-separate) + (fill-paragraph justify)))) + t)) + + +(defun py-fill-string (start &optional justify) + "Fill the paragraph around (point) in the string starting at start" + ;; basic strategy: narrow to the string and call the default + ;; implementation + (let (;; the start of the string's contents + string-start + ;; the end of the string's contents + string-end + ;; length of the string's delimiter + delim-length + ;; The string delimiter + delim + ) + + (save-excursion + (goto-char start) + (if (looking-at "\\('''\\|\"\"\"\\|'\\|\"\\)\\\\?\n?") + (setq string-start (match-end 0) + delim-length (- (match-end 1) (match-beginning 1)) + delim (buffer-substring-no-properties (match-beginning 1) + (match-end 1))) + (error "The parameter start is not the beginning of a python string")) + + ;; if the string is the first token on a line and doesn't start with + ;; a newline, fill as if the string starts at the beginning of the + ;; line. this helps with one line docstrings + (save-excursion + (beginning-of-line) + (and (/= (char-before string-start) ?\n) + (looking-at (concat "[ \t]*" delim)) + (setq string-start (point)))) + + (forward-sexp (if (= delim-length 3) 2 1)) + + ;; with both triple quoted strings and single/double quoted strings + ;; we're now directly behind the first char of the end delimiter + ;; (this doesn't work correctly when the triple quoted string + ;; contains the quote mark itself). The end of the string's contents + ;; is one less than point + (setq string-end (1- (point)))) + + ;; Narrow to the string's contents and fill the current paragraph + (save-restriction + (narrow-to-region string-start string-end) + (let ((ends-with-newline (= (char-before (point-max)) ?\n))) + (fill-paragraph justify) + (if (and (not ends-with-newline) + (= (char-before (point-max)) ?\n)) + ;; the default fill-paragraph implementation has inserted a + ;; newline at the end. Remove it again. + (save-excursion + (goto-char (point-max)) + (delete-char -1))))) + + ;; return t to indicate that we've done our work + t)) + +(defun py-fill-paragraph (&optional justify) + "Like \\[fill-paragraph], but handle Python comments and strings. +If any of the current line is a comment, fill the comment or the +paragraph of it that point is in, preserving the comment's indentation +and initial `#'s. +If point is inside a string, narrow to that string and fill. +" + (interactive "P") + (let* ((bod (py-point 'bod)) + (pps (parse-partial-sexp bod (point)))) + (cond + ;; are we inside a comment or on a line with only whitespace before + ;; the comment start? + ((or (nth 4 pps) + (save-excursion (beginning-of-line) (looking-at "[ \t]*#"))) + (py-fill-comment justify)) + ;; are we inside a string? + ((nth 3 pps) + (py-fill-string (nth 8 pps))) + ;; otherwise use the default + (t + (fill-paragraph justify))))) + + + +(provide 'python-mode) +;;; python-mode.el ends here diff --git a/sys/src/cmd/python/Misc/python.man b/sys/src/cmd/python/Misc/python.man new file mode 100644 index 000000000..865d6497c --- /dev/null +++ b/sys/src/cmd/python/Misc/python.man @@ -0,0 +1,397 @@ +.TH PYTHON "1" "$Date: 2005-03-21 01:16:03 +1100 (Mon, 21 Mar 2005) $" + +./" To view this file while editing, run it through groff: +./" groff -Tascii -man python.man | less + +.SH NAME +python \- an interpreted, interactive, object-oriented programming language +.SH SYNOPSIS +.B python +[ +.B \-d +] +[ +.B \-E +] +[ +.B \-h +] +[ +.B \-i +] +[ +.B \-m +.I module-name +] +[ +.B \-O +] +.br + [ +.B -Q +.I argument +] +[ +.B \-S +] +[ +.B \-t +] +[ +.B \-u +] +.br + [ +.B \-v +] +[ +.B \-V +] +[ +.B \-W +.I argument +] +[ +.B \-x +] +.br + [ +.B \-c +.I command +| +.I script +| +\- +] +[ +.I arguments +] +.SH DESCRIPTION +Python is an interpreted, interactive, object-oriented programming +language that combines remarkable power with very clear syntax. +For an introduction to programming in Python you are referred to the +Python Tutorial. +The Python Library Reference documents built-in and standard types, +constants, functions and modules. +Finally, the Python Reference Manual describes the syntax and +semantics of the core language in (perhaps too) much detail. +(These documents may be located via the +.B "INTERNET RESOURCES" +below; they may be installed on your system as well.) +.PP +Python's basic power can be extended with your own modules written in +C or C++. +On most systems such modules may be dynamically loaded. +Python is also adaptable as an extension language for existing +applications. +See the internal documentation for hints. +.PP +Documentation for installed Python modules and packages can be +viewed by running the +.B pydoc +program. +.SH COMMAND LINE OPTIONS +.TP +.BI "\-c " command +Specify the command to execute (see next section). +This terminates the option list (following options are passed as +arguments to the command). +.TP +.B \-d +Turn on parser debugging output (for wizards only, depending on +compilation options). +.TP +.B \-E +Ignore environment variables like PYTHONPATH and PYTHONHOME that modify +the behavior of the interpreter. +.TP +.B \-h +Prints the usage for the interpreter executable and exits. +.TP +.B \-i +When a script is passed as first argument or the \fB\-c\fP option is +used, enter interactive mode after executing the script or the +command. It does not read the $PYTHONSTARTUP file. This can be +useful to inspect global variables or a stack trace when a script +raises an exception. +.TP +.BI "\-m " module-name +Searches +.I sys.path +for the named module and runs the corresponding +.I .py +file as a script. +.TP +.B \-O +Turn on basic optimizations. This changes the filename extension for +compiled (bytecode) files from +.I .pyc +to \fI.pyo\fP. Given twice, causes docstrings to be discarded. +.TP +.BI "\-Q " argument +Division control; see PEP 238. The argument must be one of "old" (the +default, int/int and long/long return an int or long), "new" (new +division semantics, i.e. int/int and long/long returns a float), +"warn" (old division semantics with a warning for int/int and +long/long), or "warnall" (old division semantics with a warning for +all use of the division operator). For a use of "warnall", see the +Tools/scripts/fixdiv.py script. +.TP +.B \-S +Disable the import of the module +.I site +and the site-dependent manipulations of +.I sys.path +that it entails. +.TP +.B \-t +Issue a warning when a source file mixes tabs and spaces for +indentation in a way that makes it depend on the worth of a tab +expressed in spaces. Issue an error when the option is given twice. +.TP +.B \-u +Force stdin, stdout and stderr to be totally unbuffered. On systems +where it matters, also put stdin, stdout and stderr in binary mode. +Note that there is internal buffering in xreadlines(), readlines() and +file-object iterators ("for line in sys.stdin") which is not +influenced by this option. To work around this, you will want to use +"sys.stdin.readline()" inside a "while 1:" loop. +.TP +.B \-v +Print a message each time a module is initialized, showing the place +(filename or built-in module) from which it is loaded. When given +twice, print a message for each file that is checked for when +searching for a module. Also provides information on module cleanup +at exit. +.TP +.B \-V +Prints the Python version number of the executable and exits. +.TP +.BI "\-W " argument +Warning control. Python sometimes prints warning message to +.IR sys.stderr . +A typical warning message has the following form: +.IB file ":" line ": " category ": " message. +By default, each warning is printed once for each source line where it +occurs. This option controls how often warnings are printed. +Multiple +.B \-W +options may be given; when a warning matches more than one +option, the action for the last matching option is performed. +Invalid +.B \-W +options are ignored (a warning message is printed about invalid +options when the first warning is issued). Warnings can also be +controlled from within a Python program using the +.I warnings +module. + +The simplest form of +.I argument +is one of the following +.I action +strings (or a unique abbreviation): +.B ignore +to ignore all warnings; +.B default +to explicitly request the default behavior (printing each warning once +per source line); +.B all +to print a warning each time it occurs (this may generate many +messages if a warning is triggered repeatedly for the same source +line, such as inside a loop); +.B module +to print each warning only only the first time it occurs in each +module; +.B once +to print each warning only the first time it occurs in the program; or +.B error +to raise an exception instead of printing a warning message. + +The full form of +.I argument +is +.IB action : message : category : module : line. +Here, +.I action +is as explained above but only applies to messages that match the +remaining fields. Empty fields match all values; trailing empty +fields may be omitted. The +.I message +field matches the start of the warning message printed; this match is +case-insensitive. The +.I category +field matches the warning category. This must be a class name; the +match test whether the actual warning category of the message is a +subclass of the specified warning category. The full class name must +be given. The +.I module +field matches the (fully-qualified) module name; this match is +case-sensitive. The +.I line +field matches the line number, where zero matches all line numbers and +is thus equivalent to an omitted line number. +.TP +.B \-x +Skip the first line of the source. This is intended for a DOS +specific hack only. Warning: the line numbers in error messages will +be off by one! +.SH INTERPRETER INTERFACE +The interpreter interface resembles that of the UNIX shell: when +called with standard input connected to a tty device, it prompts for +commands and executes them until an EOF is read; when called with a +file name argument or with a file as standard input, it reads and +executes a +.I script +from that file; +when called with +.B \-c +.I command, +it executes the Python statement(s) given as +.I command. +Here +.I command +may contain multiple statements separated by newlines. +Leading whitespace is significant in Python statements! +In non-interactive mode, the entire input is parsed before it is +executed. +.PP +If available, the script name and additional arguments thereafter are +passed to the script in the Python variable +.I sys.argv , +which is a list of strings (you must first +.I import sys +to be able to access it). +If no script name is given, +.I sys.argv[0] +is an empty string; if +.B \-c +is used, +.I sys.argv[0] +contains the string +.I '-c'. +Note that options interpreted by the Python interpreter itself +are not placed in +.I sys.argv. +.PP +In interactive mode, the primary prompt is `>>>'; the second prompt +(which appears when a command is not complete) is `...'. +The prompts can be changed by assignment to +.I sys.ps1 +or +.I sys.ps2. +The interpreter quits when it reads an EOF at a prompt. +When an unhandled exception occurs, a stack trace is printed and +control returns to the primary prompt; in non-interactive mode, the +interpreter exits after printing the stack trace. +The interrupt signal raises the +.I Keyboard\%Interrupt +exception; other UNIX signals are not caught (except that SIGPIPE is +sometimes ignored, in favor of the +.I IOError +exception). Error messages are written to stderr. +.SH FILES AND DIRECTORIES +These are subject to difference depending on local installation +conventions; ${prefix} and ${exec_prefix} are installation-dependent +and should be interpreted as for GNU software; they may be the same. +The default for both is \fI/usr/local\fP. +.IP \fI${exec_prefix}/bin/python\fP +Recommended location of the interpreter. +.PP +.I ${prefix}/lib/python<version> +.br +.I ${exec_prefix}/lib/python<version> +.RS +Recommended locations of the directories containing the standard +modules. +.RE +.PP +.I ${prefix}/include/python<version> +.br +.I ${exec_prefix}/include/python<version> +.RS +Recommended locations of the directories containing the include files +needed for developing Python extensions and embedding the +interpreter. +.RE +.IP \fI~/.pythonrc.py\fP +User-specific initialization file loaded by the \fIuser\fP module; +not used by default or by most applications. +.SH ENVIRONMENT VARIABLES +.IP PYTHONHOME +Change the location of the standard Python libraries. By default, the +libraries are searched in ${prefix}/lib/python<version> and +${exec_prefix}/lib/python<version>, where ${prefix} and ${exec_prefix} +are installation-dependent directories, both defaulting to +\fI/usr/local\fP. When $PYTHONHOME is set to a single directory, its value +replaces both ${prefix} and ${exec_prefix}. To specify different values +for these, set $PYTHONHOME to ${prefix}:${exec_prefix}. +.IP PYTHONPATH +Augments the default search path for module files. +The format is the same as the shell's $PATH: one or more directory +pathnames separated by colons. +Non-existent directories are silently ignored. +The default search path is installation dependent, but generally +begins with ${prefix}/lib/python<version> (see PYTHONHOME above). +The default search path is always appended to $PYTHONPATH. +If a script argument is given, the directory containing the script is +inserted in the path in front of $PYTHONPATH. +The search path can be manipulated from within a Python program as the +variable +.I sys.path . +.IP PYTHONSTARTUP +If this is the name of a readable file, the Python commands in that +file are executed before the first prompt is displayed in interactive +mode. +The file is executed in the same name space where interactive commands +are executed so that objects defined or imported in it can be used +without qualification in the interactive session. +You can also change the prompts +.I sys.ps1 +and +.I sys.ps2 +in this file. +.IP PYTHONY2K +Set this to a non-empty string to cause the \fItime\fP module to +require dates specified as strings to include 4-digit years, otherwise +2-digit years are converted based on rules described in the \fItime\fP +module documentation. +.IP PYTHONOPTIMIZE +If this is set to a non-empty string it is equivalent to specifying +the \fB\-O\fP option. If set to an integer, it is equivalent to +specifying \fB\-O\fP multiple times. +.IP PYTHONDEBUG +If this is set to a non-empty string it is equivalent to specifying +the \fB\-d\fP option. If set to an integer, it is equivalent to +specifying \fB\-d\fP multiple times. +.IP PYTHONINSPECT +If this is set to a non-empty string it is equivalent to specifying +the \fB\-i\fP option. +.IP PYTHONUNBUFFERED +If this is set to a non-empty string it is equivalent to specifying +the \fB\-u\fP option. +.IP PYTHONVERBOSE +If this is set to a non-empty string it is equivalent to specifying +the \fB\-v\fP option. If set to an integer, it is equivalent to +specifying \fB\-v\fP multiple times. +.SH AUTHOR +The Python Software Foundation: http://www.python.org/psf +.SH INTERNET RESOURCES +Main website: http://www.python.org/ +.br +Documentation: http://docs.python.org/ +.br +Community website: http://starship.python.net/ +.br +Developer resources: http://www.python.org/dev/ +.br +FTP: ftp://ftp.python.org/pub/python/ +.br +Module repository: http://www.vex.net/parnassus/ +.br +Newsgroups: comp.lang.python, comp.lang.python.announce +.SH LICENSING +Python is distributed under an Open Source license. See the file +"LICENSE" in the Python source distribution for information on terms & +conditions for accessing and otherwise using Python and for a +DISCLAIMER OF ALL WARRANTIES. diff --git a/sys/src/cmd/python/Misc/setuid-prog.c b/sys/src/cmd/python/Misc/setuid-prog.c new file mode 100644 index 000000000..d850b47bc --- /dev/null +++ b/sys/src/cmd/python/Misc/setuid-prog.c @@ -0,0 +1,176 @@ +/* + Template for a setuid program that calls a script. + + The script should be in an unwritable directory and should itself + be unwritable. In fact all parent directories up to the root + should be unwritable. The script must not be setuid, that's what + this program is for. + + This is a template program. You need to fill in the name of the + script that must be executed. This is done by changing the + definition of FULL_PATH below. + + There are also some rules that should be adhered to when writing + the script itself. + + The first and most important rule is to never, ever trust that the + user of the program will behave properly. Program defensively. + Check your arguments for reasonableness. If the user is allowed to + create files, check the names of the files. If the program depends + on argv[0] for the action it should perform, check it. + + Assuming the script is a Bourne shell script, the first line of the + script should be + #!/bin/sh - + The - is important, don't omit it. If you're using esh, the first + line should be + #!/usr/local/bin/esh -f + and for ksh, the first line should be + #!/usr/local/bin/ksh -p + The script should then set the variable IFS to the string + consisting of <space>, <tab>, and <newline>. After this (*not* + before!), the PATH variable should be set to a reasonable value and + exported. Do not expect the PATH to have a reasonable value, so do + not trust the old value of PATH. You should then set the umask of + the program by calling + umask 077 # or 022 if you want the files to be readable + If you plan to change directories, you should either unset CDPATH + or set it to a good value. Setting CDPATH to just ``.'' (dot) is a + good idea. + If, for some reason, you want to use csh, the first line should be + #!/bin/csh -fb + You should then set the path variable to something reasonable, + without trusting the inherited path. Here too, you should set the + umask using the command + umask 077 # or 022 if you want the files to be readable +*/ + +#include <unistd.h> +#include <stdlib.h> +#include <stdio.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <string.h> + +/* CONFIGURATION SECTION */ + +#ifndef FULL_PATH /* so that this can be specified from the Makefile */ +/* Uncomment the following line: +#define FULL_PATH "/full/path/of/script" +* Then comment out the #error line. */ +#error "You must define FULL_PATH somewhere" +#endif +#ifndef UMASK +#define UMASK 077 +#endif + +/* END OF CONFIGURATION SECTION */ + +#if defined(__STDC__) && defined(__sgi) +#define environ _environ +#endif + +/* don't change def_IFS */ +char def_IFS[] = "IFS= \t\n"; +/* you may want to change def_PATH, but you should really change it in */ +/* your script */ +#ifdef __sgi +char def_PATH[] = "PATH=/usr/bsd:/usr/bin:/bin:/usr/local/bin:/usr/sbin"; +#else +char def_PATH[] = "PATH=/usr/ucb:/usr/bin:/bin:/usr/local/bin"; +#endif +/* don't change def_CDPATH */ +char def_CDPATH[] = "CDPATH=."; +/* don't change def_ENV */ +char def_ENV[] = "ENV=:"; + +/* + This function changes all environment variables that start with LD_ + into variables that start with XD_. This is important since we + don't want the script that is executed to use any funny shared + libraries. + + The other changes to the environment are, strictly speaking, not + needed here. They can safely be done in the script. They are done + here because we don't trust the script writer (just like the script + writer shouldn't trust the user of the script). + If IFS is set in the environment, set it to space,tab,newline. + If CDPATH is set in the environment, set it to ``.''. + Set PATH to a reasonable default. +*/ +void +clean_environ(void) +{ + char **p; + extern char **environ; + + for (p = environ; *p; p++) { + if (strncmp(*p, "LD_", 3) == 0) + **p = 'X'; + else if (strncmp(*p, "_RLD", 4) == 0) + **p = 'X'; + else if (strncmp(*p, "PYTHON", 6) == 0) + **p = 'X'; + else if (strncmp(*p, "IFS=", 4) == 0) + *p = def_IFS; + else if (strncmp(*p, "CDPATH=", 7) == 0) + *p = def_CDPATH; + else if (strncmp(*p, "ENV=", 4) == 0) + *p = def_ENV; + } + putenv(def_PATH); +} + +int +main(int argc, char **argv) +{ + struct stat statb; + gid_t egid = getegid(); + uid_t euid = geteuid(); + + /* + Sanity check #1. + This check should be made compile-time, but that's not possible. + If you're sure that you specified a full path name for FULL_PATH, + you can omit this check. + */ + if (FULL_PATH[0] != '/') { + fprintf(stderr, "%s: %s is not a full path name\n", argv[0], + FULL_PATH); + fprintf(stderr, "You can only use this wrapper if you\n"); + fprintf(stderr, "compile it with an absolute path.\n"); + exit(1); + } + + /* + Sanity check #2. + Check that the owner of the script is equal to either the + effective uid or the super user. + */ + if (stat(FULL_PATH, &statb) < 0) { + perror("stat"); + exit(1); + } + if (statb.st_uid != 0 && statb.st_uid != euid) { + fprintf(stderr, "%s: %s has the wrong owner\n", argv[0], + FULL_PATH); + fprintf(stderr, "The script should be owned by root,\n"); + fprintf(stderr, "and shouldn't be writeable by anyone.\n"); + exit(1); + } + + if (setregid(egid, egid) < 0) + perror("setregid"); + if (setreuid(euid, euid) < 0) + perror("setreuid"); + + clean_environ(); + + umask(UMASK); + + while (**argv == '-') /* don't let argv[0] start with '-' */ + (*argv)++; + execv(FULL_PATH, argv); + fprintf(stderr, "%s: could not execute the script\n", argv[0]); + exit(1); +} diff --git a/sys/src/cmd/python/Misc/valgrind-python.supp b/sys/src/cmd/python/Misc/valgrind-python.supp new file mode 100644 index 000000000..4a6710e74 --- /dev/null +++ b/sys/src/cmd/python/Misc/valgrind-python.supp @@ -0,0 +1,349 @@ +# +# This is a valgrind suppression file that should be used when using valgrind. +# +# Here's an example of running valgrind: +# +# cd python/dist/src +# valgrind --tool=memcheck --suppressions=Misc/valgrind-python.supp \ +# ./python -E -tt ./Lib/test/regrtest.py -u bsddb,network +# +# You must edit Objects/obmalloc.c and uncomment Py_USING_MEMORY_DEBUGGER +# to use the preferred suppressions with Py_ADDRESS_IN_RANGE. +# +# If you do not want to recompile Python, you can uncomment +# suppressions for PyObject_Free and PyObject_Realloc. +# +# See Misc/README.valgrind for more information. + +# all tool names: Addrcheck,Memcheck,cachegrind,helgrind,massif +{ + ADDRESS_IN_RANGE/Invalid read of size 4 + Memcheck:Addr4 + fun:Py_ADDRESS_IN_RANGE +} + +{ + ADDRESS_IN_RANGE/Invalid read of size 4 + Memcheck:Value4 + fun:Py_ADDRESS_IN_RANGE +} + +{ + ADDRESS_IN_RANGE/Invalid read of size 8 (x86_64 aka amd64) + Memcheck:Value8 + fun:Py_ADDRESS_IN_RANGE +} + +{ + ADDRESS_IN_RANGE/Conditional jump or move depends on uninitialised value + Memcheck:Cond + fun:Py_ADDRESS_IN_RANGE +} + +# +# Leaks (including possible leaks) +# Hmmm, I wonder if this masks some real leaks. I think it does. +# Will need to fix that. +# + +{ + Handle PyMalloc confusing valgrind (possibly leaked) + Memcheck:Leak + fun:realloc + fun:_PyObject_GC_Resize + fun:COMMENT_THIS_LINE_TO_DISABLE_LEAK_WARNING +} + +{ + Handle PyMalloc confusing valgrind (possibly leaked) + Memcheck:Leak + fun:malloc + fun:_PyObject_GC_New + fun:COMMENT_THIS_LINE_TO_DISABLE_LEAK_WARNING +} + +{ + Handle PyMalloc confusing valgrind (possibly leaked) + Memcheck:Leak + fun:malloc + fun:_PyObject_GC_NewVar + fun:COMMENT_THIS_LINE_TO_DISABLE_LEAK_WARNING +} + +# +# Non-python specific leaks +# + +{ + Handle pthread issue (possibly leaked) + Memcheck:Leak + fun:calloc + fun:allocate_dtv + fun:_dl_allocate_tls_storage + fun:_dl_allocate_tls +} + +{ + Handle pthread issue (possibly leaked) + Memcheck:Leak + fun:memalign + fun:_dl_allocate_tls_storage + fun:_dl_allocate_tls +} + +###{ +### ADDRESS_IN_RANGE/Invalid read of size 4 +### Memcheck:Addr4 +### fun:PyObject_Free +###} +### +###{ +### ADDRESS_IN_RANGE/Invalid read of size 4 +### Memcheck:Value4 +### fun:PyObject_Free +###} +### +###{ +### ADDRESS_IN_RANGE/Conditional jump or move depends on uninitialised value +### Memcheck:Cond +### fun:PyObject_Free +###} + +###{ +### ADDRESS_IN_RANGE/Invalid read of size 4 +### Memcheck:Addr4 +### fun:PyObject_Realloc +###} +### +###{ +### ADDRESS_IN_RANGE/Invalid read of size 4 +### Memcheck:Value4 +### fun:PyObject_Realloc +###} +### +###{ +### ADDRESS_IN_RANGE/Conditional jump or move depends on uninitialised value +### Memcheck:Cond +### fun:PyObject_Realloc +###} + +### +### All the suppressions below are for errors that occur within libraries +### that Python uses. The problems to not appear to be related to Python's +### use of the libraries. +### + +{ + Generic gentoo ld problems + Memcheck:Cond + obj:/lib/ld-2.3.4.so + obj:/lib/ld-2.3.4.so + obj:/lib/ld-2.3.4.so + obj:/lib/ld-2.3.4.so +} + +{ + DBM problems, see test_dbm + Memcheck:Param + write(buf) + fun:write + obj:/usr/lib/libdb1.so.2 + obj:/usr/lib/libdb1.so.2 + obj:/usr/lib/libdb1.so.2 + obj:/usr/lib/libdb1.so.2 + fun:dbm_close +} + +{ + DBM problems, see test_dbm + Memcheck:Value8 + fun:memmove + obj:/usr/lib/libdb1.so.2 + obj:/usr/lib/libdb1.so.2 + obj:/usr/lib/libdb1.so.2 + obj:/usr/lib/libdb1.so.2 + fun:dbm_store + fun:dbm_ass_sub +} + +{ + DBM problems, see test_dbm + Memcheck:Cond + obj:/usr/lib/libdb1.so.2 + obj:/usr/lib/libdb1.so.2 + obj:/usr/lib/libdb1.so.2 + fun:dbm_store + fun:dbm_ass_sub +} + +{ + DBM problems, see test_dbm + Memcheck:Cond + fun:memmove + obj:/usr/lib/libdb1.so.2 + obj:/usr/lib/libdb1.so.2 + obj:/usr/lib/libdb1.so.2 + obj:/usr/lib/libdb1.so.2 + fun:dbm_store + fun:dbm_ass_sub +} + +{ + GDBM problems, see test_gdbm + Memcheck:Param + write(buf) + fun:write + fun:gdbm_open + +} + +{ + ZLIB problems, see test_gzip + Memcheck:Cond + obj:/lib/libz.so.1.2.3 + obj:/lib/libz.so.1.2.3 + fun:deflate +} + +{ + Avoid problems w/readline doing a putenv and leaking on exit + Memcheck:Leak + fun:malloc + fun:xmalloc + fun:sh_set_lines_and_columns + fun:_rl_get_screen_size + fun:_rl_init_terminal_io + obj:/lib/libreadline.so.4.3 + fun:rl_initialize +} + +### +### These occur from somewhere within the SSL, when running +### test_socket_sll. They are too general to leave on by default. +### +###{ +### somewhere in SSL stuff +### Memcheck:Cond +### fun:memset +###} +###{ +### somewhere in SSL stuff +### Memcheck:Value4 +### fun:memset +###} +### +###{ +### somewhere in SSL stuff +### Memcheck:Cond +### fun:MD5_Update +###} +### +###{ +### somewhere in SSL stuff +### Memcheck:Value4 +### fun:MD5_Update +###} + +# +# All of these problems come from using test_socket_ssl +# +{ + from test_socket_ssl + Memcheck:Cond + fun:BN_bin2bn +} + +{ + from test_socket_ssl + Memcheck:Cond + fun:BN_num_bits_word +} + +{ + from test_socket_ssl + Memcheck:Value4 + fun:BN_num_bits_word +} + +{ + from test_socket_ssl + Memcheck:Cond + fun:BN_mod_exp_mont_word +} + +{ + from test_socket_ssl + Memcheck:Cond + fun:BN_mod_exp_mont +} + +{ + from test_socket_ssl + Memcheck:Param + write(buf) + fun:write + obj:/usr/lib/libcrypto.so.0.9.7 +} + +{ + from test_socket_ssl + Memcheck:Cond + fun:RSA_verify +} + +{ + from test_socket_ssl + Memcheck:Value4 + fun:RSA_verify +} + +{ + from test_socket_ssl + Memcheck:Value4 + fun:DES_set_key_unchecked +} + +{ + from test_socket_ssl + Memcheck:Value4 + fun:DES_encrypt2 +} + +{ + from test_socket_ssl + Memcheck:Cond + obj:/usr/lib/libssl.so.0.9.7 +} + +{ + from test_socket_ssl + Memcheck:Value4 + obj:/usr/lib/libssl.so.0.9.7 +} + +{ + from test_socket_ssl + Memcheck:Cond + fun:BUF_MEM_grow_clean +} + +{ + from test_socket_ssl + Memcheck:Cond + fun:memcpy + fun:ssl3_read_bytes +} + +{ + from test_socket_ssl + Memcheck:Cond + fun:SHA1_Update +} + +{ + from test_socket_ssl + Memcheck:Value4 + fun:SHA1_Update +} + + diff --git a/sys/src/cmd/python/Misc/vgrindefs b/sys/src/cmd/python/Misc/vgrindefs new file mode 100644 index 000000000..bc6eba175 --- /dev/null +++ b/sys/src/cmd/python/Misc/vgrindefs @@ -0,0 +1,10 @@ +# vgrind is a pretty-printer that takes source code and outputs +# eye-pleasing postscript. The entry below should be added to your +# local vgrindefs file. Contributed by Neale Pickett <neale@lanl.gov>. + +python|Python|py:\ + :pb=^\d?(def|class)\d\p(\d|\\|\(|\:):\ + :cb=#:ce=$:sb=":se=\e":lb=':le=\e':\ + :kw=assert and break class continue def del elif else except\ + exec finally for from global if import in is lambda not or\ + pass print raise return try while yield: |