What’s New In Python 3.7

Editor:
Elvis Pranskevichus <

This article explains the new features in Python 3.7, compared to 3.6. Python 3.7 was released on June 27, 2018. For full details, see the
changelog.

Summary – Release Highlights

New syntax features:
•
PEP 563, postponed evaluation of type annotations.

Backwards incompatible syntax changes:
•
async and await are now reserved keywords.

New library modules:
•
contextvars: PEP 567 – Context Variables
•
dataclasses: PEP 557 – Data Classes
•
importlib.resources

New built-in features:
•
PEP 553, the new breakpoint() function.

Python data model improvements:
•
PEP 562, customization of access to module attributes.
•
PEP 560, core support for typing module and generic types.
•
the insertion-order preservation nature of dict objects has been declared to be an official part of the Python language spec.

Significant improvements in the standard library:
•
The asyncio module has received new features, significant usability and performance improvements.
•
The time module gained support for functions with nanosecond resolution.

CPython implementation improvements:
•
Avoiding the use of ASCII as a default text encoding:
◦
PEP 538, legacy C locale coercion
◦
PEP 540, forced UTF-8 runtime mode
•
PEP 552, deterministic .pycs
•
New Python Development Mode
•
PEP 565, improved DeprecationWarning handling

C API improvements:
•
PEP 539, new C API for thread-local storage

Documentation improvements:
•
PEP 545, Python documentation translations
•
New documentation translations: Japanese, French, and Korean.

This release features notable performance improvements in many areas. The
Optimizations section lists them in detail.

For a list of changes that may affect compatibility with previous Python releases please refer to the
Porting to Python 3.7 section.

New Features

PEP 563: Postponed Evaluation of Annotations

The advent of type hints in Python uncovered two glaring usability issues with the functionality of annotations added in PEP 3107 and refined further in PEP 526:
•
annotations could only use names which were already available in the current scope, in other words they didn’t support forward references of any kind; and
•
annotating source code had adverse effects on startup time of Python programs.

Both of these issues are fixed by postponing the evaluation of annotations. Instead of compiling code which executes expressions in annotations at their definition time, the compiler stores the annotation in a string form equivalent to the AST of the expression in question. If needed, annotations can be resolved at runtime using typing.get_type_hints(). In the common case where this is not required, the annotations are cheaper to store (since short strings are interned by the interpreter) and make startup time faster.

Usability-wise, annotations now support forward references, making the following syntax valid:

class C:

@classmethod

def from_string(cls, source: str) -> C:
    ...

def validate_b(self, obj: B) -> bool:
    ...

class B:
...

Since this change breaks compatibility, the new behavior needs to be enabled on a per-module basis in Python 3.7 using a future import:

from future import annotations

It will become the default in Python 3.10.

See also
PEP 563 – Postponed evaluation of annotations – PEP 563 – Postponed evaluation of annotations

PEP 538: Legacy C Locale Coercion

An ongoing challenge within the Python 3 series has been determining a sensible default strategy for handling the “7-bit ASCII” text encoding assumption currently implied by the use of the default C or POSIX locale on non-Windows platforms.

PEP 538 updates the default interpreter command line interface to automatically coerce that locale to an available UTF-8 based locale as described in the documentation of the new PYTHONCOERCECLOCALE environment variable. Automatically setting LC_CTYPE this way means that both the core interpreter and locale-aware C extensions (such as readline) will assume the use of UTF-8 as the default text encoding, rather than ASCII.

The platform support definition in PEP 11 has also been updated to limit full text handling support to suitably configured non-ASCII based locales.

As part of this change, the default error handler for stdin and stdout is now surrogateescape (rather than strict) when using any of the defined coercion target locales (currently C.UTF-8, C.utf8, and UTF-8). The default error handler for stderr continues to be backslashreplace, regardless of locale.

Locale coercion is silent by default, but to assist in debugging potentially locale related integration problems, explicit warnings (emitted directly on stderr) can be requested by setting PYTHONCOERCECLOCALE=warn. This setting will also cause the Python runtime to emit a warning if the legacy C locale remains active when the core interpreter is initialized.

While PEP 538’s locale coercion has the benefit of also affecting extension modules (such as GNU readline), as well as child processes (including those running non-Python applications and older versions of Python), it has the downside of requiring that a suitable target locale be present on the running system. To better handle the case where no suitable target locale is available (as occurs on RHEL/CentOS 7, for example), Python 3.7 also implements PEP 540: Forced UTF-8 Runtime Mode.

See also
PEP 538 – Coercing the legacy C locale to a UTF-8 based locale – PEP 538 – Coercing the legacy C locale to a UTF-8 based locale

PEP 540: Forced UTF-8 Runtime Mode

The new -X utf8 command line option and PYTHONUTF8 environment variable can be used to enable the Python UTF-8 Mode.

When in UTF-8 mode, CPython ignores the locale settings, and uses the UTF-8 encoding by default. The error handlers for sys.stdin and sys.stdout streams are set to surrogateescape.

The forced UTF-8 mode can be used to change the text handling behavior in an embedded Python interpreter without changing the locale settings of an embedding application.

While PEP 540’s UTF-8 mode has the benefit of working regardless of which locales are available on the running system, it has the downside of having no effect on extension modules (such as GNU readline), child processes running non-Python applications, and child processes running older versions of Python. To reduce the risk of corrupting text data when communicating with such components, Python 3.7 also implements PEP 540: Forced UTF-8 Runtime Mode.

The UTF-8 mode is enabled by default when the locale is C or POSIX, and the PEP 538 locale coercion feature fails to change it to a UTF-8 based alternative (whether that failure is due to PYTHONCOERCECLOCALE=0 being set, LC_ALL being set, or the lack of a suitable target locale).

See also
PEP 540 – Add a new UTF-8 mode – PEP 540 – Add a new UTF-8 mode

PEP 553: Built-in breakpoint()

PEP 553: Built-in breakpoint()

Python 3.7 includes the new built-in breakpoint() function as an easy and consistent way to enter the Python debugger.

Built-in breakpoint() calls sys.breakpointhook(). By default, the latter imports pdb and then calls pdb.set_trace(). By binding sys.breakpointhook() to the function of your choosing, breakpoint() can enter any debugger. Additionally, the environment variable PYTHONBREAKPOINT can be set to the callable of your debugger of choice. Set PYTHONBREAKPOINT=0 to completely disable built-in breakpoint().

See also
PEP 553 – Built-in breakpoint() – PEP 553 – Built-in breakpoint()

PEP 539: New C API for Thread-Local Storage

While Python provides a C API for thread-local storage support; the existing Thread Local Storage (TLS) API has used int to represent TLS keys across all platforms. This has not generally been a problem for officially support platforms, but that is neither POSIX-compliant, nor portable in any practical sense.

PEP 539 changes this by providing a new Thread Specific Storage (TSS) API to CPython which supersedes use of the existing TLS API within the CPython interpreter, while deprecating the existing API. The TSS API uses a new type Py_tss_t instead of int to represent TSS keys–an opaque type the definition of which may depend on the underlying TLS implementation. Therefore, this will allow to build CPython on platforms where the native TLS key is defined in a way that cannot be safely cast to int.

Note that on platforms where the native TLS key is defined in a way that cannot be safely cast to int, all functions of the existing TLS API will be no-op and immediately return failure. This indicates clearly that the old API is not supported on platforms where it cannot be used reliably, and that no effort will be made to add such support.

See also
PEP 539 – A New C-API for Thread-Local Storage in CPython – PEP 539 – A New C-API for Thread-Local Storage in CPython

PEP 562: Customization of Access to Module Attributes

Python 3.7 allows defining getattr() on modules and will call it whenever a module attribute is otherwise not found. Defining dir() on modules is now also allowed.

A typical example of where this may be useful is module attribute deprecation and lazy loading.

See also
PEP 562 – Module getattr and dir – PEP 562 – Module getattr and dir

PEP 564: New Time Functions With Nanosecond Resolution

The resolution of clocks in modern systems can exceed the limited precision of a floating-point number returned by the time.time() function and its variants. To avoid loss of precision, PEP 564 adds six new “nanosecond” variants of the existing timer functions to the time module:
•
time.clock_gettime_ns()
•
time.clock_settime_ns()
•
time.monotonic_ns()
•
time.perf_counter_ns()
•
time.process_time_ns()
•
time.time_ns()

The new functions return the number of nanoseconds as an integer value.

Measurements show that on Linux and Windows the resolution of time.time_ns() is approximately 3 times better than that of time.time().

See also
PEP 564 – Add new time functions with nanosecond resolution – PEP 564 – Add new time functions with nanosecond resolution

PEP 565: Show DeprecationWarning in main

PEP 565: Show DeprecationWarning in main

The default handling of DeprecationWarning has been changed such that these warnings are once more shown by default, but only when the code triggering them is running directly in the main module. As a result, developers of single file scripts and those using Python interactively should once again start seeing deprecation warnings for the APIs they use, but deprecation warnings triggered by imported application, library and framework modules will continue to be hidden by default.

As a result of this change, the standard library now allows developers to choose between three different deprecation warning behaviours:
•
FutureWarning: always displayed by default, recommended for warnings intended to be seen by application end users (e.g. for deprecated application configuration settings).
•
DeprecationWarning: displayed by default only in main and when running tests, recommended for warnings intended to be seen by other Python developers where a version upgrade may result in changed behaviour or an error.
•
PendingDeprecationWarning: displayed by default only when running tests, intended for cases where a future version upgrade will change the warning category to DeprecationWarning or FutureWarning.

Previously both DeprecationWarning and PendingDeprecationWarning were only visible when running tests, which meant that developers primarily writing single file scripts or using Python interactively could be surprised by breaking changes in the APIs they used.

See also
PEP 565 – Show DeprecationWarning in main – PEP 565 – Show DeprecationWarning in main

PEP 560: Core Support for typing module and Generic Types

PEP 560: Core Support for typing module and Generic Types

Initially PEP 484 was designed in such way that it would not introduce any changes to the core CPython interpreter. Now type hints and the typing module are extensively used by the community, so this restriction is removed. The PEP introduces two special methods class_getitem() and mro_entries(), these methods are now used by most classes and special constructs in typing. As a result, the speed of various operations with types increased up to 7 times, the generic types can be used without metaclass conflicts, and several long standing bugs in typing module are fixed.

See also
PEP 560 – Core support for typing module and generic types – PEP 560 – Core support for typing module and generic types

PEP 552: Hash-based .pyc Files

Python has traditionally checked the up-to-dateness of bytecode cache files (i.e., .pyc files) by comparing the source metadata (last-modified timestamp and size) with source metadata saved in the cache file header when it was generated. While effective, this invalidation method has its drawbacks. When filesystem timestamps are too coarse, Python can miss source updates, leading to user confusion. Additionally, having a timestamp in the cache file is problematic for build reproducibility and content-based build systems.

PEP 552 extends the pyc format to allow the hash of the source file to be used for invalidation instead of the source timestamp. Such .pyc files are called “hash-based”. By default, Python still uses timestamp-based invalidation and does not generate hash-based .pyc files at runtime. Hash-based .pyc files may be generated with py_compile or compileall.

Hash-based .pyc files come in two variants: checked and unchecked. Python validates checked hash-based .pyc files against the corresponding source files at runtime but doesn’t do so for unchecked hash-based pycs. Unchecked hash-based .pyc files are a useful performance optimization for environments where a system external to Python (e.g., the build system) is responsible for keeping .pyc files up-to-date.

See Cached bytecode invalidation for more information.

See also
PEP 552 – Deterministic pycs – PEP 552 – Deterministic pycs

PEP 545: Python Documentation Translations

PEP 545 describes the process of creating and maintaining Python documentation translations.

Three new translations have been added:
•
Japanese: https://docs.python.org/ja/
•
French: https://docs.python.org/fr/
•
Korean: https://docs.python.org/ko/

See also
PEP 545 – Python Documentation Translations – PEP 545 – Python Documentation Translations

Python Development Mode (-X dev)

The new -X dev command line option or the new PYTHONDEVMODE environment variable can be used to enable Python Development Mode. When in development mode, Python performs additional runtime checks that are too expensive to be enabled by default. See Python Development Mode documentation for the full description.

Other Language Changes

•
An await expression and comprehensions containing an async for clause were illegal in the expressions in formatted string literals due to a problem with the implementation. In Python 3.7 this restriction was lifted.
•
More than 255 arguments can now be passed to a function, and a function can now have more than 255 parameters. (Contributed by Serhiy Storchaka.)
•
bytes.fromhex() and bytearray.fromhex() now ignore all ASCII whitespace, not only spaces. (Contributed by Robert Xiao.)
•
str, bytes, and bytearray gained support for the new isascii() method, which can be used to test if a string or bytes contain only the ASCII characters. (Contributed by INADA Naoki.)
•
ImportError now displays module name and module file path when from ... import ... fails. (Contributed by Matthias Bussonnier.)
•
Circular imports involving absolute imports with binding a submodule to a name are now supported. (Contributed by Serhiy Storchaka.)
•
object.format(x, '') is now equivalent to str(x) rather than format(str(self), ''). (Contributed by Serhiy Storchaka.)
•
types.TracebackType can now be instantiated from Python code, and the tb_next attribute on tracebacks is now writable. (Contributed by Nathaniel J. Smith.)
•
When using the -m switch, sys.path[0] is now eagerly expanded to the full starting directory path, rather than being left as the empty directory. (Contributed by Nick Coghlan.)
•
The new -X importtime option or the PYTHONPROFILEIMPORTTIME environment variable can be used to show the timing of each module import. (Contributed by Inada Naoki.)

New Modules

contextvars

The new contextvars module and a set of new C APIs introduce support for context variables. Context variables are conceptually similar to thread-local variables. Unlike TLS, context variables support asynchronous code correctly.

The asyncio and decimal modules have been updated to use and support context variables out of the box. Particularly the active decimal context is now stored in a context variable, which allows decimal operations to work with the correct context in asynchronous code.

See also
PEP 567 – Context Variables – PEP 567 – Context Variables

dataclasses

The new dataclass() decorator provides a way to declare data classes. A data class describes its attributes using class variable annotations. Its constructor and other magic methods, such as repr, eq, and hash, are generated automatically.

Example:

@dataclass
class Point:
x: float
y: float
z: float = 0.0

p = Point(1.5, 2.5)
print(p)

produces "Point(x=1.5, y=2.5, z=0.0)"

See also
PEP 557 – Data Classes – PEP 557 – Data Classes

importlib.resources

The new importlib.resources module provides several new APIs and one new ABC for access to, opening, and reading resources inside packages. Resources are roughly similar to files inside packages, but they needn’t be actual files on the physical file system. Module loaders can provide a get_resource_reader() function which returns a importlib.abc.ResourceReader instance to support this new API. Built-in file path loaders and zip file loaders both support this.

Contributed by Barry Warsaw and Brett Cannon.

See also
importlib_resources – a PyPI backport for earlier Python versions.

Improved Modules

(argparse, asyncio, binascii, calendar, collections, compileall, concurrent.futures, contextlib, cProfile, crypt, datetime, dbm, decimal, dis, distutils, enum, functools, gc, hmac, http.client, http.server, idlelib and IDLE, importlib, io, ipaddress, itertools, locale, logging, math, mimetypes, msilib, multiprocessing, os, pathlib, pdb, py_compile, pydoc, queue, re, signal, socket, socketserver, sqlite3, ssl, string, subprocess, sys, time, tkinter, tracemalloc, types, unicodedata, unittest, unittest.mock, urllib.parse, uu, uuid, warnings, xml, xml.etree, xmlrpc.server, zipapp, zipfile)

C API Changes

A new API for thread-local storage has been implemented. See PEP 539 and Thread-specific storage API for a complete reference.

The new context variables functionality exposes a number of new C APIs.

The new PyImport_GetModule() function returns the previously imported module with the given name. (Contributed by Eric Snow.)

The new Py_RETURN_RICHCOMPARE macro eases writing rich comparison functions. (Contributed by Petr Victorin.)

The new Py_UNREACHABLE macro can be used to mark unreachable code paths. (Contributed by Barry Warsaw.)

The tracemalloc now exposes a C API through the new PyTraceMalloc_Track() and PyTraceMalloc_Untrack() functions. (Contributed by Victor Stinner.)

The new import__find__load__start and import__find__load__done static markers can be used to trace module imports. (Contributed by Christian Heimes.)

Various other C API changes and additions are described.

Build Changes

Support for building --without-threads has been removed. The threading module is now always available.

A full copy of libffi is no longer bundled for use when building the _ctypes module on non-OSX UNIX platforms. An installed copy of libffi is now required when building _ctypes on such platforms.

The Windows build process no longer depends on Subversion to pull in external sources, a Python script is used to download zipfiles from GitHub instead.

The ssl module requires OpenSSL 1.0.2 or 1.1 compatible libssl. OpenSSL 1.0.1 has reached end of lifetime and is no longer supported.

Optimizations

The overhead of calling many methods of various standard library classes implemented in C has been significantly reduced by porting more code to use the METH_FASTCALL convention.

Various optimizations have reduced Python startup time by 10% on Linux and up to 30% on macOS.

Method calls are now up to 20% faster due to the bytecode changes which avoid creating bound method instances.

asyncio module received a number of notable optimizations (get_event_loop reimplemented in C, Future callback management optimized, gather faster, sleep faster when delay is zero, etc.).

As a result of PEP 560 work, the import time of typing has been reduced by a factor of 7.

sorted() and list.sort() have been optimized for common cases to be up to 40-75% faster.

dict.copy() is now up to 5.5 times faster.

hasattr() and getattr() are now about 4 times faster when name is not found and obj does not override object.getattr or object.getattribute.

Many other optimizations described.

Other CPython Implementation Changes

Trace hooks may now opt out of receiving the line and opt into receiving the opcode events by setting f_trace_lines and f_trace_opcodes attributes on the frame being traced.

Fixed some consistency problems with namespace package module attributes.

The locals() dictionary now displays in the lexical order that variables were defined.

Various other implementation changes.

Deprecated Python Behavior

Yield expressions in comprehensions and generator expressions are now deprecated (will be SyntaxError in future).

Returning a subclass of complex from object.complex() is deprecated and will be an error in future Python versions.

Deprecated Python modules, functions and methods

(aifc, asyncio, collections, dbm, enum, gettext, importlib, locale, macpath, threading, socket, ssl, sunau, sys, wave) - many specific deprecations listed.

Deprecated functions and types of the C API

Various deprecated C API functions listed.

Platform Support Removals

FreeBSD 9 and older are no longer officially supported.

For full Unicode support, *nix platforms are now expected to provide at least one of C.UTF-8, C.utf8 or UTF-8 locales.

OpenSSL 0.9.8 and 1.0.1 are no longer supported.

API and Feature Removals

List of removed features and APIs (os.stat_float_times(), unknown escape handling in re.sub(), ntpath.splitunc(), verbose param of collections.namedtuple(), removal of keyword args for builtins like bool(), float(), list(), tuple(), etc.).

Module Removals

fpectl module has been removed.

Windows-only Changes

Improvements to the python launcher (py.exe) and related behavior.

Porting to Python 3.7

This section lists previously described changes and other bugfixes that may require changes to your code.

(Notable changes in Python 3.7.x maintenance releases and security fixes are listed, including changes in 3.7.1, 3.7.2, 3.7.6, 3.7.10, 3.7.11, and a notable security feature in 3.7.14.)

©

Last updated on Mar 17, 2026 (09:57 UTC).

Original source Report a problem

This article explains the new features in Python 3.14, compared to 3.13. Python 3.14 was released on 7 October 2025. For full details, see the changelog.

See also
PEP 745 – Python 3.14 release schedule

Summary – Release highlights
Python 3.14 is the latest stable release of the Python programming language, with a mix of changes to the language, the implementation, and the standard library. The biggest changes include template string literals, deferred evaluation of annotations, and support for subinterpreters in the standard library.

The library changes include significantly improved capabilities for introspection in asyncio, support for Zstandard via a new compression.zstd module, syntax highlighting in the REPL, as well as the usual deprecations and removals, and improvements in user-friendliness and correctness.

This article doesn’t attempt to provide a complete specification of all new features, but instead gives a convenient overview. For full details refer to the documentation, such as the Library Reference and Language Reference. To understand the complete implementation and design rationale for a change, refer to the PEP for a particular new feature; but note that PEPs usually are not kept up-to-date once a feature has been fully implemented. See Porting to Python 3.14 for guidance on upgrading from earlier versions of Python.

Interpreter improvements:
• PEP 649 and PEP 749: Deferred evaluation of annotations
• PEP 734: Multiple interpreters in the standard library
• PEP 750: Template strings
• PEP 758: Allow except and except* expressions without brackets
• PEP 765: Control flow in finally blocks
• PEP 768: Safe external debugger interface for CPython
• A new type of interpreter
• Free-threaded mode improvements
• Improved error messages

Significant improvements in the standard library:
• PEP 784: Zstandard support in the standard library
• Asyncio introspection capabilities
• Concurrent safe warnings control
• Syntax highlighting in the default interactive shell, and color output in several standard library CLIs

C API improvements:
• PEP 741: Python configuration C API

Platform support:
• PEP 776: Emscripten is now an officially supported platform, at tier 3.

Release changes:
• PEP 779: Free-threaded Python is officially supported
• PEP 761: PGP signatures have been discontinued for official releases
• Windows and macOS binary releases now support the experimental just-in-time compiler
• Binary releases for Android are now provided

New features

PEP 649 & PEP 749: Deferred evaluation of annotations
The annotations on functions, classes, and modules are no longer evaluated eagerly. Instead, annotations are stored in special-purpose annotate functions and evaluated only when necessary (except if from future import annotations is used).

This change is designed to improve performance and usability of annotations in Python in most circumstances. The runtime cost for defining annotations is minimized, but it remains possible to introspect annotations at runtime. It is no longer necessary to enclose annotations in strings if they contain forward references.

The new annotationlib module provides tools for inspecting deferred annotations. Annotations may be evaluated in the VALUE format (which evaluates annotations to runtime values, similar to the behavior in earlier Python versions), the FORWARDREF format (which replaces undefined names with special markers), and the STRING format (which returns annotations as strings).

This example shows how these formats behave:

from annotationlib import get_annotations, Format

def func(arg: Undefined):
pass

get_annotations(func, format=Format.VALUE)
Traceback (most recent call last):
...
NameError: name 'Undefined' is not defined

get_annotations(func, format=Format.FORWARDREF)
{'arg': ForwardRef('Undefined', owner=<function func at 0x...>)}

get_annotations(func, format=Format.STRING)
{'arg': 'Undefined'}

The porting section contains guidance on changes that may be needed due to these changes, though in the majority of cases, code will continue working as-is.

(Contributed by Jelle Zijlstra in PEP 749 and gh-119180; PEP 649 was written by Larry Hastings.)

See also
PEP 649
Deferred Evaluation Of Annotations Using Descriptors
PEP 749
Implementing PEP 649

PEP 734: Multiple interpreters in the standard library
The CPython runtime supports running multiple copies of Python in the same process simultaneously and has done so for over 20 years. Each of these separate copies is called an ‘interpreter’. However, the feature had been available only through the C-API.

That limitation is removed in Python 3.14, with the new concurrent.interpreters module.

There are at least two notable reasons why using multiple interpreters has significant benefits:
• they support a new (to Python), human-friendly concurrency model
• true multi-core parallelism

For some use cases, concurrency in software improves efficiency and can simplify design, at a high level. At the same time, implementing and maintaining all but the simplest concurrency is often a struggle for the human brain. That especially applies to plain threads (for example, threading), where all memory is shared between all threads.

With multiple isolated interpreters, you can take advantage of a class of concurrency models, like Communicating Sequential Processes (CSP) or the actor model, that have found success in other programming languages, like Smalltalk, Erlang, Haskell, and Go. Think of multiple interpreters as threads but with opt-in sharing.

Regarding multi-core parallelism: as of Python 3.12, interpreters are now sufficiently isolated from one another to be used in parallel (see PEP 684). This unlocks a variety of CPU-intensive use cases for Python that were limited by the GIL.

Using multiple interpreters is similar in many ways to multiprocessing, in that they both provide isolated logical “processes” that can run in parallel, with no sharing by default. However, when using multiple interpreters, an application will use fewer system resources and will operate more efficiently (since it stays within the same process). Think of multiple interpreters as having the isolation of processes with the efficiency of threads.

While the feature has been around for decades, multiple interpreters have not been used widely, due to low awareness and the lack of a standard library module. Consequently, they currently have several notable limitations, which are expected to improve significantly now that the feature is going mainstream.

Current limitations:
• starting each interpreter has not been optimized yet
• each interpreter uses more memory than necessary (work continues on extensive internal sharing between interpreters)
• there aren’t many options yet for truly sharing objects or other data between interpreters (other than memoryview)
• many third-party extension modules on PyPI are not yet compatible with multiple interpreters (all standard library extension modules are compatible)
• the approach to writing applications that use multiple isolated interpreters is mostly unfamiliar to Python users, for now

The impact of these limitations will depend on future CPython improvements, how interpreters are used, and what the community solves through PyPI packages. Depending on the use case, the limitations may not have much impact, so try it out!

Furthermore, future CPython releases will reduce or eliminate overhead and provide utilities that are less appropriate on PyPI. In the meantime, most of the limitations can also be addressed through extension modules, meaning PyPI packages can fill any gap for 3.14, and even back to 3.12 where interpreters were finally properly isolated and stopped sharing the GIL. Likewise, libraries on PyPI are expected to emerge for high-level abstractions on top of interpreters.

Regarding extension modules, work is in progress to update some PyPI projects, as well as tools like Cython, pybind11, nanobind, and PyO3. The steps for isolating an extension module are found at Isolating Extension Modules. Isolating a module has a lot of overlap with what is required to support free-threading, so the ongoing work in the community in that area will help accelerate support for multiple interpreters.

Also added in 3.14: concurrent.futures.InterpreterPoolExecutor.

(Contributed by Eric Snow in gh-134939.)

See also
PEP 734

PEP 750: Template string literals
Template strings are a new mechanism for custom string processing. They share the familiar syntax of f-strings but, unlike f-strings, return an object representing the static and interpolated parts of the string, instead of a simple str.

To write a t-string, use a 't' prefix instead of an 'f':

variety = 'Stilton'

template = t'Try some {variety} cheese!'

type(template)
<class 'string.templatelib.Template'>

Template objects provide access to the static and interpolated (in curly braces) parts of a string before they are combined. Iterate over Template instances to access their parts in order:

list(template)
['Try some ', Interpolation('Stilton', 'variety', None, ''), ' cheese!']

It’s easy to write (or call) code to process Template instances. For example, here’s a function that renders static parts lowercase and Interpolation instances uppercase:

from string.templatelib import Interpolation

def lower_upper(template):
"""Render static parts lowercase and interpolations uppercase."""
parts = []
for part in template:
if isinstance(part, Interpolation):
parts.append(str(part.value).upper())
else:
parts.append(part.lower())
return ''.join(parts)

name = 'Wenslydale'
template = t'Mister {name}'
assert lower_upper(template) == 'mister WENSLYDALE'

Because Template instances distinguish between static strings and interpolations at runtime, they can be useful for sanitising user input. Writing a html() function that escapes user input in HTML is an exercise left to the reader! Template processing code can provide improved flexibility. For instance, a more advanced html() function could accept a dict of HTML attributes directly in the template:

attributes = {'src': 'limburger.jpg', 'alt': 'lovely cheese'}
template = t'<img {attributes}>'
assert html(template) == ''

Of course, template processing code does not need to return a string-like result. An even more advanced html() could return a custom type representing a DOM-like structure.

With t-strings in place, developers can write systems that sanitise SQL, make safe shell operations, improve logging, tackle modern ideas in web development (HTML, CSS, and so on), and implement lightweight custom business DSLs.

(Contributed by Jim Baker, Guido van Rossum, Paul Everitt, Koudai Aono, Lysandros Nikolaou, Dave Peck, Adam Turner, Jelle Zijlstra, Bénédikt Tran, and Pablo Galindo Salgado in gh-132661.)

See also
PEP 750.

PEP 768: Safe external debugger interface
Python 3.14 introduces a zero-overhead debugging interface that allows debuggers and profilers to safely attach to running Python processes without stopping or restarting them. This is a significant enhancement to Python’s debugging capabilities, meaning that unsafe alternatives are no longer required.

The new interface provides safe execution points for attaching debugger code without modifying the interpreter’s normal execution path or adding any overhead at runtime. Due to this, tools can now inspect and interact with Python applications in real-time, which is a crucial capability for high-availability systems and production environments.

For convenience, this interface is implemented in the sys.remote_exec() function. For example:
import sys
from tempfile import NamedTemporaryFile
with NamedTemporaryFile(mode='w', suffix='.py', delete=False) as f:
script_path = f.name
f.write(f'import my_debugger; my_debugger.connect({os.getpid()})')

Execute in process with PID 1234

print('Behold! An offering:')
sys.remote_exec(1234, script_path)

This function allows sending Python code to be executed in a target process at the next safe execution point. However, tool authors can also implement the protocol directly as described in the PEP, which details the underlying mechanisms used to safely attach to running processes.

The debugging interface has been carefully designed with security in mind and includes several mechanisms to control access:
• A PYTHON_DISABLE_REMOTE_DEBUG environment variable.
• A -X disable-remote-debug command-line option.
• A --without-remote-debug configure flag to completely disable the feature at build time.

(Contributed by Pablo Galindo Salgado, Matt Wozniski, and Ivona Stojanovic in gh-131591.)

See also
PEP 768.

A new type of interpreter
A new type of interpreter has been added to CPython. It uses tail calls between small C functions that implement individual Python opcodes, rather than one large C case statement. For certain newer compilers, this interpreter provides significantly better performance. Preliminary benchmarks suggest a geometric mean of 3-5% faster on the standard pyperformance benchmark suite, depending on platform and architecture. The baseline is Python 3.14 built with Clang 19, without this new interpreter.

This interpreter currently only works with Clang 19 and newer on x86-64 and AArch64 architectures. However, a future release of GCC is expected to support this as well.

This feature is opt-in for now. Enabling profile-guided optimization is highly recommendeded when using the new interpreter as it is the only configuration that has been tested and validated for improved performance. For further information, see --with-tail-call-interp.

Note
This is not to be confused with tail call optimization of Python functions, which is currently not implemented in CPython.

This new interpreter type is an internal implementation detail of the CPython interpreter. It doesn’t change the visible behavior of Python programs at all. It can improve their performance, but doesn’t change anything else.

(Contributed by Ken Jin in gh-128563, with ideas on how to implement this in CPython by Mark Shannon, Garrett Gu, Haoran Xu, and Josh Haberman.)

Free-threaded mode improvements
CPython’s free-threaded mode (PEP 703), initially added in 3.13, has been significantly improved in Python 3.14. The implementation described in PEP 703 has been finished, including C API changes, and temporary workarounds in the interpreter were replaced with more permanent solutions. The specializing adaptive interpreter (PEP 659) is now enabled in free-threaded mode, which along with many other optimizations greatly improves its performance. The performance penalty on single-threaded code in free-threaded mode is now roughly 5-10%, depending on the platform and C compiler used.

From Python 3.14, when compiling extension modules for the free-threaded build of CPython on Windows, the preprocessor variable Py_GIL_DISABLED now needs to be specified by the build backend, as it will no longer be determined automatically by the C compiler. For a running interpreter, the setting that was used at compile time can be found using sysconfig.get_config_var().

The new -X context_aware_warnings flag controls if concurrent safe warnings control is enabled. The flag defaults to true for the free-threaded build and false for the GIL-enabled build.

A new thread_inherit_context flag has been added, which if enabled means that threads created with threading.Thread start with a copy of the Context() of the caller of start(). Most significantly, this makes the warning filtering context established by catch_warnings be “inherited” by threads (or asyncio tasks) started within that context. It also affects other modules that use context variables, such as the decimal context manager. This flag defaults to true for the free-threaded build and false for the GIL-enabled build.

(Contributed by Sam Gross, Matt Page, Neil Schemenauer, Thomas Wouters, Donghee Na, Kirill Podoprigora, Ken Jin, Itamar Oren, Brett Simmers, Dino Viehland, Nathan Goldbaum, Ralf Gommers, Lysandros Nikolaou, Kumar Aditya, Edgar Margffoy, and many others. Some of these contributors are employed by Meta, which has continued to provide significant engineering resources to support this project.)

Improved error messages
• The interpreter now provides helpful suggestions when it detects typos in Python keywords. When a word that closely resembles a Python keyword is encountered, the interpreter will suggest the correct keyword in the error message. This feature helps programmers quickly identify and fix common typing mistakes. For example:

whille True:
...
pass
Traceback (most recent call last): File "", line 1
whille True:
^^^^^^
SyntaxError: invalid syntax. Did you mean 'while'?

While the feature focuses on the most common cases, some variations of misspellings may still result in regular syntax errors. (Contributed by Pablo Galindo in gh-132449.)
• elif statements that follow an else block now have a specific error message. (Contributed by Steele Farnsworth in gh-129902.)

if who == "me":
...
print("It's me!")
else:
...
print("It's not me!")
elif who is None:
...
print("Who is it?")
File "", line 5
elif who is None:
^^^^
SyntaxError: 'elif' block follows an 'else' block
• If a statement is passed to the Conditional expressions after else, or one of pass, break, or continue is passed before if, then the error message highlights where the expression is required. (Contributed by Sergey Miryanov in gh-129515.)

x = 1
if True else pass
Traceback (most recent call last): File "", line 1
^^^
SyntaxError: expected expression after 'else', but statement is given

x = continue if True else break
Traceback (most recent call last): File "", line 1
^^^^^^^^
SyntaxError: expected expression before 'if', but statement is given
• When incorrectly closed strings are detected, the error message suggests that the string may be intended to be part of the string. (Contributed by Pablo Galindo in gh-88535.)

"The interesting object " The important object " is very important"
Traceback (most recent call last):
SyntaxError: invalid syntax. Is this intended to be part of the string?
• When strings have incompatible prefixes, the error now shows which prefixes are incompatible. (Contributed by Nikita Sobolev in gh-133197.)

ub'abc'
File "", line 1
^^
SyntaxError: 'u' and 'b' prefixes are incompatible
• Improved error messages when using as with incompatible targets in:

• Imports: import ... as ...
• From imports: from ... import ... as ...
• Except handlers: except ... as ...
• Pattern-match cases: case ... as ...
  (Contributed by Nikita Sobolev in gh-123539, gh-123562, and gh-123440.)
  • Improved error message when trying to add an instance of an unhashable type to a dict or set. (Contributed by CF Bolz-Tereick and Victor Stinner in gh-132828.) s = set()
  s.add({'pages': 12, 'grade': 'A'})
  Traceback (most recent call last): File "", line 1, in
  s.add({'pages': 12, 'grade': 'A'})
  TypeError: cannot use 'dict' as a set element (unhashable type: 'dict') d = {}
  l = [1, 2, 3]
  d[l] = 12
  Traceback (most recent call last): File "", line 1, in
  d[l] = 12
  TypeError: cannot use 'list' as a dict key (unhashable type: 'list')
  • Improved error message when an object supporting the synchronous context manager protocol is entered using async with instead of with, and vice versa for the asynchronous context manager protocol. (Contributed by Bénédikt Tran in gh-128398.)

PEP 784: Zstandard support in the standard library
The new compression package contains modules compression.lzma, compression.bz2, compression.gzip, and compression.zlib which re-export the lzma, bz2, gzip, and zlib modules respectively. The new import names under compression are the preferred names for importing these compression modules from Python 3.14. However, the existing modules names have not been deprecated. Any deprecation or removal of the existing compression modules will occur no sooner than five years after the release of 3.14.

The new compression.zstd module provides compression and decompression APIs for the Zstandard format via bindings to Meta’s zstd library. Zstandard is a widely adopted, highly efficient, and fast compression format. In addition to the APIs introduced in compression.zstd, support for reading and writing Zstandard compressed archives has been added to the tarfile, zipfile, and shutil modules.

Here’s an example of using the new module to compress some data:
from compression import zstd
import math
data = str(math.pi).encode() * 20
compressed = zstd.compress(data)
ratio = len(compressed) / len(data)
print(f"Achieved compression ratio of {ratio} ")

As can be seen, the API is similar to the APIs of the lzma and bz2 modules.

(Contributed by Emma Harper Smith, Adam Turner, Gregory P. Smith, Tomas Roun, Victor Stinner, and Rogdham in gh-132983.)

See also
PEP 784.

Asyncio introspection capabilities
Added a new command-line interface to inspect running Python processes using asynchronous tasks, available via python -m asyncio ps PID or python -m asyncio pstree PID.

The ps subcommand inspects the given process ID (PID) and displays information about currently running asyncio tasks. It outputs a task table: a flat listing of all tasks, their names, their coroutine stacks, and which tasks are awaiting them.

The pstree subcommand fetches the same information, but instead renders a visual async call tree, showing coroutine relationships in a hierarchical format. This command is particularly useful for debugging long-running or stuck asynchronous programs. It can help developers quickly identify where a program is blocked, what tasks are pending, and how coroutines are chained together.

For example given this code:
import asyncio
async def play_track(track):
await asyncio.sleep(5)
print(f'🎵 Finished: {track}')

async def play_album(name, tracks):
async with asyncio.TaskGroup() as tg:
for track in tracks:
tg.create_task(play_track(track), name=track)

async def main():
async with asyncio.TaskGroup() as tg:
tg.create_task(play_album('Sundowning', ['TNDNBTG', 'Levitate']), name='Sundowning')
tg.create_task(play_album('TMBTE', ['DYWTYLM', 'Aqua Regia']), name='TMBTE')

if name == 'main':
asyncio.run(main())

Executing the new tool on the running process will yield a table like this:
python -m asyncio ps 12345
tid task id task name coroutine stack awaiter chain awaiter name awaiter id
...

or a tree like this:
python -m asyncio pstree 12345
└── (T) Task-1
└── main example.py:13
└── TaskGroup.aexit Lib/asyncio/taskgroups.py:72
└── TaskGroup._aexit Lib/asyncio/taskgroups.py:121
├── (T) Sundowning
│ └── album example.py:8
│ └── TaskGroup.aexit Lib/asyncio/taskgroups.py:72
│ └── TaskGroup._aexit Lib/asyncio/taskgroups.py:121
├── (T) TMBTE
│ └── album example.py:8
│ └── TaskGroup.aexit Lib/asyncio/taskgroups.py:72
│ └── TaskGroup._aexit Lib/asyncio/taskgroups.py:121
├── (T) TNDNBTG
│ └── sleep
│ └── play
│ └── TaskGroup._aexit
└── (T) Levitate
└── sleep
└── play

If a cycle is detected in the async await graph (which could indicate a programming issue), the tool raises an error and lists the cycle paths that prevent tree construction:
python -m asyncio pstree 12345 ERROR: await-graph contains cycles - cannot print a tree!

cycle:
Task-2 → Task-3 → Task-2

(Contributed by Pablo Galindo, Łukasz Langa, Yury Selivanov, and Marta Gomez Macias in gh-91048.)

Concurrent safe warnings control
The warnings.catch_warnings context manager will now optionally use a context variable for warning filters. This is enabled by setting the context_aware_warnings flag, either with the -X command-line option or an environment variable. This gives predictable warnings control when using catch_warnings combined with multiple threads or asynchronous tasks. The flag defaults to true for the free-threaded build and false for the GIL-enabled build.

(Contributed by Neil Schemenauer and Kumar Aditya in gh-130010.)

Other language changes
• All Windows code pages are now supported as ‘cpXXX’ codecs on Windows. (Contributed by Serhiy Storchaka in gh-123803.)
• Implement mixed-mode arithmetic rules combining real and complex numbers as specified by the C standard since C99. (Contributed by Sergey B Kirpichev in gh-69639.)
• More syntax errors are now detected regardless of optimisation and the -O command-line option. This includes writes to debug, incorrect use of await, and asynchronous comprehensions outside asynchronous functions. For example, python -O -c 'assert (debug := 1)' or python -O -c 'assert await 1' now produce SyntaxErrors. (Contributed by Irit Katriel and Jelle Zijlstra in gh-122245 & gh-121637.)
• When subclassing a pure C type, the C slots for the new type are no longer replaced with a wrapped version on class creation if they are not explicitly overridden in the subclass. (Contributed by Tomasz Pytel in gh-132284.)

Built-ins
• The bytes.fromhex() and bytearray.fromhex() methods now accept ASCII bytes and bytes-like objects. (Contributed by Daniel Pope in gh-129349.)
• Add class methods float.from_number() and complex.from_number() to convert a number to float or complex type correspondingly. They raise a TypeError if the argument is not a real number. (Contributed by Serhiy Storchaka in gh-84978.)
• Support underscore and comma as thousands separators in the fractional part for floating-point presentation types of the new-style string formatting (with format() or f-strings). (Contributed by Sergey B Kirpichev in gh-87790.)
• The int() function no longer delegates to trunc(). Classes that want to support conversion to int() must implement either int() or index(). (Contributed by Mark Dickinson in gh-119743.)
• The map() function now has an optional keyword-only strict flag like zip() to check that all the iterables are of equal length. (Contributed by Wannes Boeykens in gh-119793.)
• The memoryview type now supports subscription, making it a generic type. (Contributed by Brian Schubert in gh-126012.)
• Using NotImplemented in a boolean context will now raise a TypeError. This has raised a DeprecationWarning since Python 3.9. (Contributed by Jelle Zijlstra in gh-118767.)
• Three-argument pow() now tries calling rpow() if necessary. Previously it was only called in two-argument pow() and the binary power operator. (Contributed by Serhiy Storchaka in gh-130104.)
• super objects are now copyable and pickleable. (Contributed by Serhiy Storchaka in gh-125767.)

Command line and environment
• The import time flag can now track modules that are already loaded (‘cached’), via the new -X importtime=2. When such a module is imported, the self and cumulative times are replaced by the string cached.
Values above 2 for -X importtime are now reserved for future use.
(Contributed by Noah Kim and Adam Turner in gh-118655.)
• The command-line option -c now automatically dedents its code argument before execution. The auto-dedentation behavior mirrors textwrap.dedent(). (Contributed by Jon Crall and Steven Sun in gh-103998.)
• -J is no longer a reserved flag for Jython, and now has no special meaning. (Contributed by Adam Turner in gh-133336.)

PEP 758: Allow except and except* expressions without brackets
The except and except* expressions now allow brackets to be omitted when there are multiple exception types and the as clause is not used. For example:
try:
connect_to_server()
except TimeoutError, ConnectionRefusedError:
print('The network has ceased to be!')

(Contributed by Pablo Galindo and Brett Cannon in PEP 758 and gh-131831.)

PEP 765: Control flow in finally blocks
The compiler now emits a SyntaxWarning when a return, break, or continue statement have the effect of leaving a finally block. This change is specified in PEP 765.

In situations where this change is inconvenient (such as those where the warnings are redundant due to code linting), the warning filter can be used to turn off all syntax warnings by adding ignore::SyntaxWarning as a filter. This can be specified in combination with a filter that converts other warnings to errors (for example, passing -Werror -Wignore::SyntaxWarning as CLI options, or setting PYTHONWARNINGS=error,ignore::SyntaxWarning).

Note that applying such a filter at runtime using the warnings module will only suppress the warning in code that is compiled after the filter is adjusted. Code that is compiled prior to the filter adjustment (for example, when a module is imported) will still emit the syntax warning.

(Contributed by Irit Katriel in gh-130080.)

Incremental garbage collection
The cycle garbage collector is now incremental. This means that maximum pause times are reduced by an order of magnitude or more for larger heaps.

There are now only two generations: young and old. When gc.collect() is not called directly, the GC is invoked a little less frequently. When invoked, it collects the young generation and an increment of the old generation, instead of collecting one or more generations.

The behavior of gc.collect() changes slightly:
• gc.collect(1): Performs an increment of garbage collection, rather than collecting generation 1.
• Other calls to gc.collect() are unchanged.

(Contributed by Mark Shannon in gh-108362.)

Default interactive shell
• The default interactive shell now highlights Python syntax. The feature is enabled by default, save if PYTHON_BASIC_REPL or any other environment variable that disables colour is set. See Controlling color for details.
The default color theme for syntax highlighting strives for good contrast and exclusively uses the 4-bit VGA standard ANSI color codes for maximum compatibility. The theme can be customized using an experimental API _colorize.set_theme(). This can be called interactively or in the PYTHONSTARTUP script. Note that this function has no stability guarantees, and may change or be removed.
(Contributed by Łukasz Langa in gh-131507.)
• The default interactive shell now supports import auto-completion. This means that typing import co and pressing will suggest modules starting with co. Similarly, typing from concurrent import i will suggest submodules of concurrent starting with i. Note that autocompletion of module attributes is not currently supported. (Contributed by Tomas Roun in gh-69605.)

New modules
• annotationlib: For introspecting annotations. See PEP 749 for more details. (Contributed by Jelle Zijlstra in gh-119180.)
• compression (including compression.zstd): A package for compression-related modules, including a new module to support the Zstandard compression format. See PEP 784 for more details. (Contributed by Emma Harper Smith, Adam Turner, Gregory P. Smith, Tomas Roun, Victor Stinner, and Rogdham in gh-132983.)
• concurrent.interpreters: Support for multiple interpreters in the standard library. See PEP 734 for more details. (Contributed by Eric Snow in gh-134939.)
• string.templatelib: Support for template string literals (t-strings). See PEP 750 for more details. (Contributed by Jim Baker, Guido van Rossum, Paul Everitt, Koudai Aono, Lysandros Nikolaou, Dave Peck, Adam Turner, Jelle Zijlstra, Bénédikt Tran, and Pablo Galindo Salgado in gh-132661.)

Original source Report a problem

This article explains the new features in Python 3.13, compared to 3.12. Python 3.13 was released on October 7, 2024. For full details, see the changelog.

Summary – Release Highlights

Python 3.13 is a stable release of the Python programming language, with a mix of changes to the language, the implementation and the standard library. The biggest changes include a new interactive interpreter, experimental support for running in a free-threaded mode (PEP 703), and a Just-In-Time compiler (PEP 744).

Error messages continue to improve, with tracebacks now highlighted in color by default. The locals() builtin now has defined semantics for changing the returned mapping, and type parameters now support default values.

The library changes contain removal of deprecated APIs and modules, as well as the usual improvements in user-friendliness and correctness. Several legacy standard library modules have now been removed following their deprecation in Python 3.11 (PEP 594).

This article doesn’t attempt to provide a complete specification of all new features, but instead gives a convenient overview. For full details refer to the documentation, such as the Library Reference and Language Reference. To understand the complete implementation and design rationale for a change, refer to the PEP for a particular new feature; but note that PEPs usually are not kept up-to-date once a feature has been fully implemented. See Porting to Python 3.13 for guidance on upgrading from earlier versions of Python.

Interpreter improvements

• A greatly improved interactive interpreter and improved error messages.
• PEP 667: The locals() builtin now has defined semantics when mutating the returned mapping. Python debuggers and similar tools may now more reliably update local variables in optimized scopes even during concurrent code execution.
• PEP 703: CPython 3.13 has experimental support for running with the global interpreter lock disabled. See Free-threaded CPython for more details.
• PEP 744: A basic JIT compiler was added. It is currently disabled by default (though we may turn it on later). Performance improvements are modest – we expect to improve this over the next few releases.
• Color support in the new interactive interpreter, as well as in tracebacks and doctest output. This can be disabled through the PYTHON_COLORS and NO_COLOR environment variables.

Python data model improvements

• static_attributes stores the names of attributes accessed through self.X in any function in a class body.
• firstlineno records the first line number of a class definition.

Significant improvements in the standard library

• Add a new PythonFinalizationError exception, raised when an operation is blocked during finalization.
• The argparse module now supports deprecating command-line options, positional arguments, and subcommands.
• The new functions base64.z85encode() and base64.z85decode() support encoding and decoding Z85 data.
• The copy module now has a copy.replace() function, with support for many builtin types and any class defining the replace() method.
• The new dbm.sqlite3 module is now the default dbm backend.
• The os module has a suite of new functions for working with Linux’s timer notification file descriptors.
• The random module now has a command-line interface.

Security improvements

• ssl.create_default_context() sets ssl.VERIFY_X509_PARTIAL_CHAIN and ssl.VERIFY_X509_STRICT as default flags.

C API improvements

• The Py_mod_gil slot is now used to indicate that an extension module supports running with the GIL disabled.
• The PyTime C API has been added, providing access to system clocks.
• PyMutex is a new lightweight mutex that occupies a single byte.
• There is a new suite of functions for generating PEP 669 monitoring events in the C API.

New typing features

• PEP 696: Type parameters (typing.TypeVar, typing.ParamSpec, and typing.TypeVarTuple) now support defaults.
• PEP 702: The new warnings.deprecated() decorator adds support for marking deprecations in the type system and at runtime.
• PEP 705: typing.ReadOnly can be used to mark an item of a typing.TypedDict as read-only for type checkers.
• PEP 742: typing.TypeIs provides more intuitive type narrowing behavior, as an alternative to typing.TypeGuard.

Platform support

• PEP 730: Apple’s iOS is now an officially supported platform, at tier 3.
• PEP 738: Android is now an officially supported platform, at tier 3.
• wasm32-wasi is now supported as a tier 2 platform.
• wasm32-emscripten is no longer an officially supported platform.

Important removals

• PEP 594: The remaining 19 “dead batteries” (legacy stdlib modules) have been removed from the standard library: aifc, audioop, cgi, cgitb, chunk, crypt, imghdr, mailcap, msilib, nis, nntplib, ossaudiodev, pipes, sndhdr, spwd, sunau, telnetlib, uu, and xdrlib.
• Remove the 2to3 tool and lib2to3 module (deprecated in Python 3.11).
• Remove the tkinter.tix module (deprecated in Python 3.6).
• Remove the locale.resetlocale() function.
• Remove the typing.io and typing.re namespaces.
• Remove chained classmethod descriptors.

Release schedule changes

PEP 602 (“Annual Release Cycle for Python”) has been updated to extend the full support (‘bugfix’) period for new releases to two years. This updated policy means that:

• Python 3.9–3.12 have one and a half years of full support, followed by three and a half years of security fixes.
• Python 3.13 and later have two years of full support, followed by three years of security fixes.

Original source Report a problem

Adam Turner

This article explains the new features in Python 3.12, compared to 3.11. Python 3.12 was released on October 2, 2023. For full details, see the changelog.

See also
PEP 693 – Python 3.12 Release Schedule

Summary – Release highlights

Python 3.12 is a stable release of the Python programming language, with a mix of changes to the language and the standard library. The library changes focus on cleaning up deprecated APIs, usability, and correctness. Of note, the
distutils package has been removed from the standard library. Filesystem support in os and pathlib has seen a number of improvements, and several modules have better performance.

The language changes focus on usability, as f-strings have had many limitations removed and ‘Did you mean …’ suggestions continue to improve. The new type parameter syntax and type statement improve ergonomics for using generic types and type aliases with static type checkers.

This article doesn’t attempt to provide a complete specification of all new features, but instead gives a convenient overview. For full details, you should refer to the documentation, such as the Library Reference and Language Reference. If you want to understand the complete implementation and design rationale for a change, refer to the PEP for a particular new feature; but note that PEPs usually are not kept up-to-date once a feature has been fully implemented.

New syntax features

• PEP 695, type parameter syntax and the type statement

New grammar features

• PEP 701, f-strings in the grammar

Interpreter improvements

• PEP 684, a unique per-interpreter GIL
• PEP 669, low impact monitoring
• Improved ‘Did you mean …’ suggestions for NameError, ImportError, and SyntaxError exceptions

Python data model improvements

• PEP 688, using the buffer protocol from Python

Significant improvements in the standard library

• The pathlib.Path class now supports subclassing
• The os module received several improvements for Windows support
• A command-line interface has been added to the sqlite3 module
• isinstance() checks against runtime-checkable protocols enjoy a speed up of between two and 20 times
• The asyncio package has had a number of performance improvements, with some benchmarks showing a 75% speed up.
• A command-line interface has been added to the uuid module
• Due to the changes in PEP 701, producing tokens via the tokenize module is up to 64% faster.

Security improvements

• Replace the builtin hashlib implementations of SHA1, SHA3, SHA2-384, SHA2-512, and MD5 with formally verified code from the HACL* project. These builtin implementations remain as fallbacks that are only used when OpenSSL does not provide them.

C API improvements

• PEP 697, unstable C API tier
• PEP 683, immortal objects

CPython implementation improvements

• PEP 709, comprehension inlining
• CPython support for the Linux perf profiler
• Implement stack overflow protection on supported platforms

New typing features

• PEP 692, using TypedDict to annotate **kwargs
• PEP 698, typing.override() decorator

Important deprecations, removals or restrictions

• PEP 623: Remove wstr from Unicode objects in Python’s C API
• PEP 632: Remove the distutils package. See the migration guide for advice replacing the APIs it provided. The third-party Setuptools package continues to provide distutils if still required.
• gh-95299: Do not pre-install setuptools in virtual environments created with venv. distutils, setuptools, pkg_resources, and easy_install will no longer be available by default; to access these run pip install setuptools in the activated virtual environment.
• The asynchat, asyncore, and imp modules have been removed, along with several unittest.TestCase method aliases.

[Further sections in the document include detailed New Features (PEP 695, PEP 701, PEP 684, PEP 669, PEP 688, PEP 709), Improved Error Messages, New Features Related to Type Hints (PEP 692, PEP 698), Other Language Changes, New Modules (None), Improved Modules (array, asyncio, calendar, csv, dis, fractions, importlib.resources, inspect, itertools, math, os, os.path, pathlib, platform, pdb, random, shutil, sqlite3, statistics, sys, tempfile, threading, tkinter, tokenize, types, typing, unicodedata, unittest, uuid), Optimizations, CPython bytecode changes, Demos and Tools, Deprecated and Pending removals across multiple future versions, Porting to Python 3.12, Build Changes, C API Changes, and Removed modules (asynchat, asyncore, configparser changes, distutils removal, ensurepip changes, enum, ftplib, gzip, hashlib, importlib, imp removal, io, locale, smtpd, sqlite3, ssl, unittest removals, webbrowser, xml.etree.ElementTree, zipimport, and others).]

Last updated on Mar 15, 2026 (22:57 UTC).

Original source Report a problem

This article explains the new features in Python 3.11, compared to 3.10. Python 3.11 was released on October 24, 2022. For full details, see the changelog.

Summary – Release highlights
Python 3.11 is between 10-60% faster than Python 3.10. On average, we measured a 1.25x speedup on the standard benchmark suite. See Faster CPython for details.

New syntax features:
PEP 654: Exception Groups and except*

New built-in features:
PEP 678: Exceptions can be enriched with notes

New standard library modules:
PEP 680: tomllib — Support for parsing TOML in the Standard Library

Interpreter improvements:
PEP 657: Fine-grained error locations in tracebacks
New -P command line option and PYTHONSAFEPATH environment variable to disable automatically prepending potentially unsafe paths to sys.path

New typing features:
PEP 646: Variadic generics
PEP 655: Marking individual TypedDict items as required or not-required
PEP 673: Self type
PEP 675: Arbitrary literal string type
PEP 681: Data class transforms

Important deprecations, removals and restrictions:
PEP 594: Many legacy standard library modules have been deprecated and will be removed in Python 3.13
PEP 624: Py_UNICODE encoder APIs have been removed
PEP 670: Macros converted to static inline functions

New Features
PEP 657: Fine-grained error locations in tracebacks
When printing tracebacks, the interpreter will now point to the exact expression that caused the error, instead of just the line. For example:

Previous versions of the interpreter would point to just the line, making it ambiguous which object was None. These enhanced errors can also be helpful when dealing with deeply nested dict objects and multiple function calls:

As well as complex arithmetic expressions:

Additionally, the information used by the enhanced traceback feature is made available via a general API, that can be used to correlate bytecode instructions with source code location. This information can be retrieved using:
The codeobject.co_positions() method in Python.
The PyCode_Addr2Location() function in the C API.
See PEP 657 for more details. (Contributed by Pablo Galindo, Batuhan Taskaya and Ammar Askar in bpo-43950.)

Note
This feature requires storing column positions in Code Objects, which may result in a small increase in interpreter memory usage and disk usage for compiled Python files. To avoid storing the extra information and deactivate printing the extra traceback information, use the -X no_debug_ranges command line option or the PYTHONNODEBUGRANGES environment variable.

PEP 654: Exception Groups and except*
PEP 654 introduces language features that enable a program to raise and handle multiple unrelated exceptions simultaneously. The builtin types ExceptionGroup and BaseExceptionGroup make it possible to group exceptions and raise them together, and the new except* syntax generalizes except to match subgroups of exception groups.
See PEP 654 for more details.
(Contributed by Irit Katriel in bpo-45292. PEP written by Irit Katriel, Yury Selivanov and Guido van Rossum.)

PEP 678: Exceptions can be enriched with notes
The add_note() method is added to BaseException. It can be used to enrich exceptions with context information that is not available at the time when the exception is raised. The added notes appear in the default traceback.
See PEP 678 for more details.
(Contributed by Irit Katriel in bpo-45607. PEP written by Zac Hatfield-Dodds.)

Windows py.exe launcher improvements
The copy of the Python install manager included with Python 3.11 has been significantly updated. It now supports company/tag syntax as defined in PEP 514 using the -V:/ argument instead of the limited -.. This allows launching distributions other than PythonCore, the one hosted on python.org.
When using -V: selectors, either company or tag can be omitted, but all installs will be searched. For example, -V:OtherPython/ will select the “best” tag registered for OtherPython, while -V:3.11 or -V:/3.11 will select the “best” distribution with tag 3.11.
When using the legacy -, -., -- or -.- arguments, all existing behaviour should be preserved from past versions, and only releases from PythonCore will be selected. However, the -64 suffix now implies “not 32-bit” (not necessarily x86-64), as there are multiple supported 64-bit platforms. 32-bit runtimes are detected by checking the runtime’s tag for a -32 suffix. All releases of Python since 3.5 have included this in their 32-bit builds.

New Features Related to Type Hints
This section covers major changes affecting PEP 484 type hints and the typing module.

PEP 646: Variadic generics
PEP 484 previously introduced TypeVar, enabling creation of generics parameterised with a single type. PEP 646 adds TypeVarTuple, enabling parameterisation with an arbitrary number of types. In other words, a TypeVarTuple is a variadic type variable, enabling variadic generics.
This enables a wide variety of use cases. In particular, it allows the type of array-like structures in numerical computing libraries such as NumPy and TensorFlow to be parameterised with the array shape. Static type checkers will now be able to catch shape-related bugs in code that uses these libraries.
See PEP 646 for more details.
(Contributed by Matthew Rahtz in bpo-43224, with contributions by Serhiy Storchaka and Jelle Zijlstra. PEP written by Mark Mendoza, Matthew Rahtz, Pradeep Kumar Srinivasan, and Vincent Siles.)

PEP 655: Marking individual TypedDict items as required or not-required
Required and NotRequired provide a straightforward way to mark whether individual items in a TypedDict must be present. Previously, this was only possible using inheritance.
All fields are still required by default, unless the total parameter is set to False, in which case all fields are still not-required by default. For example, the following specifies a TypedDict with one required and one not-required key:

The following definition is equivalent:

See PEP 655 for more details.
(Contributed by David Foster and Jelle Zijlstra in bpo-47087. PEP written by David Foster.)

PEP 673: Self type
The new Self annotation provides a simple and intuitive way to annotate methods that return an instance of their class. This behaves the same as the TypeVar-based approach specified in PEP 484, but is more concise and easier to follow.
Common use cases include alternative constructors provided as classmethod s, and enter() methods that return self:

Self can also be used to annotate method parameters or attributes of the same type as their enclosing class.
See PEP 673 for more details.
(Contributed by James Hilton-Balfe in bpo-46534. PEP written by Pradeep Kumar Srinivasan and James Hilton-Balfe.)

PEP 675: Arbitrary literal string type
The new LiteralString annotation may be used to indicate that a function parameter can be of any literal string type. This allows a function to accept arbitrary literal string types, as well as strings created from other literal strings. Type checkers can then enforce that sensitive functions, such as those that execute SQL statements or shell commands, are called only with static arguments, providing protection against injection attacks.
For example, a SQL query function could be annotated as follows:

See PEP 675 for more details.
(Contributed by Jelle Zijlstra in bpo-47088. PEP written by Pradeep Kumar Srinivasan and Graham Bleaney.)

PEP 681: Data class transforms
@dataclass_transform() may be used to decorate a class, metaclass, or a function that is itself a decorator. The presence of @dataclass_transform() tells a static type checker that the decorated object performs runtime “magic” that transforms a class, giving it dataclass-like behaviors.
For example:

See PEP 681 for more details.
(Contributed by Jelle Zijlstra in gh-91860. PEP written by Erik De Bonte and Eric Traut.)

PEP 563 may not be the future
PEP 563 Postponed Evaluation of Annotations (the from future import annotations future statement) that was originally planned for release in Python 3.10 has been put on hold indefinitely. See this message from the Steering Council for more information.

Other Language Changes
Starred unpacking expressions can now be used in for statements. (See bpo-46725 for more details.)
Asynchronous comprehensions are now allowed inside comprehensions in asynchronous functions. Outer comprehensions implicitly become asynchronous in this case. (Contributed by Serhiy Storchaka in bpo-33346.)
A TypeError is now raised instead of an AttributeError in with statements and contextlib.ExitStack.enter_context() for objects that do not support the context manager protocol, and in async with statements and contextlib.AsyncExitStack.enter_async_context() for objects not supporting the asynchronous context manager protocol. (Contributed by Serhiy Storchaka in bpo-12022 and bpo-44471.)
Added object.getstate(), which provides the default implementation of the getstate() method. copy ing and pickle ing instances of subclasses of builtin types bytearray, set, frozenset, collections.OrderedDict, collections.deque, weakref.WeakSet, and datetime.tzinfo now copies and pickles instance attributes implemented as slots. This change has an unintended side effect: It trips up a small minority of existing Python projects not expecting object.getstate() to exist. See the later comments on gh-70766 for discussions of what workarounds such code may need. (Contributed by Serhiy Storchaka in bpo-26579.)
Added a -P command line option and a PYTHONSAFEPATH environment variable, which disable the automatic prepending to sys.path of the script’s directory when running a script, or the current directory when using -c and -m . This ensures only stdlib and installed modules are picked up by import , and avoids unintentionally or maliciously shadowing modules with those in a local (and typically user-writable) directory. (Contributed by Victor Stinner in gh-57684.)
A "z" option was added to the Format specification mini-language that coerces negative to positive zero after rounding to the format precision. See PEP 682 for more details. (Contributed by John Belmonte in gh-90153.)
Bytes are no longer accepted on sys.path . Support broke sometime between Python 3.2 and 3.6, with no one noticing until after Python 3.10.0 was released. In addition, bringing back support would be problematic due to interactions between -b and sys.path_importer_cache when there is a mixture of str and bytes keys. (Contributed by Thomas Grainger in gh-91181.)

Other CPython Implementation Changes
The special methods complex() for complex and bytes() for bytes are implemented to support the typing.SupportsComplex and typing.SupportsBytes protocols. (Contributed by Mark Dickinson and Donghee Na in bpo-24234.)
siphash13 is added as a new internal hashing algorithm. It has similar security properties as siphash24 , but it is slightly faster for long inputs. str , bytes , and some other types now use it as the default algorithm for hash() . PEP 552 hash-based .pyc files now use siphash13 too. (Contributed by Inada Naoki in bpo-29410.)
When an active exception is re-raised by a raise statement with no parameters, the traceback attached to this exception is now always sys.exc_info()[1].traceback . This means that changes made to the traceback in the current except clause are reflected in the re-raised exception. (Contributed by Irit Katriel in bpo-45711.)
The interpreter state’s representation of handled exceptions (aka exc_info or _PyErr_StackItem ) now only has the exc_value field; exc_type and exc_traceback have been removed, as they can be derived from exc_value . (Contributed by Irit Katriel in bpo-45711.)
A new command line option, AppendPath , has been added for the Windows installer. It behaves similarly to PrependPath , but appends the install and scripts directories instead of prepending them. (Contributed by Bastian Neuburger in bpo-44934.)
The PyConfig.module_search_paths_set field must now be set to 1 for initialization to use PyConfig.module_search_paths to initialize sys.path . Otherwise, initialization will recalculate the path and replace any values added to module_search_paths .
The output of the --help option now fits in 50 lines/80 columns. Information about Python environment variables and -X options is now available using the respective --help-env and --help-xoptions flags, and with the new --help-all . (Contributed by Éric Araujo in bpo-46142.)
Converting between int and str in bases other than 2 (binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal) now raises a ValueError if the number of digits in string form is above a limit to avoid potential denial of service attacks due to the algorithmic complexity. This is a mitigation for CVE 2020-10735 . This limit can be configured or disabled by environment variable, command line flag, or sys APIs. See the integer string conversion length limitation documentation. The default limit is 4300 digits in string form.

New Modules
tomllib : For parsing TOML. See PEP 680 for more details. (Contributed by Taneli Hukkinen in bpo-40059.)
wsgiref.types : WSGI-specific types for static type checking. (Contributed by Sebastian Rittau in bpo-42012.)

Improved Modules
asyncio
Added the TaskGroup class, an asynchronous context manager holding a group of tasks that will wait for all of them upon exit. For new code this is recommended over using create_task() and gather() directly. (Contributed by Yury Selivanov and others in gh-90908.)
Added timeout() , an asynchronous context manager for setting a timeout on asynchronous operations. For new code this is recommended over using wait_for() directly. (Contributed by Andrew Svetlov in gh-90927.)
Added the Runner class, which exposes the machinery used by run() . (Contributed by Andrew Svetlov in gh-91218.)
Added the Barrier class to the synchronization primitives in the asyncio library, and the related BrokenBarrierError exception. (Contributed by Yves Duprat and Andrew Svetlov in gh-87518.)
Added keyword argument all_errors to asyncio.loop.create_connection() so that multiple connection errors can be raised as an ExceptionGroup .
Added the asyncio.StreamWriter.start_tls() method for upgrading existing stream-based connections to TLS. (Contributed by Ian Good in bpo-34975.)
Added raw datagram socket functions to the event loop: sock_sendto() , sock_recvfrom() and sock_recvfrom_into() . These have implementations in SelectorEventLoop and ProactorEventLoop . (Contributed by Alex Grönholm in bpo-46805.)
Added cancelling() and uncancel() methods to Task . These are primarily intended for internal use, notably by TaskGroup .

contextlib
Added non parallel-safe chdir() context manager to change the current working directory and then restore it on exit. Simple wrapper around chdir() . (Contributed by Filipe Laíns in bpo-25625)

dataclasses
Change field default mutability check, allowing only defaults which are hashable instead of any object which is not an instance of dict , list or set . (Contributed by Eric V. Smith in bpo-44674.)

datetime
Add datetime.UTC , a convenience alias for datetime.timezone.utc . (Contributed by Kabir Kwatra in gh-91973.)
datetime.date.fromisoformat() , datetime.time.fromisoformat() and datetime.datetime.fromisoformat() can now be used to parse most ISO 8601 formats (barring only those that support fractional hours and minutes). (Contributed by Paul Ganssle in gh-80010.)

enum
Renamed EnumMeta to EnumType (EnumMeta kept as an alias).
Added StrEnum , with members that can be used as (and must be) strings.
Added ReprEnum , which only modifies the repr() of members while returning their literal values (rather than names) for str() and format() (used by str() , format() and f-strings).
Changed Enum.format() (the default for format() , str.format() and f-strings) to always produce the same result as Enum.str() : for enums inheriting from ReprEnum it will be the member’s value; for all other enums it will be the enum and member name (e.g. Color.RED ).
Added a new boundary class parameter to Flag enums and the FlagBoundary enum with its options, to control how to handle out-of-range flag values.
Added the verify() enum decorator and the EnumCheck enum with its options, to check enum classes against several specific constraints.
Added the member() and nonmember() decorators, to ensure the decorated object is/is not converted to an enum member.
Added the property() decorator, which works like property() except for enums. Use this instead of types.DynamicClassAttribute() .
Added the global_enum() enum decorator, which adjusts repr() and str() to show values as members of their module rather than the enum class. For example, 're.ASCII' for the ASCII member of re.RegexFlag rather than 'RegexFlag.ASCII' .
Enhanced Flag to support len() , iteration and in / not in on its members. For example, the following now works: len(AFlag(3)) == 2 and list(AFlag(3)) == (AFlag.ONE, AFlag.TWO)
Changed Enum and Flag so that members are now defined before init_subclass() is called; dir() now includes methods, etc., from mixed-in data types.
Changed Flag to only consider primary values (power of two) canonical while composite values ( 3 , 6 , 10 , etc.) are considered aliases; inverted flags are coerced to their positive equivalent.

fcntl
On FreeBSD, the F_DUP2FD and F_DUP2FD_CLOEXEC flags respectively are supported, the former equals to dup2 usage while the latter set the FD_CLOEXEC flag in addition.

fractions
Support PEP 515 -style initialization of Fraction from string. (Contributed by Sergey B Kirpichev in bpo-44258.)
Fraction now implements an int method, so that an isinstance(some_fraction, typing.SupportsInt) check passes. (Contributed by Mark Dickinson in bpo-44547.)

functools
functools.singledispatch() now supports types.UnionType and typing.Union as annotations to the dispatch argument.

gzip
The gzip.compress() function is now faster when used with the mtime=0 argument as it delegates the compression entirely to a single zlib.compress() operation. There is one side effect of this change: The gzip file header contains an “OS” byte in its header. That was traditionally always set to a value of 255 representing “unknown” by the gzip module. Now, when using compress() with mtime=0 , it may be set to a different value by the underlying zlib C library Python was linked against. (See gh-112346 for details on the side effect.)

hashlib
hashlib.blake2b() and hashlib.blake2s() now prefer libb2 over Python’s vendored copy. (Contributed by Christian Heimes in bpo-47095.)
The internal _sha3 module with SHA3 and SHAKE algorithms now uses tiny_sha3 instead of the Keccak Code Package to reduce code and binary size. The hashlib module prefers optimized SHA3 and SHAKE implementations from OpenSSL. The change affects only installations without OpenSSL support. (Contributed by Christian Heimes in bpo-47098.)
Add hashlib.file_digest() , a helper function for efficient hashing of files or file-like objects. (Contributed by Christian Heimes in gh-89313.)

IDLE and idlelib
Apply syntax highlighting to .pyi files. (Contributed by Alex Waygood and Terry Jan Reedy in bpo-45447.)
Include prompts when saving Shell with inputs and outputs. (Contributed by Terry Jan Reedy in gh-95191.)

inspect
Add getmembers_static() to return all members without triggering dynamic lookup via the descriptor protocol. (Contributed by Weipeng Hong in bpo-30533.)
Add ismethodwrapper() for checking if the type of an object is a MethodWrapperType . (Contributed by Hakan Çelik in bpo-29418.)
Change the frame-related functions in the inspect module to return new FrameInfo and Traceback class instances (backwards compatible with the previous named tuple-like interfaces) that includes the extended PEP 657 position information (end line number, column and end column). The affected functions are:
inspect.getframeinfo()
inspect.getouterframes()
inspect.getinnerframes()
inspect.stack()
inspect.trace()
(Contributed by Pablo Galindo in gh-88116.)

locale
Add locale.getencoding() to get the current locale encoding. It is similar to locale.getpreferredencoding(False) but ignores the Python UTF-8 Mode .

logging
Added getLevelNamesMapping() to return a mapping from logging level names (e.g. 'CRITICAL' ) to the values of their corresponding Logging Levels (e.g. 50 , by default). (Contributed by Andrei Kulakovin in gh-88024.)
Added a createSocket() method to SysLogHandler , to match SocketHandler.createSocket() . It is called automatically during handler initialization and when emitting an event, if there is no active socket. (Contributed by Kirill Pinchuk in gh-88457.)

math
Add math.exp2() : return 2 raised to the power of x. (Contributed by Gideon Mitchell in bpo-45917.)
Add math.cbrt() : return the cube root of x. (Contributed by Ajith Ramachandran in bpo-44357.)
The behaviour of two math.pow() corner cases was changed, for consistency with the IEEE 754 specification. The operations math.pow(0.0, -math.inf) and math.pow(-0.0, -math.inf) now return inf . Previously they raised ValueError . (Contributed by Mark Dickinson in bpo-44339.)
The math.nan value is now always available. (Contributed by Victor Stinner in bpo-46917.)

operator
A new function operator.call has been added, such that operator.call(obj, *args, **kwargs) == obj(*args, **kwargs) . (Contributed by Antony Lee in bpo-44019.)

os
On Windows, os.urandom() now uses BCryptGenRandom() , instead of CryptGenRandom() which is deprecated. (Contributed by Donghee Na in bpo-44611.)

pathlib
glob() and rglob() return only directories if pattern ends with a pathname components separator: sep or altsep . (Contributed by Eisuke Kawasima in bpo-22276 and bpo-33392.)

re
Atomic grouping ((?>...) ) and possessive quantifiers ( *+ , ++ , ?+ , {m,n}+ ) are now supported in regular expressions. (Contributed by Jeffrey C. Jacobs and Serhiy Storchaka in bpo-433030.)

shutil
Add optional parameter dir_fd in shutil.rmtree() . (Contributed by Serhiy Storchaka in bpo-46245.)

socket
Add CAN Socket support for NetBSD. (Contributed by Thomas Klausner in bpo-30512.)
create_connection() has an option to raise, in case of failure to connect, an ExceptionGroup containing all errors instead of only raising the last error. (Contributed by Irit Katriel in bpo-29980.)

sqlite3
You can now disable the authorizer by passing None to set_authorizer() . (Contributed by Erlend E. Aasland in bpo-44491.)
Collation name create_collation() can now contain any Unicode character. Collation names with invalid characters now raise UnicodeEncodeError instead of sqlite3.ProgrammingError . (Contributed by Erlend E. Aasland in bpo-44688.)
sqlite3 exceptions now include the SQLite extended error code as sqlite_errorcode and the SQLite error name as sqlite_errorname . (Contributed by Aviv Palivoda, Daniel Shahaf, and Erlend E. Aasland in bpo-16379 and bpo-24139.)
Add setlimit() and getlimit() to sqlite3.Connection for setting and getting SQLite limits by connection basis. (Contributed by Erlend E. Aasland in bpo-45243.)
sqlite3 now sets sqlite3.threadsafety based on the default threading mode the underlying SQLite library has been compiled with. (Contributed by Erlend E. Aasland in bpo-45613.)
sqlite3 C callbacks now use unraisable exceptions if callback tracebacks are enabled. Users can now register an unraisable hook handler to improve their debug experience. (Contributed by Erlend E. Aasland in bpo-45828.)
Fetch across rollback no longer raises InterfaceError . Instead we leave it to the SQLite library to handle these cases. (Contributed by Erlend E. Aasland in bpo-44092.)
Add serialize() and deserialize() to sqlite3.Connection for serializing and deserializing databases. (Contributed by Erlend E. Aasland in bpo-41930.)
Add create_window_function() to sqlite3.Connection for creating aggregate window functions. (Contributed by Erlend E. Aasland in bpo-34916.)
Add blobopen() to sqlite3.Connection . sqlite3.Blob allows incremental I/O operations on blobs. (Contributed by Aviv Palivoda and Erlend E. Aasland in bpo-24905.)

string
Add get_identifiers() and is_valid() to string.Template , which respectively return all valid placeholders, and whether any invalid placeholders are present. (Contributed by Ben Kehoe in gh-90465.)

sys
sys.exc_info() now derives the type and traceback fields from the value (the exception instance), so when an exception is modified while it is being handled, the changes are reflected in the results of subsequent calls to exc_info() . (Contributed by Irit Katriel in bpo-45711.)
Add sys.exception() which returns the active exception instance (equivalent to sys.exc_info()[1] ). (Contributed by Irit Katriel in bpo-46328.)
Add the sys.flags.safe_path flag. (Contributed by Victor Stinner in gh-57684.)

sysconfig
Three new installation schemes (posix_venv , nt_venv and venv ) were added and are used when Python creates new virtual environments or when it is running from a virtual environment. The first two schemes ( posix_venv and nt_venv ) are OS-specific for non-Windows and Windows, the venv is essentially an alias to one of them according to the OS Python runs on. This is useful for downstream distributors who modify sysconfig.get_preferred_scheme() . Third party code that creates new virtual environments should use the new venv installation scheme to determine the paths, as does venv . (Contributed by Miro Hrončok in bpo-45413.)

tempfile
SpooledTemporaryFile objects now fully implement the methods of io.BufferedIOBase or io.TextIOBase (depending on file mode). This lets them work correctly with APIs that expect file-like objects, such as compression modules. (Contributed by Carey Metcalfe in gh-70363.)

threading
On Unix, if the sem_clockwait() function is available in the C library (glibc 2.30 and newer), the threading.Lock.acquire() method now uses the monotonic clock ( time.CLOCK_MONOTONIC ) for the timeout, rather than using the system clock ( time.CLOCK_REALTIME ), to not be affected by system clock changes. (Contributed by Victor Stinner in bpo-41710.)

time
On Unix, time.sleep() now uses the clock_nanosleep() or nanosleep() function, if available, which has a resolution of 1 nanosecond (10^-9 seconds), rather than using select() which has a resolution of 1 microsecond (10^-6 seconds). (Contributed by Benjamin Szőke and Victor Stinner in bpo-21302.)
On Windows 8.1 and newer, time.sleep() now uses a waitable timer based on high-resolution timers which has a resolution of 100 nanoseconds (10^-7 seconds). Previously, it had a resolution of 1 millisecond (10^-3 seconds). (Contributed by Benjamin Szőke, Donghee Na, Eryk Sun and Victor Stinner in bpo-21302 and bpo-45429.)

tkinter
Added method info_patchlevel() which returns the exact version of the Tcl library as a named tuple similar to sys.version_info . (Contributed by Serhiy Storchaka in gh-91827.)

traceback
Add traceback.StackSummary.format_frame_summary() to allow users to override which frames appear in the traceback, and how they are formatted. (Contributed by Ammar Askar in bpo-44569.)
Add traceback.TracebackException.print() , which prints the formatted TracebackException instance to a file. (Contributed by Irit Katriel in bpo-33809.)

typing
For major changes, see New Features Related to Type Hints.
Add typing.assert_never() and typing.Never . typing.assert_never() is useful for asking a type checker to confirm that a line of code is not reachable. At runtime, it raises an AssertionError . (Contributed by Jelle Zijlstra in gh-90633.)
Add typing.reveal_type() . This is useful for asking a type checker what type it has inferred for a given expression. At runtime it prints the type of the received value. (Contributed by Jelle Zijlstra in gh-90572.)
Add typing.assert_type() . This is useful for asking a type checker to confirm that the type it has inferred for a given expression matches the given type. At runtime it simply returns the received value. (Contributed by Jelle Zijlstra in gh-90638.)
typing.TypedDict types can now be generic. (Contributed by Samodya Abeysiriwardane in gh-89026.)
NamedTuple types can now be generic. (Contributed by Serhiy Storchaka in bpo-43923.)
Allow subclassing of typing.Any . This is useful for avoiding type checker errors related to highly dynamic class, such as mocks. (Contributed by Shantanu Jain in gh-91154.)
The typing.final() decorator now sets the final attributed on the decorated object. (Contributed by Jelle Zijlstra in gh-90500.)
The typing.get_overloads() function can be used for introspecting the overloads of a function. typing.clear_overloads() can be used to clear all registered overloads of a function. (Contributed by Jelle Zijlstra in gh-89263.)
The init() method of Protocol subclasses is now preserved. (Contributed by Adrian Garcia Badarasco in gh-88970.)
The representation of empty tuple types ( Tuple[()] ) is simplified. This affects introspection, e.g. get_args(Tuple[()]) now evaluates to () instead of ((),) . (Contributed by Serhiy Storchaka in gh-91137.)
Loosen runtime requirements for type annotations by removing the callable check in the private typing._type_check function. (Contributed by Gregory Beauregard in gh-90802.)
typing.get_type_hints() now supports evaluating strings as forward references in PEP 585 generic aliases . (Contributed by Niklas Rosenstein in gh-85542.)
typing.get_type_hints() no longer adds Optional to parameters with None as a default. (Contributed by Nikita Sobolev in gh-90353.)
typing.get_type_hints() now supports evaluating bare stringified ClassVar annotations. (Contributed by Gregory Beauregard in gh-90711.)
typing.no_type_check() no longer modifies external classes and functions. It also now correctly marks classmethods as not to be type checked. (Contributed by Nikita Sobolev in gh-90729.)

unicodedata
The Unicode database has been updated to version 14.0.0. (Contributed by Benjamin Peterson in bpo-45190).

unittest
Added methods enterContext() and enterClassContext() of class TestCase , method enterAsyncContext() of class IsolatedAsyncioTestCase and function unittest.enterModuleContext() . (Contributed by Serhiy Storchaka in bpo-45046.)

venv
When new Python virtual environments are created, the venv installation scheme is used to determine the paths inside the environment. When Python runs in a virtual environment, the same installation scheme is the default. That means that downstream distributors can change the default sysconfig install scheme without changing behavior of virtual environments. Third party code that also creates new virtual environments should do the same. (Contributed by Miro Hrončok in bpo-45413.)

warnings
warnings.catch_warnings() now accepts arguments for warnings.simplefilter() , providing a more concise way to locally ignore warnings or convert them to errors. (Contributed by Zac Hatfield-Dodds in bpo-47074.)

zipfile
Added support for specifying member name encoding for reading metadata in a ZipFile’s directory and file headers. (Contributed by Stephen J. Turnbull and Serhiy Storchaka in bpo-28080.)
Added ZipFile.mkdir() for creating new directories inside ZIP archives. (Contributed by Sam Ezeh in gh-49083.)
Added stem , suffix and suffixes to zipfile.Path . (Contributed by Miguel Brito in gh-88261.)

Optimizations
This section covers specific optimizations independent of the Faster CPython project, which is covered in its own section.
The compiler now optimizes simple printf-style % formatting on string literals containing only the format codes %s , %r and %a and makes it as fast as a corresponding f-string expression. (Contributed by Serhiy Storchaka in bpo-28307.)
Integer division ( // ) is better tuned for optimization by compilers. It is now around 20% faster on x86-64 when dividing an int by a value smaller than 230 . (Contributed by Gregory P. Smith and Tim Peters in gh-90564.)
sum() is now nearly 30% faster for integers smaller than 230 . (Contributed by Stefan Behnel in gh-68264.)
Resizing lists is streamlined for the common case, speeding up list.append() by ≈15% and simple list comprehension s by up to 20-30% (Contributed by Dennis Sweeney in gh-91165.)
Dictionaries don’t store hash values when all keys are Unicode objects, decreasing dict size. For example, sys.getsizeof(dict.fromkeys("abcdefg")) is reduced from 352 bytes to 272 bytes (23% smaller) on 64-bit platforms. (Contributed by Inada Naoki in bpo-46845.)
Using asyncio.DatagramProtocol is now orders of magnitude faster when transferring large files over UDP, with speeds over 100 times higher for a ≈60 MiB file. (Contributed by msoxzw in gh-91487.)
math functions comb() and perm() are now ≈10 times faster for large arguments (with a larger speedup for larger k ). (Contributed by Serhiy Storchaka in bpo-37295.)
The statistics functions mean() , variance() and stdev() now consume iterators in one pass rather than converting them to a list first. This is twice as fast and can save substantial memory. (Contributed by Raymond Hettinger in gh-90415.)
unicodedata.normalize() now normalizes pure-ASCII strings in constant time. (Contributed by Donghee Na in bpo-44987.)

Faster CPython
CPython 3.11 is an average of 25% faster than CPython 3.10 as measured with the pyperformance benchmark suite, when compiled with GCC on Ubuntu Linux. Depending on your workload, the overall speedup could be 10-60%.
This project focuses on two major areas in Python: Faster Startup and Faster Runtime. Optimizations not covered by this project are listed separately under Optimizations .

Faster Startup
Frozen imports / Static code objects
Python caches bytecode in the pycache directory to speed up module loading.
Previously in 3.10, Python module execution looked like this:
Read pycache -> Unmarshal -> Heap allocated code object -> Evaluate
In Python 3.11, the core modules essential for Python startup are “frozen”. This means that their Code Objects (and bytecode) are statically allocated by the interpreter. This reduces the steps in module execution process to:
Statically allocated code object -> Evaluate
Interpreter startup is now 10-15% faster in Python 3.11. This has a big impact for short-running programs using Python.
(Contributed by Eric Snow, Guido van Rossum and Kumar Aditya in many issues.)

Faster Runtime
Cheaper, lazy Python frames
Python frames, holding execution information, are created whenever Python calls a Python function. The following are new frame optimizations:
Streamlined the frame creation process.
Avoided memory allocation by generously re-using frame space on the C stack.
Streamlined the internal frame struct to contain only essential information. Frames previously held extra debugging and memory management information.
Old-style frame objects are now created only when requested by debuggers or by Python introspection functions such as sys._getframe() and inspect.currentframe() . For most user code, no frame objects are created at all. As a result, nearly all Python functions calls have sped up significantly. We measured a 3-7% speedup in pyperformance.
(Contributed by Mark Shannon in bpo-44590.)

Inlined Python function calls
During a Python function call, Python will call an evaluating C function to interpret that function’s code. This effectively limits pure Python recursion to what’s safe for the C stack.
In 3.11, when CPython detects Python code calling another Python function, it sets up a new frame, and “jumps” to the new code inside the new frame. This avoids calling the C interpreting function altogether.
Most Python function calls now consume no C stack space, speeding them up. In simple recursive functions like fibonacci or factorial, we observed a 1.7x speedup. This also means recursive functions can recurse significantly deeper (if the user increases the recursion limit with sys.setrecursionlimit() ). We measured a 1-3% improvement in pyperformance.
(Contributed by Pablo Galindo and Mark Shannon in bpo-45256.)

PEP 659: Specializing Adaptive Interpreter
PEP 659 is one of the key parts of the Faster CPython project. The general idea is that while Python is a dynamic language, most code has regions where objects and types rarely change. This concept is known as type stability .
At runtime, Python will try to look for common patterns and type stability in the executing code. Python will then replace the current operation with a more specialized one. This specialized operation uses fast paths available only to those use cases/types, which generally outperform their generic counterparts. This also brings in another concept called inline caching , where Python caches the results of expensive operations directly in the bytecode .
The specializer will also combine certain common instruction pairs into one superinstruction, reducing the overhead during execution.
Python will only specialize when it sees code that is “hot” (executed multiple times). This prevents Python from wasting time on run-once code. Python can also de-specialize when code is too dynamic or when the use changes. Specialization is attempted periodically, and specialization attempts are not too expensive, allowing specialization to adapt to new circumstances.
(PEP written by Mark Shannon, with ideas inspired by Stefan Brunthaler. See PEP 659 for more information. Implementation by Mark Shannon and Brandt Bucher, with additional help from Irit Katriel and Dennis Sweeney.)

Misc
Objects now require less memory due to lazily created object namespaces. Their namespace dictionaries now also share keys more freely. (Contributed Mark Shannon in bpo-45340 and bpo-40116.)
“Zero-cost” exceptions are implemented, eliminating the cost of try statements when no exception is raised. (Contributed by Mark Shannon in bpo-40222.)
A more concise representation of exceptions in the interpreter reduced the time required for catching an exception by about 10%. (Contributed by Irit Katriel in bpo-45711.)
re ’s regular expression matching engine has been partially refactored, and now uses computed gotos (or “threaded code”) on supported platforms. As a result, Python 3.11 executes the pyperformance regular expression benchmarks up to 10% faster than Python 3.10. (Contributed by Brandt Bucher in gh-91404.)

FAQ
How should I write my code to utilize these speedups?
Write Pythonic code that follows common best practices; you don’t have to change your code. The Faster CPython project optimizes for common code patterns we observe.
Will CPython 3.11 use more memory?
Maybe not; we don’t expect memory use to exceed 20% higher than 3.10. This is offset by memory optimizations for frame objects and object dictionaries as mentioned above.
I don’t see any speedups in my workload. Why?
Certain code won’t have noticeable benefits. If your code spends most of its time on I/O operations, or already does most of its computation in a C extension library like NumPy, there won’t be significant speedups. This project currently benefits pure-Python workloads the most.
Furthermore, the pyperformance figures are a geometric mean. Even within the pyperformance benchmarks, certain benchmarks have slowed down slightly, while others have sped up by nearly 2x!
Is there a JIT compiler?
No. We’re still exploring other optimizations.

About
Faster CPython explores optimizations for CPython . The main team is funded by Microsoft to work on this full-time. Pablo Galindo Salgado is also funded by Bloomberg LP to work on the project part-time. Finally, many contributors are volunteers from the community.

CPython bytecode changes
The bytecode now contains inline cache entries, which take the form of the newly-added CACHE instructions. Many opcodes expect to be followed by an exact number of caches, and instruct the interpreter to skip over them at runtime. Populated caches can look like arbitrary instructions, so great care should be taken when reading or modifying raw, adaptive bytecode containing quickened data.

New opcodes
ASYNC_GEN_WRAP , RETURN_GENERATOR and SEND , used in generators and co-routines.
COPY_FREE_VARS , which avoids needing special caller-side code for closures.
JUMP_BACKWARD_NO_INTERRUPT , for use in certain loops where handling interrupts is undesirable.
MAKE_CELL , to create Cell Objects .
CHECK_EG_MATCH and PREP_RERAISE_STAR , to handle the new exception groups and except* added in PEP 654 .
PUSH_EXC_INFO , for use in exception handlers.
RESUME , a no-op, for internal tracing, debugging and optimization checks.

Replaced opcodes
BINARY_* INPLACE_* -> BINARY_OP
CALL_FUNCTION CALL_FUNCTION_KW CALL_METHOD -> CALL KW_NAMES PRECALL PUSH_NULL
DUP_TOP DUP_TOP_TWO ROT_TWO ROT_THREE ROT_FOUR ROT_N -> COPY SWAP
JUMP_IF_NOT_EXC_MATCH -> CHECK_EXC_MATCH
JUMP_ABSOLUTE POP_JUMP_IF_FALSE POP_JUMP_IF_TRUE -> JUMP_BACKWARD POP_JUMP_BACKWARD_IF_* POP_JUMP_FORWARD_IF_*
SETUP_WITH SETUP_ASYNC_WITH -> BEFORE_WITH

Changed/removed opcodes
Changed MATCH_CLASS and MATCH_KEYS to no longer push an additional boolean value to indicate success/failure. Instead, None is pushed on failure in place of the tuple of extracted values.
Changed opcodes that work with exceptions to reflect them now being represented as one item on the stack instead of three (see gh-89874).
Removed COPY_DICT_WITHOUT_KEYS , GEN_START , POP_BLOCK , SETUP_FINALLY and YIELD_FROM .

Deprecated
This section lists Python APIs that have been deprecated in Python 3.11.
Deprecated C APIs are listed separately.

Language/Builtins
Chaining classmethod descriptors (introduced in bpo-19072) is now deprecated. It can no longer be used to wrap other descriptors such as property . The core design of this feature was flawed and caused a number of downstream problems. To “pass-through” a classmethod , consider using the wrapped attribute that was added in Python 3.10. (Contributed by Raymond Hettinger in gh-89519.)
Octal escapes in string and bytes literals with values larger than 0o377 (255 in decimal) now produce a DeprecationWarning . In a future Python version, they will raise a SyntaxWarning and eventually a SyntaxError . (Contributed by Serhiy Storchaka in gh-81548.)
The delegation of int() to trunc() is now deprecated. Calling int(a) when type(a) implements trunc() but not int() or index() now raises a DeprecationWarning . (Contributed by Zackery Spytz in bpo-44977.)

Modules
PEP 594 led to the deprecations of the following modules slated for removal in Python 3.13:
aifc, chunk, msilib, pipes, telnetlib
audioop, crypt, nis, sndhdr, uu
cgi, imghdr, nntplib, spwd, xdrlib
cgitb, mailcap, ossaudiodev, sunau
asynchat, asyncore and smtpd modules have been deprecated since at least Python 3.6. Their documentation and deprecation warnings have now been updated to note they will be removed in Python 3.12.
lib2to3 package and 2to3 tool are now deprecated and may not be able to parse Python 3.10 or newer.
Undocumented modules sre_compile , sre_constants and sre_parse are now deprecated.

Standard Library
The following have been deprecated in configparser since Python 3.2. Their deprecation warnings have now been updated to note they will be removed in Python 3.12:
the configparser.SafeConfigParser class
the configparser.ParsingError.filename property
the configparser.RawConfigParser.readfp() method
configparser.LegacyInterpolation has been deprecated in the docstring since Python 3.2 ...
[content truncated]

Original source Report a problem

What’s New In Python 3.10

Editor:
Pablo Galindo Salgado

This article explains the new features in Python 3.10, compared to 3.9. Python 3.10 was released on October 4, 2021. For full details, see the changelog.

Summary – Release highlights

New syntax features:

• PEP 634, Structural Pattern Matching: Specification
• PEP 635, Structural Pattern Matching: Motivation and Rationale
• PEP 636, Structural Pattern Matching: Tutorial
• bpo-12782, Parenthesized context managers are now officially allowed.

New features in the standard library:

• PEP 618, Add Optional Length-Checking To zip.

Interpreter improvements:

• PEP 626, Precise line numbers for debugging and other tools.

New typing features:

• PEP 604, Allow writing union types as X | Y
• PEP 612, Parameter Specification Variables
• PEP 613, Explicit Type Aliases
• PEP 647, User-Defined Type Guards

Important deprecations, removals or restrictions:

• PEP 644, Require OpenSSL 1.1.1 or newer
• PEP 632, Deprecate distutils module.
• PEP 623, Deprecate and prepare for the removal of the wstr member in PyUnicodeObject.
• PEP 624, Remove Py_UNICODE encoder APIs
• PEP 597, Add optional EncodingWarning

New Features

Parenthesized context managers

Using enclosing parentheses for continuation across multiple lines in context managers is now supported. This allows formatting a long collection of context managers in multiple lines in a similar way as it was previously possible with import statements. For instance, all these examples are now valid:
with (
CtxManager()
as example):
...
with (
CtxManager1(),
CtxManager2()
):
...
with (
CtxManager1()
as example,
CtxManager2()):
...
with (
CtxManager1(),
CtxManager2()
as example):
...
with (
CtxManager1()
as example1,
CtxManager2()
as example2
):
...

it is also possible to use a trailing comma at the end of the enclosed group:
with (
CtxManager1()
as example1,
CtxManager2()
as example2,
CtxManager3()
as example3,
):
...

This new syntax uses the non LL(1) capacities of the new parser. Check PEP 617 for more details.

(Contributed by Guido van Rossum, Pablo Galindo and Lysandros Nikolaou in bpo-12782 and bpo-40334.)

Better error messages SyntaxErrors

When parsing code that contains unclosed parentheses or brackets the interpreter now includes the location of the unclosed bracket of parentheses instead of displaying SyntaxError: unexpected EOF while parsing or pointing to some incorrect location. For instance, consider the following code (notice the unclosed ‘{‘):
expected = {
9:1,
18:2,
19:2,
27:3,
28:3,
29:3,
36:4,
37:4,
38:4,
39:4,
45:5,
46:5,
47:5,
48:5,
49:5,
54:6
some_other_code = foo()

Previous versions of the interpreter reported confusing places as the location of the syntax error:
File "example.py", line 3
some_other_code = foo()
^
SyntaxError: invalid syntax

but in Python 3.10 a more informative error is emitted:
File "example.py", line 1
expected = {
9:1,
18:2,
19:2,
27:3,
28:3,
29:3,
36:4,
37:4,
38:4,
^
SyntaxError: '{' was never closed

In a similar way, errors involving unclosed string literals (single and triple quoted) now point to the start of the string instead of reporting EOF/EOL.

These improvements are inspired by previous work in the PyPy interpreter.

(Contributed by Pablo Galindo in bpo-42864 and Batuhan Taskaya in bpo-40176.)

SyntaxError exceptions raised by the interpreter will now highlight the full error range of the expression that constitutes the syntax error itself, instead of just where the problem is detected. In this way, instead of displaying (before Python 3.10):

foo(x, z for z in range(10), t, w)
File "", line 1

foo(x, z for z in range(10), t, w)
^
SyntaxError: Generator expression must be parenthesized

now Python 3.10 will display the exception as:

foo(x, z for z in range(10), t, w)
File "", line 1

foo(x, z for z in range(10), t, w)
^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized

This improvement was contributed by Pablo Galindo in bpo-43914.

A considerable amount of new specialized messages for SyntaxError exceptions have been incorporated. Some of the most notable ones are as follows:

• Missing : before blocks: if rocket.position > event_horizon
  File "", line 1
  if rocket.position > event_horizon
  ^
  SyntaxError: expected ':'
  (Contributed by Pablo Galindo in bpo-42997.)
• Unparenthesised tuples in comprehensions targets:

  {x, y for x, y in zip('abcd', '1234')}
  File "", line 1
  {x, y for x, y in zip('abcd', '1234')}
  ^
  SyntaxError: did you forget parentheses around the comprehension target?
  (Contributed by Pablo Galindo in bpo-43017.)

• Missing commas in collection literals and between expressions: items = {
  ...
  x:1,
  ...
  y:2,
  ...
  z:3,
  }
  File "", line 3
  y:2
  ^
  SyntaxError: invalid syntax. Perhaps you forgot a comma?
  (Contributed by Pablo Galindo in bpo-43822.)
• Multiple Exception types without parentheses: try:
  ...
  build_dyson_sphere()
  ...
  except NotEnoughScienceError, NotEnoughResourcesError:
  File "", line 3
  except NotEnoughScienceError, NotEnoughResourcesError:
  ^
  SyntaxError: multiple exception types must be parenthesized
  (Contributed by Pablo Galindo in bpo-43149.)
• Missing : and values in dictionary literals: values = {
  x:1,
  y:2,
  z w:3
  }
  File "", line 1
  values = { x:1, y:2, z w:3 }
  ^
  SyntaxError: ':' expected after dictionary key values = { x:1, y:2, z w:3 }
  File "", line 1
  values = { x:1, y:2, z w:3 }
  ^
  SyntaxError: ':' expected after dictionary key
  (Contributed by Pablo Galindo in bpo-43823.)
• try blocks without except or finally blocks: try:
  ...
  x = 2
  ...
  something = 3
  File "", line 3
  something = 3
  ^^^^^^^^^
  SyntaxError: expected 'except' or 'finally' block
  (Contributed by Pablo Galindo in bpo-44305.)
• Usage of = instead of == in comparisons: if rocket.position = event_horizon:
  File "", line 1
  if rocket.position = event_horizon:
  ^
  SyntaxError: cannot assign to attribute here. Maybe you meant '==' instead of '='?
  (Contributed by Pablo Galindo in bpo-43797.)
• Usage of * in f-strings: f"Black holes {*all_black_holes} and revelations"
  File "", line 1
  (*all_black_holes)
  ^
  SyntaxError: f-string: cannot use starred expression here
  (Contributed by Pablo Galindo in bpo-41064.)

IndentationErrors
Many IndentationError exceptions now have more context regarding what kind of block was expecting an indentation, including the location of the statement:

def foo():
...
if lel:
...
x = 2
File "", line 3
x = 2
^
IndentationError: expected an indented block after 'if' statement in line 2

AttributeErrors
When printing AttributeError, PyErr_Display() will offer suggestions of similar attribute names in the object that the exception was raised from:

collections.namedtoplo
Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'collections' has no attribute 'namedtoplo'. Did you mean: namedtuple?

(Contributed by Pablo Galindo in bpo-38530.)

Warning
Notice this won’t work if PyErr_Display() is not called to display the error which can happen if some other custom error display function is used. This is a common scenario in some REPLs like IPython.

NameErrors
When printing NameError raised by the interpreter, PyErr_Display() will offer suggestions of similar variable names in the function that the exception was raised from:

schwarzschild_black_hole = None

schwarschild_black_hole
Traceback (most recent call last):
File "", line 1, in
NameError: name 'schwarschild_black_hole' is not defined. Did you mean: schwarzschild_black_hole?

(Contributed by Pablo Galindo in bpo-38530.)

Warning
Notice this won’t work if PyErr_Display() is not called to display the error, which can happen if some other custom error display function is used. This is a common scenario in some REPLs like IPython.

PEP 626: Precise line numbers for debugging and other tools
PEP 626 brings more precise and reliable line numbers for debugging, profiling and coverage tools. Tracing events, with the correct line number, are generated for all lines of code executed and only for lines of code that are executed.

The f_lineno attribute of frame objects will always contain the expected line number.

The co_lnotab attribute of code objects is deprecated and will be removed in 3.12. Code that needs to convert from offset to line number should use the new co_lines() method instead.

PEP 634: Structural Pattern Matching
Structural pattern matching has been added in the form of a match statement and case statements of patterns with associated actions. Patterns consist of sequences, mappings, primitive data types as well as class instances. Pattern matching enables programs to extract information from complex data types, branch on the structure of data, and apply specific actions based on different forms of data.

Syntax and operations

The generic syntax of pattern matching is:
match subject:
case :
case :
case :
case _:

A match statement takes an expression and compares its value to successive patterns given as one or more case blocks. Specifically, pattern matching operates by:

1. using data with type and shape (the subject)
2. evaluating the subject in the match statement
3. comparing the subject with each pattern in a case statement from top to bottom until a match is confirmed.
4. executing the action associated with the pattern of the confirmed match
5. If an exact match is not confirmed, the last case, a wildcard _, if provided, will be used as the matching case. If an exact match is not confirmed and a wildcard case does not exist, the entire match block is a no-op.

Declarative approach

Readers may be aware of pattern matching through the simple example of matching a subject (data object) to a literal (pattern) with the switch statement found in C, Java or JavaScript (and many other languages). Often the switch statement is used for comparison of an object/expression with case statements containing literals.

More powerful examples of pattern matching can be found in languages such as Scala and Elixir. With structural pattern matching, the approach is “declarative” and explicitly states the conditions (the patterns) for data to match.

While an “imperative” series of instructions using nested “if” statements could be used to accomplish something similar to structural pattern matching, it is less clear than the “declarative” approach. Instead the “declarative” approach states the conditions to meet for a match and is more readable through its explicit patterns. While structural pattern matching can be used in its simplest form comparing a variable to a literal in a case statement, its true value for Python lies in its handling of the subject’s type and shape.

Simple pattern: match to a literal

Let’s look at this example as pattern matching in its simplest form: a value, the subject, being matched to several literals, the patterns. In the example below, status is the subject of the match statement. The patterns are each of the case statements, where literals represent request status codes. The associated action to the case is executed after a match:

def http_error(status):
  match status:
   case 400:
    return "Bad request"
   case 404:
    return "Not found"
   case 418:
    return "I'm a teapot"
   case _:
    return "Something's wrong with the internet"

If the above function is passed a status of 418, “I’m a teapot” is returned. If the above function is passed a status of 500, the case statement with _ will match as a wildcard, and “Something’s wrong with the internet” is returned. Note the last block: the variable name, _, acts as a wildcard and insures the subject will always match. The use of _ is optional.

You can combine several literals in a single pattern using | (“or”):

case 401 | 403 | 404:
  return "Not allowed"

Behavior without the wildcard

If we modify the above example by removing the last case block, the example becomes:

def http_error(status):
  match status:
   case 400:
    return "Bad request"
   case 404:
    return "Not found"
   case 418:
    return "I'm a teapot"

Without the use of _ in a case statement, a match may not exist. If no match exists, the behavior is a no-op. For example, if status of 500 is passed, a no-op occurs.

Patterns with a literal and variable

Patterns can look like unpacking assignments, and a pattern may be used to bind variables. In this example, a data point can be unpacked to its x-coordinate and y-coordinate:

# point is an (x, y) tuple
match point:
  case (0, 0):
   print("Origin")
  case (0, y):
   print(f"Y={y}")
  case (x, 0):
   print(f"X={x}")
  case (x, y):
   print(f"X={x}, Y={y}")
  case _:
   raise ValueError("Not a point")

The first pattern has two literals, (0, 0), and may be thought of as an extension of the literal pattern shown above. The next two patterns combine a literal and a variable, and the variable binds a value from the subject (point). The fourth pattern captures two values, which makes it conceptually similar to the unpacking assignment (x, y) = point.

Patterns and classes

If you are using classes to structure your data, you can use as a pattern the class name followed by an argument list resembling a constructor. This pattern has the ability to capture instance attributes into variables:

class Point:
  def __init__(self, x, y):
   self.x = x
   self.y = y
def location(point):
  match point:
   case Point(x = 0, y = 0):
    print("Origin is the point's location.")
   case Point(x = 0, y = y):
    print(f"Y={y} and the point is on the y-axis.")
   case Point(x = x, y = 0):
    print(f"X={x} and the point is on the x-axis.")
   case Point():
    print("The point is located somewhere else on the plane.")
   case _:
    print("Not a point")

Patterns with positional parameters

You can use positional parameters with some builtin classes that provide an ordering for their attributes (e.g. dataclasses). You can also define a specific position for attributes in patterns by setting the match_args special attribute in your classes. If it’s set to ("x", "y"), the following patterns are all equivalent (and all bind the y attribute to the var variable):

• Point(1, var)
• Point(1, y = var)
• Point(x = 1, y = var)
• Point(y = var, x = 1)

Nested patterns

Patterns can be arbitrarily nested. For example, if our data is a short list of points, it could be matched like this:

match points:
  case []:
   print("No points in the list.")
  case [Point(0, 0)]:
   print("The origin is the only point in the list.")
  case [Point(x, y)]:
   print(f"A single point {x}, {y} is in the list.")
  case [Point(0, y1), Point(0, y2)]:
   print(f"Two points on the Y axis at {y1}, {y2} are in the list.")
  case _:
   print("Something else is found in the list.")

Complex patterns and the wildcard

To this point, the examples have used _ alone in the last case statement. A wildcard can be used in more complex patterns, such as ('error', code, _) . For example:

match test_variable:
  case ('warning', code, 40):
   print("A warning has been received.")
  case ('error', code, _):
   print(f"An error {code} occurred.")

In the above case, test_variable will match for (‘error’, code, 100) and (‘error’, code, 800).

Guard

We can add an if clause to a pattern, known as a “guard”. If the guard is false, match goes on to try the next case block. Note that value capture happens before the guard is evaluated:

match point:
  case Point(x, y) if x == y:
   print(f"The point is located on the diagonal Y=X at {x}.")
  case Point(x, y):
   print(f"Point is not on the diagonal.")

Other Key Features

Several other key features:

• Like unpacking assignments, tuple and list patterns have exactly the same meaning and actually match arbitrary sequences. Technically, the subject must be a sequence. Therefore, an important exception is that patterns don’t match iterators. Also, to prevent a common mistake, sequence patterns don’t match strings.
• Sequence patterns support wildcards: [x, y, *rest] and (x, y, *rest) work similar to wildcards in unpacking assignments. The name after * may also be _ , so (x, y, *_) matches a sequence of at least two items without binding the remaining items.
• Mapping patterns: {"bandwidth": b, "latency": l} captures the "bandwidth" and "latency" values from a dict. Unlike sequence patterns, extra keys are ignored. A wildcard **rest is also supported. (But **_ would be redundant, so is not allowed.)
• Subpatterns may be captured using the as keyword:

 ...

This binds x1, y1, x2, y2 like you would expect without the as clause, and p2 to the entire second item of the subject.

• Most literals are compared by equality. However, the singletons True, False, and None are compared by identity.
• Named constants may be used in patterns. These named constants must be dotted names to prevent the constant from being interpreted as a capture variable: (example with Enum Color)

For the full specification see PEP 634. Motivation and rationale are in PEP 635, and a longer tutorial is in PEP 636.

Optional EncodingWarning and encoding="locale" option

Optional EncodingWarning and encoding="locale" option

The default encoding of TextIOWrapper and open() is platform and locale dependent. Since UTF-8 is used on most Unix platforms, omitting encoding option when opening UTF-8 files (e.g. JSON, YAML, TOML, Markdown) is a very common bug. For example:

BUG: "rb" mode or encoding="utf-8" should be used.

with open("data.json") as f:
data = json.load(f)

To find this type of bug, an optional EncodingWarning is added. It is emitted when sys.flags.warn_default_encoding is true and locale-specific default encoding is used.

-X warn_default_encoding option and PYTHONWARNDEFAULTENCODING are added to enable the warning.

See Text Encoding for more information.

New Features Related to Type Hints

This section covers major changes affecting PEP 484 type hints and the typing module.

PEP 604: New Type Union Operator

A new type union operator was introduced which enables the syntax X | Y. This provides a cleaner way of expressing ‘either type X or type Y’ instead of using typing.Union, especially in type hints.

(Examples and details omitted for brevity in this content extraction.)

PEP 612: Parameter Specification Variables

Two new options to improve the information provided to static type checkers for PEP 484 's Callable have been added to the typing module.

PEP 613: TypeAlias

Now the typing module has a special value TypeAlias which lets you declare type aliases more explicitly.

PEP 647: User-Defined Type Guards

TypeGuard has been added to the typing module to annotate type guard functions and improve information provided to static type checkers during type narrowing.

Other Language Changes

(Extensive list of language changes, new/optimized modules, deprecations, removals, porting notes, C API changes, build changes, optimizations, and notable security fixes are included in the full document.)

Notable security feature in 3.10.7
Converting between int and str in bases other than 2, 4, 8, 16, or 32 such as base 10 now raises a ValueError if the number of digits in string form is above a limit to avoid potential denial of service attacks due to the algorithmic complexity. This is a mitigation for CVE 2020-10735. This limit can be configured or disabled by environment variable, command line flag, or sys APIs. The default limit is 4300 digits in string form.

Notable security feature in 3.10.8
The deprecated mailcap module now refuses to inject unsafe text (filenames, MIME types, parameters) into shell commands. Instead of using such text, it will warn and act as if a match was not found (or for test commands, as if the test failed).

Notable changes in 3.10.12
tarfile

• The extraction methods in tarfile, and shutil.unpack_archive(), have a new a filter argument that allows limiting tar features than may be surprising or dangerous, such as creating files outside the destination directory. See Extraction filters for details. In Python 3.12, use without the filter argument will show a DeprecationWarning. In Python 3.14, the default will switch to 'data'.

(End of extracted main content.)

Original source Report a problem

This article explains the new features in Python 3.9, compared to 3.8. Python 3.9 was released on October 5, 2020. For full details, see the changelog.

See also
PEP 596 - Python 3.9 Release Schedule

Summary – Release highlights
New syntax features:
• PEP 584, union operators added to dict;
• PEP 585, type hinting generics in standard collections;
• PEP 614, relaxed grammar restrictions on decorators.
New built-in features:
• PEP 616, string methods to remove prefixes and suffixes.
New features in the standard library:
• PEP 593, flexible function and variable annotations;
• os.pidfd_open() added that allows process management without races and signals.
Interpreter improvements:
• PEP 573, fast access to module state from methods of C extension types;
• PEP 617, CPython now uses a new parser based on PEG;
• a number of Python builtins (range, tuple, set, frozenset, list, dict) are now sped up using PEP 590 vectorcall;
• garbage collection does not block on resurrected objects;
• a number of Python modules (_abc, audioop, _bz2, _codecs, _contextvars, _crypt, _functools, _json, _locale, math, operator, resource, time, _weakref) now use multiphase initialization as defined by PEP 489;
• a number of standard library modules (audioop, ast, grp, _hashlib, pwd, _posixsubprocess, random, select, struct, termios, zlib) are now using the stable ABI defined by PEP 384.
New library modules:
• PEP 615, the IANA Time Zone Database is now present in the standard library in the zoneinfo module;
• an implementation of a topological sort of a graph is now provided in the new graphlib module.
Release process changes:
• PEP 602, CPython adopts an annual release cycle.

You should check for DeprecationWarning in your code
When Python 2.7 was still supported, a lot of functionality in Python 3 was kept for backward compatibility with Python 2.7. With the end of Python 2 support, these backward compatibility layers have been removed, or will be removed soon. Most of them emitted a DeprecationWarning warning for several years. For example, using collections.Mapping instead of collections.abc.Mapping emits a DeprecationWarning since Python 3.3, released in 2012.
Test your application with the -W default command-line option to see DeprecationWarning and PendingDeprecationWarning, or even with -W error to treat them as errors. Warnings Filter can be used to ignore warnings from third-party code.
Python 3.9 is the last version providing those Python 2 backward compatibility layers, to give more time to Python projects maintainers to organize the removal of the Python 2 support and add support for Python 3.9.
Aliases to Abstract Base Classes in the collections module, like collections.Mapping alias to collections.abc.Mapping, are kept for one last release for backward compatibility. They will be removed from Python 3.10.
More generally, try to run your tests in the Python Development Mode which helps to prepare your code to make it compatible with the next Python version.
Note: a number of pre-existing deprecations were removed in this version of Python as well. Consult the Removed section.

New Features
Dictionary Merge & Update Operators
Merge (|) and update (|=) operators have been added to the built-in dict class. Those complement the existing dict.update and {**d1, **d2} methods of merging dictionaries.
Example:

x = {
"key1": "value1 from x",
"key2": "value2 from x"
}

y = {
"key2": "value2 from y",
"key3": "value3 from y"
}

x | y {'key1': 'value1 from x', 'key2': 'value2 from y', 'key3': 'value3 from y'}
y | x {'key2': 'value2 from x', 'key3': 'value3 from y', 'key1': 'value1 from x'}
See PEP 584 for a full description. (Contributed by Brandt Bucher in bpo-36144.)

New String Methods to Remove Prefixes and Suffixes
str.removeprefix(prefix) and str.removesuffix(suffix) have been added to easily remove an unneeded prefix or a suffix from a string. Corresponding bytes, bytearray, and collections.UserString methods have also been added. See PEP 616 for a full description. (Contributed by Dennis Sweeney in bpo-39939.)

Type Hinting Generics in Standard Collections
In type annotations you can now use built-in collection types such as list and dict as generic types instead of importing the corresponding capitalized types (e.g. List or Dict) from typing. Some other types in the standard library are also now generic, for example queue.Queue.
Example:
def greet_all(names: list[str]) -> None:
for name in names:
print("Hello", name)
See PEP 585 for more details. (Contributed by Guido van Rossum, Ethan Smith, and Batuhan Taşkaya in bpo-39481.)

New Parser
Python 3.9 uses a new parser, based on PEG instead of LL(1). The new parser’s performance is roughly comparable to that of the old parser, but the PEG formalism is more flexible than LL(1) when it comes to designing new language features. We’ll start using this flexibility in Python 3.10 and later.
The ast module uses the new parser and produces the same AST as the old parser.
In Python 3.10, the old parser will be deleted and so will all functionality that depends on it (primarily the parser module, which has long been deprecated). In Python 3.9 only, you can switch back to the LL(1) parser using a command line switch (-X oldparser) or an environment variable (PYTHONOLDPARSER=1).
See PEP 617 for more details. (Contributed by Guido van Rossum, Pablo Galindo and Lysandros Nikolaou in bpo-40334.)

Other Language Changes
• import() now raises ImportError instead of ValueError, which used to occur when a relative import went past its top-level package. (Contributed by Ngalim Siregar in bpo-37444.)
• Python now gets the absolute path of the script filename specified on the command line (ex: python3 script.py): the file attribute of the main module became an absolute path, rather than a relative path. These paths now remain valid after the current directory is changed by os.chdir(). As a side effect, the traceback also displays the absolute path for main module frames in this case. (Contributed by Victor Stinner in bpo-20443.)
• In the Python Development Mode and in debug build, the encoding and errors arguments are now checked for string encoding and decoding operations. Examples: open(), str.encode() and bytes.decode().
• By default, for best performance, the errors argument is only checked at the first encoding/decoding error and the encoding argument is sometimes ignored for empty strings. (Contributed by Victor Stinner in bpo-37388.)
• "".replace("", s, n) now returns s instead of an empty string for all non-zero n. It is now consistent with "".replace("", s). There are similar changes for bytes and bytearray objects. (Contributed by Serhiy Storchaka in bpo-28029.)
• Any valid expression can now be used as a decorator. Previously, the grammar was much more restrictive. See PEP 614 for details. (Contributed by Brandt Bucher in bpo-39702.)
• Improved help for the typing module. Docstrings are now shown for all special forms and special generic aliases (like Union and List). Using help() with generic alias like List[int] will show the help for the correspondent concrete type (list in this case). (Contributed by Serhiy Storchaka in bpo-40257.)
• Parallel running of aclose()/asend()/athrow() is now prohibited, and ag_running now reflects the actual running status of the async generator. (Contributed by Yury Selivanov in bpo-30773.)
• Unexpected errors in calling the iter method are no longer masked by TypeError in the in operator and functions contains(), indexOf() and countOf() of the operator module. (Contributed by Serhiy Storchaka in bpo-40824.)
• Unparenthesized lambda expressions can no longer be the expression part in an if clause in comprehensions and generator expressions. See bpo-41848 and bpo-43755 for details.

New Modules
zoneinfo
The zoneinfo module brings support for the IANA time zone database to the standard library. It adds zoneinfo.ZoneInfo, a concrete datetime.tzinfo implementation backed by the system’s time zone data.
Example:

from zoneinfo import ZoneInfo
import datetime as dt

Daylight saving time

when = dt.datetime(2020, 10, 31, 12, tzinfo=ZoneInfo("America/Los_Angeles"))
print(when)
2020-10-31 12:00:00-07:00

when.tzname()
'PDT'

Standard time

when += dt.timedelta(days=7)
print(when)
2020-11-07 12:00:00-08:00

print(when.tzname())
PST
As a fall-back source of data for platforms that don’t ship the IANA database, the tzdata module was released as a first-party package – distributed via PyPI and maintained by the CPython core team.
See also
PEP 615 – Support for the IANA Time Zone Database in the Standard Library
PEP written and implemented by Paul Ganssle

graphlib
A new module, graphlib, was added that contains the graphlib.TopologicalSorter class to offer functionality to perform topological sorting of graphs. (Contributed by Pablo Galindo, Tim Peters and Larry Hastings in bpo-17005.)

Improved Modules
ast
Added the indent option to dump() which allows it to produce a multiline indented output. (Contributed by Serhiy Storchaka in bpo-37995.)
Added ast.unparse() as a function in the ast module that can be used to unparse an ast.AST object and produce a string with code that would produce an equivalent ast.AST object when parsed. (Contributed by Pablo Galindo and Batuhan Taskaya in bpo-38870.)
Added docstrings to AST nodes that contains the ASDL signature used to construct that node. (Contributed by Batuhan Taskaya in bpo-39638.)

asyncio
Due to significant security concerns, the reuse_address parameter of asyncio.loop.create_datagram_endpoint() is no longer supported. This is because of the behavior of the socket option SO_REUSEADDR in UDP. For more details, see the documentation for loop.create_datagram_endpoint(). (Contributed by Kyle Stanley, Antoine Pitrou, and Yury Selivanov in bpo-37228.)
Added a new coroutine shutdown_default_executor() that schedules a shutdown for the default executor that waits on the ThreadPoolExecutor to finish closing. Also, asyncio.run() has been updated to use the new coroutine. (Contributed by Kyle Stanley in bpo-34037.)
Added asyncio.PidfdChildWatcher, a Linux-specific child watcher implementation that polls process file descriptors. (bpo-38692)
Added a new coroutine asyncio.to_thread(). It is mainly used for running IO-bound functions in a separate thread to avoid blocking the event loop, and essentially works as a high-level version of run_in_executor() that can directly take keyword arguments. (Contributed by Kyle Stanley and Yury Selivanov in bpo-32309.)
When cancelling the task due to a timeout, asyncio.wait_for() will now wait until the cancellation is complete also in the case when timeout is <= 0, like it does with positive timeouts. (Contributed by Elvis Pranskevichus in bpo-32751.)
asyncio now raises TypeError when calling incompatible methods with an ssl.SSLSocket socket. (Contributed by Ido Michael in bpo-37404.)

compileall
Added new possibility to use hardlinks for duplicated .pyc files: hardlink_dupes parameter and –hardlink-dupes command line option. (Contributed by Lumír ‘Frenzy’ Balhar in bpo-40495.)
Added new options for path manipulation in resulting .pyc files: stripdir, prependdir, limit_sl_dest parameters and -s, -p, -e command line options. Added the possibility to specify the option for an optimization level multiple times. (Contributed by Lumír ‘Frenzy’ Balhar in bpo-38112.)

concurrent.futures
Added a new cancel_futures parameter to concurrent.futures.Executor.shutdown() that cancels all pending futures which have not started running, instead of waiting for them to complete before shutting down the executor. (Contributed by Kyle Stanley in bpo-39349.)
Removed daemon threads from ThreadPoolExecutor and ProcessPoolExecutor. This improves compatibility with subinterpreters and predictability in their shutdown processes. (Contributed by Kyle Stanley in bpo-39812.)
Workers in ProcessPoolExecutor are now spawned on demand, only when there are no available idle workers to reuse. This optimizes startup overhead and reduces the amount of lost CPU time to idle workers. (Contributed by Kyle Stanley in bpo-39207.)

curses
Added curses.get_escdelay(), curses.set_escdelay(), curses.get_tabsize(), and curses.set_tabsize() functions. (Contributed by Anthony Sottile in bpo-38312.)

datetime
The isocalendar() of datetime.date and isocalendar() of datetime.datetime methods now returns a namedtuple() instead of a tuple. (Contributed by Donghee Na in bpo-24416.)

distutils
The upload command now creates SHA2-256 and Blake2b-256 hash digests. It skips MD5 on platforms that block MD5 digest. (Contributed by Christian Heimes in bpo-40698.)

fcntl
Added constants fcntl.F_OFD_GETLK, fcntl.F_OFD_SETLK and fcntl.F_OFD_SETLKW. (Contributed by Donghee Na in bpo-38602.)

ftplib
FTP and FTP_TLS now raise a ValueError if the given timeout for their constructor is zero to prevent the creation of a non-blocking socket. (Contributed by Donghee Na in bpo-39259.)

gc
When the garbage collector makes a collection in which some objects resurrect (they are reachable from outside the isolated cycles after the finalizers have been executed), do not block the collection of all objects that are still unreachable. (Contributed by Pablo Galindo and Tim Peters in bpo-38379.)
Added a new function gc.is_finalized() to check if an object has been finalized by the garbage collector. (Contributed by Pablo Galindo in bpo-39322.)

hashlib
The hashlib module can now use SHA3 hashes and SHAKE XOF from OpenSSL when available. (Contributed by Christian Heimes in bpo-37630.)
Builtin hash modules can now be disabled with ./configure --without-builtin-hashlib-hashes or selectively enabled with e.g. ./configure --with-builtin-hashlib-hashes=sha3,blake2 to force use of OpenSSL based implementation. (Contributed by Christian Heimes in bpo-40479.)

http
HTTP status codes 103 EARLY_HINTS, 418 IM_A_TEAPOT and 425 TOO_EARLY are added to http.HTTPStatus. (Contributed by Donghee Na in bpo-39509 and Ross Rhodes in bpo-39507.)

IDLE and idlelib
Added option to toggle cursor blink off. (Contributed by Zackery Spytz in bpo-4603.)
Escape key now closes IDLE completion windows. (Contributed by Johnny Najera in bpo-38944.)
Added keywords to module name completion list. (Contributed by Terry J. Reedy in bpo-37765.)
New in 3.9 maintenance releases
Make IDLE invoke sys.excepthook() (when started without ‘-n’). User hooks were previously ignored. (Contributed by Ken Hilton in bpo-43008.)
The changes above have been backported to 3.8 maintenance releases.
Rearrange the settings dialog. Split the General tab into Windows and Shell/Ed tabs. Move help sources, which extend the Help menu, to the Extensions tab. Make space for new options and shorten the dialog. The latter makes the dialog better fit small screens. (Contributed by Terry Jan Reedy in bpo-40468.) Move the indent space setting from the Font tab to the new Windows tab. (Contributed by Mark Roseman and Terry Jan Reedy in bpo-33962.)
Apply syntax highlighting to .pyi files. (Contributed by Alex Waygood and Terry Jan Reedy in bpo-45447.)

imaplib
IMAP4 and IMAP4_SSL now have an optional timeout parameter for their constructors. Also, the open() method now has an optional timeout parameter with this change. The overridden methods of IMAP4_SSL and IMAP4_stream were applied to this change. (Contributed by Donghee Na in bpo-38615.)
imaplib.IMAP4.unselect() is added. imaplib.IMAP4.unselect() frees server’s resources associated with the selected mailbox and returns the server to the authenticated state. This command performs the same actions as imaplib.IMAP4.close(), except that no messages are permanently removed from the currently selected mailbox. (Contributed by Donghee Na in bpo-40375.)

importlib
To improve consistency with import statements, importlib.util.resolve_name() now raises ImportError instead of ValueError for invalid relative import attempts. (Contributed by Ngalim Siregar in bpo-37444.)
Import loaders which publish immutable module objects can now publish immutable packages in addition to individual modules. (Contributed by Dino Viehland in bpo-39336.)
Added importlib.resources.files() function with support for subdirectories in package data, matching backport in importlib_resources version 1.5. (Contributed by Jason R. Coombs in bpo-39791.)
Refreshed importlib.metadata from importlib_metadata version 1.6.1.

inspect
inspect.BoundArguments.arguments is changed from OrderedDict to regular dict. (Contributed by Inada Naoki in bpo-36350 and bpo-39775.)

ipaddress
ipaddress now supports IPv6 Scoped Addresses (IPv6 address with suffix %).
Scoped IPv6 addresses can be parsed using ipaddress.IPv6Address. If present, scope zone ID is available through the scope_id attribute. (Contributed by Oleksandr Pavliuk in bpo-34788.)
Starting with Python 3.9.5 the ipaddress module no longer accepts any leading zeros in IPv4 address strings. (Contributed by Christian Heimes in bpo-36384).

math
Expanded the math.gcd() function to handle multiple arguments. Formerly, it only supported two arguments. (Contributed by Serhiy Storchaka in bpo-39648.)
Added math.lcm(): return the least common multiple of specified arguments. (Contributed by Mark Dickinson, Ananthakrishnan and Serhiy Storchaka in bpo-39479 and bpo-39648.)
Added math.nextafter(): return the next floating-point value after x towards y. (Contributed by Victor Stinner in bpo-39288.)
Added math.ulp(): return the value of the least significant bit of a float. (Contributed by Victor Stinner in bpo-39310.)

multiprocessing
The multiprocessing.SimpleQueue class has a new close() method to explicitly close the queue. (Contributed by Victor Stinner in bpo-30966.)

nntplib
NNTP and NNTP_SSL now raise a ValueError if the given timeout for their constructor is zero to prevent the creation of a non-blocking socket. (Contributed by Donghee Na in bpo-39259.)

os
Added CLD_KILLED and CLD_STOPPED for si_code. (Contributed by Donghee Na in bpo-38493.)
Exposed the Linux-specific os.pidfd_open() (bpo-38692) and os.P_PIDFD (bpo-38713) for process management with file descriptors.
The os.unsetenv() function is now also available on Windows. (Contributed by Victor Stinner in bpo-39413.)
The os.putenv() and os.unsetenv() functions are now always available. (Contributed by Victor Stinner in bpo-39395.)
Added os.waitstatus_to_exitcode() function: convert a wait status to an exit code. (Contributed by Victor Stinner in bpo-40094.)

pathlib
Added pathlib.Path.readlink() which acts similarly to os.readlink(). (Contributed by Girts Folkmanis in bpo-30618)

pdb
On Windows now Pdb supports ~/.pdbrc. (Contributed by Tim Hopper and Dan Lidral-Porter in bpo-20523.)

poplib
POP3 and POP3_SSL now raise a ValueError if the given timeout for their constructor is zero to prevent the creation of a non-blocking socket. (Contributed by Donghee Na in bpo-39259.)

pprint
pprint can now pretty-print types.SimpleNamespace. (Contributed by Carl Bordum Hansen in bpo-37376.)

pydoc
The documentation string is now shown not only for class, function, method etc, but for any object that has its own doc attribute. (Contributed by Serhiy Storchaka in bpo-40257.)

random
Added a new random.Random.randbytes() method: generate random bytes. (Contributed by Victor Stinner in bpo-40286.)

signal
Exposed the Linux-specific signal.pidfd_send_signal() for sending to signals to a process using a file descriptor instead of a pid. (bpo-38712)

smtplib
SMTP and SMTP_SSL now raise a ValueError if the given timeout for their constructor is zero to prevent the creation of a non-blocking socket. (Contributed by Donghee Na in bpo-39259.)
LMTP constructor now has an optional timeout parameter. (Contributed by Donghee Na in bpo-39329.)

socket
The socket module now exports the CAN_RAW_JOIN_FILTERS constant on Linux 4.1 and greater. (Contributed by Stefan Tatschner and Zackery Spytz in bpo-25780.)
The socket module now supports the CAN_J1939 protocol on platforms that support it. (Contributed by Karl Ding in bpo-40291.)
The socket module now has the socket.send_fds() and socket.recv_fds() functions. (Contributed by Joannah Nanjekye, Shinya Okano and Victor Stinner in bpo-28724.)

time
On AIX, thread_time() is now implemented with thread_cputime() which has nanosecond resolution, rather than clock_gettime(CLOCK_THREAD_CPUTIME_ID) which has a resolution of 10 milliseconds. (Contributed by Batuhan Taskaya in bpo-40192)

sys
Added a new sys.platlibdir attribute: name of the platform-specific library directory. It is used to build the path of standard library and the paths of installed extension modules. It is equal to "lib" on most platforms. On Fedora and SuSE, it is equal to "lib64" on 64-bit platforms. (Contributed by Jan Matějek, Matěj Cepl, Charalampos Stratakis and Victor Stinner in bpo-1294959.)
Previously, sys.stderr was block-buffered when non-interactive. Now stderr defaults to always being line-buffered. (Contributed by Jendrik Seipp in bpo-13601.)

tracemalloc
Added tracemalloc.reset_peak() to set the peak size of traced memory blocks to the current size, to measure the peak of specific pieces of code. (Contributed by Huon Wilson in bpo-40630.)

typing
PEP 593 introduced an typing.Annotated type to decorate existing types with context-specific metadata and new include_extras parameter to typing.get_type_hints() to access the metadata at runtime. (Contributed by Till Varoquaux and Konstantin Kashin.)

unicodedata
The Unicode database has been updated to version 13.0.0. (bpo-39926).

venv
The activation scripts provided by venv now all specify their prompt customization consistently by always using the value specified by VENV_PROMPT. Previously some scripts unconditionally used VENV_PROMPT, others only if it happened to be set (which was the default case), and one used VENV_NAME instead. (Contributed by Brett Cannon in bpo-37663.)

xml
White space characters within attributes are now preserved when serializing xml.etree.ElementTree to XML file. EOLNs are no longer normalized to “n”. This is the result of discussion about how to interpret section 2.11 of XML spec. (Contributed by Mefistotelis in bpo-39011.)

Optimizations
• Optimized the idiom for assignment a temporary variable in comprehensions. Now for y in [expr] in comprehensions is as fast as a simple assignment y = expr . For example:
sums = [s for s in [0] for x in data for s in [s + x]]
Unlike the := operator this idiom does not leak a variable to the outer scope.
(Contributed by Serhiy Storchaka in bpo-32856.)
• Optimized signal handling in multithreaded applications. If a thread different than the main thread gets a signal, the bytecode evaluation loop is no longer interrupted at each bytecode instruction to check for pending signals which cannot be handled. Only the main thread of the main interpreter can handle signals.
Previously, the bytecode evaluation loop was interrupted at each instruction until the main thread handles signals. (Contributed by Victor Stinner in bpo-40010.)
• Optimized the subprocess module on FreeBSD using closefrom(). (Contributed by Ed Maste, Conrad Meyer, Kyle Evans, Kubilay Kocak and Victor Stinner in bpo-38061.)
• PyLong_FromDouble() is now up to 1.87x faster for values that fit into long. (Contributed by Sergey Fedoseev in bpo-37986.)
• A number of Python builtins (range, tuple, set, frozenset, list, dict) are now sped up by using PEP 590 vectorcall protocol. (Contributed by Donghee Na, Mark Shannon, Jeroen Demeyer and Petr Viktorin in bpo-37207.)
• Optimized set.difference_update() for the case when the other set is much larger than the base set. (Suggested by Evgeny Kapun with code contributed by Michele Orrù in bpo-8425.)
• Python’s small object allocator (obmalloc.c) now allows (no more than) one empty arena to remain available for immediate reuse, without returning it to the OS. This prevents thrashing in simple loops where an arena could be created and destroyed anew on each iteration. (Contributed by Tim Peters in bpo-37257.)
• floor division of float operation now has a better performance. Also the message of ZeroDivisionError for this operation is updated. (Contributed by Donghee Na in bpo-39434.)
• Decoding short ASCII strings with UTF-8 and ascii codecs is now about 15% faster. (Contributed by Inada Naoki in bpo-37348.)
Here’s a summary of performance improvements from Python 3.4 through Python 3.9:
Python version 3.4 3.5 3.6 3.7 3.8 3.9

Variable and attribute read access:
read_local 7.1 7.1 5.4 5.1 3.9 3.9
read_nonlocal 7.1 8.1 5.8 5.4 4.4 4.5
read_global 15.5 19.0 14.3 13.6 7.6 7.8
read_builtin 21.1 21.6 18.5 19.0 7.5 7.8
read_classvar_from_class 25.6 26.5 20.7 19.5 18.4 17.9
read_classvar_from_instance 22.8 23.5 18.8 17.1 16.4 16.9
read_instancevar 32.4 33.1 28.0 26.3 25.4 25.3
read_instancevar_slots 27.8 31.3 20.8 20.8 20.2 20.5
read_namedtuple 73.8 57.5 45.0 46.8 18.4 18.7
read_boundmethod 37.6 37.9 29.6 26.9 27.7 41.1

Variable and attribute write access:
write_local 8.7 9.3 5.5 5.3 4.3 4.3
write_nonlocal 10.5 11.1 5.6 5.5 4.7 4.8
write_global 19.7 21.2 18.0 18.0 15.8 16.7
write_classvar 92.9 96.0 104.6 102.1 39.2 39.8
write_instancevar 44.6 45.8 40.0 38.9 35.5 37.4
write_instancevar_slots 35.6 36.1 27.3 26.6 25.7 25.8

Data structure read access:
read_list 24.2 24.5 20.8 20.8 19.0 19.5
read_deque 24.7 25.5 20.2 20.6 19.8 20.2
read_dict 24.3 25.7 22.3 23.0 21.0 22.4
read_strdict 22.6 24.3 19.5 21.2 18.9 21.5

Data structure write access:
write_list 27.1 28.5 22.5 21.6 20.0 20.0
write_deque 28.7 30.1 22.7 21.8 23.5 21.7
write_dict 31.4 33.3 29.3 29.2 24.7 25.4
write_strdict 28.4 29.9 27.5 25.2 23.1 24.5

Stack (or queue) operations:
list_append_pop 93.4 112.7 75.4 74.2 50.8 50.6
deque_append_pop 43.5 57.0 49.4 49.2 42.5 44.2
deque_append_popleft 43.7 57.3 49.7 49.7 42.8 46.4

Timing loop:
loop_overhead 0.5 0.6 0.4 0.3 0.3 0.3
These results were generated from the variable access benchmark script at:
Tools/scripts/var_access_benchmark.py . The benchmark script displays timings in nanoseconds. The benchmarks were measured on an Intel® Core™ i7-4960HQ processor running the macOS 64-bit builds found at python.org.

Deprecated
• The distutils bdist_msi command is now deprecated, use bdist_wheel (wheel packages) instead. (Contributed by Hugo van Kemenade in bpo-39586.)
• Currently math.factorial() accepts float instances with non-negative integer values (like 5.0). It raises a ValueError for non-integral and negative floats. It is now deprecated. In future Python versions it will raise a TypeError for all floats. (Contributed by Serhiy Storchaka in bpo-37315.)
• The parser and symbol modules are deprecated and will be removed in future versions of Python. For the majority of use cases, users can leverage the Abstract Syntax Tree (AST) generation and compilation stage, using the ast module.
• The Public C API functions PyParser_SimpleParseStringFlags(), PyParser_SimpleParseStringFlagsFilename(), PyParser_SimpleParseFileFlags() and PyNode_Compile() are deprecated and will be removed in Python 3.10 together with the old parser.
• Using NotImplemented in a boolean context has been deprecated, as it is almost exclusively the result of incorrect rich comparator implementations. It will be made a TypeError in a future version of Python. (Contributed by Josh Rosenberg in bpo-35712.)
• The random module currently accepts any hashable type as a possible seed value. Unfortunately, some of those types are not guaranteed to have a deterministic hash value. After Python 3.9, the module will restrict its seeds to None, int, float, str, bytes, and bytearray.
• Opening the GzipFile file for writing without specifying the mode argument is deprecated. In future Python versions it will always be opened for reading by default. Specify the mode argument for opening it for writing and silencing a warning. (Contributed by Serhiy Storchaka in bpo-28286.)
• Deprecated the split() method of _tkinter.TkappType in favour of the splitlist() method which has more consistent and predictable behavior. (Contributed by Serhiy Storchaka in bpo-38371.)
• The explicit passing of coroutine objects to asyncio.wait() has been deprecated and will be removed in version 3.11. (Contributed by Yury Selivanov and Kyle Stanley in bpo-34790.)
• binhex4 and hexbin4 standards are now deprecated. The binhex module and the following binascii functions are now deprecated:
• b2a_hqx(), a2b_hqx()
• rlecode_hqx(), rledecode_hqx()
(Contributed by Victor Stinner in bpo-39353.)
• ast classes slice, Index and ExtSlice are considered deprecated and will be removed in future Python versions. value itself should be used instead of Index(value). Tuple(slices, Load()) should be used instead of ExtSlice(slices). (Contributed by Serhiy Storchaka in bpo-34822.)
• ast classes Suite, Param, AugLoad and AugStore are considered deprecated and will be removed in future Python versions. They were not generated by the parser and not accepted by the code generator in Python 3. (Contributed by Batuhan Taskaya in bpo-39639 and bpo-39969 and Serhiy Storchaka in bpo-39988.)
• The PyEval_InitThreads() and PyEval_ThreadsInitialized() functions are now deprecated and will be removed in Python 3.11. Calling PyEval_InitThreads() now does nothing. The GIL is initialized by Py_Initialize() since Python 3.7. (Contributed by Victor Stinner in bpo-39877.)
• Passing None as the first argument to the shlex.split() function has been deprecated. (Contributed by Zackery Spytz in bpo-33262.)
• smtpd.MailmanProxy() is now deprecated as it is unusable without an external module, mailman. (Contributed by Samuel Colvin in bpo-35800.)
• The lib2to3 module now emits a PendingDeprecationWarning. Python 3.9 switched to a PEG parser (see PEP 617), and Python 3.10 may include new language syntax that is not parsable by lib2to3’s LL(1) parser. The lib2to3 module may be removed from the standard library in a future Python version. Consider third-party alternatives such as LibCST or parso. (Contributed by Carl Meyer in bpo-40360.)
• The random parameter of random.shuffle() has been deprecated. (Contributed by Raymond Hettinger in bpo-40465)

Removed
• The erroneous version at unittest.mock.version has been removed.
• nntplib.NNTP: xpath() and xgtitle() methods have been removed. These methods are deprecated since Python 3.3. Generally, these extensions are not supported or not enabled by NNTP server administrators. For xgtitle(), please use nntplib.NNTP.descriptions() or nntplib.NNTP.description() instead. (Contributed by Donghee Na in bpo-39366.)
• array.array: tostring() and fromstring() methods have been removed. They were aliases to tobytes() and frombytes(), deprecated since Python 3.2. (Contributed by Victor Stinner in bpo-38916.)
• The undocumented sys.callstats() function has been removed. Since Python 3.7, it was deprecated and always returned None. It required a special build option CALL_PROFILE which was already removed in Python 3.7. (Contributed by Victor Stinner in bpo-37414.)
• The sys.getcheckinterval() and sys.setcheckinterval() functions have been removed. They were deprecated since Python 3.2. Use sys.getswitchinterval() and sys.setswitchinterval() instead. (Contributed by Victor Stinner in bpo-37392.)
• The C function PyImport_Cleanup() has been removed. It was documented as: “Empty the module table. For internal use only.” (Contributed by Victor Stinner in bpo-36710.)
• _dummy_thread and dummy_threading modules have been removed. These modules were deprecated since Python 3.7 which requires threading support. (Contributed by Victor Stinner in bpo-37312.)
• aifc.openfp() alias to aifc.open(), sunau.openfp() alias to sunau.open(), and wave.openfp() alias to wave.open() have been removed. They were deprecated since Python 3.7. (Contributed by Victor Stinner in bpo-37320.)
• The isAlive() method of threading.Thread has been removed. It was deprecated since Python 3.8. Use is_alive() instead. (Contributed by Donghee Na in bpo-37804.)
• Methods getchildren() and getiterator() of classes ElementTree and Element in the ElementTree module have been removed. They were deprecated in Python 3.2. Use iter(x) or list(x) instead of x.getchildren() and x.iter() or list(x.iter()) instead of x.getiterator(). (Contributed by Serhiy Storchaka in bpo-36543.)
• The old plistlib API has been removed, it was deprecated since Python 3.4. Use the load(), loads(), dump(), and dumps() functions. Additionally, the use_builtin_types parameter was removed, standard bytes objects are always used instead. (Contributed by Jon Janzen in bpo-36409.)
• The C function PyGen_NeedsFinalizing has been removed. It was not documented, tested, or used anywhere within CPython after the implementation of PEP 442. Patch by Joannah Nanjekye. (Contributed by Joannah Nanjekye in bpo-15088)
• base64.encodestring() and base64.decodestring(), aliases deprecated since Python 3.1, have been removed: use base64.encodebytes() and base64.decodebytes() instead. (Contributed by Victor Stinner in bpo-39351.)
• fractions.gcd() function has been removed, it was deprecated since Python 3.5 (bpo-22486): use math.gcd() instead. (Contributed by Victor Stinner in bpo-39350.)
• The buffering parameter of bz2.BZ2File has been removed. Since Python 3.0, it was ignored and using it emitted a DeprecationWarning. Pass an open file object to control how the file is opened. (Contributed by Victor Stinner in bpo-39357.)
• The encoding parameter of json.loads() has been removed. As of Python 3.1, it was deprecated and ignored; using it has emitted a DeprecationWarning since Python 3.8. (Contributed by Inada Naoki in bpo-39377)
• with (await asyncio.lock): and with (yield from asyncio.lock): statements are not longer supported, use async with lock instead. The same is correct for asyncio.Condition and asyncio.Semaphore. (Contributed by Andrew Svetlov in bpo-34793.)
• The sys.getcounts() function, the -X showalloccount command line option and the show_alloc_count field of the C structure PyConfig have been removed. They required a special Python build by defining COUNT_ALLOCS macro. (Contributed by Victor Stinner in bpo-39489.)
• The _field_types attribute of the typing.NamedTuple class has been removed. It was deprecated since Python 3.8. Use the annotations attribute instead. (Contributed by Serhiy Storchaka in bpo-40182.)
• The symtable.SymbolTable.has_exec() method has been removed. It was deprecated since 2006, and only returning False when it’s called. (Contributed by Batuhan Taskaya in bpo-40208)
• The asyncio.Task.current_task() and asyncio.Task.all_tasks() have been removed. They were deprecated since Python 3.7 and you can use asyncio.current_task() and asyncio.all_tasks() instead. (Contributed by Rémi Lapeyre in bpo-40967)
• The unescape() method in the html.parser.HTMLParser class has been removed (it was deprecated since Python 3.4). html.unescape() should be used for converting character references to the corresponding unicode characters.

Porting to Python 3.9
This section lists previously described changes and other bugfixes that may require changes to your code.

Changes in the Python API
• import() and importlib.util.resolve_name() now raise ImportError where it previously raised ValueError. Callers catching the specific exception type and supporting both Python 3.9 and earlier versions will need to catch both using except (ImportError, ValueError):.
• The venv activation scripts no longer special-case when VENV_PROMPT is set to "".
• The select.epoll.unregister() method no longer ignores the EBADF error. (Contributed by Victor Stinner in bpo-39239.)
• The compresslevel parameter of bz2.BZ2File became keyword-only, since the buffering parameter has been removed. (Contributed by Victor Stinner in bpo-39357.)
• Simplified AST for subscription. Simple indices will be represented by their value, extended slices will be represented as tuples. Index(value) will return a value itself, ExtSlice(slices) will return Tuple(slices, Load()). (Contributed by Serhiy Storchaka in bpo-34822.)
• The importlib module now ignores the PYTHONCASEOK environment variable when the -E or -I command line options are being used.
• The encoding parameter has been added to the classes ftplib.FTP and ftplib.FTP_TLS as a keyword-only parameter, and the default encoding is changed from Latin-1 to UTF-8 to follow RFC 2640.
• asyncio.loop.shutdown_default_executor() has been added to AbstractEventLoop, meaning alternative event loops that inherit from it should have this method defined. (Contributed by Kyle Stanley in bpo-34037.)
• The constant values of future flags in the future module is updated in order to prevent collision with compiler flags. Previously PyCF_ALLOW_TOP_LEVEL_AWAIT was clashing with CO_FUTURE_DIVISION. (Contributed by Batuhan Taskaya in bpo-39562.)
• array('u') now uses wchar_t as C type instead of Py_UNICODE. This change doesn’t affect to its behavior because Py_UNICODE is alias of wchar_t since Python 3.3. (Contributed by Inada Naoki in bpo-34538.)
• The logging.getLogger() API now returns the root logger when passed the name 'root', whereas previously it returned a non-root logger named 'root'. This could affect cases where user code explicitly wants a non-root logger named 'root', or instantiates a logger using logging.getLogger(name) in some top-level module called 'root.py'. (Contributed by Vinay Sajip in bpo-37742.)
• Division handling of PurePath now returns NotImplemented instead of raising a TypeError when passed something other than an instance of str or PurePath. This allows creating compatible classes that don’t inherit from those mentioned types. (Contributed by Roger Aiudi in bpo-34775).
• Starting with Python 3.9.5 the ipaddress module no longer accepts any leading zeros in IPv4 address strings. Leading zeros are ambiguous and interpreted as octal notation by some libraries. For example the legacy function socket.inet_aton() treats leading zeros as octal notatation. glibc implementation of modern inet_pton() does not accept any leading zeros. (Contributed by Christian Heimes in bpo-36384).
• codecs.lookup() now normalizes the encoding name the same way as encodings.normalize_encoding(), except that codecs.lookup() also converts the name to lower case. For example, "latex+latin1" encoding name is now normalized to "latex_latin1". (Contributed by Jordon Xu in bpo-37751.)

Changes in the C API
• Instances of heap-allocated types (such as those created with PyType_FromSpec() and similar APIs) hold a reference to their type object since Python 3.8. As indicated in the “Changes in the C API” of Python 3.8, for the vast majority of cases, there should be no side effect but for types that have a custom tp_traverse function, ensure that all custom tp_traverse functions of heap-allocated types visit the object’s type.
Example:
int foo_traverse(PyObject *self, visitproc visit, void *arg)
{
// Rest of the traverse function
#if PY_VERSION_HEX >= 0x03090000
// This was not needed before Python 3.9 (Python issue 35810 and 40217)
Py_VISIT(Py_TYPE(self));
#endif
}
If your traverse function delegates to tp_traverse of its base class (or another type), ensure that Py_TYPE(self) is visited only once. Note that only heap type are expected to visit the type in tp_traverse.
For example, if your tp_traverse function includes:
base->tp_traverse(self, visit, arg)
then add:
#if PY_VERSION_HEX >= 0x03090000
// This was not needed before Python 3.9 (bpo-35810 and bpo-40217)
if (base->tp_flags & Py_TPFLAGS_HEAPTYPE) {
// a heap type's tp_traverse already visited Py_TYPE(self)
}
else
{
Py_VISIT(Py_TYPE(self));
}
#else

(See bpo-35810 and bpo-40217 for more information.)
• The functions PyEval_CallObject, PyEval_CallFunction, PyEval_CallMethod and PyEval_CallObjectWithKeywords are deprecated. Use PyObject_Call() and its variants instead. (See more details in bpo-29548.)

CPython bytecode changes
• The LOAD_ASSERTION_ERROR opcode was added for handling the assert statement. Previously, the assert statement would not work correctly if the AssertionError exception was being shadowed. (Contributed by Zackery Spytz in bpo-34880.)
• The COMPARE_OP opcode was split into four distinct instructions:
• COMPARE_OP for rich comparisons
• IS_OP for ‘is’ and ‘is not’ tests
• CONTAINS_OP for ‘in’ and ‘not in’ tests
• JUMP_IF_NOT_EXC_MATCH for checking exceptions in ‘try-except’ statements.
(Contributed by Mark Shannon in bpo-39156.)

Build Changes
• Added --with-platlibdir option to the configure script: name of the platform-specific library directory, stored in the new sys.platlibdir attribute. See sys.platlibdir attribute for more information. (Contributed by Jan Matějek, Matěj Cepl, Charalampos Stratakis and Victor Stinner in bpo-1294959.)
• The COUNT_ALLOCS special build macro has been removed. (Contributed by Victor Stinner in bpo-39489.)
• On non-Windows platforms, the setenv() and unsetenv() functions are now required to build Python. (Contributed by Victor Stinner in bpo-39395.)
• On non-Windows platforms, creating bdist_wininst installers is now officially unsupported. (See bpo-10945 for more details.)
• When building Python on macOS from source, _tkinter now links with non-system Tcl and Tk frameworks if they are installed in /Library/Frameworks, as had been the case on older releases of macOS. If a macOS SDK is explicitly configured, by using --enable-universalsdk or -isysroot, only the SDK itself is searched. The default behavior can still be overridden with --with-tcltk-includes and --with-tcltk-libs. (Contributed by Ned Deily in bpo-34956.)
• Python can now be built for Windows 10 ARM64. (Contributed by Steve Dower in bpo-33125.)
• Some individual tests are now skipped when --pgo is used. The tests in question increased the PGO task time significantly and likely didn’t help improve optimization of the final executable. This speeds up the task by a factor of about 15x. Running the full unit test suite is slow. This change may result in a slightly less optimized build since not as many code branches will be executed. If you are willing to wait for the much slower build, the old behavior can be restored using ./configure [..] PROFILE_TASK="-m test --pgo-extended". We make no guarantees as to which PGO task set produces a faster build. Users who care should run their own relevant benchmarks as results can depend on the environment, workload, and compiler tool chain. (See bpo-36044 and bpo-37707 for more details.)

C API Changes
New Features
• PEP 573: Added PyType_FromModuleAndSpec() to associate a module with a class; PyType_GetModule() and PyType_GetModuleState() to retrieve the module and its state; and PyCMethod and METH_METHOD to allow a method to access the class it was defined in. (Contributed by Marcel Plch and Petr Viktorin in bpo-38787.)
• Added PyFrame_GetCode() function: get a frame code. Added PyFrame_GetBack() function: get the frame next outer frame. (Contributed by Victor Stinner in bpo-40421.)
• Added PyFrame_GetLineNumber() to the limited C API. (Contributed by Victor Stinner in bpo-40421.)
• Added PyThreadState_GetInterpreter() and PyInterpreterState_Get() functions to get the interpreter. Added PyThreadState_GetFrame() function to get the current frame of a Python thread state. Added PyThreadState_GetID() function: get the unique identifier of a Python thread state. (Contributed by Victor Stinner in bpo-39947.)
• Added a new public PyObject_CallNoArgs() function to the C API, which calls a callable Python object without any arguments. It is the most efficient way to call a callable Python object without any argument. (Contributed by Victor Stinner in bpo-37194.)
• Changes in the limited C API (if Py_LIMITED_API macro is defined):
• Provide Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() as regular functions for the limited API. Previously, there were defined as macros, but these macros didn’t compile with the limited C API which cannot access PyThreadState.recursion_depth field (the structure is opaque in the limited C API).
• PyObject_INIT() and PyObject_INIT_VAR() become regular “opaque” function to hide implementation details.
(Contributed by Victor Stinner in bpo-38644 and bpo-39542.)
• The PyModule_AddType() function is added to help adding a type to a module. (Contributed by Donghee Na in bpo-40024.)
• Added the functions PyObject_GC_IsTracked() and PyObject_GC_IsFinalized() to the public API to allow to query if Python objects are being currently tracked or have been already finalized by the garbage collector respectively. (Contributed by Pablo Galindo Salgado in bpo-40241.)
• Added _PyObject_FunctionStr() to get a user-friendly string representation of a function-like object. (Patch by Jeroen Demeyer in bpo-37645.)
• Added PyObject_CallOneArg() for calling an object with one positional argument (Patch by Jeroen Demeyer in bpo-37483.)

Porting to Python 3.9
• PyInterpreterState.eval_frame (PEP 523) now requires a new mandatory tstate parameter (PyThreadState*). (Contributed by Victor Stinner in bpo-38500.)
• Extension modules: m_traverse, m_clear and m_free functions of PyModuleDef are no longer called if the module state was requested but is not allocated yet. This is the case immediately after the module is created and before the module is executed (Py_mod_exec function). More precisely, these functions are not called if m_size is greater than 0 and the module state (as returned by PyModule_GetState()) is NULL.
Extension modules without module state (m_size <= 0) are not affected.
• If Py_AddPendingCall() is called in a subinterpreter, the function is now scheduled to be called from the subinterpreter, rather than being called from the main interpreter. Each subinterpreter now has its own list of scheduled calls. (Contributed by Victor Stinner in bpo-39984.)
• The Windows registry is no longer used to initialize sys.path when the -E option is used (if PyConfig.use_environment is set to 0). This is significant when embedding Python on Windows. (Contributed by Zackery Spytz in bpo-8901.)
• The global variable PyStructSequence_UnnamedField is now a constant and refers to a constant string. (Contributed by Serhiy Storchaka in bpo-38650.)
• The PyGC_Head structure is now opaque. It is only defined in the internal C API (pycore_gc.h). (Contributed by Victor Stinner in bpo-40241.)
• The Py_UNICODE_COPY, Py_UNICODE_FILL, PyUnicode_WSTR_LENGTH, PyUnicode_FromUnicode(), PyUnicode_AsUnicode(), _PyUnicode_AsUnicode and PyUnicode_AsUnicodeAndSize() are marked as deprecated in C. They have been deprecated by PEP 393 since Python 3.3. (Contributed by Inada Naoki in bpo-36346.)
• The Py_FatalError() function is replaced with a macro which logs automatically the name of the current function, unless the Py_LIMITED_API macro is defined. (Contributed by Victor Stinner in bpo-39882.)
• The vectorcall protocol now requires that the caller passes only strings as keyword names. (See bpo-37540 for more information.)
• Implementation details of a number of macros and functions are now hidden:
• PyObject_IS_GC() macro was converted to a function.
• The PyObject_NEW() macro becomes an alias to the PyObject_New macro, and the PyObject_NEW_VAR() macro becomes an alias to the PyObject_NewVar macro. They no longer access directly the PyTypeObject.tp_basicsize member.
• PyObject_GET_WEAKREFS_LISTPTR() macro was converted to a function: the macro accessed directly the PyTypeObject.tp_weaklistoffset member.
• PyObject_CheckBuffer() macro was converted to a function: the macro accessed directly the PyTypeObject.tp_as_buffer member.
• PyIndex_Check() is now always declared as an opaque function to hide implementation details: removed the PyIndex_Check() macro. The macro accessed directly the PyTypeObject.tp_as_number member.
(See bpo-40170 for more details.)

Removed
• Excluded PyFPE_START_PROTECT() and PyFPE_END_PROTECT() macros of pyfpe.h from the limited C API. (Contributed by Victor Stinner in bpo-38835.)
• The tp_print slot of PyTypeObject has been removed. It was used for printing objects to files in Python 2.7 and before. Since Python 3.0, it has been ignored and unused. (Contributed by Jeroen Demeyer in bpo-36974.)
• Changes in the limited C API (if Py_LIMITED_API macro is defined):
• Excluded the following functions from the limited C API:
• PyThreadState_DeleteCurrent() (Contributed by Joannah Nanjekye in bpo-37878.)
• _Py_CheckRecursionLimit
• _Py_NewReference()
• _Py_ForgetReference()
• _PyTraceMalloc_NewReference()
• _Py_GetRefTotal()
• The trashcan mechanism which never worked in the limited C A

Original source Report a problem

What’s New In Python 3.8

Editor:
Raymond Hettinger

This article explains the new features in Python 3.8, compared to 3.7. Python 3.8 was released on October 14, 2019. For full details, see the changelog.

Summary – Release highlights

New Features

Assignment expressions

There is new syntax
:=
that assigns values to variables as part of a larger expression. It is affectionately known as “the walrus operator” due to its resemblance to the eyes and tusks of a walrus.

In this example, the assignment expression helps avoid calling
len()
twice:

if (n := len(a)) > 10:
print(f"List is too long ({n} elements, expected <= 10)")

A similar benefit arises during regular expression matching where match objects are needed twice, once to test whether a match occurred and another to extract a subgroup:

discount = 0.0
if (mo := re.search(r'(\d+)% d iscount', advertisement)):
discount = float(mo.group(1)) / 100.0

The operator is also useful with while-loops that compute a value to test loop termination and then need that same value again in the body of the loop:

Loop over fixed length blocks

while (block := f.read(256)) != '':
process(block)

Another motivating use case arises in list comprehensions where a value computed in a filtering condition is also needed in the expression body:

[clean_name.title() for name in names if (clean_name := normalize('NFC', name)) in allowed_names]

Try to limit use of the walrus operator to clean cases that reduce complexity and improve readability.

See PEP 572 for a full description.

(Contributed by Emily Morehouse in bpo-35224.)

Positional-only parameters

There is a new function parameter syntax
/
to indicate that some function parameters must be specified positionally and cannot be used as keyword arguments. This is the same notation shown by
help()
for C functions annotated with Larry Hastings’ Argument Clinic tool.

In the following example, parameters a and b are positional-only, while c or d can be positional or keyword, and e or f are required to be keywords:

def f(a, b, /, c, d, *, e, f):
print(a, b, c, d, e, f)

The following is a valid call:

f(10, 20, 30, d=40, e=50, f=60)

However, these are invalid calls:

f(10, b=20, c=30, d=40, e=50, f=60) # b cannot be a keyword argument
f(10, 20, 30, 40, 50, f=60) # e must be a keyword argument

One use case for this notation is that it allows pure Python functions to fully emulate behaviors of existing C coded functions. For example, the built-in
divmod()
function does not accept keyword arguments:

def divmod(a, b, /):
"Emulate the built in divmod() function"
return (a // b, a % b)

Another use case is to preclude keyword arguments when the parameter name is not helpful. For example, the builtin
len()
function has the signature
len(obj, /).
This precludes awkward calls such as:

len(obj='hello')

The "obj" keyword argument impairs readability

A further benefit of marking a parameter as positional-only is that it allows the parameter name to be changed in the future without risk of breaking client code. For example, in the
statistics
module, the parameter
dist
may be changed in the future. This was made possible with the following function specification:

def quantiles(dist, /, *, n=4, method='exclusive'):
...

Since the parameters to the left of
/
are not exposed as possible keywords, the parameters names remain available for use in
**kwargs
:

def f(a, b, /, **kwargs):
... print(a, b, kwargs)
...

f(10, 20, a=1, b=2, c=3)

a and b are used in two ways

10 20 {'a': 1, 'b': 2, 'c': 3}

This greatly simplifies the implementation of functions and methods that need to accept arbitrary keyword arguments. For example, here is an excerpt from code in the
collections
module:

class Counter(dict):
def init(self, iterable=None, /, **kwds):
# Note "iterable" is a possible keyword argument
...

See PEP 570 for a full description.

(Contributed by Pablo Galindo in bpo-36540.)

Parallel filesystem cache for compiled bytecode files

The new
PYTHONPYCACHEPREFIX
setting (also available as
-X pycache_prefix
) configures the implicit bytecode cache to use a separate parallel filesystem tree, rather than the default
pycache
subdirectories within each source directory.

The location of the cache is reported in
sys.pycache_prefix
(None indicates the default location in
pycache
subdirectories).

(Contributed by Carl Meyer in bpo-33499.)

Debug build uses the same ABI as release build

Python now uses the same ABI whether it’s built in release or debug mode. On Unix, when Python is built in debug mode, it is now possible to load C extensions built in release mode and C extensions built using the stable ABI.

Release builds and debug builds are now ABI compatible: defining the
Py_DEBUG
macro no longer implies the
Py_TRACE_REFS
macro, which introduces the only ABI incompatibility. The
Py_TRACE_REFS
macro, which adds the
sys.getobjects()
function and the
PYTHONDUMPREFS
environment variable, can be set using the new
./configure --with-trace-refs
build option. (Contributed by Victor Stinner in bpo-36465.)

On Unix, C extensions are no longer linked to libpython except on Android and Cygwin. It is now possible for a statically linked Python to load a C extension built using a shared library Python. (Contributed by Victor Stinner in bpo-21536.)

On Unix, when Python is built in debug mode, import now also looks for C extensions compiled in release mode and for C extensions compiled with the stable ABI. (Contributed by Victor Stinner in bpo-36722.)

To embed Python into an application, a new
--embed
option must be passed to
python3-config --libs --embed
to get
-lpython3.8
(link the application to libpython). To support both 3.8 and older, try
python3-config --libs --embed
first and fallback to
python3-config --libs
(without
--embed
) if the previous command fails.

Add a pkg-config
python-3.8-embed
module to embed Python into an application:

pkg-config python-3.8-embed --libs includes -lpython3.8. To support both 3.8 and older, try
pkg-config python-X.Y-embed --libs
first and fallback to
pkg-config python-X.Y --libs
(without
--embed
) if the previous command fails (replace
X.Y
with the Python version).

On the other hand,
pkg-config python3.8 --libs
no longer contains
-lpython3.8
. C extensions must not be linked to libpython (except on Android and Cygwin, whose cases are handled by the script); this change is backward incompatible on purpose. (Contributed by Victor Stinner in bpo-36721.)

f-strings support = for self-documenting expressions and debugging

Added an

specifier to f-strings. An f-string such as
f'{expr=}'
will expand to the text of the expression, an equal sign, then the representation of the evaluated expression. For example:

user = 'eric_idle'
member_since = date(1975, 7, 31)
f'{user=} {member_since=}'
"user='eric_idle' member_since=datetime.date(1975, 7, 31)"

The usual f-string format specifiers allow more control over how the result of the expression is displayed:

delta = date.today() - member_since
f'{user=!s} {delta.days=:,d}'
'user=eric_idle delta.days=16,075'

The = specifier will display the whole expression so that calculations can be shown:

print(f'{theta=} {cos(radians(theta)) =:.3f}')
theta=30 cos(radians(theta))=0.866

(Contributed by Eric V. Smith and Larry Hastings in bpo-36817.)

PEP 578: Python Runtime Audit Hooks

The PEP adds an Audit Hook and Verified Open Hook. Both are available from Python and native code, allowing applications and frameworks written in pure Python code to take advantage of extra notifications, while also allowing embedders or system administrators to deploy builds of Python where auditing is always enabled.

See PEP 578 for full details.

PEP 587: Python Initialization Configuration

The PEP 587 adds a new C API to configure the Python Initialization providing finer control on the whole configuration and better error reporting.

New structures:
• PyConfig
• PyPreConfig
• PyStatus
• PyWideStringList

New functions:
• PyConfig_Clear()
• PyConfig_InitIsolatedConfig()
• PyConfig_InitPythonConfig()
• PyConfig_Read()
• PyConfig_SetArgv()
• PyConfig_SetBytesArgv()
• PyConfig_SetBytesString()
• PyConfig_SetString()
• PyPreConfig_InitIsolatedConfig()
• PyPreConfig_InitPythonConfig()
• PyStatus_Error()
• PyStatus_Exception()
• PyStatus_Exit()
• PyStatus_IsError()
• PyStatus_IsExit()
• PyStatus_NoMemory()
• PyStatus_Ok()
• PyWideStringList_Append()
• PyWideStringList_Insert()
• Py_BytesMain()
• Py_ExitStatusException()
• Py_InitializeFromConfig()
• Py_PreInitialize()
• Py_PreInitializeFromArgs()
• Py_PreInitializeFromBytesArgs()
• Py_RunMain()

This PEP also adds _PyRuntimeState.preconfig (PyPreConfig type) and PyInterpreterState.config (PyConfig type) fields to these internal structures. PyInterpreterState.config becomes the new reference configuration, replacing global configuration variables and other private variables.

See Python Initialization Configuration for the documentation.

See PEP 587 for a full description.

(Contributed by Victor Stinner in bpo-36763.)

PEP 590: Vectorcall: a fast calling protocol for CPython

The Vectorcall Protocol is added to the Python/C API. It is meant to formalize existing optimizations which were already done for various classes. Any static type implementing a callable can use this protocol.

This is currently provisional. The aim is to make it fully public in Python 3.9.

See PEP 590 for a full description.

(Contributed by Jeroen Demeyer, Mark Shannon and Petr Viktorin in bpo-36974.)

Pickle protocol 5 with out-of-band data buffers

When
pickle
is used to transfer large data between Python processes in order to take advantage of multi-core or multi-machine processing, it is important to optimize the transfer by reducing memory copies, and possibly by applying custom techniques such as data-dependent compression.

The
pickle
protocol 5 introduces support for out-of-band buffers where PEP 3118-compatible data can be transmitted separately from the main pickle stream, at the discretion of the communication layer.

See PEP 574 for a full description.

(Contributed by Antoine Pitrou in bpo-36785.)

Other Language Changes

A continue statement was illegal in the finally clause due to a problem with the implementation. In Python 3.8 this restriction was lifted. (Contributed by Serhiy Storchaka in bpo-32489.)

The bool, int, and fractions.Fraction types now have an as_integer_ratio() method like that found in float and decimal.Decimal. This minor API extension makes it possible to write numerator, denominator = x.as_integer_ratio() and have it work across multiple numeric types. (Contributed by Lisa Roach in bpo-33073 and Raymond Hettinger in bpo-37819.)

Constructors of int, float and complex will now use the index() special method, if available and the corresponding method int(), float, or complex() is not available. (Contributed by Serhiy Storchaka in bpo-20092.)

Added support of \N{name} escapes in regular expressions:

notice = 'Copyright © 2019'
copyright_year_pattern = re.compile(r'\N{copyright sign}\s*(\d{4})')
int(copyright_year_pattern.search(notice).group(1))
2019

(Contributed by Jonathan Eunice and Serhiy Storchaka in bpo-30688.)

Dict and dictviews are now iterable in reversed insertion order using
reversed().
(Contributed by Rémi Lapeyre in bpo-33462.)

The syntax allowed for keyword names in function calls was further restricted. In particular, f((keyword)=arg) is no longer allowed. It was never intended to permit more than a bare name on the left-hand side of a keyword argument assignment term. (Contributed by Benjamin Peterson in bpo-34641.)

Generalized iterable unpacking in
yield
and
return
statements no longer requires enclosing parentheses. This brings the yield and return syntax into better agreement with normal assignment syntax.

(Contributed by David Cuthbert and Jordan Chapman in bpo-32117.)

When a comma is missed in code such as [(10, 20)(30, 40)], the compiler displays a SyntaxWarning with a helpful suggestion. This improves on just having a TypeError indicating that the first tuple was not callable. (Contributed by Serhiy Storchaka in bpo-15248.)

Arithmetic operations between subclasses of datetime.date or datetime.datetime and datetime.timedelta objects now return an instance of the subclass, rather than the base class. This also affects the return type of operations whose implementation (directly or indirectly) uses datetime.timedelta arithmetic, such as astimezone(). (Contributed by Paul Ganssle in bpo-32417.)

When the Python interpreter is interrupted by Ctrl-C (SIGINT) and the resulting KeyboardInterrupt exception is not caught, the Python process now exits via a SIGINT signal or with the correct exit code such that the calling process can detect that it died due to a Ctrl-C. Shells on POSIX and Windows use this to properly terminate scripts in interactive sessions. (Contributed by Google via Gregory P. Smith in bpo-1054041.)

Some advanced styles of programming require updating the types.CodeType object for an existing function. Since code objects are immutable, a new code object needs to be created, one that is modeled on the existing code object. With 19 parameters, this was somewhat tedious. Now, the new replace() method makes it possible to create a clone with a few altered parameters.

(Contributed by Victor Stinner in bpo-37032.)

For integers, the three-argument form of the pow() function now permits the exponent to be negative in the case where the base is relatively prime to the modulus. It then computes a modular inverse to the base when the exponent is -1, and a suitable power of that inverse for other negative exponents.

(Contributed by Mark Dickinson in bpo-36027.)

Dict comprehensions have been synced-up with dict literals so that the key is computed first and the value second. The guaranteed execution order is helpful with assignment expressions because variables assigned in the key expression will be available in the value expression. (Contributed by Jörn Heissler in bpo-35224.)

The object.reduce() method can now return a tuple from two to six elements long. Formerly, five was the limit. The new, optional sixth element is a callable with a (obj, state) signature. This allows the direct control over the state-updating behavior of a specific object. If not None, this callable will have priority over the object’s setstate() method. (Contributed by Pierre Glaser and Olivier Grisel in bpo-35900.)

New Modules

The new importlib.metadata module provides (provisional) support for reading metadata from third-party packages. For example, it can extract an installed package’s version number, list of entry points, and more.

(Contributed by Barry Warsaw and Jason R. Coombs in bpo-34632.)

Improved Modules

ast

AST nodes now have end_lineno and end_col_offset attributes, which give the precise location of the end of the node. New function ast.get_source_segment() returns the source code for a specific AST node. (Contributed by Ivan Levkivskyi in bpo-33416.)

asyncio

asyncio.run() has graduated from the provisional to stable API. This function can be used to execute a coroutine and return the result while automatically managing the event loop. Running python -m asyncio launches a natively async REPL. The exception asyncio.CancelledError now inherits from BaseException rather than Exception. On Windows, the default event loop is now ProactorEventLoop. ProactorEventLoop now also supports UDP and can be interrupted by KeyboardInterrupt. Added asyncio.Task.get_coro(), task naming, Happy Eyeballs support, and more. (Contributed by Yury Selivanov and others.)

builtins

The compile() built-in has been improved to accept the ast.PyCF_ALLOW_TOP_LEVEL_AWAIT flag. With this new flag passed, compile() will allow top-level await, async for and async with constructs that are usually considered invalid syntax. (Contributed by Matthias Bussonnier.)

collections

The _asdict() method for collections.namedtuple() now returns a dict instead of a collections.OrderedDict. (Contributed by Raymond Hettinger in bpo-35864.)

cProfile

The cProfile.Profile class can now be used as a context manager. (Contributed by Scott Sanderson in bpo-29235.)

csv

The csv.DictReader now returns instances of dict instead of a collections.OrderedDict. The tool is now faster and uses less memory while still preserving the field order. (Contributed by Michael Selik in bpo-34003.)

ctypes

On Windows, CDLL and subclasses now accept a winmode parameter to specify flags for the underlying LoadLibraryEx call. The default flags are set to only load DLL dependencies from trusted locations. (Contributed by Steve Dower in bpo-36085.)

datetime

Added new alternate constructors datetime.date.fromisocalendar() and datetime.datetime.fromisocalendar(). (Contributed by Paul Ganssle in bpo-36004.)

functools

functools.lru_cache() can now be used as a straight decorator. Added functools.cached_property() and functools.singledispatchmethod(). (Contributed by Raymond Hettinger and others.)

gc

get_objects() can now receive an optional generation parameter. (Contributed by Pablo Galindo in bpo-36016.)

gettext

Added pgettext() and its variants. (Contributed by Franz Glasner, Éric Araujo, and Cheryl Sabella in bpo-2504.)

gzip

Added the mtime parameter to gzip.compress() for reproducible output. A BadGzipFile exception is now raised instead of OSError for certain invalid gzip files. (Contributed by Guo Ci Teo; Filip Gruszczyński et al.)

IDLE and idlelib

Multiple UI and behavior improvements; see details. New in 3.8.1: toggle cursor blink, Escape key closes completion windows. (Contributed by various contributors.)

inspect

inspect.getdoc() can now find docstrings for slots if that attribute is a dict where the values are docstrings. (Contributed by Raymond Hettinger in bpo-36326.)

io

In development mode and in debug build, the io.IOBase finalizer now logs the exception if close() fails. (Contributed by Victor Stinner in bpo-18748.)

itertools

itertools.accumulate() added an initial keyword argument. (Contributed by Lisa Roach in bpo-34659.)

json.tool

Add option --json-lines to parse every input line as a separate JSON object. (Contributed by Weipeng Hong in bpo-31553.)

logging

Added a force keyword argument to logging.basicConfig(). (Suggested by Raymond Hettinger, implemented by Donghee Na.)

math

Added math.dist(), expanded math.hypot() to multiple dimensions, added math.prod(), math.perm(), math.comb(), math.isqrt(), and stricter argument types for math.factorial(). (Contributed by various contributors.)

mmap

mmap.mmap class now has a madvise() method. (Contributed by Zackery Spytz in bpo-32941.)

multiprocessing

Added new multiprocessing.shared_memory module. On macOS, spawn start method is now default. (Contributed by Davin Potts and Victor Stinner.)

os

Added add_dll_directory() on Windows, os.memfd_create(), improved reparse point handling on Windows, os.readlink() can read directory junctions. (Contributed by Steve Dower and others.)

os.path / pathlib

Functions that return boolean results now return False instead of raising ValueError or Unicode errors for unrepresentable paths. expanduser() on Windows prefers USERPROFILE. pathlib.Path.link_to() added. (Contributed by Serhiy Storchaka et al.)

pickle

pickle extensions subclassing the C-optimized Pickler can now override reducer_override(). (Contributed by Pierre Glaser and Olivier Grisel.)

plistlib

Added plistlib.UID and NSKeyedArchiver support. (Contributed by Jon Janzen.)

pprint

Added sort_dicts parameter and pprint.pp() convenience function. (Contributed by Rémi Lapeyre.)

py_compile

py_compile.compile() now supports silent mode. (Contributed by Joannah Nanjekye.)

shlex

Added shlex.join(). (Contributed by Bo Bayles.)

shutil

shutil.copytree() now accepts dirs_exist_ok; shutil.make_archive() defaults to pax; rmtree on Windows removes directory junctions without recursing. (Contributed by various.)

socket

Added create_server() and has_dualstack_ipv6(), implemented if_* functions on Windows. (Contributed by Giampaolo Rodolà and Zackery Spytz.)

ssl

Added post_handshake_auth and verify_client_post_handshake() for TLS 1.3. (Contributed by Christian Heimes.)

statistics

Added statistics.fmean(), geometric_mean(), multimode(), quantiles(), NormalDist, and examples. (Contributed by Raymond Hettinger and others.)

sys

Add new sys.unraisablehook() function to control handling of "unraisable exceptions". (Contributed by Victor Stinner.)

tarfile

tarfile module now defaults to pax format for new archives. (Contributed by C.A.M. Gerlach.)

threading

Add threading.excepthook(), threading.get_native_id(), and native_id attribute. (Contributed by Victor Stinner and Jake Tesler.)

tokenize

tokenize now implicitly emits a NEWLINE token when input lacks a trailing newline. (Contributed by Ammar Askar.)

tkinter

Added selection_* and moveto(), transparency_get/set, and other improvements. (Contributed by various.)

time

Added CLOCK_UPTIME_RAW for macOS 10.12. (Contributed by Joannah Nanjekye.)

typing

Multiple typing enhancements: TypedDict (PEP 589), Literal types (PEP 586), Final (PEP 591), Protocols (PEP 544), typing.SupportsIndex, typing.get_origin(), typing.get_args(), and more.

unicodedata

Upgraded to Unicode 12.1.0 and added is_normalized(). (Contributed by Max Belanger et al.)

unittest

Added AsyncMock, addModuleCleanup, addClassCleanup, support for coroutines as test cases (IsolatedAsyncioTestCase), and examples. (Contributed by Lisa Roach and others.)

venv

venv now includes an Activate.ps1 script on all platforms for PowerShell Core. (Contributed by Brett Cannon.)

weakref

weakref.proxy() now supports matrix multiplication operators @ and @=. (Contributed by Mark Dickinson.)

xml / xmlrpc

Security mitigations: xml.dom.minidom and xml.sax no longer process external entities by default; ElementTree improvements and canonicalize() added; xmlrpc.client.ServerProxy supports headers argument.

Optimizations

Multiple performance improvements across subprocess, shutil, pickle, UUID memory footprint, operator.itemgetter(), collections.namedtuple lookups, list constructor allocation, class variable writes, builtin call overhead reductions, and LOAD_GLOBAL opcode caching (~40% faster).

Build and C API Changes

Numerous changes: sys.abiflags default empty, header reorganization, macros converted to inline functions, PyByteArray_Init/Fini removed, integer conversion now prefers index(), PyObject_Init() reference changes for heap-allocated types, new PyCode_NewWithPosOnlyArgs(), Py_SetPath() sets sys.executable to full path, and many more.

Deprecated

Multiple deprecations including distutils bdist_wininst, ElementTree getchildren/getiterator, passing non-instance to loop.set_default_executor(), deprecated getitem implementations, typing.NamedTuple._field_types, ast Num/Str/Bytes/NameConstant/Ellipsis classes, @asyncio.coroutine decorator, explicit loop parameter in many asyncio APIs, and others.

API and Feature Removals

Several removals: macpath, platform.popen(), time.clock(), pyvenv script, cgi functions moved, tarfile.filemode removed, XMLParser html argument removed, unicode_internal codec removed, sqlite3 Cache/Statement objects not exposed, fileinput.bufsize removed, sys.set_coroutine_wrapper/get_coroutine_wrapper removed, and others.

Porting to Python 3.8

Guidance on changes in Python behavior, Python API, C API, CPython bytecode changes, demos and tools, and notable changes in maintenance releases 3.8.1 through 3.8.17 (security fixes, macOS Big Sur and Apple Silicon support backports, urllib.parse safety fixes, tarfile extraction filters, integer conversion limits for CVE mitigation, etc.).

Last updated on Mar 17, 2026 (09:57 UTC).

Original source Report a problem

This article explains the new features in Python 3.6, compared to 3.5. Python 3.6 was released on December 23, 2016. See the changelog for a full list of changes.

See also
PEP 494 - Python 3.6 Release Schedule

Summary – Release highlights
New syntax features:
• PEP 498, formatted string literals.
• PEP 515, underscores in numeric literals.
• PEP 526, syntax for variable annotations.
• PEP 525, asynchronous generators.
• PEP 530: asynchronous comprehensions.

New library modules:
• secrets: PEP 506 – Adding A Secrets Module To The Standard Library.

CPython implementation improvements:
• The dict type has been reimplemented to use a more compact representation based on a proposal by Raymond Hettinger and similar to the PyPy dict implementation. This resulted in dictionaries using 20% to 25% less memory when compared to Python 3.5.
• Customization of class creation has been simplified with the new protocol.
• The class attribute definition order is now preserved.
• The order of elements in **kwargs now corresponds to the order in which keyword arguments were passed to the function.
• DTrace and SystemTap probing support has been added.
• The new PYTHONMALLOC environment variable can now be used to debug the interpreter memory allocation and access errors.

Significant improvements in the standard library:
• The asyncio module has received new features, significant usability and performance improvements, and a fair amount of bug fixes. Starting with Python 3.6 the asyncio module is no longer provisional and its API is considered stable.
• A new file system path protocol has been implemented to support path-like objects. All standard library functions operating on paths have been updated to work with the new protocol.
• The datetime module has gained support for Local Time Disambiguation.
• The typing module received a number of improvements.
• The tracemalloc module has been significantly reworked and is now used to provide better output for ResourceWarning as well as provide better diagnostics for memory allocation errors. See the PYTHONMALLOC section for more information.

Security improvements:
• The new secrets module has been added to simplify the generation of cryptographically strong pseudo-random numbers suitable for managing secrets such as account authentication, tokens, and similar.
• On Linux, os.urandom() now blocks until the system urandom entropy pool is initialized to increase the security. See the PEP 524 for the rationale.
• The hashlib and ssl modules now support OpenSSL 1.1.0.
• The default settings and feature set of the ssl module have been improved.
• The hashlib module received support for the BLAKE2, SHA-3 and SHAKE hash algorithms and the scrypt() key derivation function.

Windows improvements:
• PEP 528 and PEP 529, Windows filesystem and console encoding changed to UTF-8.
• The py.exe launcher, when used interactively, no longer prefers Python 2 over Python 3 when the user doesn’t specify a version (via command line arguments or a config file). Handling of shebang lines remains unchanged - “python” refers to Python 2 in that case.
• python.exe and pythonw.exe have been marked as long-path aware, which means that the 260 character path limit may no longer apply. See removing the MAX_PATH limitation for details.
• A ._pth file can be added to force isolated mode and fully specify all search paths to avoid registry and environment lookup. See the documentation for more information.
• A python36.zip file now works as a landmark to infer PYTHONHOME. See the documentation for more information.

PEP 498: Formatted string literals
PEP 498 introduces a new kind of string literals: f-strings, or formatted string literals.
Formatted string literals are prefixed with 'f' and are similar to the format strings accepted by str.format(). They contain replacement fields surrounded by curly braces. The replacement fields are expressions, which are evaluated at run time, and then formatted using the format() protocol:

name = "Fred"

f"He said his name is {name}."
'He said his name is Fred.'

width = 10

precision = 4

value = decimal.Decimal("12.34567")

f"result: {value:{width}.{precision}}" # nested fields
'result: 12.35'

See also
PEP 498 – Literal String Interpolation.
PEP written and implemented by Eric V. Smith.
Feature documentation.

PEP 526: Syntax for variable annotations
PEP 484 introduced the standard for type annotations of function parameters, a.k.a. type hints. This PEP adds syntax to Python for annotating the types of variables including class variables and instance variables:
primes:
List[int]

[]
captain:
str

Note: no initial value!

class
Starship:
stats:
Dict[str, int]

{}
Just as for function annotations, the Python interpreter does not attach any particular meaning to variable annotations and only stores them in the annotations attribute of a class or module.
In contrast to variable declarations in statically typed languages, the goal of annotation syntax is to provide an easy way to specify structured type metadata for third party tools and libraries via the abstract syntax tree and the annotations attribute.

See also
PEP 526 – Syntax for variable annotations.
PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa Roach, and Guido van Rossum. Implemented by Ivan Levkivskyi.
Tools that use or will use the new syntax: mypy, pytype, PyCharm, etc.

PEP 515: Underscores in Numeric Literals
PEP 515 adds the ability to use underscores in numeric literals for improved readability. For example:

1_000_000_000_000_000
1000000000000000

0x_FF_FF_FF_FF
4294967295
Single underscores are allowed between digits and after any base specifier. Leading, trailing, or multiple underscores in a row are not allowed.
The string formatting language also now has support for the '_' option to signal the use of an underscore for a thousands separator for floating-point presentation types and for integer presentation type 'd'. For integer presentation types 'b', 'o', 'x', and 'X', underscores will be inserted every 4 digits:

'{:_}'.format(1000000)
'1_000_000'

'{:_x}'.format(0xFFFFFFFF)
'ffff_ffff'

See also
PEP 515 – Underscores in Numeric Literals
PEP written by Georg Brandl and Serhiy Storchaka.

PEP 525: Asynchronous Generators
PEP 492 introduced support for native coroutines and async/await syntax to Python 3.5. A notable limitation of the Python 3.5 implementation is that it was not possible to use await and yield in the same function body. In Python 3.6 this restriction has been lifted, making it possible to define asynchronous generators:
async def ticker(delay, to):
"""Yield numbers from 0 to to every delay seconds."""
for i in range(to):
yield i
await asyncio.sleep(delay)
The new syntax allows for faster and more concise code.

See also
PEP 525 – Asynchronous Generators
PEP written and implemented by Yury Selivanov.

PEP 530: Asynchronous Comprehensions
PEP 530 adds support for using async for in list, set, dict comprehensions and generator expressions:
result = [i async for i in aiter() if i % 2]
Additionally, await expressions are supported in all kinds of comprehensions:
result = [await fun() for fun in funcs if await condition()]

See also
PEP 530 – Asynchronous Comprehensions
PEP written and implemented by Yury Selivanov.

PEP 487: Simpler customization of class creation
It is now possible to customize subclass creation without using a metaclass. The new init_subclass classmethod will be called on the base class whenever a new subclass is created:
class PluginBase:
subclasses = []
def init_subclass(cls, **kwargs):
super().init_subclass(**kwargs)
cls.subclasses.append(cls)
class Plugin1(PluginBase):
pass
class Plugin2(PluginBase):
pass
In order to allow zero-argument super() calls to work correctly from init_subclass() implementations, custom metaclasses must ensure that the new classcell namespace entry is propagated to type.new (as described in Creating the class object).

See also
PEP 487 – Simpler customization of class creation
PEP written and implemented by Martin Teichmann.
Feature documentation.

PEP 487: Descriptor Protocol Enhancements
PEP 487 extends the descriptor protocol to include the new optional set_name() method. Whenever a new class is defined, the new method will be called on all descriptors included in the definition, providing them with a reference to the class being defined and the name given to the descriptor within the class namespace. In other words, instances of descriptors can now know the attribute name of the descriptor in the owner class:
class IntField:
def get(self, instance, owner):
return instance.dict[self.name]
def set(self, instance, value):
if not isinstance(value, int):
raise ValueError(f'expecting integer in {self.name}')
instance.dict[self.name] = value

this is the new initializer:

def set_name(self, owner, name):
self.name = name
class Model:
int_field = IntField()

See also
PEP 487 – Simpler customization of class creation
PEP written and implemented by Martin Teichmann.
Feature documentation.

PEP 519: Adding a file system path protocol
File system paths have historically been represented as str or bytes objects. This has led to people who write code which operate on file system paths to assume that such objects are only one of those two types (an int representing a file descriptor does not count as that is not a file path). Unfortunately that assumption prevents alternative object representations of file system paths like pathlib from working with pre-existing code, including Python’s standard library.
To fix this situation, a new interface represented by os.PathLike has been defined. By implementing the fspath() method, an object signals that it represents a path. An object can then provide a low-level representation of a file system path as a str or bytes object. This means an object is considered path-like if it implements os.PathLike or is a str or bytes object which represents a file system path. Code can use os.fspath(), os.fsdecode(), or os.fsencode() to explicitly get a str and/or bytes representation of a path-like object.
The built-in open() function has been updated to accept os.PathLike objects, as have all relevant functions in the os and os.path modules, and most other functions and classes in the standard library. The os.DirEntry class and relevant classes in pathlib have also been updated to implement os.PathLike.
The hope is that updating the fundamental functions for operating on file system paths will lead to third-party code to implicitly support all path-like objects without any code changes, or at least very minimal ones (e.g. calling os.fspath() at the beginning of code before operating on a path-like object).
Here are some examples of how the new interface allows for pathlib.Path to be used more easily and transparently with pre-existing code:

import pathlib

with open(pathlib.Path("README")) as f:
...
contents = f.read()
...

import os.path

os.path.splitext(pathlib.Path("some_file.txt"))
('some_file', '.txt')

os.path.join("/a/b", pathlib.Path("c"))
'/a/b/c'

import os

os.fspath(pathlib.Path("some_file.txt"))
'some_file.txt'
(Implemented by Brett Cannon, Ethan Furman, Dusty Phillips, and Jelle Zijlstra.)

See also
PEP 519 – Adding a file system path protocol
PEP written by Brett Cannon and Koos Zevenhoven.

PEP 495: Local Time Disambiguation
In most world locations, there have been and will be times when local clocks are moved back. In those times, intervals are introduced in which local clocks show the same time twice in the same day. In these situations, the information displayed on a local clock (or stored in a Python datetime instance) is insufficient to identify a particular moment in time.
PEP 495 adds the new fold attribute to instances of datetime.datetime and datetime.time classes to differentiate between two moments in time for which local times are the same:

u0 = datetime(2016, 11, 6, 4, tzinfo=timezone.utc)

for i in range(4):
...
u = u0 + i * HOUR
...
t = u.astimezone(Eastern)
...
print(u.time(), 'UTC =', t.time(), t.tzname(), t.fold)
...
04:00:00 UTC = 00:00:00 EDT 0
05:00:00 UTC = 01:00:00 EDT 0
06:00:00 UTC = 01:00:00 EST 1
07:00:00 UTC = 02:00:00 EST 0
The values of the fold attribute have the value 0 for all instances except those that represent the second (chronologically) moment in time in an ambiguous case.

See also
PEP 495 – Local Time Disambiguation
PEP written by Alexander Belopolsky and Tim Peters, implementation by Alexander Belopolsky.

PEP 529: Change Windows filesystem encoding to UTF-8
Representing filesystem paths is best performed with str (Unicode) rather than bytes. However, there are some situations where using bytes is sufficient and correct.
Prior to Python 3.6, data loss could result when using bytes paths on Windows. With this change, using bytes to represent paths is now supported on Windows, provided those bytes are encoded with the encoding returned by sys.getfilesystemencoding(), which now defaults to 'utf-8'.
Applications that do not use str to represent paths should use os.fsencode() and os.fsdecode() to ensure their bytes are correctly encoded. To revert to the previous behaviour, set PYTHONLEGACYWINDOWSFSENCODING or call sys._enablelegacywindowsfsencoding().
See PEP 529 for more information and discussion of code modifications that may be required.

PEP 528: Change Windows console encoding to UTF-8
The default console on Windows will now accept all Unicode characters and provide correctly read str objects to Python code. sys.stdin, sys.stdout, and sys.stderr now default to utf-8 encoding.
This change only applies when using an interactive console, and not when redirecting files or pipes. To revert to the previous behaviour for interactive console use, set PYTHONLEGACYWINDOWSSTDIO.

See also
PEP 528 – Change Windows console encoding to UTF-8
PEP written and implemented by Steve Dower.

PEP 520: Preserving Class Attribute Definition Order
Attributes in a class definition body have a natural ordering: the same order in which the names appear in the source. This order is now preserved in the new class’s dict attribute.
Also, the effective default class execution namespace (returned from type.prepare() ) is now an insertion-order-preserving mapping.

See also
PEP 520 – Preserving Class Attribute Definition Order
PEP written and implemented by Eric Snow.

PEP 468: Preserving Keyword Argument Order
**kwargs in a function signature is now guaranteed to be an insertion-order-preserving mapping.

See also
PEP 468 – Preserving Keyword Argument Order
PEP written and implemented by Eric Snow.

New dict implementation
New dict implementation
The dict type now uses a “compact” representation based on a proposal by Raymond Hettinger which was first implemented by PyPy. The memory usage of the new dict() is between 20% and 25% smaller compared to Python 3.5.
The order-preserving aspect of this new implementation is considered an implementation detail and should not be relied upon (this may change in the future, but it is desired to have this new dict implementation in the language for a few releases before changing the language spec to mandate order-preserving semantics for all current and future Python implementations; this also helps preserve backwards-compatibility with older versions of the language where random iteration order is still in effect, e.g. Python 3.5).
(Contributed by INADA Naoki in bpo-27350. Idea originally suggested by Raymond Hettinger.)

PEP 523: Adding a frame evaluation API to CPython
While Python provides extensive support to customize how code executes, one place it has not done so is in the evaluation of frame objects. If you wanted some way to intercept frame evaluation in Python there really wasn’t any way without directly manipulating function pointers for defined functions.
PEP 523 changes this by providing an API to make frame evaluation pluggable at the C level. This will allow for tools such as debuggers and JITs to intercept frame evaluation before the execution of Python code begins. This enables the use of alternative evaluation implementations for Python code, tracking frame evaluation, etc.
This API is not part of the limited C API and is marked as private to signal that usage of this API is expected to be limited and only applicable to very select, low-level use-cases. Semantics of the API will change with Python as necessary.

See also
PEP 523 – Adding a frame evaluation API to CPython
PEP written by Brett Cannon and Dino Viehland.

PYTHONMALLOC environment variable
The new PYTHONMALLOC environment variable allows setting the Python memory allocators and installing debug hooks.
It is now possible to install debug hooks on Python memory allocators on Python compiled in release mode using PYTHONMALLOC=debug. Effects of debug hooks:
• Newly allocated memory is filled with the byte 0xCB
• Freed memory is filled with the byte 0xDB
• Detect violations of the Python memory allocator API. For example, PyObject_Free() called on a memory block allocated by PyMem_Malloc().
• Detect writes before the start of a buffer (buffer underflows)
• Detect writes after the end of a buffer (buffer overflows)
• Check that the GIL is held when allocator functions of PYMEM_DOMAIN_OBJ (ex: PyObject_Malloc() ) and PYMEM_DOMAIN_MEM (ex: PyMem_Malloc() ) domains are called.
Checking if the GIL is held is also a new feature of Python 3.6.
See the PyMem_SetupDebugHooks() function for debug hooks on Python memory allocators.
It is now also possible to force the usage of the malloc() allocator of the C library for all Python memory allocations using PYTHONMALLOC=malloc. This is helpful when using external memory debuggers like Valgrind on a Python compiled in release mode.
On error, the debug hooks on Python memory allocators now use the tracemalloc module to get the traceback where a memory block was allocated.
Example of fatal error on buffer overflow using python3.6 -X tracemalloc=5 (store 5 frames in traces):
Debug memory block at address p = 0x7fbcd41666f8: API 'o' 4 bytes originally requested The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected. The 8 pad bytes at tail = 0x7fbcd41666fc are not all FORBIDDENBYTE (0xfb): at tail+0: 0x02 *** OUCH at tail+1: 0xfb at tail+2: 0xfb at tail+3: 0xfb at tail+4: 0xfb at tail+5: 0xfb at tail+6: 0xfb at tail+7: 0xfb The block was made by call #1233329 to debug malloc/realloc. Data at p: 1 a 2 b 30 00 Memory block allocated at (most recent call first): File "test/test_bytes.py", line 323 File "unittest/case.py", line 600 File "unittest/case.py", line 648 File "unittest/suite.py", line 122 File "unittest/suite.py", line 84 Fatal Python error : bad trailing pad byte Current thread 0x00007fbcdbd32700 (most recent call first): File "test/test_bytes.py", line 323 in test_hex File "unittest/case.py", line 600 in run File "unittest/case.py", line 648 in call File "unittest/suite.py", line 122 in run File "unittest/suite.py", line 84 in call ...
(Contributed by Victor Stinner in bpo-26516 and bpo-26564.)

DTrace and SystemTap probing support
Python can now be built --with-dtrace which enables static markers for the following events in the interpreter:
• function call/return
• garbage collection started/finished
• line of code executed.
This can be used to instrument running interpreters in production, without the need to recompile specific debug builds or providing application-specific profiling/debugging code.
More details in Instrumenting CPython with DTrace and SystemTap.
The current implementation is tested on Linux and macOS. Additional markers may be added in the future.
(Contributed by Łukasz Langa in bpo-21590, based on patches by Jesús Cea Avión, David Malcolm, and Nikhil Benesch.)

Other Language Changes
Some smaller changes made to the core Python language are:
• A global or nonlocal statement must now textually appear before the first use of the affected name in the same scope. Previously this was a SyntaxWarning.
• It is now possible to set a special method to None to indicate that the corresponding operation is not available. For example, if a class sets iter() to None, the class is not iterable. (Contributed by Andrew Barnert and Ivan Levkivskyi in bpo-25958.)
• Long sequences of repeated traceback lines are now abbreviated as "[Previous line repeated {count} more times]" (see traceback for an example). (Contributed by Emanuel Barry in bpo-26823.)
• Import now raises the new exception ModuleNotFoundError (subclass of ImportError) when it cannot find a module. Code that currently checks for ImportError (in try-except) will still work. (Contributed by Eric Snow in bpo-15767.)
• Class methods relying on zero-argument super() will now work correctly when called from metaclass methods during class creation. (Contributed by Martin Teichmann in bpo-23722.)

New Modules
secrets
The main purpose of the new secrets module is to provide an obvious way to reliably generate cryptographically strong pseudo-random values suitable for managing secrets, such as account authentication, tokens, and similar.
Warning
Note that the pseudo-random generators in the random module should NOT be used for security purposes. Use secrets on Python 3.6+ and os.urandom() on Python 3.5 and earlier.

See also
PEP 506 – Adding A Secrets Module To The Standard Library
PEP written and implemented by Steven D’Aprano.

Improved Modules
array
Exhausted iterators of array.array will now stay exhausted even if the iterated array is extended. This is consistent with the behavior of other mutable sequences.
Contributed by Serhiy Storchaka in bpo-26492.

ast
The new ast.Constant AST node has been added. It can be used by external AST optimizers for the purposes of constant folding.
Contributed by Victor Stinner in bpo-26146.

asyncio
Starting with Python 3.6 the asyncio module is no longer provisional and its API is considered stable.
Notable changes in the asyncio module since Python 3.5.0 (all backported to 3.5.x due to the provisional status):
• The get_event_loop() function has been changed to always return the currently running loop when called from coroutines and callbacks. (Contributed by Yury Selivanov in bpo-28613.)
• The ensure_future() function and all functions that use it, such as loop.run_until_complete(), now accept all kinds of awaitable objects. (Contributed by Yury Selivanov.)
• New run_coroutine_threadsafe() function to submit coroutines to event loops from other threads. (Contributed by Vincent Michel.)
• New Transport.is_closing() method to check if the transport is closing or closed. (Contributed by Yury Selivanov.)
• The loop.create_server() method can now accept a list of hosts. (Contributed by Yann Sionneau.)
• New loop.create_future() method to create Future objects. This allows alternative event loop implementations, such as uvloop, to provide a faster asyncio.Future implementation. (Contributed by Yury Selivanov in bpo-27041.)
• New loop.get_exception_handler() method to get the current exception handler. (Contributed by Yury Selivanov in bpo-27040.)
• New StreamReader.readuntil() method to read data from the stream until a separator bytes sequence appears. (Contributed by Mark Korenberg.)
• The performance of StreamReader.readexactly() has been improved. (Contributed by Mark Korenberg in bpo-28370.)
• The loop.getaddrinfo() method is optimized to avoid calling the system getaddrinfo function if the address is already resolved. (Contributed by A. Jesse Jiryu Davis.)
• The loop.stop() method has been changed to stop the loop immediately after the current iteration. Any new callbacks scheduled as a result of the last iteration will be discarded. (Contributed by Guido van Rossum in bpo-25593.)
• Future.set_exception will now raise TypeError when passed an instance of the StopIteration exception. (Contributed by Chris Angelico in bpo-26221.)
• New loop.connect_accepted_socket() method to be used by servers that accept connections outside of asyncio, but that use asyncio to handle them. (Contributed by Jim Fulton in bpo-27392.)
• TCP_NODELAY flag is now set for all TCP transports by default. (Contributed by Yury Selivanov in bpo-27456.)
• New loop.shutdown_asyncgens() to properly close pending asynchronous generators before closing the loop. (Contributed by Yury Selivanov in bpo-28003.)
• Future and Task classes now have an optimized C implementation which makes asyncio code up to 30% faster. (Contributed by Yury Selivanov and INADA Naoki in bpo-26081 and bpo-28544.)

binascii
The b2a_base64() function now accepts an optional newline keyword argument to control whether the newline character is appended to the return value. (Contributed by Victor Stinner in bpo-25357.)

cmath
The new cmath.tau (τ) constant has been added. (Contributed by Lisa Roach in bpo-12345, see PEP 628 for details.)
New constants: cmath.inf and cmath.nan to match math.inf and math.nan, and also cmath.infj and cmath.nanj to match the format used by complex repr. (Contributed by Mark Dickinson in bpo-23229.)

collections
The new Collection abstract base class has been added to represent sized iterable container classes. (Contributed by Ivan Levkivskyi, docs by Neil Girdhar in bpo-27598.)
The new Reversible abstract base class represents iterable classes that also provide the reversed() method. (Contributed by Ivan Levkivskyi in bpo-25987.)
The new AsyncGenerator abstract base class represents asynchronous generators. (Contributed by Yury Selivanov in bpo-28720.)
The namedtuple() function now accepts an optional keyword argument module , which, when specified, is used for the module attribute of the returned named tuple class. (Contributed by Raymond Hettinger in bpo-17941.)
The verbose and rename arguments for namedtuple() are now keyword-only. (Contributed by Raymond Hettinger in bpo-25628.)
Recursive collections.deque instances can now be pickled. (Contributed by Serhiy Storchaka in bpo-26482.)

concurrent.futures
The ThreadPoolExecutor class constructor now accepts an optional thread_name_prefix argument to make it possible to customize the names of the threads created by the pool. (Contributed by Gregory P. Smith in bpo-27664.)

contextlib
The contextlib.AbstractContextManager class has been added to provide an abstract base class for context managers. It provides a sensible default implementation for enter() which returns self and leaves exit() an abstract method. A matching class has been added to the typing module as typing.ContextManager . (Contributed by Brett Cannon in bpo-25609.)

datetime
The datetime and time classes have the new fold attribute used to disambiguate local time when necessary. Many functions in the datetime have been updated to support local time disambiguation. See Local Time Disambiguation section for more information. (Contributed by Alexander Belopolsky in bpo-24773.)
The datetime.strftime() and date.strftime() methods now support ISO 8601 date directives %G , %u and %V . (Contributed by Ashley Anderson in bpo-12006.)
The datetime.isoformat() function now accepts an optional timespec argument that specifies the number of additional components of the time value to include. (Contributed by Alessandro Cucci and Alexander Belopolsky in bpo-19475.)
The datetime.combine() now accepts an optional tzinfo argument. (Contributed by Alexander Belopolsky in bpo-27661.)

decimal
New Decimal.as_integer_ratio() method that returns a pair (n, d) of integers that represent the given Decimal instance as a fraction, in lowest terms and with a positive denominator:

Decimal('-3.14').as_integer_ratio()
(-157, 50)
(Contributed by Stefan Krah amd Mark Dickinson in bpo-25928.)

distutils
The default_format attribute has been removed from distutils.command.sdist.sdist and the formats attribute defaults to ['gztar'] . Although not anticipated, any code relying on the presence of default_format may need to be adapted. See bpo-27819 for more details.

email
The new email API, enabled via the policy keyword to various constructors, is no longer provisional. The email documentation has been reorganized and rewritten to focus on the new API, while retaining the old documentation for the legacy API. (Contributed by R. David Murray in bpo-24277.)
The email.mime classes now all accept an optional policy keyword. (Contributed by Berker Peksag in bpo-27331.)
The DecodedGenerator now supports the policy keyword.
There is a new policy attribute, message_factory , that controls what class is used by default when the parser creates new message objects. For the email.policy.compat32 policy this is Message , for the new policies it is EmailMessage . (Contributed by R. David Murray in bpo-20476.)

encodings
On Windows, added the 'oem' encoding to use CP_OEMCP , and the 'ansi' alias for the existing 'mbcs' encoding, which uses the CP_ACP code page. (Contributed by Steve Dower in bpo-27959.)

enum
Two new enumeration base classes have been added to the enum module: Flag and IntFlag . Both are used to define constants that can be combined using the bitwise operators. (Contributed by Ethan Furman in bpo-23591.)
Many standard library modules have been updated to use the IntFlag class for their constants.
The new enum.auto value can be used to assign values to enum members automatically:

from enum import Enum, auto

class Color(Enum):
...
red = auto()
...
blue = auto()
...
green = auto()

list(Color)
[<Color.red: 1>, <Color.blue: 2>, <Color.green: 3>]

faulthandler
On Windows, the faulthandler module now installs a handler for Windows exceptions: see faulthandler.enable() . (Contributed by Victor Stinner in bpo-23848.)

fileinput
hook_encoded() now supports the errors argument. (Contributed by Joseph Hackman in bpo-25788.)

hashlib
hashlib supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2. (Contributed by Christian Heimes in bpo-26470.)
BLAKE2 hash functions were added to the module. blake2b() and blake2s() are always available and support the full feature set of BLAKE2. (Contributed by Christian Heimes in bpo-26798 based on code by Dmitry Chestnykh and Samuel Neves. Documentation written by Dmitry Chestnykh.)
The SHA-3 hash functions sha3_224() , sha3_256() , sha3_384() , sha3_512() , and SHAKE hash functions shake_128() and shake_256() were added. (Contributed by Christian Heimes in bpo-16113. Keccak Code Package by Guido Bertoni, Joan Daemen, Michaël Peeters, Gilles Van Assche, and Ronny Van Keer.)
The password-based key derivation function scrypt() is now available with OpenSSL 1.1.0 and newer. (Contributed by Christian Heimes in bpo-27928.)

http.client
The HTTPConnection.request() and endheaders() both now support chunked encoding request bodies. (Contributed by Demian Brecht and Rolf Krahl in bpo-12319.)

idlelib and IDLE
The idlelib package is being modernized and refactored to make IDLE look and work better and to make the code easier to understand, test, and improve. Part of making IDLE look better, especially on Linux and Mac, is using ttk widgets, mostly in the dialogs. As a result, IDLE no longer runs with tcl/tk 8.4. It now requires tcl/tk 8.5 or 8.6. We recommend running the latest release of either.
‘Modernizing’ includes renaming and consolidation of idlelib modules. The renaming of files with partial uppercase names is similar to the renaming of, for instance, Tkinter and TkFont to tkinter and tkinter.font in 3.0. As a result, imports of idlelib files that worked in 3.5 will usually not work in 3.6. At least a module name change will be needed (see idlelib/README.txt), sometimes more. (Name changes contributed by Al Swiegart and Terry Reedy in bpo-24225. Most idlelib patches since have been and will be part of the process.)
In compensation, the eventual result with be that some idlelib classes will be easier to use, with better APIs and docstrings explaining them. Additional useful information will be added to idlelib when available.
New in 3.6.2:
Multiple fixes for autocompletion. (Contributed by Louie Lu in bpo-15786.)
New in 3.6.3:
Module Browser (on the File menu, formerly called Class Browser), now displays nested functions and classes in addition to top-level functions and classes. (Contributed by Guilherme Polo, Cheryl Sabella, and Terry Jan Reedy in bpo-1612262.)
The IDLE features formerly implemented as extensions have been reimplemented as normal features. Their settings have been moved from the Extensions tab to other dialog tabs. (Contributed by Charles Wohlganger and Terry Jan Reedy in bpo-27099.)
The Settings dialog (Options, Configure IDLE) has been partly rewritten to improve both appearance and function. (Contributed by Cheryl Sabella and Terry Jan Reedy in multiple issues.)
New in 3.6.4:
The font sample now includes a selection of non-Latin characters so that users can better see the effect of selecting a particular font. (Contributed by Terry Jan Reedy in bpo-13802.) The sample can be edited to include other characters. (Contributed by Serhiy Storchaka in bpo-31860.)
New in 3.6.6:
Editor code context option revised. Box displays all context lines up to maxlines. Clicking on a context line jumps the editor to that line. Context colors for custom themes is added to Highlights tab of Settings dialog. (Contributed by Cheryl Sabella and Terry Jan Reedy in bpo-33642, bpo-33768, and bpo-33679.)
On Windows, a new API call tells Windows that tk scales for DPI. On Windows 8.1+ or 10, with DPI compatibility properties of the Python binary unchanged, and a monitor resolution greater than 96 DPI, this should make text and lines sharper. It should otherwise have no effect. (Contributed by Terry Jan Reedy in bpo-33656.)
New in 3.6.7:
Output over N lines (50 by default) is squeezed down to a button. N can be changed in the PyShell section of the General page of the Settings dialog. Fewer, but possibly extra long, lines can be squeezed by right clicking on the output. Squeezed output can be expanded in place by double-clicking the button or into the clipboard or a separate window by right-clicking the button. (Contributed by Tal Einat in bpo-1529353.)

importlib
Import now raises the new exception ModuleNotFoundError (subclass of ImportError) when it cannot find a module. Code that current checks for ImportError (in try-except) will still work. (Contributed by Eric Snow in bpo-15767.)
importlib.util.LazyLoader now calls create_module() on the wrapped loader, removing the restriction that importlib.machinery.BuiltinImporter and importlib.machinery.ExtensionFileLoader couldn’t be used with importlib.util.LazyLoader .
importlib.util.cache_from_source() , importlib.util.source_from_cache() , and importlib.util.spec_from_file_location() now accept a path-like object.

inspect
The inspect.signature() function now reports the implicit .0 parameters generated by the compiler for comprehension and generator expression scopes as if they were positional-only parameters called implicit0 . (Contributed by Jelle Zijlstra in bpo-19611.)
To reduce code churn when upgrading from Python 2.7 and the legacy inspect.getargspec() API, the previously documented deprecation of inspect.getfullargspec() has been reversed. While this function is convenient for single/source Python 2/3 code bases, the richer inspect.signature() interface remains the recommended approach for new code. (Contributed by Nick Coghlan in bpo-27172)

json
json.load() and json.loads() now support binary input. Encoded JSON should be represented using either UTF-8, UTF-16, or UTF-32. (Contributed by Serhiy Storchaka in bpo-17909.)

logging
The new WatchedFileHandler.reopenIfNeeded() method has been added to add the ability to check if the log file needs to be reopened. (Contributed by Marian Horban in bpo-24884.)

math
The tau (τ) constant has been added to the math and cmath modules. (Contributed by Lisa Roach in bpo-12345, see PEP 628 for details.)

multiprocessing
Proxy Objects returned by multiprocessing.Manager() can now be nested. (Contributed by Davin Potts in bpo-6766.)

os
See the summary of PEP 519 for details on how the os and os.path modules now support path-like objects.
scandir() now supports bytes paths on Windows.
A new close() method allows explicitly closing a scandir() iterator. The scandir() iterator now supports the context manager protocol. If a scandir() iterator is neither exhausted nor explicitly closed a ResourceWarning will be emitted in its destructor. (Contributed by Serhiy Storchaka in bpo-25994.)
On Linux, os.urandom() now blocks until the system urandom entropy pool is initialized to increase the security. See the PEP 524 for the rationale.
The Linux getrandom() syscall (get random bytes) is now exposed as the new os.getrandom() function. (Contributed by Victor Stinner, part of the PEP 524)

pathlib
pathlib now supports path-like objects. (Contributed by Brett Cannon in bpo-27186.)
See the summary of PEP 519 for details.

pdb
The Pdb class constructor has a new optional readrc argument to control whether .pdbrc files should be read.

pickle
Objects that need new called with keyword arguments can now be pickled using pickle protocols older than protocol version 4. Protocol version 4 already supports this case. (Contributed by Serhiy Storchaka in bpo-24164.)

pickletools
pickletools.dis() now outputs the implicit memo index for the MEMOIZE opcode. (Contributed by Serhiy Storchaka in bpo-25382.)

pydoc
The pydoc module has learned to respect the MANPAGER environment variable. (Contributed by Matthias Klose in bpo-8637.)
help() and pydoc can now list named tuple fields in the order they were defined rather than alphabetically. (Contributed by Raymond Hettinger in bpo-24879.)

random
The new choices() function returns a list of elements of specified size from the given population with optional weights. (Contributed by Raymond Hettinger in bpo-18844.)

re
Added support of modifier spans in regular expressions. Examples:
'(?i:p)ython' matches 'python' and 'Python', but not 'PYTHON'; '(?i)g(?-i:v)r' matches 'GvR' and 'gvr', but not 'GVR'. (Contributed by Serhiy Storchaka in bpo-433028.)
Match object groups can be accessed by getitem, which is equivalent to group(). So mo['name'] is now equivalent to mo.group('name'). (Contributed by Eric Smith in bpo-24454.)
Match objects now support index-like objects as group indices. (Contributed by Jeroen Demeyer and Xiang Zhang in bpo-27177.)

readline
Added set_auto_history() to enable or disable automatic addition of input to the history list. (Contributed by Tyler Crompton in bpo-26870.)

rlcompleter
Private and special attribute names now are omitted unless the prefix starts with underscores. A space or a colon is added after some completed keywords. (Contributed by Serhiy Storchaka in bpo-25011 and bpo-25209.)

shlex
The shlex has much improved shell compatibility through the new punctuation_chars argument to control which characters are treated as punctuation. (Contributed by Vinay Sajip in bpo-1521950.)

site
When specifying paths to add to sys.path in a .pth file, you may now specify file paths on top of directories (e.g. zip files). (Contributed by Wolfgang Langner in bpo-26587).

sqlite3
sqlite3.Cursor.lastrowid now supports the REPLACE statement. (Contributed by Alex LordThorsen in bpo-16864.)

socket
The ioctl() function now supports the SIO_LOOPBACK_FAST_PATH control code. (Contributed by Daniel Stokes in bpo-26536.)
The getsockopt() constants SO_DOMAIN , SO_PROTOCOL , SO_PEERSEC , and SO_PASSSEC are now supported. (Contributed by Christian Heimes in bpo-26907.)
The setsockopt() now supports the setsockopt(level, optname, None, optlen: int) form. (Contributed by Christian Heimes in bpo-27744.)
The socket module now supports the address family AF_ALG to interface with Linux Kernel crypto API. ALG_* , SOL_ALG and sendmsg_afalg() were added. (Contributed by Christian Heimes in bpo-27744 with support from Victor Stinner.)
New Linux constants TCP_USER_TIMEOUT and TCP_CONGESTION were added. (Contributed by Omar Sandoval, bpo-26273).

socketserver
Servers based on the socketserver module, including those defined in http.server , xmlrpc.server and wsgiref.simple_server , now support the context manager protocol. (Contributed by Aviv Palivoda in bpo-26404.)
The wfile attribute of StreamRequestHandler classes now implements the io.BufferedIOBase writable interface. In particular, calling write() is now guaranteed to send the data in full. (Contributed by Martin Panter in bpo-26721.)

ssl
ssl supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2. (Contributed by Christian Heimes in bpo-26470.)
3DES has been removed from the default cipher suites and ChaCha20 Poly1305 cipher suites have been added. (Contributed by Christian Heimes in bpo-27850 and bpo-27766.)
SSLContext has better default configuration for options and ciphers. (Contributed by Christian Heimes in bpo-28043.)
SSL session can be copied from one client-side connection to another with the new SSLSession class. TLS session resumption can speed up the initial handshake, reduce latency and improve performance (Contributed by Christian Heimes in bpo-19500 based on a draft by Alex Warhawk.)
The new get_ciphers() method can be used to get a list of enabled ciphers in order of cipher priority.
All constants and flags have been converted to IntEnum and IntFlag . (Contributed by Christian Heimes in bpo-28025.)
Server and client-side specific TLS protocols for SSLContext were added. (Contributed by Christian Heimes in bpo-28085.)
Added ssl.SSLContext.post_handshake_auth to enable and ssl.SSLSocket.verify_client_post_handshake() to initiate TLS 1.3 post-handshake authentication. (Contributed by Christian Heimes in gh-78851.)

statistics
A new harmonic_mean() function has been added. (Contributed by Steven D’Aprano in bpo-27181.)

struct
struct now supports IEEE 754 half-precision floats via the 'e' format specifier. (Contributed by Eli Stevens, Mark Dickinson in bpo-11734.)

subprocess
subprocess.Popen destructor now emits a ResourceWarning warning if the child process is still running. Use the context manager protocol ( with proc: ... ) or explicitly call the wait() method to read the exit status of the child process. (Contributed by Victor Stinner in bpo-26741.)
The subprocess.Popen constructor and all functions that pass arguments through to it now accept encoding and errors arguments. Specifying either of these will enable text mode for the stdin , stdout and stderr streams. (Contributed by Steve Dower in bpo-6135.)

sys
The new getfilesystemencodeerrors() function returns the name of the error mode used to convert between Unicode filenames and bytes filenames. (Contributed by Steve Dower in bpo-27781.)
On Windows the return value of the getwindowsversion() function now includes the platform_version field which contains the accurate major version, minor version and build number of the current operating system, rather than the version that is being emulated for the process (Contributed by Steve Dower in bpo-27932.)

telnetlib
telnetlib.Telnet is now a context manager (contributed by Stéphane Wirtel in bpo-25485).

time
The struct_time attributes tm_gmtoff and tm_zone are now available on all platforms.

timeit
The new Timer.autorange() convenience method has been added to call Timer.timeit() repeatedly so that the total run time is greater or equal to 200 milliseconds. (Contributed by Steven D’Aprano in bpo-6422.)
timeit now warns when there is substantial (4x) variance between best and worst times. (Contributed by Serhiy Storchaka in bpo-23552.)

tkinter
Added methods Variable.trace_add() , Variable.trace_remove() , and trace_info() in the tkinter.Variable class. They replace old methods trace_variable() , trace() , trace_vdelete() and trace_vinfo() that use obsolete Tcl commands and might not work in future versions of Tcl. (Contributed by Serhiy Storchaka in bpo-22115).

traceback
Both the traceback module and the interpreter’s builtin exception display now abbreviate long sequences of repeated lines in tracebacks as shown in the following example:

def f():
f()
...

f()
Traceback (most recent call last):
File "", line 1, in
File "", line 1, in f
File "", line 1, in f
File "", line 1, in f
[Previous line repeated 995 more times]
RecursionError: maximum recursion depth exceeded
(Contributed by Emanuel Barry in bpo-26823.)

tracemalloc
The tracemalloc module now supports tracing memory allocations in multiple different address spaces.
The new DomainFilter filter class has been added to filter block traces by their address space (domain).
(Contributed by Victor Stinner in bpo-26588.)

typing
Since the typing module is provisional, all changes introduced in Python 3.6 have also been backported to Python 3.5.x.
The typing module has a much improved support for generic type aliases. For example Dict[str, Tuple[S, T]] is now a valid type annotation. (Contributed by Guido van Rossum in Github #195.)
The typing.ContextManager class has been added for representing contextlib.AbstractContextManager . (Contributed by Brett Cannon in bpo-25609.)
The typing.Collection class has been added for representing collections.abc.Collection . (Contributed by Ivan Levkivskyi in bpo-27598.)
The typing.ClassVar type construct has been added to mark class variables. As introduced in PEP 526 , a variable annotation wrapped in ClassVar indicates that a given attribute is intended to be used as a class variable and should not be set on instances of that class. (Contributed by Ivan Levkivskyi in Github #280.)
A new TYPE_CHECKING constant that is assumed to be True by the static type checkers, but is False at runtime. (Contributed by Guido van Rossum in Github #230.)
A new NewType() helper function has been added to create lightweight distinct types for annotations:
from typing import NewType
UserId = NewType('UserId', int)
some_id = UserId(524313)
The static type checker will treat the new type as if it were a subclass of the original type. (Contributed by Ivan Levkivskyi in Github #189.)

unicodedata
The unicodedata module now uses data from Unicode 9.0.0. (Contributed by Benjamin Peterson.)

unittest.mock
The Mock class has the following improvements:
• Two new methods, Mock.assert_called() and Mock.assert_called_once() to check if the mock object was called. (Contributed by Amit Saha in bpo-26323.)
• The Mock.reset_mock() method now has two optional keyword only arguments: return_value and side_effect . (Contributed by Kushal Das in bpo-21271.)

urllib.request
If a HTTP request has a file or iterable body (other than a bytes object) but no Content-Length header, rather than throwing an error, AbstractHTTPHandler now falls back to use chunked transfer encoding. (Contributed by Demian Brecht and Rolf Krahl in bpo-12319.)

urllib.robotparser
RobotFileParser now supports the Crawl-delay and Request-rate extensions. (Contributed by Nikolay Bogoychev in bpo-16099.)

venv
venv accepts a new parameter --prompt . This parameter provides an alternative prefix for the virtual environment. (Proposed by Łukasz Balcerzak and ported to 3.6 by Stéphane Wirtel in bpo-22829.)

warnings
A new optional source parameter has been added to the warnings.warn_explicit() function: the destroyed object which emitted a ResourceWarning . A source attribute has also been added to warnings.WarningMessage (contributed by Victor Stinner in bpo-26568 and bpo-26567).
When a ResourceWarning warning is logged, the tracemalloc module is now used to try to retrieve the traceback where the destroyed object was allocated.
Example with the script example.py :
import warnings
def func():
return open(file)
f = func()
f = None
Output of the command python3.6 -Wd -X tracemalloc=5 example.py :
example.py: 7 : ResourceWarning : unclosed file < _io .TextIOWrapper name = 'example.py' mode = 'r' encoding = 'UTF-8' > f = None Object allocated at (most recent call first): File "example.py", lineno 4 return open(file) File "example.py", lineno 6 f = func()
The “Object allocated at” traceback is new and is only displayed if tracemalloc is tracing Python memory allocations and if the warnings module was already imported.

winreg
Added the 64-bit integer type REG_QWORD . (Contributed by Clement Rouault in bpo-23026.)

winsound
Allowed keyword arguments to be passed to Beep , MessageBeep , and PlaySound . (bpo-27982).

xmlrpc.client
The xmlrpc.client module now supports unmarshalling additional data types used by the Apache XML-RPC implementation for numerics and None . (Contributed by Serhiy Storchaka in bpo-26885.)

zipfile
A new ZipInfo.from_file() class method allows making a ZipInfo instance from a filesystem file. A new ZipInfo.is_dir() method can be used to check if the ZipInfo instance represents a directory. (Contributed by Thomas Kluyver in bpo-26039.)
The ZipFile.open() method can now be used to write data into a ZIP file, as well as for extracting data. (Contributed by Thomas Kluyver in bpo-26039.)

zlib
The compress() and decompress() functions now accept keyword arguments. (Contributed by Aviv Palivoda in bpo-26243 and Xiang Zhang in bpo-16764 respectively.)

Optimizations
• The Python interpreter now uses a 16-bit wordcode instead of bytecode which made a number of opcode optimizations possible. (Contributed by Demur Rumed with input and reviews from Serhiy Storchaka and Victor Stinner in bpo-26647 and bpo-28050.)
• The asyncio.Future class now has an optimized C implementation. (Contributed by Yury Selivanov and INADA Naoki in bpo-26081.)
• The asyncio.Task class now has an optimized C implementation. (Contributed by Yury Selivanov in bpo-28544.)
• Various implementation improvements in the typing module (such as caching of generic types) allow up to 30 times performance improvements and reduced memory footprint.
• The ASCII deco

Original source Report a problem

This article explains the new features in Python 3.5, compared to 3.4. Python 3.5 was released on September 13, 2015. See the changelog for a full list of changes.

See also
PEP 478 - Python 3.5 Release Schedule

Summary – Release highlights
New syntax features:
• PEP 492, coroutines with async and await syntax.
• PEP 465, a new matrix multiplication operator: a @ b.
• PEP 448, additional unpacking generalizations.

New library modules:
• typing: PEP 484 – Type Hints.
• zipapp: PEP 441 Improving Python ZIP Application Support.

New built-in features:
• bytes % args, bytearray % args: PEP 461 – Adding % formatting to bytes and bytearray.
• New bytes.hex(), bytearray.hex() and memoryview.hex() methods. (Contributed by Arnon Yaari in bpo-9951.)
• memoryview now supports tuple indexing (including multi-dimensional). (Contributed by Antoine Pitrou in bpo-23632.)
• Generators have a new gi_yieldfrom attribute, which returns the object being iterated by yield from expressions. (Contributed by Benno Leslie and Yury Selivanov in bpo-24450.)
• A new RecursionError exception is now raised when maximum recursion depth is reached. (Contributed by Georg Brandl in bpo-19235.)

CPython implementation improvements:
• When the LC_TYPE locale is the POSIX locale (C locale), sys.stdin and sys.stdout now use the surrogateescape error handler, instead of the strict error handler. (Contributed by Victor Stinner in bpo-19977.)
• .pyo files are no longer used and have been replaced by a more flexible scheme that includes the optimization level explicitly in .pyc name. (See PEP 488 overview.)
• Builtin and extension modules are now initialized in a multi-phase process, which is similar to how Python modules are loaded. (See PEP 489 overview.)

Significant improvements in the standard library:
• collections.OrderedDict is now implemented in C, which makes it 4 to 100 times faster.
• The ssl module gained support for Memory BIO, which decouples SSL protocol handling from network IO.
• The new os.scandir() function provides a better and significantly faster way of directory traversal.
• functools.lru_cache() has been mostly reimplemented in C, yielding much better performance.
• The new subprocess.run() function provides a streamlined way to run subprocesses.
• The traceback module has been significantly enhanced for improved performance and developer convenience.

Security improvements:
• SSLv3 is now disabled throughout the standard library. It can still be enabled by instantiating a ssl.SSLContext manually. (See bpo-22638 for more details; this change was backported to CPython 3.4 and 2.7.)
• HTTP cookie parsing is now stricter, in order to protect against potential injection attacks. (Contributed by Antoine Pitrou in bpo-22796.)

Windows improvements:
• A new installer for Windows has replaced the old MSI. See Using Python on Windows for more information.
• Windows builds now use Microsoft Visual C++ 14.0, and extension modules should use the same.

Please read on for a comprehensive list of user-facing changes, including many other smaller improvements, CPython optimizations, deprecations, and potential porting issues.

New Features
PEP 492 - Coroutines with async and await syntax
PEP 492 greatly improves support for asynchronous programming in Python by adding awaitable objects, coroutine functions, asynchronous iteration, and asynchronous context managers.
Coroutine functions are declared using the new async def syntax:

async def coro():
... return 'spam'
Inside a coroutine function, the new await expression can be used to suspend coroutine execution until the result is available. Any object can be awaited, as long as it implements the awaitable protocol by defining the await() method.
PEP 492 also adds async for statement for convenient iteration over asynchronous iterables.
An example of a rudimentary HTTP client written using the new syntax:
import asyncio
async def http_get(domain):
reader, writer = await asyncio.open_connection(domain, 80)
writer.write(b'\r\n'.join([b'GET / HTTP/1.1', b'Host: %b' % domain.encode('latin-1'), b'Connection: close', b'', b'']))
async for line in reader:
print('>>>', line)
writer.close()
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(http_get('example.com'))
finally:
loop.close()

Similarly to asynchronous iteration, there is a new syntax for asynchronous context managers. The following script:
import asyncio
async def coro(name, lock):
print('coro {}: waiting for lock'.format(name))
async with lock:
print('coro {}: holding the lock'.format(name))
await asyncio.sleep(1)
print('coro {}: releasing the lock'.format(name))
loop = asyncio.get_event_loop()
lock = asyncio.Lock()
coros = asyncio.gather(coro(1, lock), coro(2, lock))
try:
loop.run_until_complete(coros)
finally:
loop.close()
will output:
coro 2: waiting for lock
coro 2: holding the lock
coro 1: waiting for lock
coro 2: releasing the lock
coro 1: holding the lock
coro 1: releasing the lock
Note that both async for and async with can only be used inside a coroutine function declared with async def.
Coroutine functions are intended to be run inside a compatible event loop, such as the asyncio loop.

Changed in version 3.5.2:
Starting with CPython 3.5.2, aiter can directly return asynchronous iterators. Returning an awaitable object will result in a PendingDeprecationWarning.
See more details in the Asynchronous Iterators documentation section.

See also
PEP 492 – Coroutines with async and await syntax
PEP written and implemented by Yury Selivanov.

PEP 465 - A dedicated infix operator for matrix multiplication
PEP 465 adds the @ infix operator for matrix multiplication. Currently, no builtin Python types implement the new operator, however, it can be implemented by defining matmul(), rmatmul(), and imatmul() for regular, reflected, and in-place matrix multiplication. The semantics of these methods is similar to that of methods defining other infix arithmetic operators.
Matrix multiplication is a notably common operation in many fields of mathematics, science, engineering, and the addition of @ allows writing cleaner code:
S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r)
instead of:
S = dot((dot(H, beta) - r).T, dot(inv(dot(H, V), H.T)), dot(H, beta) - r))
NumPy 1.10 has support for the new operator:

import numpy
x = numpy.ones(3)
x
array([ 1., 1., 1.])

m = numpy.eye(3)
m
array([[ 1., 0., 0.],
[ 0., 1., 0.],
[ 0., 0., 1.]])

x @ m
array([ 1., 1., 1.])

See also
PEP 465 – A dedicated infix operator for matrix multiplication
PEP written by Nathaniel J. Smith; implemented by Benjamin Peterson.

PEP 448 - Additional Unpacking Generalizations
PEP 448 extends the allowed uses of the * iterable unpacking operator and ** dictionary unpacking operator. It is now possible to use an arbitrary number of unpackings in function calls:

print(*[1], *[2], 3, *[4, 5])
1 2 3 4 5

def fn(a, b, c, d):
... print(a, b, c, d)
...

fn(**{'a': 1, 'c': 3}, **{'b': 2, 'd': 4})
1 2 3 4
Similarly, tuple, list, set, and dictionary displays allow multiple unpackings (see Expression lists and Dictionary displays):

*range(4), 4
(0, 1, 2, 3, 4)

[*range(4), 4]
[0, 1, 2, 3, 4]

{*range(4), 4, *(5, 6, 7)}
{0, 1, 2, 3, 4, 5, 6, 7}

{'x': 1, **{'y': 2}}
{'x': 1, 'y': 2}

See also
PEP 448 – Additional Unpacking Generalizations
PEP written by Joshua Landau; implemented by Neil Girdhar, Thomas Wouters, and Joshua Landau.

PEP 461 - percent formatting support for bytes and bytearray
PEP 461 adds support for the % interpolation operator to bytes and bytearray.
While interpolation is usually thought of as a string operation, there are cases where interpolation on bytes or bytearrays makes sense, and the work needed to make up for this missing functionality detracts from the overall readability of the code. This issue is particularly important when dealing with wire format protocols, which are often a mixture of binary and ASCII compatible text.
Examples:

b'Hello %b!' % b'World'
b'Hello World!'

b'x=%i y=%f' % (1, 2.5)
b'x=1 y=2.500000'
Unicode is not allowed for %b, but it is accepted by %a (equivalent of repr(obj).encode('ascii', 'backslashreplace')):

b'Hello %b!' % 'World'
Traceback (most recent call last):
File "", line 1, in
TypeError: %b requires bytes, or an object that implements bytes, not 'str'

b'price: %a' % '10€'
b"price: '10\u20ac'"
Note that %s and %r conversion types, although supported, should only be used in codebases that need compatibility with Python 2.

See also
PEP 461 – Adding % formatting to bytes and bytearray
PEP written by Ethan Furman; implemented by Neil Schemenauer and Ethan Furman.

PEP 484 - Type Hints
Function annotation syntax has been a Python feature since version 3.0 (PEP 3107), however the semantics of annotations has been left undefined.
Experience has shown that the majority of function annotation uses were to provide type hints to function parameters and return values. It became evident that it would be beneficial for Python users, if the standard library included the base definitions and tools for type annotations.
PEP 484 introduces a provisional module to provide these standard definitions and tools, along with some conventions for situations where annotations are not available.
For example, here is a simple function whose argument and return type are declared in the annotations:
def greeting(name: str) -> str:
return 'Hello ' + name
While these annotations are available at runtime through the usual annotations attribute, no automatic type checking happens at runtime. Instead, it is assumed that a separate off-line type checker (e.g. mypy) will be used for on-demand source code analysis.
The type system supports unions, generic types, and a special type named Any which is consistent with (i.e. assignable to and from) all types.

See also
typing module documentation
PEP 484 – Type Hints
PEP written by Guido van Rossum, Jukka Lehtosalo, and Łukasz Langa; implemented by Guido van Rossum.
PEP 483 – The Theory of Type Hints
PEP written by Guido van Rossum

PEP 471 - os.scandir() function – a better and faster directory iterator
PEP 471 adds a new directory iteration function, os.scandir(), to the standard library. Additionally, os.walk() is now implemented using scandir, which makes it 3 to 5 times faster on POSIX systems and 7 to 20 times faster on Windows systems. This is largely achieved by greatly reducing the number of calls to os.stat() required to walk a directory tree.
Additionally, scandir returns an iterator, as opposed to returning a list of file names, which improves memory efficiency when iterating over very large directories.
The following example shows a simple use of os.scandir() to display all the files (excluding directories) in the given path that don’t start with '.'. The entry.is_file() call will generally not make an additional system call:
for entry in os.scandir(path):
if not entry.name.startswith('.') and entry.is_file():
print(entry.name)

See also
PEP 471 – os.scandir() function – a better and faster directory iterator
PEP written and implemented by Ben Hoyt with the help of Victor Stinner.

PEP 475: Retry system calls failing with EINTR
An errno.EINTR error code is returned whenever a system call, that is waiting for I/O, is interrupted by a signal. Previously, Python would raise InterruptedError in such cases. This meant that, when writing a Python application, the developer had two choices:

1. Ignore the InterruptedError.
2. Handle the InterruptedError and attempt to restart the interrupted system call at every call site.
   The first option makes an application fail intermittently. The second option adds a large amount of boilerplate that makes the code nearly unreadable. Compare:
   print("Hello World")
   and:
   while True:
   try:
   print("Hello World")
   break
   except InterruptedError:
   continue

PEP 475 implements automatic retry of system calls on EINTR. This removes the burden of dealing with EINTR or InterruptedError in user code in most situations and makes Python programs, including the standard library, more robust. Note that the system call is only retried if the signal handler does not raise an exception.
Below is a list of functions which are now retried when interrupted by a signal:
• open() and io.open();
• functions of the faulthandler module;
• os functions: fchdir(), fchmod(), fchown(), fdatasync(), fstat(), fstatvfs(), fsync(), ftruncate(), mkfifo(), mknod(), open(), posix_fadvise(), posix_fallocate(), pread(), pwrite(), read(), readv(), sendfile(), wait3(), wait4(), wait(), waitid(), waitpid(), write(), writev();
• special cases: os.close() and os.dup2() now ignore EINTR errors; the syscall is not retried (see the PEP for the rationale);
• select functions: devpoll.poll(), epoll.poll(), kqueue.control(), poll.poll(), select();
• methods of the socket class: accept(), connect() (except for non-blocking sockets), recv(), recvfrom(), recvmsg(), send(), sendall(), sendmsg(), sendto();
• signal.sigtimedwait() and signal.sigwaitinfo();
• time.sleep().

See also
PEP 475 – Retry system calls failing with EINTR
PEP and implementation written by Charles-François Natali and Victor Stinner, with the help of Antoine Pitrou (the French connection).

PEP 479: Change StopIteration handling inside generators
The interaction of generators and StopIteration in Python 3.4 and earlier was sometimes surprising, and could conceal obscure bugs. Previously, StopIteration raised accidentally inside a generator function was interpreted as the end of the iteration by the loop construct driving the generator.
PEP 479 changes the behavior of generators: when a StopIteration exception is raised inside a generator, it is replaced with a RuntimeError before it exits the generator frame. The main goal of this change is to ease debugging in the situation where an unguarded next() call raises StopIteration and causes the iteration controlled by the generator to terminate silently. This is particularly pernicious in combination with the yield from construct.
This is a backwards incompatible change, so to enable the new behavior, a future import is necessary:

from future import generator_stop
def gen(): ...
next(iter([])) ...
yield ...
next(gen())
Traceback (most recent call last):
File "", line 2, in gen
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "", line 1, in
RuntimeError: generator raised StopIteration
Without a future import, a PendingDeprecationWarning will be raised whenever a StopIteration exception is raised inside a generator.

See also
PEP 479 – Change StopIteration handling inside generators
PEP written by Chris Angelico and Guido van Rossum. Implemented by Chris Angelico, Yury Selivanov and Nick Coghlan.

PEP 485: A function for testing approximate equality
PEP 485 adds the math.isclose() and cmath.isclose() functions which tell whether two values are approximately equal or “close” to each other. Whether or not two values are considered close is determined according to given absolute and relative tolerances. Relative tolerance is the maximum allowed difference between isclose arguments, relative to the larger absolute value:

import math
a = 5.0
b = 4.99998
math.isclose(a, b, rel_tol=1e-5)
True

math.isclose(a, b, rel_tol=1e-6)
False
It is also possible to compare two values using absolute tolerance, which must be a non-negative value:

import math
a = 5.0
b = 4.99998
math.isclose(a, b, abs_tol=0.00003)
True

math.isclose(a, b, abs_tol=0.00001)
False

See also
PEP 485 – A function for testing approximate equality
PEP written by Christopher Barker; implemented by Chris Barker and Tal Einat.

PEP 486: Make the Python Launcher aware of virtual environments
PEP 486 makes the Windows launcher (see PEP 397) aware of an active virtual environment. When the default interpreter would be used and the VIRTUAL_ENV environment variable is set, the interpreter in the virtual environment will be used.

See also
PEP 486 – Make the Python Launcher aware of virtual environments
PEP written and implemented by Paul Moore.

PEP 488: Elimination of PYO files
PEP 488 does away with the concept of .pyo files. This means that .pyc files represent both unoptimized and optimized bytecode. To prevent the need to constantly regenerate bytecode files, .pyc files now have an optional opt- tag in their name when the bytecode is optimized. This has the side-effect of no more bytecode file name clashes when running under either -O or -OO . Consequently, bytecode files generated from -O , and -OO may now exist simultaneously. importlib.util.cache_from_source() has an updated API to help with this change.

See also
PEP 488 – Elimination of PYO files
PEP written and implemented by Brett Cannon.

PEP 489: Multi-phase extension module initialization
PEP 489 updates extension module initialization to take advantage of the two step module loading mechanism introduced by PEP 451 in Python 3.4.
This change brings the import semantics of extension modules that opt-in to using the new mechanism much closer to those of Python source and bytecode modules, including the ability to use any valid identifier as a module name, rather than being restricted to ASCII.

See also
PEP 489 – Multi-phase extension module initialization
PEP written by Petr Viktorin, Stefan Behnel, and Nick Coghlan; implemented by Petr Viktorin.

Other Language Changes
Some smaller changes made to the core Python language are:
• Added the "namereplace" error handlers. The "backslashreplace" error handlers now work with decoding and translating. (Contributed by Serhiy Storchaka in bpo-19676 and bpo-22286.)
• The -b option now affects comparisons of bytes with int. (Contributed by Serhiy Storchaka in bpo-23681.)
• New Kazakh kz1048 and Tajik koi8_t codecs. (Contributed by Serhiy Storchaka in bpo-22682 and bpo-22681.)
• Property docstrings are now writable. This is especially useful for collections.namedtuple() docstrings. (Contributed by Berker Peksag in bpo-24064.)
• Circular imports involving relative imports are now supported. (Contributed by Brett Cannon and Antoine Pitrou in bpo-17636.)

New Modules
typing
The new typing provisional module provides standard definitions and tools for function type annotations. See Type Hints for more information.

zipapp
The new zipapp module (specified in PEP 441) provides an API and command line tool for creating executable Python Zip Applications, which were introduced in Python 2.6 in bpo-1739468, but which were not well publicized, either at the time or since.
With the new module, bundling your application is as simple as putting all the files, including a main.py file, into a directory myapp and running:
$ python -m zipapp myapp
$ python myapp.pyz
The module implementation has been contributed by Paul Moore in bpo-23491.

See also
PEP 441 – Improving Python ZIP Application Support

Improved Modules
argparse
The ArgumentParser class now allows disabling abbreviated usage of long options by setting allow_abbrev to False. (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson in bpo-14910.)

asyncio
Since the asyncio module is provisional, all changes introduced in Python 3.5 have also been backported to Python 3.4.x.
Notable changes in the asyncio module since Python 3.4.0:
• New debugging APIs: loop.set_debug() and loop.get_debug() methods. (Contributed by Victor Stinner.)
• The proactor event loop now supports SSL. (Contributed by Antoine Pitrou and Victor Stinner in bpo-22560.)
• A new loop.is_closed() method to check if the event loop is closed. (Contributed by Victor Stinner in bpo-21326.)
• A new loop.create_task() to conveniently create and schedule a new Task for a coroutine. The create_task method is also used by all asyncio functions that wrap coroutines into tasks, such as asyncio.wait(), asyncio.gather(), etc. (Contributed by Victor Stinner.)
• A new transport.get_write_buffer_limits() method to inquire for high- and low-water limits of the flow control. (Contributed by Victor Stinner.)
• The async() function is deprecated in favor of ensure_future(). (Contributed by Yury Selivanov.)
• New loop.set_task_factory() and loop.get_task_factory() methods to customize the task factory that loop.create_task() method uses. (Contributed by Yury Selivanov.)
• New Queue.join() and Queue.task_done() queue methods. (Contributed by Victor Stinner.)
• The JoinableQueue class was removed, in favor of the asyncio.Queue class. (Contributed by Victor Stinner.)
Updates in 3.5.1:
• The ensure_future() function and all functions that use it, such as loop.run_until_complete(), now accept all kinds of awaitable objects. (Contributed by Yury Selivanov.)
• New run_coroutine_threadsafe() function to submit coroutines to event loops from other threads. (Contributed by Vincent Michel.)
• New Transport.is_closing() method to check if the transport is closing or closed. (Contributed by Yury Selivanov.)
• The loop.create_server() method can now accept a list of hosts. (Contributed by Yann Sionneau.)
Updates in 3.5.2:
• New loop.create_future() method to create Future objects. This allows alternative event loop implementations, such as uvloop, to provide a faster asyncio.Future implementation. (Contributed by Yury Selivanov.)
• New loop.get_exception_handler() method to get the current exception handler. (Contributed by Yury Selivanov.)
• New StreamReader.readuntil() method to read data from the stream until a separator bytes sequence appears. (Contributed by Mark Korenberg.)
• The loop.create_connection() and loop.create_server() methods are optimized to avoid calling the system getaddrinfo function if the address is already resolved. (Contributed by A. Jesse Jiryu Davis.)
• The loop.sock_connect(sock, address) no longer requires the address to be resolved prior to the call. (Contributed by A. Jesse Jiryu Davis.)

bz2
The BZ2Decompressor.decompress method now accepts an optional max_length argument to limit the maximum size of decompressed data. (Contributed by Nikolaus Rath in bpo-15955.)

cgi
The FieldStorage class now supports the context manager protocol. (Contributed by Berker Peksag in bpo-20289.)

cmath
A new function isclose() provides a way to test for approximate equality. (Contributed by Chris Barker and Tal Einat in bpo-24270.)

code
The InteractiveInterpreter.showtraceback() method now prints the full chained traceback, just like the interactive interpreter. (Contributed by Claudiu Popa in bpo-17442.)

collections
The OrderedDict class is now implemented in C, which makes it 4 to 100 times faster. (Contributed by Eric Snow in bpo-16991.)
OrderedDict.items(), OrderedDict.keys(), and OrderedDict.values() views now support reversed() iteration. (Contributed by Serhiy Storchaka in bpo-19505.)
The deque class now defines index(), insert(), and copy(), and supports the + and * operators. This allows deques to be recognized as a MutableSequence and improves their substitutability for lists. (Contributed by Raymond Hettinger in bpo-23704.)
Docstrings produced by namedtuple() can now be updated:
Point = namedtuple('Point', ['x', 'y'])
Point.doc += ': Cartesian coordinate'
Point.x.doc = 'abscissa'
Point.y.doc = 'ordinate'
(Contributed by Berker Peksag in bpo-24064.)
The UserString class now implements the getnewargs(), rmod(), casefold(), format_map(), isprintable(), and maketrans() methods to match the corresponding methods of str. (Contributed by Joe Jevnik in bpo-22189.)

collections.abc
The Sequence.index() method now accepts start and stop arguments to match the corresponding methods of tuple, list, etc. (Contributed by Devin Jeanpierre in bpo-23086.)
A new Generator abstract base class. (Contributed by Stefan Behnel in bpo-24018.)
New Awaitable, Coroutine, AsyncIterator, and AsyncIterable abstract base classes. (Contributed by Yury Selivanov in bpo-24184.)
For earlier Python versions, a backport of the new ABCs is available in an external PyPI package.

compileall
A new compileall option, -j N, allows running N workers simultaneously to perform parallel bytecode compilation. The compile_dir() function has a corresponding workers parameter. (Contributed by Claudiu Popa in bpo-16104.)
Another new option, -r, allows controlling the maximum recursion level for subdirectories. (Contributed by Claudiu Popa in bpo-19628.)
The -q command line option can now be specified more than once, in which case all output, including errors, will be suppressed. The corresponding quiet parameter in compile_dir(), compile_file(), and compile_path() can now accept an integer value indicating the level of output suppression. (Contributed by Thomas Kluyver in bpo-21338.)

concurrent.futures
The Executor.map() method now accepts a chunksize argument to allow batching of tasks to improve performance when ProcessPoolExecutor() is used. (Contributed by Dan O’Reilly in bpo-11271.)
The number of workers in the ThreadPoolExecutor constructor is optional now. The default value is 5 times the number of CPUs. (Contributed by Claudiu Popa in bpo-21527.)

configparser
configparser now provides a way to customize the conversion of values by specifying a dictionary of converters in the ConfigParser constructor, or by defining them as methods in ConfigParser subclasses. Converters defined in a parser instance are inherited by its section proxies.
Example:
import configparser
conv = {}
conv['list'] = lambda v: [e.strip() for e in v.split() if e.strip()]
cfg = configparser.ConfigParser(converters=conv)
cfg.read_string("""
[s]
list = a b c d e f g
""")
cfg.get('s', 'list')
'a b c d e f g'
cfg.getlist('s', 'list')
['a', 'b', 'c', 'd', 'e', 'f', 'g']
section = cfg['s']
section.getlist('list')
['a', 'b', 'c', 'd', 'e', 'f', 'g']
(Contributed by Łukasz Langa in bpo-18159.)

contextlib
The new redirect_stderr() context manager (similar to redirect_stdout()) makes it easier for utility scripts to handle inflexible APIs that write their output to sys.stderr and don’t provide any options to redirect it:
import contextlib, io, logging
f = io.StringIO()
with contextlib.redirect_stderr(f):
...
logging.warning('warning')
...
f.getvalue()
'WARNING:root:warning\n'
(Contributed by Berker Peksag in bpo-22389.)

csv
The writerow() method now supports arbitrary iterables, not just sequences. (Contributed by Serhiy Storchaka in bpo-23171.)

curses
The new update_lines_cols() function updates the LINES and COLS module variables. This is useful for detecting manual screen resizing. (Contributed by Arnon Yaari in bpo-4254.)

dbm
dumb.open always creates a new database when the flag has the value "n". (Contributed by Claudiu Popa in bpo-18039.)

difflib
The charset of HTML documents generated by HtmlDiff.make_file() can now be customized by using a new charset keyword-only argument. The default charset of HTML document changed from "ISO-8859-1" to "utf-8". (Contributed by Berker Peksag in bpo-2052.)
The diff_bytes() function can now compare lists of byte strings. This fixes a regression from Python 2. (Contributed by Terry J. Reedy and Greg Ward in bpo-17445.)

distutils
Both the build and build_ext commands now accept a -j option to enable parallel building of extension modules. (Contributed by Antoine Pitrou in bpo-5309.)
The distutils module now supports xz compression, and can be enabled by passing xztar as an argument to bdist --format. (Contributed by Serhiy Storchaka in bpo-16314.)

doctest
The DocTestSuite() function returns an empty unittest.TestSuite if module contains no docstrings, instead of raising ValueError. (Contributed by Glenn Jones in bpo-15916.)
email
A new policy option Policy.mangle_from_ controls whether or not lines that start with "From " in email bodies are prefixed with a ">" character by generators. The default is True for compat32 and False for all other policies. (Contributed by Milan Oberkirch in bpo-20098.)
A new Message.get_content_disposition() method provides easy access to a canonical value for the Content-Disposition header. (Contributed by Abhilash Raj in bpo-21083.)
A new policy option EmailPolicy.utf8 can be set to True to encode email headers using the UTF-8 charset instead of using encoded words. This allows Messages to be formatted according to RFC 6532 and used with an SMTP server that supports the RFC 6531 SMTPUTF8 extension. (Contributed by R. David Murray in bpo-24211.)
The mime.text.MIMEText constructor now accepts a charset.Charset instance. (Contributed by Claude Paroz and Berker Peksag in bpo-16324.)

enum
The Enum callable has a new parameter start to specify the initial number of enum values if only names are provided:

Animal = enum.Enum('Animal', 'cat dog', start=10)
Animal.cat
<Animal.cat: 10>

Animal.dog
<Animal.dog: 11>
(Contributed by Ethan Furman in bpo-21706.)

faulthandler
The enable(), register(), dump_traceback(), and dump_traceback_later() functions now accept file descriptors in addition to file-like objects. (Contributed by Wei Wu in bpo-23566.)

functools
Most of the lru_cache() machinery is now implemented in C, making it significantly faster. (Contributed by Matt Joiner, Alexey Kachayev, and Serhiy Storchaka in bpo-14373.)

glob
The iglob() and glob() functions now support recursive search in subdirectories, using the "**" pattern. (Contributed by Serhiy Storchaka in bpo-13968.)

gzip
The mode argument of the GzipFile constructor now accepts "x" to request exclusive creation. (Contributed by Tim Heaney in bpo-19222.)

heapq
Element comparison in merge() can now be customized by passing a key function in a new optional key keyword argument, and a new optional reverse keyword argument can be used to reverse element comparison:

import heapq
a = ['9', '777', '55555']
b = ['88', '6666']
list(heapq.merge(a, b, key=len))
['9', '88', '777', '6666', '55555']

list(heapq.merge(reversed(a), reversed(b), key=len, reverse=True))
['55555', '6666', '777', '88', '9']
(Contributed by Raymond Hettinger in bpo-13742.)

http
A new HTTPStatus enum that defines a set of HTTP status codes, reason phrases and long descriptions written in English. (Contributed by Demian Brecht in bpo-21793.)

http.client
HTTPConnection.getresponse() now raises a RemoteDisconnected exception when a remote server connection is closed unexpectedly. Additionally, if a ConnectionError (of which RemoteDisconnected is a subclass) is raised, the client socket is now closed automatically, and will reconnect on the next request:
import http.client
conn = http.client.HTTPConnection('www.python.org')
for retries in range(3):
try:
conn.request('GET', '/')
resp = conn.getresponse()
except http.client.RemoteDisconnected:
pass
(Contributed by Martin Panter in bpo-3566.)

idlelib and IDLE
Since idlelib implements the IDLE shell and editor and is not intended for import by other programs, it gets improvements with every release. See Lib/idlelib/NEWS.txt for a cumulative list of changes since 3.4.0, as well as changes made in future 3.5.x releases. This file is also available from the IDLE Help ‣ About IDLE dialog.

imaplib
The IMAP4 class now supports the context manager protocol. When used in a with statement, the IMAP4 LOGOUT command will be called automatically at the end of the block. (Contributed by Tarek Ziadé and Serhiy Storchaka in bpo-4972.)
The imaplib module now supports RFC 5161 (ENABLE Extension) and RFC 6855 (UTF-8 Support) via the IMAP4.enable() method. A new IMAP4.utf8_enabled attribute tracks whether or not RFC 6855 support is enabled. (Contributed by Milan Oberkirch, R. David Murray, and Maciej Szulik in bpo-21800.)
The imaplib module now automatically encodes non-ASCII string usernames and passwords using UTF-8, as recommended by the RFCs. (Contributed by Milan Oberkirch in bpo-21800.)

imghdr
The what() function now recognizes the OpenEXR format (contributed by Martin Vignali and Claudiu Popa in bpo-20295), and the WebP format (contributed by Fabrice Aneche and Claudiu Popa in bpo-20197.)

importlib
The util.LazyLoader class allows for lazy loading of modules in applications where startup time is important. (Contributed by Brett Cannon in bpo-17621.)
The abc.InspectLoader.source_to_code() method is now a static method. This makes it easier to initialize a module object with code compiled from a string by running exec(code, module.dict). (Contributed by Brett Cannon in bpo-21156.)
The new util.module_from_spec() function is now the preferred way to create a new module. As opposed to creating a types.ModuleType instance directly, this new function will set the various import-controlled attributes based on the passed-in spec object. (Contributed by Brett Cannon in bpo-20383.)

inspect
Both the Signature and Parameter classes are now picklable and hashable. (Contributed by Yury Selivanov in bpo-20726 and bpo-20334.)
A new BoundArguments.apply_defaults() method provides a way to set default values for missing arguments:

def foo(a, b='ham', *args): pass
ba = inspect.signature(foo).bind('spam')
ba.apply_defaults()
ba.arguments
OrderedDict([('a', 'spam'), ('b', 'ham'), ('args', ())])
(Contributed by Yury Selivanov in bpo-24190.)
A new class method Signature.from_callable() makes subclassing of Signature easier. (Contributed by Yury Selivanov and Eric Snow in bpo-17373.)
The signature() function now accepts a follow_wrapped optional keyword argument, which, when set to False, disables automatic following of wrapped links. (Contributed by Yury Selivanov in bpo-20691.)
A set of new functions to inspect coroutine functions and coroutine objects has been added: iscoroutine(), iscoroutinefunction(), isawaitable(), getcoroutinelocals(), and getcoroutinestate(). (Contributed by Yury Selivanov in bpo-24017 and bpo-24400.)
The stack(), trace(), getouterframes(), and getinnerframes() functions now return a list of named tuples. (Contributed by Daniel Shahaf in bpo-16808.)

io
A new BufferedIOBase.readinto1() method, that uses at most one call to the underlying raw stream’s RawIOBase.read() or RawIOBase.readinto() methods. (Contributed by Nikolaus Rath in bpo-20578.)

ipaddress
Both the IPv4Network and IPv6Network classes now accept an (address, netmask) tuple argument, so as to easily construct network objects from existing addresses:

import ipaddress
ipaddress.IPv4Network(('127.0.0.0', 8))
IPv4Network('127.0.0.0/8')

ipaddress.IPv4Network(('127.0.0.0', '255.0.0.0'))
IPv4Network('127.0.0.0/8')
(Contributed by Peter Moody and Antoine Pitrou in bpo-16531.)
A new reverse_pointer attribute for the IPv4Address and IPv6Address classes returns the name of the reverse DNS PTR record:

import ipaddress
addr = ipaddress.IPv4Address('127.0.0.1')
addr.reverse_pointer
'1.0.0.127.in-addr.arpa'

addr6 = ipaddress.IPv6Address('::1')
addr6.reverse_pointer
'1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa'
(Contributed by Leon Weber in bpo-20480.)

json
The json.tool command line interface now preserves the order of keys in JSON objects passed in input. The new --sort-keys option can be used to sort the keys alphabetically. (Contributed by Berker Peksag in bpo-21650.)
JSON decoder now raises JSONDecodeError instead of ValueError to provide better context information about the error. (Contributed by Serhiy Storchaka in bpo-19361.)

linecache
A new lazycache() function can be used to capture information about a non-file-based module to permit getting its lines later via getline(). This avoids doing I/O until a line is actually needed, without having to carry the module globals around indefinitely. (Contributed by Robert Collins in bpo-17911.)

locale
A new delocalize() function can be used to convert a string into a normalized number string, taking the LC_NUMERIC settings into account:

import locale
locale.setlocale(locale.LC_NUMERIC, 'de_DE.UTF-8')
'de_DE.UTF-8'

locale.delocalize('1.234,56')
'1234.56'

locale.setlocale(locale.LC_NUMERIC, 'en_US.UTF-8')
'en_US.UTF-8'

locale.delocalize('1,234.56')
'1234.56'
(Contributed by Cédric Krier in bpo-13918.)

logging
All logging methods (Logger.log(), exception(), critical(), debug(), etc.), now accept exception instances as an exc_info argument, in addition to boolean values and exception tuples:

import logging
try:
... 1 / 0
... except ZeroDivisionError as ex:
... logging.error('exception', exc_info=ex)
ERROR:root:exception
(Contributed by Yury Selivanov in bpo-20537.)
The handlers.HTTPHandler class now accepts an optional ssl.SSLContext instance to configure SSL settings used in an HTTP connection. (Contributed by Alex Gaynor in bpo-22788.)
The handlers.QueueListener class now takes a respect_handler_level keyword argument which, if set to True, will pass messages to handlers taking handler levels into account. (Contributed by Vinay Sajip.)

lzma
The LZMADecompressor.decompress() method now accepts an optional max_length argument to limit the maximum size of decompressed data. (Contributed by Martin Panter in bpo-15955.)

math
Two new constants have been added to the math module:
inf and nan. (Contributed by Mark Dickinson in bpo-23185.)
A new function isclose() provides a way to test for approximate equality. (Contributed by Chris Barker and Tal Einat in bpo-24270.)
A new gcd() function has been added. The fractions.gcd() function is now deprecated. (Contributed by Mark Dickinson and Serhiy Storchaka in bpo-22486.)

multiprocessing
sharedctypes.synchronized() objects now support the context manager protocol. (Contributed by Charles-François Natali in bpo-21565.)

operator
attrgetter(), itemgetter(), and methodcaller() objects now support pickling. (Contributed by Josh Rosenberg and Serhiy Storchaka in bpo-22955.)
New matmul() and imatmul() functions to perform matrix multiplication. (Contributed by Benjamin Peterson in bpo-21176.)

os
The new scandir() function returning an iterator of DirEntry objects has been added. If possible, scandir() extracts file attributes while scanning a directory, removing the need to perform subsequent system calls to determine file type or attributes, which may significantly improve performance. (Contributed by Ben Hoyt with the help of Victor Stinner in bpo-22524.)
On Windows, a new stat_result.st_file_attributes attribute is now available. It corresponds to the dwFileAttributes member of the BY_HANDLE_FILE_INFORMATION structure returned by GetFileInformationByHandle(). (Contributed by Ben Hoyt in bpo-21719.)
The urandom() function now uses the getrandom() syscall on Linux 3.17 or newer, and getentropy() on OpenBSD 5.6 and newer, removing the need to use /dev/urandom and avoiding failures due to potential file descriptor exhaustion. (Contributed by Victor Stinner in bpo-22181.)
New get_blocking() and set_blocking() functions allow getting and setting a file descriptor’s blocking mode (O_NONBLOCK.) (Contributed by Victor Stinner in bpo-22054.)
The truncate() and ftruncate() functions are now supported on Windows. (Contributed by Steve Dower in bpo-23668.)
There is a new os.path.commonpath() function returning the longest common sub-path of each passed pathname. Unlike the os.path.commonprefix() function, it always returns a valid path:

os.path.commonprefix(['/usr/lib', '/usr/local/lib'])
'/usr/l'

os.path.commonpath(['/usr/lib', '/usr/local/lib'])
'/usr'
(Contributed by Rafik Draoui and Serhiy Storchaka in bpo-10395.)

pathlib
The new Path.samefile() method can be used to check whether the path points to the same file as another path, which can be either another Path object, or a string:

import pathlib
p1 = pathlib.Path('/etc/hosts')
p2 = pathlib.Path('/etc/../etc/hosts')
p1.samefile(p2)
True
(Contributed by Vajrasky Kok and Antoine Pitrou in bpo-19775.)
The Path.mkdir() method now accepts a new optional exist_ok argument to match mkdir -p and os.makedirs() functionality. (Contributed by Berker Peksag in bpo-21539.)
There is a new Path.expanduser() method to expand ~ and ~user prefixes. (Contributed by Serhiy Storchaka and Claudiu Popa in bpo-19776.)
A new Path.home() class method can be used to get a Path instance representing the user’s home directory. (Contributed by Victor Salgado and Mayank Tripathi in bpo-19777.)
New Path.write_text(), Path.read_text(), Path.write_bytes(), Path.read_bytes() methods to simplify read/write operations on files.
The following code snippet will create or rewrite existing file ~/spam42:

import pathlib
p = pathlib.Path('~/spam42')
p.expanduser().write_text('ham')
3
(Contributed by Christopher Welborn in bpo-20218.)

pickle
Nested objects, such as unbound methods or nested classes, can now be pickled using pickle protocols older than protocol version 4. Protocol version 4 already supports these cases. (Contributed by Serhiy Storchaka in bpo-23611.)

poplib
A new POP3.utf8() command enables RFC 6856 (Internationalized Email) support, if a POP server supports it. (Contributed by Milan OberKirch in bpo-21804.)

re
References and conditional references to groups with fixed length are now allowed in lookbehind assertions:

import re
pat = re.compile(r'(a|b).(?<=\1)c')
pat.match('aac')
<_sre.SRE_Match object; span=(0, 3), match='aac'>

pat.match('bbc')
<_sre.SRE_Match object; span=(0, 3), match='bbc'>
(Contributed by Serhiy Storchaka in bpo-9179.)
The number of capturing groups in regular expressions is no longer limited to 100. (Contributed by Serhiy Storchaka in bpo-22437.)
The sub() and subn() functions now replace unmatched groups with empty strings instead of raising an exception. (Contributed by Serhiy Storchaka in bpo-1519638.)
The re.error exceptions have new attributes, msg, pattern, pos, lineno, and colno, that provide better context information about the error:

re.compile("""
... (?x)
... .++
... """)
Traceback (most recent call last):
...
sre_constants.error: multiple repeat at position 16 (line 3, column 7)
(Contributed by Serhiy Storchaka in bpo-22578.)

readline
A new append_history_file() function can be used to append the specified number of trailing elements in history to the given file. (Contributed by Bruno Cauet in bpo-22940.)

selectors
The new DevpollSelector supports efficient /dev/poll polling on Solaris. (Contributed by Giampaolo Rodola’ in bpo-18931.)

shutil
The move() function now accepts a copy_function argument, allowing, for example, the copy() function to be used instead of the default copy2() if there is a need to ignore file metadata when moving. (Contributed by Claudiu Popa in bpo-19840.)
The make_archive() function now supports the xztar format. (Contributed by Serhiy Storchaka in bpo-5411.)

signal
On Windows, the set_wakeup_fd() function now also supports socket handles. (Contributed by Victor Stinner in bpo-22018.)
Various SIG* constants in the signal module have been converted into Enums. This allows meaningful names to be printed during debugging, instead of integer “magic numbers”. (Contributed by Giampaolo Rodola’ in bpo-21076.)

smtpd
Both the SMTPServer and SMTPChannel classes now accept a decode_data keyword argument to determine if the DATA portion of the SMTP transaction is decoded using the "utf-8" codec or is instead provided to the SMTPServer.process_message() method as a byte string. The default is True for backward compatibility reasons, but will change to False in Python 3.6. If decode_data is set to False, the process_message method must be prepared to accept keyword arguments. (Contributed by Maciej Szulik in bpo-19662.)
The SMTPServer class now advertises the 8BITMIME extension (RFC 6152) if decode_data has been set True. If the client specifies BODY=8BITMIME on the MAIL command, it is passed to SMTPServer.process_message() via the mail_options keyword. (Contributed by Milan Oberkirch and R. David Murray in bpo-21795.)
The SMTPServer class now also supports the SMTPUTF8 extension (RFC 6531: Internationalized Email). If the client specified SMTPUTF8 BODY=8BITMIME on the MAIL command, they are passed to SMTPServer.process_message() via the mail_options keyword. It is the responsibility of the process_message method to correctly handle the SMTPUTF8 data. (Contributed by Milan Oberkirch in bpo-21725.)
It is now possible to provide, directly or via name resolution, IPv6 addresses in the SMTPServer constructor, and have it successfully connect. (Contributed by Milan Oberkirch in bpo-14758.)

smtplib
A new SMTP.auth() method provides a convenient way to implement custom authentication mechanisms. (Contributed by Milan Oberkirch in bpo-15014.)
The SMTP.set_debuglevel() method now accepts an additional debuglevel (2), which enables timestamps in debug messages. (Contributed by Gavin Chappell and Maciej Szulik in bpo-16914.)
Both the SMTP.sendmail() and SMTP.send_message() methods now support RFC 6531 (SMTPUTF8). (Contributed by Milan Oberkirch and R. David Murray in bpo-22027.)

sndhdr
The what() and whathdr() functions now return a namedtuple(). (Contributed by Claudiu Popa in bpo-18615.)

socket
Functions with timeouts now use a monotonic clock, instead of a system clock. (Contributed by Victor Stinner in bpo-22043.)
A new socket.sendfile() method allows sending a file over a socket by using the high-performance os.sendfile() function on UNIX, resulting in uploads being from 2 to 3 times faster than when using plain socket.send(). (Contributed by Giampaolo Rodola’ in bpo-17552.)
The socket.sendall() method no longer resets the socket timeout every time bytes are received or sent. The socket timeout is now the maximum total duration to send all data. (Contributed by Victor Stinner in bpo-23853.)
The backlog argument of the socket.listen() method is now optional. By default it is set to SOMAXCONN or to 128, whichever is less. (Contributed by Charles-François Natali in bpo-21455.)

ssl
Memory BIO Support
(Contributed by Geert Jansen in bpo-21965.)
The new SSLObject class has been added to provide SSL protocol support for cases when the network I/O capabilities of SSLSocket are not necessary or are suboptimal. SSLObject represents an SSL protocol instance, but does not implement any network I/O methods, and instead provides a memory buffer interface. The new MemoryBIO class can be used to pass data between Python and an SSL protocol instance.
The memory BIO SSL support is primarily intended to be used in frameworks implementing asynchronous I/O for which SSLSocket’s readiness model (“select/poll”) is inefficient.
A new SSLContext.wrap_bio() method can be used to create a new SSLObject instance.

Application-Layer Protocol Negotiation Support
(Contributed by Benjamin Peterson in bpo-20188.)
Where OpenSSL support is present, the ssl module now implements the Application-Layer Protocol Negotiation TLS extension as described in RFC 7301.
The new SSLContext.set_alpn_protocols() can be used to specify which protocols a socket should advertise during the TLS handshake.
The new SSLSocket.selected_alpn_protocol() returns the protocol that was selected during the TLS handshake. The HAS_ALPN flag indicates whether ALPN support is present.

Other Changes
There is a new SSLSocket.version() method to query the actual protocol version in use. (Contributed by Antoine Pitrou in bpo-20421.)
The SSLSocket class now implements a SSLSocket.sendfile() method. (Contributed by Giampaolo Rodola’ in bpo-17552.)
The SSLSocket.send() method now raises either the ssl.SSLWantReadError or ssl.SSLWantWriteError exception on a non-blocking socket if the operation would block. Previously, it would return 0.
The cert_time_to_seconds() function now interprets the input time as UTC and not as local time, per RFC 5280. Additionally, the return value is always an int. (Contributed by Akira Li in bpo-19940.)
New SSLObject.shared_ciphers() and SSLSocket.shared_ciphers() methods return the list of ciphers sent by the client during the handshake. (Contributed by Benjamin Peterson in bpo-23186.)
The SSLSocket.do_handshake(), SSLSocket.read(), SSLSocket.shutdown(), and SSLSocket.write() methods of the SSLSocket class no longer reset the socket timeout every time bytes are received or sent. The socket timeout is now the maximum total duration of the method. (Contributed by Victor Stinner in bpo-23853.)
The match_hostname() function now supports matching of IP addresses. (Contributed by Antoine Pitrou in bpo-23239.)

sqlite3
The Row class now fully supports the sequence protocol, in particular reversed() iteration and slice indexing. (Contributed by Claudiu Popa in bpo-10203; by Lucas Sinclair, Jessica McKellar, and Serhiy Storchaka in bpo-13583.)

subprocess
The new run() function has been added. It runs the specified command and returns a CompletedProcess object, which describes a finished process. The new API is more consistent and is the recommended approach to invoking subprocesses in Python code that does not need to maintain compatibility with earlier Python versions. (Contributed by Thomas Kluyver in bpo-23342.)
Examples:

subprocess.run(["ls", "-l"])

doesn't capture output

CompletedProcess(args=['ls', '-l'], returncode=0)

subprocess.run("exit 1", shell=True, check=True)
Traceback (most recent call last):
...
subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1

subprocess.run(["ls", "-l", "/dev/null"], stdout=subprocess.PIPE)
CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0, stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n')

sys
A new set_coroutine_wrapper() function allows setting a global hook that will be called whenever a coroutine object is created by an async def function. A corresponding get_coroutine

Original source Report a problem