Python 3.12 Overview: Past Limitations, Faster CPython's Advent, and the Excitement of New Versions

Python, since its inception, has established a unique spot in the sequence of programming languages. Revered for its simplicity and readability, it has been the foundation for numerous applications, spanning web development to scientific research.

Yet, there has always been an asterisk attached to its acclaim — its performance. For a considerable amount of time, Python bore the tag of being a "slow" language, a sentiment repeated even today in certain quarters. This reputation, however, faced a potential shift a few years ago with the emergence of Faster CPython. Focused on turbocharging Python's performance, this new incarnation aimed to not only address the age-old speed concerns but also navigate through a spectrum of legacy issues.

With this being said, let’s now dive deeper into the intricacies of this very legacy, find out what is new about one of the latest versions of the programming language, and shed light on the reasons it has been receiving so much attention in recent times.

Faster CPython: Background And How Things Are Now

Guido Van Rossum, the esteemed creator of Python, recently aligned himself with Microsoft, turning all eyes toward the developments in the latest versions of Python.

At the 2021 Python Language Summit, van Rossum unveiled his ambitious endeavor: a collaboration with a dedicated team he gathered backed by Microsoft's funding. Their mission is crystal clear — to channel performance enhancements into the interpreter while ensuring the integral C API remains intact. This move is crucial, especially to guarantee that Python extensions continue to function seamlessly.

The subject of performance augmentation for CPython is hardly new — it has, in fact, been a recurring theme at annual language summits. Mark Shannon, in his proposition, shed light on a strategic plan poised to enhance the language's speed by a staggering 5x. This plan, already manifesting its efficacy in Python 3.11, showcases a promising trajectory.

Central to these advancements is the Faster CPython project. The brilliance of it is in its innovative approach to the default CPython interpreter. By using the strength of just-in-time compiler technology, it identifies code segments that can immensely benefit from this sophisticated solution. The result is the creation of new directives that operate at a quicker rate, leading to an overall faster performance of the Python language. Notably, van Rossum called upon collaborative work on GitHub in a transparent manner. The Faster CPython repositories serve as a hub for the code, concepts, tools, issue management, feature dialogues, and more.

Real-world benchmarks already echo the success of this work. The Python 3.11 alpha 6 release boasts a remarkable performance uplift of approximately 19% compared to its predecessor, Python 3.10. And with the release of Python 3.11 alpha 7, expectations are soaring even higher. Embracing Shannon's projection of a 1.5x performance surge annually, Python 3.11 is already demonstrating the results of these labors.

Presently, we stand at the zenith of this transformative stage, with the tech community eagerly awaiting to discern the fruits of years of relentless endeavor aimed at Python's performance advancement. It’s time to take a closer look at what is awaiting in version 3.12 to assess and make further predictions about Python’s acceleration. But first, let’s recap the previous ones for the sake of continuity.

Python's Evolution: Versions 3.9 to 3.11

As was mentioned previously, with van Rossum's move to Microsoft, Python received even more attention, and this happened around version 3.9. Within this version and the consequent ones, a myriad of features and improvements emerged. Performance optimizations were especially prominent. This push for speed was not just a mere numerical increase — it was a tangible enhancement that developers felt in their everyday coding tasks. Such advancements allowed for quicker code execution, reduced resource consumption, and an overall smoother Python experience.

Additionally, these versions steered in various features that enrich the language's capabilities. From improved handling of data types to enhanced libraries, developers found themselves geared with more powerful tools at their disposal.

The community's response to these updates has been overwhelmingly positive. Forums, blogs, and discussion threads are replete with developers sharing their experiences, offering tips, and eagerly experimenting with new features.

As expected, this enthusiasm has not been confined solely to past updates. The anticipation for Python 3.12 is palpable. Speculations about its potential features, performance upgrades, and the trajectory it might set for Python's future are already gaining momentum.

Python 3.12: Major Features and Enhancements

F-strings Syntactic Makeover

Now, any valid Python expression can be used within the expression parts of f-strings. This includes using the same quotation marks as the outer f-string, extending expressions over multiple lines, adding comments, using backslashes, and incorporating Unicode escape sequences.

Comprehension Inlining

Comprehension inlining in Python has undergone optimization, where dictionary, list, and set comprehensions are now inlined, which removes the need to create a new single-use function object each time comprehension is executed, potentially doubling the speed of comprehension execution. However, inlining introduces some behavioral changes, such as modified traceback displays, alterations to the 'symtable' module, and variations in the behavior of the 'locals()' function within comprehensions.

Minimal overhead monitoring in CPython

PEP 669 introduces a new API in CPython that enables tools like profilers and debuggers to monitor various events, such as calls, returns, and exceptions, with increased efficiency. This ensures minimal overhead, making debuggers and coverage tools more performance-friendly. Notably, since Python 3.12, the interpreter can be operated in a special mode that integrates with the Linux 'perf' profiler, a powerful tool with a robust ecosystem, allowing Python functions to be visible in its output.

Improved Modules: Asyncio

Asyncio's socket writing performance has seen a considerable boost. It now minimizes redundant copying during socket writes and utilizes sendmsg() where the platform allows.

Enhanced Specialisations

Specialization in virtual machines for dynamic languages, like Python, is the process of tailoring the code execution based on the specific types and values present in the program.

A meticulous analysis has been conducted to discern which bytecodes would gain immensely from specialization. The objective for Python 3.12 is to finalize the remaining high-impact specializations, reaping the rewards in performance.

This adaptive and speculative specialization boosts performance by potentially up to 50%, ensuring Python runs more efficiently even as it dynamically responds to changes in code patterns, without incurring high costs due to mis-specialization. Here is a central issue to monitor which bytecodes are already specialized, which ones require specialization, which could benefit from improvements, or which ones aren't suitable for specialization.

Multi-threaded Parallelism

Historically, Python's major obstruction has been its single Global Interpreter Lock per process, inhibiting multi-threaded parallelism. Python 3.12 addresses this limitation as detailed in PEP 684, transitioning to a Global Interpreter Lock (GIL) per sub-interpreter. Alongside, PEP 554 aims to allow the creation of subinterpreters directly from Python, an attribute previously exclusive to the C API. This change signifies the onset of genuine multi-threaded parallelism.

Multi-threaded parallelism represents a foundational shift for Python, setting the venue for future innovations rather than offering an immediate out-of-the-box solution. Planned for a more tangible API in Python 3.13, it currently stands as a framework awaiting further refinement. Crucially, as outlined in this part of PEP 554, this feature ensures robust isolation between interpreters; data can only be exchanged between them through serialization, like converting an ‘int’ object into its underlying raw data, thereby assuring data integrity and preventing data mingling.

Tooled Memory Management

Python 3.12 aims for efficient memory utilization. It is done by decreasing the size of objects (see the instance here) as well as by adopting a more consistent object layout. These enhancements in the memory structure optimize allocation and deallocation processes.

Deprecated Features

Continuing its tradition of maintaining a clean and efficient codebase, Python 3.12 will remove several previously deprecated features from the 'unittest' module. This list includes method aliases, certain assert methods, and specific parameters in TestLoader, ensuring that the module remains lean and up-to-date.

Furthermore, PEP 623 introduces a significant optimization to the memory footprint of Python. By removing the 'wstr' from Unicode objects in Python's C API, every 'str' object will be at least 8 bytes lighter. This change offers potential performance improvements and makes the memory usage of Python programs more efficient.

Additionally to the important deprecations, the 'distutils' package will no longer be a part of the standard library. Developers are directed to a migration guide for alternatives to the functionalities previously provided by this package. Notably, for those who still need 'distutils,’ the third-party package 'setuptools' will continue to offer it.

Impact of Faster CPython

Directly and indirectly, Faster CPython's influence on these features is undeniable. By continually pushing the boundaries of performance and optimization, it sets the stage for Python 3.12's monumental advancements.

From this, we can witness now that Python 3.12 promises a blend of speed, efficiency, and versatility. With its large number of enhancements (some of them are not mentioned here; you can find them on the official page), it's aimed at redefining the coding experience.

Reflecting on Python's trajectory and its strides to strengthen performance, the advent of Faster CPython, no doubt, emerges as a turn in its rich narrative. The sheer volume of contributions, open-source projects, and active discussions around issues showcases a vibrant community eager to push Python's boundaries, ensuring it remains at the forefront of programming excellence. The future of Python seems promising due to the way it is shaped by those engaged.