diff --git a/images/2024-12-new-graph.png b/images/2024-12-new-graph.png new file mode 100644 index 000000000..b344f65c2 Binary files /dev/null and b/images/2024-12-new-graph.png differ diff --git a/images/2024-12-old-graph.png b/images/2024-12-old-graph.png new file mode 100644 index 000000000..68ba24d9d Binary files /dev/null and b/images/2024-12-old-graph.png differ diff --git a/posts/2024/12/towards-pypy311-an-update.md b/posts/2024/12/towards-pypy311-an-update.md new file mode 100644 index 000000000..2bf1d7e69 --- /dev/null +++ b/posts/2024/12/towards-pypy311-an-update.md @@ -0,0 +1,112 @@ + + +We[^0] are steadily working towards a Python 3.11 interpreter, which will be part +of the upcoming PyPy 7.3.18 release. Along with that, we also recently updated +[speed.pypy.org](https://speed.pypy.org) to compare PyPy's performance to CPython +3.11 (it used to be CPython 3.7). + +# Why is there no PyPy for Python 3.11? +TL;DR: we are working on it and hopefully will have a beta version soon + +We started by merging the [exception groups](https://peps.python.org/pep-0654/) +work by Nico Rittinghaus, merging the CPython stdlib for Python 3.11.9, and +updating the regex engine to handle atomic groupings. I think these were the +largest changes needed to support Python3.11, maybe I missed something else? + +We then created a [milestone](https://github.com/pypy/pypy/milestone/15) with +many of the changes that might not be caught in testing. As of today that +milestone stands at 24/40 issues closed, 16 open. This is in addition to the +[v7.3.18 milestone](https://github.com/pypy/pypy/milestone/22), which has 10/23 +issues closed, 13 open. + +We also updated our infrastructure to run nightly [buildbot test of the +py3.11](https://buildbot.pypy.org/summary?branch=py3.11) branch, including adding py3.11 to our benchmarking site [speed.pypy.org](https://speed.pypy.org/). + +Then the real work started: fixing these milestone issues and failing stdlib +tests. Some of the changes were cosmetic changes to error messages, some were +more involved changes to the interpreter to behave more like CPython. For instance, +`hex(x)` where `x` is an int calls `long.format(x, "#x")` on CPython where in +PyPy we used `x.__format__("#x")`. This subtle difference caused a failure in +the `repr` of `IntEnum`. Tracking down problems like these takes time. We are +now down to about 250 failing stdlib tests, with thousands passing. For comparison, +PyPy3.10 has around 100 failing stdlib tests. + +## C-extension support +PyPy supports the Python C-API via a cpyext compatibility layer. We "mangle" +the CPython symbols to add an extra `Py` to prevent loading CPython c-extension +modules into PyPy, since the ABI is different. So a function like +`PyLong_FromLong` will be exported from the c shared object as +`PyPyLong_FromLong`. One of my long-standing goals is to remove this mangling, +but it then requires that our c declarations, inlined functions, and macros in +the headers match 1:1 the CPython headers. We can get by with no implementing +parts of the interfaces, but what is declared must be identical. This is a +long-term project, with each release the headers get closer to the CPython +versions. I hope to concentrate on the PyUnicode interfaces for this release. + +Note that the time to do this is before a new version release. Once the version +is released, we cannot change the headers significantly. + +## So what is left? +Summarizing the milestones and other things to do: +- Update to the recently released 3.11.11 stdlib +- Make sure [vmprof](https://vmprof.readthedocs.io/en/latest/) works +- Update the `time` module to use more `MONOTONIC_CLOCK`, implement + `time.sleep` differently, and clean up the many duplicate `time` like + interfaces we have across the codebase. We have the `time` module, some time + routines in the `_threading` module` and RPython threading support in the + `rpython/rlib` code. We should also make sure we are using 64-bit time + interfaces. +- Decide whether zero-cost exceptions gain us in performance and whether we + should implement them even if they do not improve performance. +- Update our [hpy](http://hpyproject.org/) backend to the latest HEAD, which + would allow running the [hpy numpy + fork](https://github.com/hpyproject/numpy-hpy/tree/graal-team/hpy#readme) +- Reintegrate the pure-python pyrepl libbrary from CPython 3.13. + +What else did I forget? + +# Why did the benchmark results get worse on speed.pypy.org? +TL;DR: running a benchmark site is hard. Something changed in the benchmark +runner, and suddenly benchmarks got 10-15% slower. + +PyPy run an instance of +[codepseed](https://github.com/python/codespeed/tree/speed.pypy.org) with a +very old [benchmarking suite](https://foss.heptapod.net/pypy/benchmarks) +that can still run Python2 (remember: that is the language of +[RPython](https://rpython.readthedocs.io/en/latest/) underlying PyPy). The site +runs on PSA infrastructure and the benchmarking machine is generously sponsored +by [Baroque Software](https://baroquesoftware.com/). On Nov 9, there was a +sudden jump in benchmarking times. For instance +[here](https://speed.pypy.org/timeline/#/?exe=21,8&base=10+2622&ben=float&revs=50&equid=off&quarts=on&extr=on&env=3) +is the result for the `float` benchmark. This happened across various benchmark +runs: both PyPy2.7 and PyPy3.11alpha, with and without the JIT. After spending +some time rerunning various benchmarks, I could only conclude the machine +itself had gotten slower, maybe due to some security update in the linux +kernel, maybe some change in the hosting platform. This "broke" the front-page +comparsison: suddenly "latest" is much slower than the historic benchmarks run +previous to the changes in the machine. + +That page also recently (as of last week) uses CPython 3.11 as a baseline for +comparison, where previously it used CPython3.7. It is common knowledge that +the newer CPython versions are faster, and we see this now quite clearly. +Diving into individual benchmarks, we can see that ones where PyPy-with-a-jit +was comparable to CPython3.7 seem to be the ones that CPython3.11 improved +greatly. So the new graph has more lines that extend past 1.5 than the old +graph. ![new graph](/images/2024-12-new-graph.png) ![old +graph](/images/2024-12-old-graph.png). + + +[^0]: These days most of the work is done by [CF + Bolz-Tereick](https://cfbolz.de/) and [me](https://github.com/mattip) on a +volunteer basis. Want to get involved? Reach out, we would love to expand the +team. Have an idea for funding the work? Fantastic, let's talk. diff --git a/requirements.txt b/requirements.txt index bee17c26d..9742edffd 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,5 +1,5 @@ nikola==8.3.1 -markdown==3.3 +markdown==3.4 jinja2 aiohttp watchdog