bpo-46564: Optimize super().meth() calls via adaptive superinstructions

[Fidget-Spinner]

They should now have almost no overhead over a corresponding self.meth() call.

Summary of changes:

• typeobject.c -- refactoring to reuse code during specialization, also use InterpreterFrame over PyFrameObject for lazy frame benefits. Some changes here are partially taken from bpo-43563 : Introduce dedicated opcodes for super calls #24936. All credits to @vladima (I've tried to properly include them in the news item too.)
• specialize.c -- specialize for the 0-argument and 2-argument form of super().
• ceval.c -- does both a CALL and LOAD_METHOD without intermediates (and both are specialized forms too).

TODO:
benchmarks!

https://bugs.python.org/issue46564

[Fidget-Spinner]

Marking as draft as I need make this work with the new CALL convention.

[markshannon]

Can't we do this at specialization time? The number of locals, the index of "self", and whether it is a cell are all known then. Likewise the nature of __class__ is also known.

[markshannon]

When can this fail?
Isn't the next item in the MRO determined solely by type(self) and __class__, both of which are known at this point?

[Fidget-Spinner]

I wanted assurance that __class__ didn't change.. Then again, I'm not sure if it can?

[markshannon]

Maybe we should merge #31002 first, as that PR is simpler.
It would also allow us to compare the performance of just the specialization, without the removal of frame object allocation.

[Fidget-Spinner]

Maybe we should merge #31002 first, as that PR is simpler. It would also allow us to compare the performance of just the specialization, without the removal of frame object allocation.

👍

[Fidget-Spinner]

Mark, I'm going to run benchmarks on deltablue first since it uses 2-argument form super. I'll address your optimization ideas for the 0-arg form once those results come back. Fingers crossed.

[arhadthedev]

A couple of indentation-related inconsistencies:

[arhadthedev]

Suggested change

	PyTypeObject **type_p, PyObject **obj_p)
	PyTypeObject **type_p, PyObject **obj_p)

The line was aligned with an opening parenthesis of a parameter list.

[arhadthedev]

Suggested change

	PyObject *kwnames, SpecializedCacheEntry *cache, PyObject *builtins,
	PyObject **stack_pointer, InterpreterFrame *frame, PyObject *names);
	PyObject *kwnames, SpecializedCacheEntry *cache, PyObject *builtins,
	PyObject **stack_pointer, InterpreterFrame *frame, PyObject *names);

as in a removed line, or even:

Suggested change

	PyObject *kwnames, SpecializedCacheEntry *cache, PyObject *builtins,
	PyObject **stack_pointer, InterpreterFrame *frame, PyObject *names);
	PyObject *kwnames, SpecializedCacheEntry *cache, PyObject *builtins,
	PyObject **stack_pointer, InterpreterFrame *frame, PyObject *names);

as in _Py_Specialize_BinaryOp right below.

[Fidget-Spinner]

Mark, I'm going to run benchmarks on deltablue first since it uses 2-argument form super. I'll address your optimization ideas for the 0-arg form once those results come back. Fingers crossed.

Well that was depressing. deltablue only shows 1.03x speedup. Looking closer at the code, super isn't called in any tight loops so that might be why. Maybe I need to pull out microbenchmarks now.

[Fidget-Spinner]

Microbenchmarks show that super() has sped up by more than 2.2x. This is faster than that other attempt because there's also speedups from the LOAD_METHOD_CACHED:
(Extremely unscientific, I'm short on time to set up pyperf right now)

import timeit

setup = """
class A:
    def f(self): pass
class B(A):
    def g(self): super().f()
    def h(self): self.f()

b = B()
"""

# super() call
print(timeit.timeit("b.g()", setup=setup, number=20_000_000))
# reference
print(timeit.timeit("b.h()", setup=setup, number=20_000_000))

Results:

# Main
5.796037399995839
2.4094066999969073

# This branch
2.4578273000006448
2.3718886000060593

So super().meth() is now only ~10% slowly than the corresponding self.meth() call whereas it was nearly 2x as slow previously. If I manage to incorporate your suggestions correctly, this will effectively just be a competition between LOAD_GLOBAL_BUILTIN (super) and LOAD_FAST (self).