gh-109118: Make comprehensions work within annotation scopes, but without inlining

[JelleZijlstra]

This is what I have so far to allow comprehensions within annotation scopes. The first commit changes the tests to what I think is the right behavior, and removes the SyntaxError on comprehensions in annotation scopes. These tests fail because of comprehension inlining; you can see that analogous tests work correctly with genexps (which are not inlined) but not with listcomps (which are). The second commit is an attempt to fix the tests, inspired by #104528, but it does not work yet.

cc @carljm

• Issue: Annotation scopes containing nested scopes #109118

[carljm]

I looked at this a bit, and my initial conclusions aren't promising. I think this may not really be possible to support, at least not without extensive hackery.

If we consider this test case:

            class C[T]:
                T = "class"
                class Inner[U](make_base([T for _ in (1,)]), make_base(T)):
                    pass

The problem is that in the annotation scope of Inner, there is already a reference to T (in make_base(T)). The goal of this PR is to inline the comprehension [T for _ in (1,)] into that same scope, but somehow have the inlined T symbol behave differently from the existing T symbol in the same scope. Since all symbol information is tracked by name, that would require renaming the inlined T, somehow tracking that the renamed symbol is "really" named T, and extending that knowledge into the compiler. That sounds quite painful.

I think the more feasible options are either to disable comprehension inlining in annotation scopes, or to accept that the visibility effects of comprehension inlining leak here.

Given that comprehension inlining is usually transparent, and that complex code in annotation scopes is unlikely, I think just disabling comprehension inlining in this case is reasonable.

(If I had a time machine, I think I would inline comprehensions only in function scopes and nowhere else, as I'd initially proposed in the first version of PEP 709.)

[JelleZijlstra]

Thanks. I can't think of a tractable solution either, so I pushed the non-inlining version.

[carljm]

Thank you!