Fix parser when using Python 3.9

[gvanrossum]

The structure of the AST for subscripts was changed in
python/cpython#9605

Fixes #8627

Also fix the majority of the errors produced when running pytest with Python 3.9, but not all.

[gvanrossum]

Hm, there are some errors in the 3.9 test run. Let me look into these.

[gvanrossum]

Okay, there are some remaining Python 3.9 test failures, but they seem to be just cases where Python 3.9 somehow produces different output. Examples:

Expected:
  main:3: error: keyword argument repeated      (diff)
Actual:
  main:3: error: "f" gets multiple values for keyword argument "a" (diff)

(The new parser produces a better error message for duplicate keyword args.)

Also a few parse trees are different, e.g.:

Alignment of first line difference:
  E:       TupleExpr:-1(
  A:       TupleExpr:1(
                     ^

(The old parser didn't have a line or column number for certain Tuples.)

Finally:

Expected:
  Daemon started                                (diff)
  mypy-daemon: error: Missing target module, package, files, or command. (diff)
  == Return code: 2
Actual:
  mypy-daemon: error: Missing target module, package, files, or command. (diff)
  Daemon started                                (diff)
  == Return code: 2

Seems the two error lines are produced in a different order.

[msullivan]

What was going on here?

[gvanrossum]

Many tests suddenly showed the full pathname in the traceback instead of simply File "driver.py", line NNN. I don't actually know why, but I could imagine that somehow an absolute path was computed by Python where it previously believed the pathname on the command line? It's always about "driver.py" which I presume is the synthetic script created by the test harness.

[gvanrossum]

Playing around, this just seems new behavior in Python 3.9, so I think my fix is the best we can do.

[msullivan]

Thanks, this looks good!

I'm not actually sure if we have any kind of principled way to deal with version mismatches like this; I think we normally just hack around it... We could add -only_post_39 and -only_pre_39 magic test name suffixes to split_test_cases, I guess, or just disable or tweak the tests.

The daemon one is bizarre and probably needs real investigation; maybe its something involving the interleaving of stdout and stderr?

[gvanrossum]

Merging now, we can iterate later.

[hauntsaninja]

Opened #8743 for fixing parse trees.
The full path name in the traceback is probably caused by https://bugs.python.org/issue20443 (as mentioned in 3.9's what's new).