regression: proc-macro derive unparseable

[Mark-Simulacrum]

From https://crater-reports.s3.amazonaws.com/beta-1.52-2/beta-2021-05-23/gh/omegablitz.gluon-bug/log.txt

[INFO] [stdout] error: expected one of `{`, lifetime, or type, found `,`
[INFO] [stdout]    --> /opt/rustwide/cargo-home/git/checkouts/gluon-9c68e1737197fd9b/77652ab/base/src/types/mod.rs:522:17
[INFO] [stdout]     |
[INFO] [stdout] 522 | #[derive(Clone, AstClone)]
[INFO] [stdout]     |                 ^^^^^^^^ unexpected token
[INFO] [stdout] ...
[INFO] [stdout] 543 |     T: TypePtr<Id = Id>,
[INFO] [stdout]     |                         - expected one of `{`, lifetime, or type
[INFO] [stdout]     |
[INFO] [stdout]     = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] error: proc-macro derive produced unparseable tokens
[INFO] [stdout]    --> /opt/rustwide/cargo-home/git/checkouts/gluon-9c68e1737197fd9b/77652ab/base/src/types/mod.rs:522:17
[INFO] [stdout]     |
[INFO] [stdout] 522 | #[derive(Clone, AstClone)]
[INFO] [stdout]     |                 ^^^^^^^^

cc @petrochenkov @Aaron1011 -- any recent parser changes you can think of that might've caused this?

[Aaron1011]

This appears to have already fixed in gluon-lang/gluon#889. I'm not sure why this is just now showing up again on Crater.

[Aaron1011]

Oh, I see - the affected derive invocation also has a #[cfg_attr] attribute on the same attribute target. This previously meant that we would pretty-print and retokenize the input, which would cause the trailing comma to get lost.

This is expected fallout from #82608 - preserving the exact TokenStream may expose bugs in user-written proc macros. Fortunately, the bug in gluon_codegen is already fixed.

[Mark-Simulacrum]

Tagging as relnotes (cc @XAMPPRocky) and removing prioritization request; I expect to close as won't fix with a note that duplicate trailing commas are no longer accepted placed in the compatibility notes.

[Mark-Simulacrum]

@Aaron1011 is https://crater-reports.s3.amazonaws.com/beta-1.52-2/beta-2021-05-23/gh/yejingchen.jieba-bot/log.txt (source code in https://docs.rs/crate/tbot/0.6.7/source/src/errors/https_webhook.rs) basically the same error as here? It looks like it's parsing the Path, as a tuple perhaps -- seems a bit different.

[Aaron1011]

@Mark-Simulacrum: Yes, the underlying cause (exposed bug due to preserving tokens when #[cfg_attr] is involved) is the same. That bug also has a fix merged in kdy1/is-macro#8.