fix: false positive E0109 for shift operators

[Aditya-PS-05]

closes #20958
also fixes #3861

Earlier when a type parameter in a macro is followed by the << shift operator, rust-analyzer incorrectly treats it as the start of generic arguments, triggering a false positive E0109 error.

Cause

In opt_path_type_args(), the parser checks if < starts a generic argument list but it incorrectly parses shift operators as generic args and also incorrectly treats nested generics.

Fix

Added a look ahead check to examine the token after <<:

• A type/path start token (ident, Self, super, crate, self) → it treats them as nested generics
• Otherwise (e.g., literals, expressions) → it treat them as shift operator

I fixed the broken test_issue_3861 and added a new test also.

[ChayimFriedman2]

No no no. The error is not in the parser, and the fix need not be in the parser, definitely not such workaround.

The error is in macro expansion, where we handle transparent groups incorrectly.

[Aditya-PS-05]

No no no. The error is not in the parser, and the fix need not be in the parser, definitely not such workaround.

The error is in macro expansion, where we handle transparent groups incorrectly.

Sir, I think this issue is not in the macro expansion, as in both rustc and rust-analyzer, macro expands to 1 as u8 << 2 from 1 as $t << 2 and then parser thinks << is generic args and works differently.

Can I get more inputs and exact location where you think the Macro expansion error is happening?

[Aditya-PS-05]

@ChayimFriedman2 please review it.

[ChayimFriedman2]

I don't think ty fragments are unique in the way they capture, and we definitely shouldn't have a very hacky special delimiters for them.

This is not an issue that can be just solved. It is a fundamental problem with our macro expander (rustc works a bit differently), and will require design work before trying to solve it. The likely solution will be to rewrite the entirety of our expander to match rustc.

[rustbot]

☔ The latest upstream changes (possibly #21804) made this pull request unmergeable. Please resolve the merge conflicts.

[ChayimFriedman2]

Closing since this is not fixing the real problem, as I explained above.