proc-macro-srv: Reimplement token trees via immutable trees

[Veykril]

For macros, rust has the concept of TokenTrees and TokenStreams which are basically a tree-like form of the underlying tokenized input with (){}[] delimiters forming Groups (internal nodes) and all other tokens forming leaf nodes of the tree.

In rust-analyzer today these trees are implemented as a flattened list using indices to reduce their memory impact as we keep a lot of these memoized (implementation). This comes at the cost of mutating operations being more expensive though and we especially feel this in proc-macro expansion.

As it turns out, one of the primitive and very commonly used operations on token streams in proc-macros is concatenation, an operation that with our model is notoriously expensive as we need to do a deepclone of the first stream, append the second stream and rewrite the internal indices. Combine this with deeply recursive concatenations as is common with the quote! macro and we run into issues where the proc-macro server can take 8 seconds to expand these 8 derive invocations of num_derive::FromPrimitive on decent machines.

Fortunately, the proc-macro server has no reason to use the same implementation of TokenStreams and TokenTrees as rust-analyzer since it doesn't persist them in memory anways. So instead, to gain good concatenation performance, we can do with what rustc does and store them as immutable trees with shared internal nodes.

Before this change cache priming took 56 seconds on Zed's codebase, with the new proc-macro server it now takes 45.

[bit-aloo]

LGTM. Just some questions.

[ChayimFriedman2]

Mostly skimmed, but LGTM.

[Veykril]

This might've introduced a bug according to https://youtrack.jetbrains.com/issue/RUST-19283