Release with the improved Prism::Translation::Ripper

[eregon]

We're using Prism::Translation::Ripper as the implementation for ::Ripper in TruffleRuby.
This seems to work fine, but currently I'm importing a commit from ruby/prism main branch.

It would be better to import a Prism release, so that e.g. gem info prism would report the right version.
Currently with that commit from main it would report itself as 1.8.0 but that's not correct since it has many extra commits.

This is not urgent and I'm more than happy to include more fixes/optimizations from @Earlopain if he plans any (do you?).

@kddnewton On a slightly related note I wonder if you'd be interested to avoid so many copies of the version number across the codebase.
I recall #1295 but that didn't work when imported in CRuby.
I could do a safer variant that just share the version for files generated by template.rb if you're interested.
It's probably not a big deal though looking at https://github.com/ruby/prism/blob/main/docs/releasing.md#preparation.

Related: #3838

[Earlopain]

This is not urgent and I'm more than happy to include more fixes/optimizations

No more fixes, it seems good enough. I was looking for more things to optimize but it was all without much impact so not really worth a PR for the extra complexity (main...Earlopain:prism:more-ripper-optimizations)

That, and the majority of time is spend here:

prism/lib/prism/lex_compat.rb

Lines 607 to 608 in 61e1207

	lineno = token.location.start_line
	column = token.location.start_column

It binary searches through the offsets incredibly often but the tokens are basically already sorted (ignoring heredocs). But I don't really have the motivation to try, overall it is also good enough for me.

[eregon]

Took a quick look at your 2 commits, the first one is I think worth it and a small/easy change. I'd just use nonzero? since it's made for that, like (line_a - line_b).nonzero? || column_a - column_b.
Could probably also be (line_a <=> line_b).nonzero? || column_a <=> column_b to use the "sorting operator".
Could you open a PR with that?

The second commit I agree seems too much trouble. I think if we want to speed that up we should look into the O(1) vs O(log n) solution for mapping offsets to line, apparently CRuby uses that internally and IIRC it's been mentioned on this issue tracker in the past.

[eregon]

(I do wonder why we actually need to sort, tests do fail if we don't, does original Ripper sort too?)

[Earlopain]

Could you open a PR with that?

Yeah, sure. #3870. This one is even better. I tried <=> like you mentioned and that was very good for YJIT but actually worse without. The PR is consistently better regardless.

I think if we want to speed that up we should look into the O(1) vs O(log n) solution for mapping offsets to line

Sounds nice. But I don't know much about the defaults. If you happen to find some reference, I'm happy to take a look.

(I do wonder why we actually need to sort, tests do fail if we don't, does original Ripper sort too?)

Most of ripper returns the tokens sorted. Ripper.lex does for example. There is Ripper::Lexer.new("<<~FOO\nFOO").parse which returns them unsorted but it is undocumented. The other Lexer methods are implemented on top of it and they always sort. That is the only API that does this as far as I remember.