Skip to content

Releases: PythonNut/superbpe

Tokenizer Transition Point Sweep

Pre-release

Choose a tag to compare

@PythonNut PythonNut released this 28 May 20:01

This is the full set of trained tokenizers from Figure 1 of our paper.