English · 简体中文 · 日本語

News · Speed · Exact answers · Quick start · Models · Options

Flide is fast inference infrastructure for Jev-style decision models. With exact state forking it is up to 9.4 times faster than vLLM and SGLang, and every answer is the one the model gives the question alone. It serves the Jev /v1/systemone API.

🔥 News

• 🔥 2026-11-02 We released our paper Flide: Accelerating Decision Inference with Exact State Forking on arXiv.
• 🔥 2026-10-29 We added three decision model families in Flide 0.1.7, including SynACK Decide, Wald and Xor, and Gemma 4 mixture-of-experts models such as KnowLine-26B-A4B.
• 🔥 2026-10-29 We added two decision model families in Flide 0.1.6, including Drex and KnowLine, and flide verify checks exact forking on your own model and requests.
• 🔥 2026-10-28 We added six decision model families in Flide 0.1.5, including Jeffan, JevAny, Kev, Lychee, pplx-decider and StartLux, and images for Lumma and mammoth.
• 🔥 2026-10-26 We added five decision model families in Flide 0.1.4, including cerno, JADE, Linne, mammoth and Sieve, and also Quyet-1.0-Large.
• 🔥 2026-10-26 We added eight decision model families in Flide 0.1.3, including AJev, Bespoke-Nimble, Darwin, jiwo, Lumma, mogan, Quyet and Senova, and LoRA adapters now serve as they are.
• 🔥 2026-10-25 We added two decision model families in Flide 0.1.2, including Intern-Decision and torchcast, and a question now takes up to 256 options.
• 🔥 2026-10-24 We released Flide 0.1.0 on PyPI, serving Qwen3.5 and Gemma 4 chat models and the OneJev and Open-Jev decision models. Welcome to have a try.

With three questions, a generation engine runs three sequences and computes 4 cells of the state again, and multi-item scoring adds 3 delimiters that change the answers. Flide runs one sequence, computes nothing twice, adds no tokens, and gives each question the answer it gets when asked alone.

Speed

On one H200 with Qwen3.5-0.8B, Flide is up to 9.4 times faster than both vLLM and SGLang (QuALITY, where it is 14 times faster than vLLM), 3.6 to 4 times faster on RACE and the Decision Index, and serves 2.4 times the throughput with 64 clients. The more questions a state carries, the bigger the gap.

Exact answers

Flide stays within 1.1 points of a float32 reference, the closest of any engine, and flips at most 2.3% of decisions at 0.5. Multi-item scoring moves answers by up to 14.8 points and flips up to 28.9%. With --batch-invariant, a request gets the same bits however it is batched.

Quick start

See the docs for every model, the API and all options.

1. Install

uv venv -p 3.12 && source .venv/bin/activate
uv pip install flide torchvision

2. Start a server

Any chat model on Qwen3.5 or Gemma 4 serves as it is and answers through its next token.

flide serve --model Qwen/Qwen3.5-4B

OneJev models read their answers their own way, and Flide serves them as they are too.

flide serve --model OmniJev/OneJev1.6-0.8B

An Open-Jev release is an adapter on a Qwen3.5 base. Merge it into a checkpoint once, then serve that.

python -m flide_open_jev.prepare ZefanCai/Open-Jev-2B open-jev-2b
flide serve --model open-jev-2b

flide serve listens on all interfaces at port 8000; pass --host 127.0.0.1 to keep it local.

3. Send a request

curl http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d '{
  "state": "Order #1182 arrived with a cracked screen. The customer asks for a refund and attached two photos.",
  "questions": {
    "action": {"type": "choice", "instructions": "What should support do next?",
               "criteria": {"refund": "refund the order", "replace": "send a replacement", "escalate": "ask a human"}},
    "urgent": {"type": "noul", "instructions": "Does this need a reply within the hour?"}
  }
}'

OneJev models also read images. Put <image:1> where the photo belongs in the state and send the photo in media.

IMG=$(base64 -w0 photo.jpg)
curl http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d @- <<EOF
{
  "state": {"ticket": "Order #1182 arrived with a cracked screen. The customer asks for a refund.", "photo": "<image:1>"},
  "media": [{"type": "image", "data": "data:image/jpeg;base64,$IMG"}],
  "questions": {
    "cracked": {"type": "noul", "instructions": "The photo shows a cracked screen"},
    "action": {"type": "choice", "instructions": "What should support do next?",
               "criteria": {"refund": "refund the order", "replace": "send a replacement", "escalate": "ask a human"}}
  }
}
EOF

Every answer carries the probability of each option, ready for a threshold.

Models

The last three columns say how much faster Flide answers one request than the engine its release ships, as the median latency over the requests of each workload on one H200, where DI is the Decision Index. The chat models run against the faster of vLLM and SGLang, and OneJev against qev, the engine it used before Flide. KnowLine 35B runs with --quantize fp8, as its release serves it in FP8. A value marked ~ is the average over the sizes tested.

Model	Inputs	DI	MultiRC	QuALITY
AJev	text	3.7×	9.3×	2.6×
Bespoke-Nimble	text	2.6×	2.1×	1.6×
cerno	text	~4.1×	~8.6×	1.1×
Darwin	text	1.6×	7.0×	1.2×
Drex	text	~3.4×	~7.9×	~1.7×
Gemma 4	text, images	~1.3×	~1.7×	~1.0×
Intern-Decision	text, images	4.4×	1.1×	1.5×
JADE	text	2.0×	9.2×	1.7×
Jeffan	text	3.1×	2.6×	1.9×
JevAny	text, images	1.5×	5.8×	1.2×
jiwo	text	~1.9×	~5.9×	1.8×
Kev	text	~7.9×	~1.5×	~1.7×
KnowLine	text	~1.4×	~5.6×	~1.5×
Linne	text	2.3×	8.6×	1.2×
Lumma	text, images	1.7×	7.1×	1.2×
Lychee	text	1.4×	1.5×	1.4×
mammoth	text, images	~4.3×	~2.8×	~1.8×
mogan	text	1.6×	8.5×	1.8×
OneJev	text, images, video	~6.8×	7.3×	4.7×
Open-Jev	text	4.2×	5.0×	~1.5×
pplx-decider	text, images	3.1×	1.4×	1.2×
Quyet	text	~3.9×	~7.1×	~1.6×
Qwen3.5	text, images	~2.1×	~4.8×	~1.3×
Senova	text	3.1×	28.1×	1.2×
Sieve	text	3.0×	14.3×	2.6×
StartLux	text	2.6×	1.1×	1.2×
SynACK Decide	text	3.3×	11.9×	3.0×
torchcast	text, images	1.6×	8.4×	1.4×
Wald	text	1.6×	12.7×	1.5×
Xor	text, images, video	2.5×	6.2×	2.8×

Options

Option	What it does
--session-cache-gb 40	keeps the states of earlier requests, so an agent's next step reads only what is new
--batch-invariant	the same bits for a request whatever it is batched with, at some cost in speed
--quantize fp8	Qwen3.5 in FP8 on Ada, Hopper and Blackwell GPUs
--frontends 2	HTTP, rendering and tokenizing in two processes of their own, for many clients

License

Apache-2.0. flide/kernels/gdn.py holds a kernel derived from two kernels of flash-linear-attention (MIT); THIRD_PARTY_NOTICES lists it with the other code Flide builds on.

Citation

If you use Flide in your research, please cite our paper.

@article{li2026flide,
  title   = {Flide: Accelerating Decision Inference with Exact State Forking},
  author  = {Li, Bobo and Song, Jiajia and Fei, Hao and Hsu, Wynne and Lee, Mong Li},
  journal = {arXiv preprint arXiv:2610.42148},
  year    = {2026}
}