Mockup. Simulated data; model loads run 10× faster.

ALBASTOV. LabLocal LLM

Local LLM

Qwen 3.8 27B is loaded on Vegeta and ready.

ModelOnly models that batch can run many sessions at once

SessionsEach one is its own conversation. Up to 64 fit in one batch.

30

PromptEvery session gets it with its own seed, so answers differ

Waiting for first tokenStreaming, with tok/sDone, with time to first token

Bars show tokens out of the 256-token limit. Select a session to read it below.

Total speed, tok/s

ModelAny model. Ones that are not loaded load first, and that wait is shown.

Prompt