Laguna-S

The sleeper: level with DeepSeek V4 Flash on the hard tasks at 67 GB of weights - under half the footprint. Matching DeepSeek at this size is the standout of the batch.

~67 GB weightsthinking modelno spec-decode (yet)VRAM-light · 4 August 2026· updated 30 August 2026

Grade (mean)

2.28

±0.22 · 50% gold · n=18

Weights

67 GB

vLLM

VRAM serving

94 GB

1 cards

Decode

81.1-104.5

tok/s across context

What to run

Reach for it when VRAM is tight. It matches DeepSeek on the hard tasks at under half the weights, serving flat at ~105-118 tok/s without speculative decode.

A much smaller model that keeps up

Laguna-S lands within noise of DeepSeek V4 Flash on the four hard tasks while carrying less than half the weights. It serves flat at about 105 to 118 tok/s without speculative decode: its DFlash draft head is unstable on these NVLink-less SM120 cards, so it is dropped for now, pending vLLM 0.26. Getting its real quality out first required turning think-mode on - without it the model under-reasons and the grades sag.

By the numbers

Everything below is read live from the benchmark, the same source as the data page. It reflects the current measurement, not a snapshot pasted in when this page was written.

Grade distribution (0-3, 95% CI)

Mean tier across all runs with its 95% interval; the bar shows how attempts spread across the four grades.Drawn against DS V4 0731 for scale.

Decode speed vs context (tok/s)

tok/s vs context

How decode holds up as context fills. no speculative (DFlash head unstable on vLLM 0.25.1 + SM120 - crashes/deadlocks; dropped, retry on 0.26.0)

VRAM while serving: weights vs reserved KV cache

Solid is weights, faint is reserved KV cache that scales with how much context you keep live.

Serving gotchas

DFlash draft head is unstable on SM120

Its speculative head crashes/deadlocks on these NVLink-less cards under vLLM 0.25.1, so it is served spec-off. Watching vLLM 0.26 to re-enable it.
Enable think-mode

The benched quality is the thinking configuration; served without it the model under-reasons and grades lower.

How it's served

Engine: vLLM

Speculative: no speculative (DFlash head unstable on vLLM 0.25.1 + SM120 - crashes/deadlocks; dropped, retry on 0.26.0)

Weights: poolside/Laguna-S-2.1-NVFP4

Download: hf download poolside/Laguna-S-2.1-NVFP4 --local-dir ~/models/Laguna-S-2.1-NVFP4

Serving compose - ~/vllm/compose-Laguna-S-2.1-NVFP4-thinking.yml

# Laguna-S-2.1 (Poolside) NVFP4 - THINKING ON, stable config (spec DROPPED).
# The model's DFlash speculative head is fundamentally unstable on vLLM 0.25.1 + SM120 (2 cards, no NVLink):
# custom-all-reduce illegal-mem crash -> cudagraph capture deadlock -> startup OOM -> runtime OOM @0.7 tok/s.
# So this variant runs the PROVEN no-think bench config (which was stable + fast) with the ONLY change being
# enable_thinking:true. Cudagraphs ON, custom all-reduce ON, util 0.93, no spec. Sliding-window hybrid attn
# still engages automatically. Sampling = model gen_config defaults (temp 1.0), matching the the suite harness.
# Spec (the MTP-equiv speed feature) is a WATCHLIST item -> retry on vLLM 0.26.0/nightly which may fix SM120 spec.
# Serve ONE at a time vs DeepSeek (both :8080; DS4 needs both cards). Stop DS4 first.
services:
  vllm:
    image: vllm/vllm-openai:v0.25.1
    restart: unless-stopped
    ipc: host
    shm_size: "16gb"
    ulimits:
      memlock: { soft: -1, hard: -1 }
      stack: 67108864
    ports:
      - "8080:8080"
    volumes:
      - ~/models/Laguna-S-2.1-NVFP4:/model:ro
      - laguna-cache:/root/.cache
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              device_ids: ['0']
              capabilities: [gpu]
    environment:
      - CUDA_VISIBLE_DEVICES=0
      - CUDA_DEVICE_ORDER=PCI_BUS_ID
      - CUTE_DSL_ARCH=sm_120a
      - VLLM_FLASHINFER_WORKSPACE_BUFFER_SIZE=2147483648
      - PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
      - HF_HUB_OFFLINE=1
      - TRANSFORMERS_OFFLINE=1
      - HF_HUB_DISABLE_TELEMETRY=1
      - DO_NOT_TRACK=1
      - VLLM_NO_USAGE_STATS=1
    command: >
      --model /model
      --chat-template /model/chat_template.jinja
      --served-model-name laguna latest /latest
      --host 0.0.0.0 --port 8080
      --tensor-parallel-size 1
      --disable-custom-all-reduce
      --tool-call-parser poolside_v1
      --reasoning-parser poolside_v1
      --enable-auto-tool-choice
      --default-chat-template-kwargs '{"enable_thinking": true}'
      --kv-cache-dtype fp8
      --enable-prefix-caching
      --enable-chunked-prefill
      --max-num-batched-tokens 8192
      --max-num-seqs 16
      --max-model-len 262144
      --gpu-memory-utilization 0.95
volumes:
  laguna-cache: