Surogate Train

Train and serve your own model.

One open-source engine to fine-tune, run RL and serve the result over the APIs your code already uses.

curl -LsSf https://github.com/invergent-ai/surogate/releases/latest/download/install.sh | bash
Docs GitHub Apache 2.0

A team scores its fine-tunes against a big model, on questions the training chats never asked. Filmed on a demo business.

Your model, for your work.

When a general model is not enough, or may not leave your network.

From your data to a served model.

Five steps, one engine, the same terminal.

  1. 01

    Your data in

    One YAML file names the base model, the recipe and your datasets. Python configures it; native C++/CUDA runs it.

    # train.yamlmodel: Qwen/Qwen3-0.6Boutput_dir: ./outputrecipe: bf16        # fp8-hybrid for FP8; nvfp4 for Blackwell FP4lora: truelora_rank: 16datasets:  - path: mlabonne/FineTome-100k    type: auto

  2. 02

    Fine-tune, distil or run RL

    Reinforcement learning with reward environments, preference training, and teacher-to-student distillation.

    • Adapters on BF16 bases or FP8, NVFP4 and BnB/NF4 quantization, including stacked LoRA.
    • Threaded data parallelism, ZeRO sharding, and Ray across nodes.
    $ surogate sft train.yaml

  3. 03

    Train in the precision your cards have

    BF16, hybrid FP8 and Blackwell NVFP4.

    • CPU offload for weights, gradients, optimizer state and activations, to train models larger than your cards.
    # reciperecipe: bf16recipe: fp8-hybrid   # FP8recipe: nvfp4        # Blackwell FP4

  4. 04

    Merge, then serve on the same engine

    Chat Completions, Completions, Responses and Messages endpoints, so existing clients connect unchanged.

    • Load and unload adapters without restarting; pick one per request.
    • K-quants, Q8_0 and IQ formats, or safetensors in BF16, FP8 and NVFP4.
    $ surogate merge --base-model Qwen/Qwen3-0.6B \  --checkpoint-dir output --output merged$ surogate serve merged --served-model-name my-finetune

  5. 05

    Answer many users at once

    Continuous batching, chunked prefill, CUDA graphs, and up to 128 active sequences per model.

    • MTP, and DFlash with a compatible drafter on a single GPU.
    • Multi-GPU layer pipelines, CPU weight offload, and GPU expert caching for MoE models.
    $ surogate serve Qwen/Qwen3.5-0.8B \  --served-model-name surogate \  --port 8080 --max-model-len 4096 \  --max-num-seqs 16 --kv-capacity auto $ curl -N http://127.0.0.1:8080/v1/chat/completions \  -H 'Content-Type: application/json' \  -d '{"model": "surogate", "stream": true,       "messages": [{"role": "user", "content": "Hello"}]}'

Prefer a window to a terminal?

Jackalope is the desktop app for the engine. Pick a model, pick a dataset, click your GPUs, hit Launch, and watch the loss fall, live. Linux, macOS and Windows.

Download Jackalope

Measured against the tools you would use instead.

Tokens a second, each row against the baseline it names.

training tok/s · 4× RTX 5090 · Qwen3-0.6B FP4 LoRA
136,200
training tok/s · 4× RTX 5090 · Qwen3-0.6B FP4 LoRA
training throughput vs. Unsloth · 1× H100 · BF16 on both
2.53×
training throughput vs. Unsloth · 1× H100 · BF16 on both
serving tok/s, one user · 1× RTX 5090 · Qwen3.5-0.8B GGUF
802
serving tok/s, one user · 1× RTX 5090 · Qwen3.5-0.8B GGUF
serving throughput vs. llama.cpp · 8× RTX 5090 · 16 users
7.0×
serving throughput vs. llama.cpp · 8× RTX 5090 · 16 users

Training

  • Qwen3-0.6B · 1× H100 80 GB2.53× vs. Unsloth
    Surogate53,900 tok/sUnsloth21,300 tok/s
  • Qwen3-0.6B · 1× RTX 50901.36× vs. Unsloth
    Surogate30,100 tok/sUnsloth22,100 tok/s
  • Qwen3-8B · 1× RTX 50901.97× vs. Unsloth
    Surogate6,900 tok/sUnsloth3,500 tok/s

Serving, on RTX 5090s

  • Qwen3.5-0.8B · 1 user · 1 GPU2.32× vs. vLLM
    Surogate802 tok/svLLM346 tok/s
  • Qwen3.5-0.8B · 8 users · 1 GPU4.05× vs. llama.cpp
    Surogate2,765 tok/sllama.cpp683 tok/s
  • Qwen3.5-4B · 100 users · 1 GPU1.19× vs. vLLM
    Surogate5,345 tok/svLLM4,481 tok/s
  • GLM-5.3-Flash · 16 users · 8 GPUs7.0× vs. llama.cpp
    Surogate292.9 tok/sllama.cpp41.9 tok/s

From the engine's README and its training and serving benchmarks.

Free to run. Or we train it for you.

The engine is Apache 2.0, free on your own GPUs. On our cloud a GPU costs $2.00 a GPU-hour. Or bring your data, and the team that builds the engine trains, evaluates and serves the model.

See pricing

Contact Sales

Questions

What can I train with it?

Pretraining and full fine-tuning, LoRA and QLoRA, GRPO, DPO and distillation, on one GPU or across nodes. Configuration is Python and YAML; execution is native C++/CUDA.

Which GPUs and precisions?

BF16, hybrid FP8 and Blackwell NVFP4, with FP8, NVFP4 and NF4 quantised bases for adapters. CPU offload lets you train models larger than your cards.

How do I serve the result?

With the same engine: surogate serve exposes Chat Completions, Completions, Responses and Messages endpoints, so existing clients connect unchanged. GGUF and Hugging Face checkpoints load natively.

How were the benchmarks measured?

Each row names its GPU, model, precision and baseline. The full tables are in the engine's repository, under docs/reference/benchmarks.md and surogate/serve/BENCHMARKS.md.

What is the licence?

Apache 2.0.

Can you train a model for us?

Yes. We train on your data, evaluate it and serve it on your hardware or ours. Contact Sales.

Install the engine.

One line installs it. Docs and examples are on GitHub.

curl -LsSf https://github.com/invergent-ai/surogate/releases/latest/download/install.sh | bash