Open source
Surogate Engine 1.5 serves what it trains
surogate serve joins training in one Apache 2.0 toolkit, on native C++/CUDA.
- training throughput vs. Unsloth, 1× H100, BF16 on both
- 2.53×
- training throughput vs. Unsloth, 1× H100, BF16 on both
- serving throughput vs. llama.cpp, 8× RTX 5090, 16 users
- 7.0×
- serving throughput vs. llama.cpp, 8× RTX 5090, 16 users
- licence
- Apache 2.0
- licence
Surogate Engine started as a trainer: pretraining, full fine-tuning, LoRA and QLoRA, then reinforcement learning. Version 1.5.0 adds surogate serve, a native HTTP server in the same toolkit, so the model you train is the model you serve, without a second stack.
What serving does
- Serves Hugging Face checkpoints and GGUF files directly.
- Speaks the Chat Completions, Completions, Responses and Messages APIs, so existing clients connect unchanged.
- Batches concurrent users, and loads LoRA adapters at runtime, one per request.
The numbers
Measured on RTX 5090s, against the baseline each row names, from the engine's README:
- 802 tokens a second for one user, one GPU, Qwen3.5-0.8B GGUF.
- 7.0× the throughput of llama.cpp at 16 users on 8 GPUs.
- 2.53× the training throughput of Unsloth on one H100, BF16 on both.
Install
curl -LsSf https://github.com/invergent-ai/surogate/releases/latest/download/install.sh | bashLinux x86_64 and an NVIDIA GPU. The code is on GitHub under Apache 2.0.