The open-source inference engine for decision models
Bulk classification and decision inference on one GPU. spinf-serve is the engine behind the hosted spinf API, free to run on your own hardware: each input is read once, every question about it is answered in the same pass, and each answer is a probability instead of generated text.
2026-10
spinf-serve is open source: the engine behind the hosted API, under Apache 2.0. pip install "spinf-serve[server]". GitHub →
2026-10
The public container image ghcr.io/spinfinc/spinf-serve: docker run --gpus all -p 8000:8000 -v spinf-cache:/root/.cache ghcr.io/spinfinc/spinf-serve. The host needs the NVIDIA driver and the NVIDIA Container Toolkit, nothing more. The image →
2026-10
spinf-prompts: the official prompt packs (content moderation, inbound triage, product listings, reviews, entertainment metadata, brand perception) as open files any server loads. The packs →
2026-10
The spinf/moderation prompt pack for content moderation, with its paper and a public benchmark. Read the paper →
2026-10
Decision mode at scale: 750,000 news articles read against 257 prediction-market questions. Decision engine →
Built for our own bulk classification, published because people asked
We built this engine for our own work: a daily market index read from millions of news articles, each article asked dozens of narrow, calibrated questions. Serving engines built for chat re-read the content for every question and generate one token at a time, which made that job slow and expensive. spinf-serve reads each input once, answers every question from that read and packs many inputs into one pass on one GPU. Bulk classification is where it started and where it is strongest; the same mechanism is what is now called a decision model, a model that returns a probability over a fixed set of answers instead of a paragraph. It is not a chat server: nothing is generated unless you ask for it, and it is built for throughput rather than for the latency of one request.
Throughput
- Each input is read once; every question about it is answered from that read.
- Scores, not generated text: no output loop.
- Inputs packed into 16k-token passes: about 10,000 processed tokens per second on one RTX 4090 with the 12B model, measured on the public job set in the repository.
Scores you can act on
- A probability for each answer you allow: yes / no, a label, a level.
- The floor of each question (the same prompt with no input) to read the lift against.
- The same input gets the same score whatever else is in the batch, bit for bit on fp8 GPUs, so thresholds stay valid over time.
Plain to run
- pip install, one command to serve, an OpenAI-compatible endpoint on your own NVIDIA GPU.
- Ready-made prompt packs for common jobs, scored in the same call as your own questions.
- The same request and response as the hosted API: a client switches by changing the base URL and the key.
One input, many questions, one read
Most real jobs ask many questions about the same content: intent, urgency and policy for a message; category and attributes for a product; sentiment and events for an article. Decision mode reads the content once and answers them all from that read, so each extra decision costs only its own tokens, in money and in time.
| General-purpose LLM serving | spinf decision mode | |
|---|---|---|
| Unit of work | One prompt, one generated answer | One input, many questions |
| Output | Generated text, parsed afterwards | A probability for each allowed answer |
| 200 questions about one document | The document is sent and read 200 times | The document is read once |
| Cost of one more question | The whole prompt again | Only the question’s own tokens |
| Thresholds and audits | Depend on sampling and on parsing the text | Calibrated, reproducible scores |
Output: a probability for each allowed answer.
200 questions about one document: the document is read once.
Cost of one more question: only the question’s own tokens.
Install, serve, score
One NVIDIA GPU with 24 GB, Python 3.12 and a CUDA 13 driver, or the public container image ghcr.io/spinfinc/spinf-serve (the host needs the NVIDIA driver and the NVIDIA Container Toolkit, nothing more). The first start downloads the checkpoint and compiles its kernels; then the server answers on port 8000 with the same API as api.spinf.com. The request below is one message, a question with five intents and one template asked three ways: eight answers scored from a single read, on your server or on the hosted API.
pip install "spinf-serve[server]"
spinf-serve serve spinf/gemma-4-12b-fp8 # downloads the checkpoint on first start; serves on port 8000
curl -s localhost:8000/readyz # 200 once the model is loaded and warmOpen-weights models, as decision models
Standard instruction models become decision models in decision mode, and dedicated decision models run as they are. The calibrated checkpoints are published on the Hugging Face Hub under spinf/ and downloaded on first start; the hosted model names work as aliases. Open weights: Gemma 4 by Google DeepMind and Qwen3.8 by the Qwen team.
Gemma 4 12B
spinf/gemma-4-12b-fp8 · one 24 GB GPU · the default
Reads text, images, audio, video frames
Gemma 4 31B
spinf/gemma-4-31b-fp8 · one 48 GB GPU
Reads text, images, video frames
Qwen3.8 27B
spinf/qwen3.8-27b-int4 · one 24 GB GPU
Reads text, images, video frames
Your decision model
a fine-tune of a supported family, served by its Hugging Face path
Reads what its base model reads
Gemma is a trademark of Google LLC. The checkpoints: huggingface.co/Spinf.
Ready-made decision sets, as open files
Each pack is a calibrated set of questions for a common business job, with the few decisions they add up to. The official packs live in spinf-prompts and are preloaded by the server; the same files run on the hosted API and in its playground, and any GitHub repository with a manifest is a pack source for your own server.
Content moderation
Harm categories with thresholds you set, for posts, comments and messages.
Inbound triage
Intent, urgency, sentiment, risk flags and phishing signals for emails, chats and tickets.
Product listings
Category, attributes and listing compliance for product pages and listings.
Entertainment and media
Genre, mood, themes, content advisories and audience, from a synopsis.
Reviews and feedback
Sentiment per aspect, feature requests and competitor mentions.
Brand perception
How a web page treats the brand you name per input, with topics for contextual advertising.
Community packs load from any GitHub repository with a prompt-packs.json: the first is Thinking Text's news packs (oil, the market read, twelve companies), with --packs github:thinkingtext/spinf-news-pack@0.1.0. Every known pack, official and community, is listed in spinf-serve's PACKS.md; publish yours and open a "List my pack" issue to have it added.
The whole kit is open
The engine, the packs, the client and the agent skills are public repositories under github.com/spinfinc, and the models are on the Hugging Face Hub. Pull requests are welcome; questions, ideas and bugs go to the issues of the relevant repository (spinf-serve when in doubt), there being no forum or chat.
spinf-serve
The engine and the server: decision mode, extraction slots, media, prompt packs, the OpenAI-compatible API. Apache 2.0, on PyPI.
spinf-prompts
The official prompt packs: the questions, their calibration, fixtures and the pack format, in files any spinf-serve preloads. Every known pack, official and community, is listed in spinf-serve's PACKS.md.
spinf-client
The Python client for both servers: batching, video frames, the moderation pack with typed results. pip install spinf-client.
Agent skills
Three skills for coding agents: spinf (calling the API), spinf-serve (running the server) and spinf-packs (authoring packs). A Claude Code plugin, or skills.sh.
Open source, hosted or private
The same engine three ways: on your own GPUs, on demand through the spinf API, or on a private deployment we run for you.
Open source
Run the engine on your own GPUs, free, under the Apache 2.0 license. The same inference as the hosted API.
Hosted API
Decision mode on demand, paid per million input tokens, with free credit to start. No GPU to run.
Private deployment
The engine on GPUs dedicated to you, sized to your volume and deadline, run by us.
Decision inference, explained
Run it on your own GPU
pip install "spinf-serve[server]", one command to serve, and the same API as the hosted service.