# GLiNER2.5-Decide puts structured AI decisions on a 340M CPU model Fastino's Apache-2.0 GLiNER2.5-Decide turns schema-defined routing, triage and guardrail decisions into a 340M open-weight model that can run locally on CPU and jointly enforce constraints across related answers. GLiNER2.5-Decide attacks the same bounded-decision layer as Jev and CLM from a much smaller encoder architecture. Its strongest benchmark claims are vendor-produced, but CPU deployment and constrained joint decoding make it a materially different option for software-facing AI decisions. - Status: Active - Published: 2026-09-26T16:00:45+12:00 - Updated: 2026-09-26T16:00:45+12:00 - Categories: Artificial Intelligence, AI Models, AI Agents, Open Models, Inference & APIs - Tags: agent routing, decision models, Fastino, GLiNER2.5-Decide, open models, structured outputs - Canonical HTML: https://beyondthe.news/dossiers/fastino-gliner2-5-decide-340m-cpu-structured-decision-model ## What changed Fastino released GLiNER2.5-Decide, a 340M-parameter open-weight encoder specialised for schema-defined decisions. Applications supply text plus typed questions and permitted answers; the model returns selected answers, probability distributions, confidence scores and feasibility metadata. It can jointly decode related questions under explicit implications, exclusions, cardinality limits and ordinal bounds. The checkpoint is Apache 2.0, runs on CPU or GPU, supports air-gapped deployment, and is available through the GLiNER2 tooling. ## Why it matters The recent System One model wave has mostly been framed around specialised decision APIs or GPU-hosted models. GLiNER2.5-Decide pushes the same software-facing pattern down to a 340M encoder that Fastino reports at 167.3 ms p50 on a 48-vCPU Xeon for short inputs. That makes bounded model routing, ticket triage, guardrails and tool selection plausible without a generative model or dedicated GPU. The joint constraint decoder is also operationally useful: downstream software can require related decisions to be mutually consistent instead of repairing contradictory classifier outputs after the fact. ## It is a classifier, not a tiny chatbot GLiNER2.5-Decide does not generate prose or reason through open-ended questions. A caller declares the legal answers in a schema and the encoder scores them. That narrower contract is the point: software receives bounded values, probabilities and confidence rather than text that must be parsed. ## Constraints are part of decoding Schemas can connect decisions with implications, exclusions, cardinality limits and ordinal bounds. Fastino's example shows why this matters: independently decoded safety and harm labels can contradict one another, while joint decoding searches for the highest-scoring assignment that satisfies the declared rules. ## CPU deployment changes the economics Fastino reports 167.3 ms p50 for a short two-head classification workload on a 48-vCPU Intel Xeon Platinum 8581C, with GPU latency around 38–47 ms on several tested accelerators. Those are vendor measurements on a specific schema and input size, but a 340M encoder is a very different deployment proposition from an 8B decision model or a frontier API. ## The benchmark is useful but internal Fastino reports a 60.1% average across its unseen 17-dataset Fast Decisions suite, ahead of an open JevK5 reproduction at 57.5% and SemIf at 56.4%. The suite is internally generated, JevK5 is not TypeSafe's production Jev, and these results therefore should not be treated as a neutral ranking of the decision-model market. ## Key details - GLiNER2.5-Decide has 340M parameters and is based on an encoder architecture rather than an autoregressive text generator. - The model accepts text plus typed questions, permitted answer sets, descriptions, examples and cross-question constraints. - Outputs include selected answers, probabilities, confidence scores and constraint-feasibility metadata. - The model is released under Apache 2.0 and can run locally on CPU, GPU or air-gapped infrastructure. - Fastino reports 167.3 ms p50 on a 48-vCPU Intel Xeon Platinum 8581C for a short 64-token workload. - Fastino reports a 60.1% average on its internal 17-dataset Fast Decisions suite and first place on 9 datasets. - The Jev comparison uses JevK5, an open reproduction, not TypeSafe's production Jev. ## Builder takeaways - Use this class of model when the application already knows the allowed choices and needs a bounded decision rather than generated prose. - Consider CPU deployment for low-volume routing, triage or guardrail workloads where adding a GPU or external model API would be disproportionate. - Encode cross-field business rules in the schema when contradictory independent classifications would create unsafe or invalid downstream states. - Benchmark on your own labels and traffic before replacing a general LLM or existing classifier; Fastino's headline accuracy and latency numbers are vendor-produced. - Keep an escalation path for low-confidence cases and for requests whose correct answer is not represented in the declared schema. ## What to watch - Independent comparisons with production Jev, CLM-8B and conventional fine-tuned classifiers on the same real workloads. - Calibration quality of the returned probabilities and confidence scores outside Fastino's benchmark suite. - Whether agent frameworks adopt the schema and constraint interface for routing and tool selection. - Real CPU throughput and cost under longer documents, larger label sets and concurrent production traffic. ## Uncertainties - The main accuracy benchmark is internally generated and reported by Fastino. - The published JevK5 baseline is an open reproduction rather than TypeSafe's production Jev. - Latency varies with input length, schema size, hardware and batching; the headline CPU figure is for a short batch-1 workload. - A bounded decision model cannot invent an action that the application failed to include in the allowed schema. ## Sources - [GLiNER2.5-Decide: An Open-Weight Model for Structured Decision Making](https://fastino.ai/blog/gliner-2-5-decide-open-weight-decision-model) — Fastino · primary · 2026-09-24T00:00:00+12:00. Primary release, architecture, benchmark methodology, latency measurements, licensing and constraints. - [Fastino Releases GLiNER2.5-Decide: A 340M Open-Weight Decision Model That Runs on CPU](https://www.marktechpost.com/2026/09/24/fastino-releases-gliner2-5-decide-a-340m-open-weight-decision-model-that-runs-on-cpu/) — MarkTechPost · secondary · 2026-09-25T00:00:00+12:00. Independent specialist summary and qualification of the model's scope and vendor benchmark claims.