OpenJev

Fast, calibrated, typed decisions from an open model. OpenJev is an open-source "System One" decision server. Send it a state and typed questions (yes/no, choice, score). It returns a probability and a confidence for each answer in tens of milliseconds. It reads the answers directly from the model's probabilities and parses no text, so an answer cannot go off-schema. Questions can also ask about images.

OpenJev uses the same wire API as TypeSafe's Jev, so their SDKs work with it unchanged. It runs DiffusionGemma 26B-A4B (Apache-2.0) through vLLM on an NVIDIA GPU, or through MLX on Apple silicon.

Hosted for free on Codiv, an inference platform for open System One models. Sign up and get 100M input tokens, no card required. https://api.codiv.ai/v1/systemone

OpenJev is an independent project. It is not affiliated with or endorsed by TypeSafe AI.

Models

diffusiongemma-26b is the same DiffusionGemma for text generation. All weights are Apache-2.0. Laya, Verdict, CLM and JevK5 are other people's models: see Small encoder models, CLM and JevK5 for details and credit.

Try it

pip install typesafe-sdk
export TYPESAFE_BASE_URL=https://api.codiv.ai   # or http://127.0.0.1:8080 for your own server
export TYPESAFE_API_KEY=sk-codiv-...

from typesafe_sdk import TypeSafeClient

client = TypeSafeClient()
r = client.system_one(
    "Everything is down and we have a demo with our biggest client at noon.",
    {
        "urgent": {"type": "noul", "instructions": "Does the customer need a reply within the hour?"},
        "team":   {"type": "choice", "instructions": "Which team should handle it?",
                   "criteria": {"outage": "service down", "billing": "charges, refunds", "feature": "requests, how-to"}},
        "tone":   {"type": "score", "instructions": "How upset is the customer?",
                   "criteria": ["calm", "annoyed", "furious"]},
    },
)
r.nouls["urgent"].noul        # 1.00
r.choices["team"].choice      # "outage", confidence 1.00
r.scores["tone"].score        # 2.00 (expected level, 0-indexed)

Or with curl:

curl https://api.codiv.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "openjev-latest", "state": "I was charged twice this month.",
       "questions": {"is_billing": {"type": "noul", "instructions": "Is this a billing issue?"}}}'

API

Question types:

• noul (yes/no): takes optional criteria: {true, false}. Returns {noul: P(yes)}.
• choice: takes criteria: {name: description}. Returns {choice, probabilities, confidence}.
• score: takes criteria: [level0, level1, …] (1–10 levels; a single level is answered directly, as for a choice with one option). Returns {score: Σ i·pᵢ, legend, probabilities, confidence}.

confidence is 1 − H(p)/ln K: 1 when the model is certain, 0 when the distribution is uniform. usage.input_tokens counts prompt tokens, image tokens included. usage.output_tokens is 0 unless you set think.

Each response has a Server-Timing header:

server-timing: model;dur=41.2, server;dur=2.8, total;dur=44.0

model is the time spent on the model, summed over the request's reads. It can be more than total when reads run in parallel. server is the remaining time: schema compile, tokenization, validation and serialization. Neither includes your network.

Errors use the same shapes as Jev, checked against the live API:

• 422 with a FastAPI validation list for a field of the wrong shape.
• 400 with a plain-text reason for a question the server cannot ask (no options, too many options, or too many score levels).
• 400 api_usage_error for an unknown model or question type.
• {"detail": {"error_type", "message"}} for auth errors (401/403).
• 429 for rate limits. 529 when the server is overloaded.

Differences from Jev:

• Model names are OpenJev's own. jev-latest and jev-preview are aliases. A pinned Jev version such as jev-1.13.0 gets 400 Unknown model.
• The server reads many questions in chunks of about 12 per read. The chunks run in parallel.

Extensions

These optional request fields are OpenJev additions. A request without them behaves exactly like Jev. TypeSafe's SDKs never send them. They come from the example server in vllm-project/vllm#57250.

curl https://api.codiv.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "openjev-latest", "state": "Look at the photo.",
       "images": ["data:image/jpeg;base64,/9j/4AAQ..."],
       "questions": {"hotdog": {"type": "noul", "instructions": "The photo shows a hot dog"}}}'

think and sequential need a text state. A request that combines them with images gets a 400.

Text generation

POST /v1/chat/completions generates text with the same model, OpenAI style. Use model diffusiongemma-26b. It supports streaming and tools. The server ignores sampling fields (temperature, seed, penalties and similar) because vLLM refuses them for diffusion models. max_tokens defaults to 1024, with a cap of 8192. At most 8 generations run at once, so reads always have room. The MLX backend ignores tools and logprobs and does not return the thought.

How it works

DiffusionGemma is a discrete diffusion model. It denoises a full canvas of tokens in each forward pass, instead of writing left to right. OpenJev uses this to read answers, not write them.

OpenJev builds a canvas in which only the answer slots are masked, one token per question:

canvas in                 one read-only pass         answer out
  q1: [?]        ──►      P(yes) 0.001        ──►    noul  0.001
  q2: [?]                 P(A) 0.000                 choice "billing"
                          P(B) 0.999                 confidence 0.997
                          P(C) 0.000
  q3: [?]                 P(0) 0.000                 score 1.00
                          P(1) 0.996
                          P(2) 0.004

Each label is one token: yes/no for a noul, A/B/C for a choice, 0/1/2 for a score. The model never writes into these slots. One read-only pass gives the probability distribution for each slot, and that distribution is the answer. The numbers above are a real read of "The invoice looks wrong again. Second time this quarter.": not urgent, billing, mildly annoyed.

The read scores only the label tokens, so an answer cannot go off-schema. The confidence comes from the model's own distribution, not from a number that the model reports about itself.

If a slot is uncertain (entropy > 0.1), OpenJev reads three more times with fresh noise and averages the four results. One uncertain question causes a re-read of all the questions in the request. These extra reads add no tokens to usage. Set samples: 1 to get one read only. Question ids never go to the model. It sees q1, q2, q3.

The vLLM part is vllm-project/vllm#57250, merged on 2026-09-22. It adds seeded canvases, read-only steps, step caps and pinned canvas positions for DiffusionGemma. openjev/engine.py adapts that PR's structured_server.py example, with async I/O, bounded concurrency and backpressure.

Run your own

Two backends serve the same /v1/systemone. Select one by hardware:

NVIDIA GPU

You need an NVIDIA GPU with at least 24 GB of memory for the NVFP4 checkpoint. We tested on an RTX PRO 6000 Blackwell (sm_120).

The prebuilt image razorback16/openjev runs vLLM and the API server in one container, on CUDA 13. It pins the upstream vLLM commit below, with the two changes in Caveats.

git clone https://github.com/razorback16/openjev && cd openjev
docker compose up -d          # OpenJev on 127.0.0.1:8080 when the model is loaded
curl localhost:8080/v1/models

Or without compose:

docker run -d --gpus all --ipc=host -p 127.0.0.1:8080:8080 \
  -v ~/.cache/huggingface:/root/.cache/huggingface razorback16/openjev:0.5.0

The first start downloads the weights (about 18 GB) into ~/.cache/huggingface. To build the images yourself, build the shared base first, then run docker compose build:

docker build -f docker/Dockerfile.base -t razorback16/openjev-base:cu130-torch2.13 .
docker compose build

vLLM listens only inside the container. Set OPENJEV_UPSTREAM to use a vLLM server that you already run.

Measured on an RTX PRO 6000 at 38% of the GPU, with 3 questions per request and cache-busted states:

One request at a time, on the same GPU, with samples: 1:

Without Docker:

git clone https://github.com/vllm-project/vllm && cd vllm
VLLM_COMMIT=1b3b88ec2b7457aa030db4d0e7d8aaf04f6d0fb8   # the commit the image pins
git checkout $VLLM_COMMIT
# a choice of more than 128 options needs a larger cap, as in the image
sed -i 's/^MAX_LOGPROB_TOKEN_IDS = 128$/MAX_LOGPROB_TOKEN_IDS = 512/' vllm/sampling_params.py
VLLM_USE_PRECOMPILED=1 VLLM_PRECOMPILED_WHEEL_COMMIT=$VLLM_COMMIT pip install -e .
vllm serve nvidia/diffusiongemma-26B-A4B-it-NVFP4 --served-model-name dgemma \
  --diffusion-config '{"canvas_length": 64}' --max-logprobs 32 --enable-prefix-caching \
  --async-scheduling --attention-backend TRITON_ATTN \
  --limit-mm-per-prompt '{"image": 8, "video": 0}' \
  --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4 \
  --override-generation-config '{"max_new_tokens": null}'
pip install -e path/to/openjev && python -m openjev

To match the image's image reads, also apply docker/patches/vision_prefix_lm.py to this checkout. Without it, vLLM prefills images causally.

Apple silicon

A Mac needs no vLLM and no Docker. OPENJEV_BACKEND=mlx runs DiffusionGemma inside the OpenJev process through MLX and mlx-vlm. The 4-bit weights need about 16 GB of memory.

pip install -e '.[mlx]'
OPENJEV_BACKEND=mlx python -m openjev     # 127.0.0.1:8080

This backend uses the same prompts, canvases and seeds as vLLM. It supports images, samples, sequential, steps, think and the automatic re-reads, and it bills the same. Each step after the first reuses one prefill of the prompt. More steps cost GPU time, not prompt tokens. The re-reads and samples of one request share one vision pass.

Reads run one at a time, so this backend is for local use, not for serving. A 3-question request takes about 0.2–0.4 s on an M3 Ultra and about 0.39 s on an M4 Max, both with the 4-bit weights. 16 concurrent requests finish at about 4 req/s.

Small encoder models

OpenJev also serves two small System One models that other people built and trained. The credit is theirs. OpenJev only puts them behind the same API. Each is a bidirectional encoder with a classification head, not a diffusion model. It reads each question in one forward pass, so an answer is still a distribution over your options.

Each model runs in its own container (OPENJEV_BACKEND=laya or verdict). docker compose up -d starts both next to the vLLM container on the same GPU. The openjev container sends their requests to them (OPENJEV_MODEL_ROUTES), so :8080 serves all three models. /v1/models lists a routed model even when its container is stopped. A request for it then gets a 503. On a GPU both models use bf16 weights and need about 3.7 GB together. Set OPENJEV_GPU_UTIL to keep that memory free.

Measured on an RTX PRO 6000 with 16 questions per request. GPU memory is the nvidia-smi value, CUDA context included. A short state is about 50 tokens. A full state fills the model's limit.

FlashAttention 2 gave no improvement. Memory was the same, speed was within 6%, and short states were slower.

To run one model alone, on a GPU or on the CPU:

docker build -f docker/Dockerfile.laya -t openjev-laya .   # after the base, as above
docker run -d --gpus all -p 127.0.0.1:8081:8080 \
  -v ~/.cache/huggingface:/root/.cache/huggingface openjev-laya
# or without Docker:
pip install -e '.[laya]' && OPENJEV_BACKEND=laya python -m openjev

A server that runs one of these models alone also accepts jev-latest and jev-preview for it.

Each image has its own Dockerfile in docker/. All three start from docker/Dockerfile.base and share its CUDA, Python and PyTorch layers (about 8.7 GB on disk). A server that runs all three stores these layers once, about 21 GB in total.

Differences from the DiffusionGemma model:

• Text only. images, steps above 1, samples above 1, think and sequential get a 400.
• The server cuts a state that is longer than the limit, with no error. usage.input_tokens counts what the model read. Each question is a separate sequence, so each question bills the state again.
• Verdict adds an "insufficient evidence" option to each question. OpenJev removes it and scales the other probabilities to a sum of 1, as Jev's answer shapes require. Verdict also ignores a noul's criteria.
• Laya rounds each probability to 4 decimal places.
• On a GPU the weights are bf16, not fp32. On 36 test answers per model, no top option changed. Probabilities moved by at most 0.021 (Laya) and 0.007 (Verdict).
• The Verdict prompt format and temperatures come from Verdict's inference engine (v1.4). For the same input, the probabilities are the same as that engine's output.

For benchmarks, training, fine-tuning and known limits, read the authors' repositories: Laya and Verdict. The laya container runs Laya's laya package. Verdict uses ModernBERT (Answer.AI, LightOn) and GLiClass (Knowledgator).

CLM

CLM (Contrastive Language Model) is Contrastive-LM's model; the credit is theirs. The CLM-v0.1-8B checkpoint is two small heads (a state head and an action head, 9.4M parameters each) on top of a frozen Qwen3-8B. Qwen3-8B turns the state (with the question appended) and each option into its last-token embedding. The heads project them to 512 dimensions, and an answer is the softmax over the scaled cosine of each option with the state.

The clm image (docker/Dockerfile.clm) runs vLLM's pooling runner for Qwen3-8B and the heads in one container. vLLM is the same pinned commit as the main image, without its changes. The prompt layout, the heads and the scoring come from Contrastive-LM's contrastive-lm package (0.1.0).

docker build -f docker/Dockerfile.clm -t openjev-clm .   # after the base, as above
docker run -d --gpus all --ipc host -p 127.0.0.1:8083:8080 \
  -v ~/.cache/huggingface:/root/.cache/huggingface openjev-clm

Next to DiffusionGemma, run docker compose --profile clm up -d and add clm-v0.1=http://clm:8080 to OPENJEV_MODEL_ROUTES. CLM runs its own vLLM, so lower the openjev service's OPENJEV_GPU_UTIL to leave it room; OPENJEV_CLM_GPU_UTIL (default 0.12) is its share of the GPU.

The default weights are Qwen/Qwen3-8B-FP8. On an RTX 3090 (Ampere, no FP8 compute) vLLM runs them as weight-only FP8 through Marlin. On 581 four-way SQuAD questions, FP8 and bf16 agreed on 98.5% of top options. The mean embedding cosine was 0.9992 and accuracy went from 89.7% to 88.8%. RedHatAI/Qwen3-8B-FP8-dynamic does not start on Ampere with this vLLM. Set OPENJEV_MODEL=Qwen/Qwen3-8B for bf16.

Measured on one RTX 3090 at OPENJEV_GPU_UTIL=0.85. Each request has a unique state (a SQuAD paragraph) and 3 questions, about 550 prompt tokens in all:

The GPU is the limit: prefill is compute-bound, and weight-only FP8 does not add compute on Ampere. What FP8 buys there is latency at low load and 2.4 times the prefix cache. --max-num-batched-tokens 8192 gave no gain.

Differences from the other models:

• Text only, as for the encoder models. A state longer than 2,048 tokens loses its start, not its end, so the question (which comes last) survives. Upstream CLM cuts the end (see CLM PR #6).
• The server keeps the embeddings and projections of recent texts (OPENJEV_CLM_EMBED_CACHE, OPENJEV_CLM_CACHE). usage.input_tokens counts only the texts it had to embed, so a repeated request reports 0.
• Score questions can ignore the state. Upstream reports one level winning whatever the state says (CLM issue #3), and in our checks a thankful customer scored "annoyed". Choice and noul questions follow the state. Evaluate score questions on your own data before you rely on them.

JevK5

JevK5 is Alibi Serikbay's model; the credit is theirs. The JevK5 checkpoint (v0.2) is Qwen3.5-4B with a LoRA distilled from Qwen3.6-27B, merged. Each question becomes a JSON prompt with its options lettered A to P, and the answer is a softmax over those letters' next-token logits under one calibration temperature (1.532, from the checkpoint's jevk5_config.json). The readout is SemIf's. A question with more than 16 options takes several passes, combined as JevK5 combines them.

The jevk5 image (docker/Dockerfile.jevk5) runs the checkpoint in bf16 on vLLM, the same pinned commit as the main image without its changes, and asks it for the letters' logprobs. The prompt and the combining of passes come from JevK5's own jevk5 package (0.2.2). JevK5's own server reads one question at a time; here vLLM batches the questions of all requests together.

docker build -f docker/Dockerfile.jevk5 -t openjev-jevk5 .   # after the base, as above
docker run -d --gpus all --ipc host -p 127.0.0.1:8084:8080 \
  -v ~/.cache/huggingface:/root/.cache/huggingface openjev-jevk5

Next to DiffusionGemma, run docker compose --profile jevk5 up -d and add jevk5-0.2=http://jevk5:8080 to OPENJEV_MODEL_ROUTES, as for CLM. OPENJEV_JEVK5_GPU_UTIL (default 0.12) is its share of the GPU.

On JevBench's 231 public items, OpenJev and JevK5's own published v0.2 run gave the same top answer on all 231 and the same input token count on all 231, so the prompts are identical. Probabilities differed by 0.0012 at the median and 0.055 at most (vLLM's kernels are not transformers'). Both scored 86.6%.

Measured on one RTX 3090 at OPENJEV_GPU_UTIL=0.85 (7.9 GB of weights, 284k KV tokens), with the same requests as for CLM:

The GPU is the limit (100% busy at its 350 W cap). --max-num-batched-tokens 8192 gave no gain. Qwen3.5's linear-attention layers make vLLM cache prompts in blocks of 528 tokens, so questions about a state shorter than that do not share its prefill; longer states do.

Differences from the other models:

• Text only, as for the encoder models. A read longer than 16,384 tokens gets a 400, never a cut.
• usage.input_tokens counts every pass, as JevK5 does: each question reads the state again.
• The image sets VLLM_USE_FLASHINFER_SAMPLER=0. A read takes one greedy token and keeps only logprobs, and FlashInfer's sampler would need a CUDA compiler the image does not have.

Settings

The server reads its settings from the environment.

Caveats

• The image pins upstream vLLM and makes two changes. A build fails if either change no longer applies.
  • It raises the limit of exact label ids per request from 128 to 512, for choices of up to 255 options.
  • docker/patches/vision_prefix_lm.py gives image tokens bidirectional attention, as the checkpoint config asks. Upstream vLLM does this for Gemma4 but not yet for DiffusionGemma.
• The clm and jevk5 images pin the same vLLM commit with neither change.
• Answer quality is the quality of DiffusionGemma 26B-A4B in this mode. Evaluate it on your own tasks before you rely on it.

Development

pip install -e '.[test]' && pytest
OPENJEV_LIVE_URL=http://127.0.0.1:8080 pytest tests/test_live.py   # end to end against a running server
OPENJEV_MLX_TEST_MODEL=path/to/weights pytest tests/test_mlx_model.py   # MLX against the real model

Run the live checks after you build an image and before a cutover. They cover each read option, images, chat, and the encoder models that the server lists.

License

Apache-2.0. The DiffusionGemma weights are Apache-2.0 (NVIDIA / Google). Laya (Nandakishor M / Convai Innovations) and Verdict (Heman10x) are Apache-2.0, weights and code. openjev/encoders.py adapts Verdict's prompt format and calibration from its repository.