OneJev, a Multimodal System One Decision Model

English · 简体中文 · 日本語

Hugging Face Demo GitHub Website License

Data API Python PyTorch Transformers

OneJev is a multimodal System One decision model. It returns calibrated probabilities for typed questions about screenshots, photos, videos and text in a single forward pass. Available in 0.8B, 4B, 9B and 27B. ## Results

Block bar charts: the four OneJev sizes against Jev 1.13, Jev-Omni 12B and Qwen3.8-27B thinking on the OneJev test set, DecisionBench hard, TypeSafe and MMStar

Results table, best score in each row in magenta, second best in light pink

Accuracy (%). The OneJev test set is held out from OneJev training. Jev 1.13 uses published text-only scores; Jev-Omni and Qwen3.8-27B thinking were evaluated by us.

Latency on one H200 for one 1280x720 screenshot: OneJev-0.8B 31 ms for 1 question, 51 ms for 10, 5.1 ms per question; OneJev-4B 64 ms for 1 question, 104 ms for 10, 10.4 ms per question; OneJev-9B 81 ms for 1 question, 131 ms for 10, 13.1 ms per question; OneJev-27B 189 ms for 1 question, 324 ms for 10, 32.4 ms per question

## Quick start Choose one backend, then run the Python example below. ### Option A: PyTorch For NVIDIA GPUs. Supports text, images and video. ```bash pip install "qev[torch] @ git+https://github.com/OmniJev/OneJev.git" qev serve --model OmniJev/OneJev-4B --port 8000 ``` ### Option B: llama.cpp For GGUF models. Supports text and images; use PyTorch for video. Install [llama.cpp](https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md) first (`brew install llama.cpp` on macOS). ```bash pip install git+https://github.com/OmniJev/OneJev.git qev serve --gguf mradermacher/OneJev-4B-GGUF:Q8_0 --port 8000 ``` ### Send a request Both backends serve the same API at `http://localhost:8000`. In another terminal, run this example with your own `screenshot.png`: ```python from qev import Client, Choice, Noul, Score from qev.media import data_uri r = Client("http://localhost:8000").system_one( state={"task": "Pay the open invoice from ACME", "screen": ""}, media=[{"type": "image", "data": data_uri("screenshot.png")}], questions={"done": Noul("The invoice has been paid"), "next": Choice("What should the agent do next?", {"click": "click an element", "type": "type text", "scroll": "scroll", "stop": "stop"}), "progress": Score("How far along is the task?", ["not started", "halfway", "almost done", "done"])}, ) r.answers["done"].noul # probability of yes r.answers["next"].probabilities # one probability per option r.answers["progress"].score # expected level ``` The API is compatible with TypeSafe System One. More examples: [video](examples/video.py), [curl](examples/curl.sh), [official TypeSafe SDK](examples/official_sdk.py). ## Training ```bash git clone https://github.com/OmniJev/OneJev.git cd OneJev pip install -e ".[train]" # Choose A or B. A is enabled below. # A. Demo: 100 examples with images (23 MB) hf download OmniJev/OneJev-Data sample/sample-100.parquet --repo-type dataset --local-dir data/onejev python -m train.unpack data/onejev --sample # B. Full dataset: 94,707 examples (17.7 GB). Uncomment these two lines instead of A. # hf download OmniJev/OneJev-Data --repo-type dataset --include "data/*.parquet" --local-dir data/onejev # python -m train.unpack data/onejev ``` Either option creates `data/onejev/train.jsonl` and extracts images and video frames into `data/onejev/media/`. [OneJev-Data](https://huggingface.co/datasets/OmniJev/OneJev-Data) releases 94,707 of the original 99,193 training questions; the remaining sources do not permit redistribution. ### Fine-tune The four models fine-tune Qwen3.5-0.8B, Qwen3.5-4B, Qwen3.5-9B and Qwen3.8-27B for one epoch with the vision tower frozen. The configs use a learning rate of `5e-6`, a 16,384-token limit, and cross-entropy plus Brier loss over answer probabilities. To train the 4B model on four GPUs: ```bash torchrun --nproc-per-node 4 -m train.sft --config train/configs/onejev_4b_full.yaml ``` Configs: [0.8B](train/configs/onejev_08b_full.yaml) · [4B](train/configs/onejev_4b_full.yaml) · [9B](train/configs/onejev_9b_full.yaml) · [27B](train/configs/onejev_27b_full.yaml). Set `--nproc-per-node` to your GPU count; the 9B and 27B configs enable FSDP to shard model and optimizer state. For your own data, change `train` and `media_root` in the config. The final 4B checkpoint is saved to `train/runs/onejev_4b/final`. Start it with: ```bash qev serve --model train/runs/onejev_4b/final --port 8000 ``` ## Citation ```bibtex @misc{onejev2026, title = {{OneJev}: A Multimodal System One Decision Model}, author = {{OmniJev Team}}, year = {2026}, howpublished = {\url{https://github.com/OmniJev/OneJev}} } ``` ## Friendly Links - [LINUX DO](https://linux.do) - [Jev](https://typesafe.ai) - [Awesome JEV](https://github.com/OmniJev/awesome-jev-gallery) - [Awesome JEV Website](https://omnijev.github.io/awesome-jev-gallery/) - [PlayJev](https://github.com/OmniJev/PlayJev) ## License [Apache 2.0](LICENSE)