WeVisOne-VL

Unified Visual Perception from Objects to Pixels

English | 简体中文

WeVisOne-VL (WeChat Vision All-in-One) is a vision-language foundation model that unifies object-level perception (object detection, referring localization, counting, pointing, and OCR) and pixel-level perception (referring segmentation and promptable alpha matting) through structured next-token prediction and a lightweight 9.6M-parameter pixel decoder—enabling localization at any user-specified granularity from points and boxes to binary masks and continuous alpha mattes, while reliably rejecting absent-target queries.

✨ Highlights

• 🎯 All-in-One: detection, referring, counting, pointing, OCR, segmentation and matting in a single model.
• 🪶 Lightweight Pixel Decoder: a 9.6M-parameter decoder outputs masks and alpha mattes, with no SAM required.
• 🚫 Reliable Rejection: returns None for absent targets instead of hallucinating.

📢 News

• [2026-10] 🎮 The demo is ready to try: 2B / 4B.
• [2026-10] 🔥🔥 We also open-sourced Mask2Alpha, the matting annotator used in the WeVisOne-VL Data Engine to turn binary masks into high-quality alpha mattes! Check out the mask2alpha branch.
• [2026-10] 🎉 We release the technical report, model weights (2B & 4B), inference package, Cookbooks, and evaluation pipeline!

📝 TODO

• Inference package and Cookbooks
• Gradio web demo
• Benchmark evaluation pipeline
• Model weights (2B & 4B) on Hugging Face
• Release our matting annotator
• Fine-tuning code

🏆 Performance at a Glance

Headline numbers for WeVisOne-VL-4B against the strongest open baseline in each benchmark (Qwen3-VL / Qwen3.5 up to 9B, Rex-Omni, LocateAnything, and task-specific segmentation / matting models).

📊 Full Benchmark Results (click each task to expand)

F1 for detection, referring and OCR is averaged over IoU thresholds 0.5:0.95 ("Mean"). Greedy decoding, ≤ 1,960 visual tokens per image.

See the technical report for all splits and metrics, and evaluation/README.md to reproduce the public benchmarks.

🏗️ How It Works

1. Data Engine. A knowledge-graph-driven engine discovers concepts, localizes them with complementary experts verified by a VLM, and extends the verified boxes to masks and alpha mattes. Over 100M images are annotated this way.

2. Unified Task Formulation. Every task is written as structured text, with coordinate tokens <0> … <1000> for boxes and points, None for absent targets, and an <OBJ> token after each box for pixel-level outputs.

3. Model Architecture. A 9.6M-parameter pixel decoder combines the hidden state of each <OBJ> token, its predicted box and multi-level Qwen3-VL ViT features to predict a full-resolution mask or alpha matte.

🚀 Quick Start

Installation

git clone https://github.com/Tencent/WeVisOne-VL.git && cd WeVisOne-VL
conda create -n wevisone-vl python=3.10 -y && conda activate wevisone-vl
pip install -r requirements.txt && pip install -v -e .

Inference

from PIL import Image
from wevisone_vl import WeVisOneVL
from wevisone_vl.visualization import save_result

model = WeVisOneVL("tencent/WeVisOne-VL-4B", backend="transformers")  # or backend="vllm"
# model = WeVisOneVL("tencent/WeVisOne-VL-2B", backend="transformers")  # Use 2B instead

# Object-level: open-vocabulary detection
image = Image.open("demo/examples/images/livingroom.jpg").convert("RGB")
result = model.predict(image, task="detection", query=["sofa", "pillow", "coffee table", "lamp"])
save_result("outputs/detection", image, result)

# Pixel-level: promptable alpha matting
image = Image.open("demo/examples/images/seabreeze.jpg").convert("RGB")
result = model.predict(image, task="matting", query="person")
save_result("outputs/matting", image, result)

Input · query="person"

Predicted alpha matte

Cutout

Supported Tasks

[!TIP] device ("auto", "cuda:0", "npu:0", "cpu"), dtype, min_pixels / max_pixels, and the usual decoding arguments (max_new_tokens, temperature, ...) are also available. The Cookbooks walk through each task in detail.

📓 Cookbooks

🎮 Gradio Demo

Try the hosted demo: 2B / 4B

Launch the demo locally:

CUDA_VISIBLE_DEVICES=0 python demo/app.py --model tencent/WeVisOne-VL-4B
# Or use 2B:
# CUDA_VISIBLE_DEVICES=0 python demo/app.py --model tencent/WeVisOne-VL-2B

# vLLM backend on a custom port, with a public share link
CUDA_VISIBLE_DEVICES=0 python demo/app.py \
    --model tencent/WeVisOne-VL-4B --backend vllm \
    --server-name 0.0.0.0 --server-port 8080 --share

📊 Evaluation

See evaluation/README.md.

🖼️ Representative Visualizations

Side-by-side comparisons with strong baselines. Click a task to expand or collapse it.

Referring Detection — locating the objects described by a natural-language expression

Box-prompted Matting — predicting the alpha matte of the target inside a given box

Text-prompted Matting — predicting the alpha matte of the target specified by a category name or referring expression

Object Detection — locating all instances of the given categories with boxes

Object Counting — counting the instances that match a category or expression

Object Pointing — marking each queried object with a point

OCR — locating and recognizing text in the image

Referring Segmentation — segmenting the objects described by a natural-language expression

📜 License

Tencent-authored code in this repository is released under the Apache License 2.0. Third-party components keep their original licenses; please check the corresponding source files.

📖 Citation

If WeVisOne-VL is helpful to your work, please consider giving us a ⭐ and citing:

@article{liang2026wevisone,
  title={WeVisOne-VL: Unified Visual Perception from Objects to Pixels},
  author={Liang, Tianming and Pan, Junyi and Li, Yuxi and Fu, Canmiao and Huang, Zhipeng and Sun, Chong and Li, Chen and Lyu, Jing},
  journal={arXiv preprint arXiv:2610.31329},
  year={2026}
}