An agentic interactive-segmentation framework that autonomously plans multiple rounds of prompts and segmentation calls, rather than asking a person to place every correction.
At a glance
| Field | Details |
|---|---|
| Model type | Agentic multi-turn segmentation framework |
| Research team / organization | The Chinese University of Hong Kong, Tencent Hunyuan and collaborators |
| Input / modality | 2D medical images from six modalities and 21 datasets in the authors’ evaluation. |
| Prompt / interaction | Natural-language target description plus agent-generated clicks; selectable Qwen or GPT grounding model and a separate segmentation backbone. |
| Code license | The GitHub repository snapshot does not include a top-level LICENSE file. Check the model card and each backbone’s terms before reuse; do not assume the Qwen agent and segmentation weights share one license. |
📚 Paper and publication
MedSAM-Agent: Empowering Interactive Medical Image Segmentation with Multi-turn Agentic Reinforcement Learning — arXiv:2602.03320 (2026 preprint).
📦 Checkpoints and access
The official Hugging Face model is Saint-lsy/MedSAM-Agent-Qwen3-VL-8B-MedSAM2. Inference also requires a segmentation checkpoint (e.g. MedSAM2) and its configuration. Dataset/trajectory releases are marked pending in the upstream README.
The code, model checkpoint and any required foundation-model checkpoint can have different terms. Review each linked source before use or redistribution.
⚙️ Official setup and inference
1 | conda create -n msagent python=3.11 |
Upstream reports its development environment as Python 3.11, CUDA 12.9 and PyTorch 2.8.0. The paper’s training used eight H20 GPUs; that is a training setup, not a guaranteed minimum inference requirement.
⚠️ Version and practical notes
This is an autonomous multi-turn method. Its action policy chooses prompts, so its interaction count is not directly comparable with a fixed manual-click budget. Record the grounding model and segmentation backbone separately.
📜 License
The GitHub repository snapshot does not include a top-level LICENSE file. Check the model card and each backbone’s terms before reuse; do not assume the Qwen agent and segmentation weights share one license.
Citation: Cite the paper above and the upstream project as directed by its authors. This repository links to the original paper; it does not redistribute the code or weights.
Source pages checked: 27 October 2026. Upstream environments, weights and access policies may change; use the linked project as the source of truth.