Official code · checkpoints · papers · setup and inference guides
A practical overview of interactive and promptable medical image segmentation models and agentic workflows, organized by primary interaction style and input modality. Institutional affiliations follow the cited project or paper; prompt capabilities can overlap.
Included models
This catalog summarizes 15 interactive or promptable medical segmentation models and agentic workflows, including institutional affiliations, prompt styles, input modalities, official code, checkpoint access and run entry points. Each guide links to the original project and paper.
Spatial and visual prompts
| Model / institutional origin | Modality / prompts | Official code | Weights | Details |
|---|---|---|---|---|
| SAM-Med3D 🏛️ Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences |
3D volume · point | code | Hugging-Face | guide |
| PRISM 🏛️ MedICL-VU · Vanderbilt University |
2D/3D · visual prompts | code | Google-Drive | guide |
| nnInteractive 🏛️ MIC-DKFZ · German Cancer Research Center (DKFZ) · Helmholtz Imaging |
3D · points, boxes, lasso | code | Hugging-Face | guide |
| MedSAM2 🏛️ University Health Network (UHN) · University of Toronto |
3D/video · spatial prompts | code | Hugging-Face | guide |
| VISTA3D 🏛️ Project MONAI · NVIDIA |
3D CT · class/point | code | Hugging-Face | guide |
| SegVol 🏛️ Beijing Academy of Artificial Intelligence (BAAI) · DCAI |
3D CT · text/point/box | code | Hugging-Face | guide |
| MedicoSAM 🏛️ Computational Cell Analytics · University of Göttingen |
2D/3D · SAM prompts | code | OwnCloud | guide |
Text and concept prompts
| Model / institutional origin | Modality / prompts | Official code | Weights | Details |
|---|---|---|---|---|
| SAM 3 🏛️ Meta AI Research |
2D/video · text/visual | code | Gated-Checkpoint | guide |
| MedSAM3-v1 🏛️ Hong Kong University of Science and Technology (Guangzhou) · University of Illinois Urbana-Champaign |
medical image · text + LoRA | code | LoRA-Weights | guide |
| Medical-SAM3 🏛️ University of Central Florida · University College London · University of Illinois Urbana-Champaign · University of Trento · The Hong Kong Polytechnic University · University of Pennsylvania |
2D + 3D v2 · text/box | code | Hugging-Face | guide |
| VoxTell v1.1 🏛️ German Cancer Research Center (DKFZ) · Helmholtz Imaging |
3D CT/MRI/PET · free text | code | Hugging-Face | guide |
| BiomedParse v2 🏛️ Microsoft Research |
3D · text, challenge format | code | Gated-Checkpoint | guide |
| DA-SAM3 🏛️ Shenzhen Research Institute, The Chinese University of Hong Kong, Shenzhen · University of Science and Technology of China · Nanyang Technological University · Imperial College London |
medical image · text + experts | code | Not listed | guide |
Agentic multi-turn interaction
| Model / institutional origin | Modality / prompts | Official code | Weights | Details |
|---|---|---|---|---|
| MedSAM-Agent 🏛️ The Chinese University of Hong Kong · Tencent Hunyuan |
2D · autonomous multi-turn | code | Hugging-Face | guide |
| nnInteractive Agent 🏛️ MIC-DKFZ · Heidelberg University · NCT Heidelberg · Helmholtz Imaging |
3D · natural-language tasks + agent-selected prompts | code | nnInteractive base weights | guide |
Practical notes
- Models differ in modality, spatial dimensionality and prompt type. Report the input, prompt policy and interaction budget when describing results.
- A project may host an original paper checkpoint, a newer release, task-specific weights or an adapter. Record the exact model version, checkpoint and preprocessing.
- Agentic methods choose prompts autonomously; their interaction rounds are not directly equivalent to a fixed number of manual clicks. nnInteractive Agent orchestrates nnInteractive and has no separate segmentation checkpoint.
- Some demo/evaluation code synthesizes prompts from ground truth. Mark those runs as GT-derived/oracle and distinguish them from human-input or deployment-style inference.
- Code and weights may use different licenses. Review both before downloading or reusing a model.
Citation and scope
This catalog links to original projects, papers and checkpoint pages. It does not redistribute third-party code, weights or paper PDFs. Cite the original method paper and follow the upstream licenses when using a model.
Upstream pages checked: 27 October 2026. Access, weights and dependencies can change; follow the original project.