PAIRS Lab

Preprint 2026

CPOR-Grasp

Calibrated probabilistic obstruction reasoning with vision-language models for grasping in clutter

Thanh-Tuan Tran1, Ngoc-Chien Chu1, Thanh Nguyen Canh2, Nak Young Chong2,3, Nguyen Viet Ha1, Xiem HoangVan1,*

1VNU University of Engineering and Technology   2Japan Advanced Institute of Science and Technology   3Hanyang University   *Corresponding author

arXiv Code Datasets Video
Retrieving a horse from clutter with a UR3: a confident but wrong VLM removes the wrong object, while CPOR-Grasp picks the correct blocker or defers; bar charts compare reliability, path reasoning and cross-VLM gains.
State-of-the-art VLMs are overconfident and pick the wrong object to remove; CPOR-Grasp is calibrated, picks a reliable next action, or defers.

01 · Abstract

Grasp the target, remove a blocker, or defer?

Retrieving a target from clutter requires deciding whether to grasp the target, remove a blocker, or defer. Existing methods typically commit to a single obstruction graph or removal strategy, ignoring uncertainty across alternative scene interpretations. They rely on miscalibrated vision-language model (VLM) predictions, can produce pairwise obstruction relations that are jointly inconsistent, and give no guarantee about how discarded hypotheses affect the final decision.

CPOR-Grasp propagates uncertainty from pairwise evidence to action decisions. It calibrates and fuses VLM, depth and amodal-mask cues into obstruction probabilities, induces a distribution over valid obstruction graphs, and marginalizes over them to estimate whether the target is graspable or a blocker should be removed. To stay tractable it keeps only the highest-probability graphs and derives a total-variation bound on the discarded mass, enabling certified decisions, adaptive stopping and principled deferral.

On synthetic and real UNOBench scenes CPOR-Grasp outperforms state-of-the-art baselines on most metrics; the explorer below shows each one, including where a baseline is better. On a UR3 robot it reaches a 77.8% average success rate.

02 · Video

CPOR-Grasp on a real robot

03 · Method

From pairwise evidence to a certified next action

Overview: local pairwise evidence from a VLM and from 3D geometry (fine-tuned UOAIS amodal masks and depth), reliability-gated log-odds fusion into calibrated edge probabilities, and global structural reasoning by Top-K feasible-graph inference that marginalizes over obstruction graphs to choose the object to grasp or remove.
Fig. 2: overview of CPOR-Grasp.
  1. 1

    Local pairwise evidence

    A VLM scores directed obstruction relations from the image and the request; fine-tuned UOAIS amodal masks and depth give geometric evidence for the same relations.

  2. 2

    Calibration

    Each source's confidence is calibrated with scene-conditioned (adaptive) Platt scaling, fit once on synthetic data and never refit per domain.

  3. 3

    Reliability-gated fusion

    Calibrated sources are fused in log-odds space, discounting unreliable evidence, into one probability per directed obstruction relation.

  4. 4

    Posterior over graphs

    The relations induce a distribution over obstruction graphs that admit a removal order (acyclic), instead of a single committed graph.

  5. 5

    Certified Top-K inference

    Only the K most probable graphs are kept; a per-scene total-variation bound certifies when the truncated decision equals exact inference.

  6. 6

    Act or defer

    Marginals give the probability that the target is free or a blocker should go; the robot grasps, removes, or defers, then FGC-GraspNet plans the 6-DoF grasp.

04 · Results

Obstruction reasoning on UNOBench

Pick a task, split, difficulty tier and metric. Path-level reasoning scores the predicted removal path; next-obstructor prediction scores the object to remove first.

Qualitative obstruction reasoning on synthetic and real UNOBench scenes: predicted removal paths drawn over cluttered bins.
Qualitative obstruction reasoning on synthetic (top) and real (bottom) UNOBench scenes.

05 · Calibration

Probabilities you can act on

Reliability diagrams at path and object level: CPOR-Grasp follows the diagonal more closely than the VLM baselines.
Reliability diagrams: a calibrated model follows the dashed diagonal.

Works across VLMs, without retraining

raw VLMwith CPOR-Grasp

06 · Certified inference

Top-K graphs with a guarantee

Exact enumeration over all order-feasible graphs is expensive. Keeping the K most probable graphs is fast, and the total-variation bound tells when the truncated decision is certified to match exact inference; otherwise K grows adaptively.

Risk and coverage curves for selective decisions with deferral.
Selective decisions: deferring on uncertain scenes trades coverage for lower risk.

07 · Real robot

UR3, 45 paired scenarios, 40 objects

CPOR-Grasp, UNOGrasp and Gemini Robotics-ER-1.6 run on matched scenes with the same grasping backend and action budget. Success needs the correct blocker removal and target retrieval within the budget.

Success rate% · higher is better · Table VIII
Real-world obstruction reasoning (a) and UR3 executions (b).
(a) Obstruction reasoning on real scenes; (b) execution on the UR3.
The 40 household objects used to build the real-robot scenarios.
The 40-object set used to build the real-robot scenarios.

08 · Cite

Citation

@misc{tran2026cporgrasp,
  title         = {{CPOR-Grasp}: Calibrated Probabilistic Obstruction Reasoning with Vision-Language Models for Grasping in Clutter},
  author        = {Tran, Thanh-Tuan and Chu, Ngoc-Chien and Canh, Thanh Nguyen and Chong, Nak Young and Ha, Nguyen Viet and HoangVan, Xiem},
  eprint        = {2609.18718},
  archivePrefix = {arXiv},
  year          = {2026}
}