PAIRS Lab

Preprint 2026 · submitted

HiPer

Which objects stay? Learning hierarchical persistence beliefs for lifelong open-vocabulary object maps

Thanh Nguyen Canh1, Haolan Zhang1, Nak Young Chong1,2,*

1Japan Advanced Institute of Science and Technology   2Hanyang University   *Corresponding author

arXiv Code Video
Each object carries its own belief that it is still where it was last seen.

01 · Abstract

After an object leaves view, is it still there?

Lifelong object maps must estimate whether an object remains at its previously observed location after it leaves view. Existing persistence filters rely on class-based survival priors, which are hard to apply reliably in open-vocabulary settings.

We introduce HiPer, a hierarchical persistence filter that learns persistence beliefs from data rather than semantic categories. For a new object, HiPer retrieves visually and geometrically similar training instances and estimates an initial survival rate from their histories, with no class labels. This initializes a conjugate Beta belief over the object's survival probability, updated in closed form from detections and misses under Rosen's persistence-filter framework. When an object is re-identified at a new location, its belief moves with it, so the map learns object-specific mobility over time.

On 120 held-out simulated sequences HiPer performs best among the compared methods; retrieval-based priors beat CLIP-derived label priors, and object-specific online learning adds further gains. On 47 held-out 3RScan environments the retrieval prior also beats state-of-the-art baselines in real-world settings.

Four sessions of a held-out THOR-Lifelong sequence (top) and the predicted probability that each object stays in place (bottom).
Fig. 1: which objects stay? Four sessions of a held-out sequence and HiPer's predicted probability that each object stays.

02 · Video

HiPer in three minutes

03 · Method

A belief per object, learned from similar objects

HiPer overview: association turns each session into one piece of evidence per map object (hit, miss or moved); a retrieval-based prior initializes a Beta belief over survival that is updated in closed form; the map is maintained by pruning objects whose belief drops.
Fig. 2: overview of HiPer on a held-out sequence.
  1. 1

    Cross-session association

    Each session becomes one piece of evidence per map object: seen in place (hit), not seen (miss) or moved.

  2. 2

    Survival prior by retrieval

    A new object retrieves training instances similar in DINOv2 and CLIP appearance and in geometry, and inherits a survival rate from their persistence histories, without class labels.

  3. 3

    Hierarchical persistence filter

    A conjugate Beta belief over the object's survival rate is updated in closed form from hits and misses, inside Rosen's persistence filter.

  4. 4

    Beliefs move with objects

    When an object is re-identified somewhere else, its belief is transferred, so the map learns how mobile that specific object is.

  5. 5

    Environment adaptation

    An optional hazard multiplier, learned online from all objects in the map, rescales every survival model for environments busier or calmer than training.

  6. 6

    Map maintenance

    After each session, objects whose predicted probability of still being there falls below a threshold are removed from the map.

04 · Results

Persistence prediction on simulated and real data

Pick a dataset and a metric. NLL, Brier and ECE measure how good the predicted probabilities are (lower is better); AUC measures how well stayed and moved objects are separated (higher is better).

05 · Qualitative

What the maps look like

Qualitative results on a THOR-Lifelong sequence: objects colored by predicted probability of remaining in place until the next session; black rings mark objects that actually moved.
Fig. 4: THOR-Lifelong. Color shows the predicted probability of staying until the next session; black rings mark objects that moved.
Qualitative results on a 3RScan environment.
Fig. 8: a 3RScan environment.
Qualitative results on an ObChange room: objects on tables and shelves seen from above, colored by predicted presence at the next visit.
Fig. 9: an ObChange room, objects colored by predicted presence at the next visit.

06 · Ablation

Which parts matter

NLL increase over full HiPer when one component is removed · longer = that part matters more

On THOR, replacing the retrieved prior with a hand-set class table and dropping appearance from association hurt the most; a single pooled rate and CLIP class rates also lose accuracy. Switch to 3RScan to compare.

07 · Cite

Citation

@misc{canh2026hiper,
  title         = {Which Objects Stay? Learning Hierarchical Persistence Beliefs for Lifelong Open-Vocabulary Object Maps},
  author        = {Canh, Thanh Nguyen and Zhang, Haolan and Chong, Nak Young},
  eprint        = {2611.07662},
  archivePrefix = {arXiv},
  year          = {2026}
}