PAIRS Lab

Preprint 2026 · Survey

Open-World Semantic SLAM

A survey on representations, uncertainty, and lifelong adaptation

Thanh Nguyen Canh1, Haolan Zhang1, Xiem HoangVan2, Nak Young Chong1

1Japan Advanced Institute of Science and Technology   2VNU University of Engineering and Technology

Paper · coming soon Paper list Explore the systems

01 · Overview

Semantic SLAM beyond a closed, static, certain world

Semantic SLAM still largely assumes a closed set of categories, a static world and deterministic labels. Four developments each relax one of these assumptions: differentiable scene representations such as neural radiance fields and 3D Gaussian splatting, vision–language foundation models, principled uncertainty quantification, and lifelong mapping. Yet the literature has grown in parallel threads.

This survey argues that these threads are interlocking pillars of a single shift toward open-world semantic SLAM: SLAM with open vocabularies, in non-stationary environments, with calibrated beliefs over geometry and meaning. It gives a unified probabilistic formulation, organizes 81 systems around four pillars, assesses them through 17 explicit claims, shows that no current system addresses all four, and closes with a reference architecture and seven open problems.

OW1

Open vocabulary

Not a fixed taxonomy of K classes: the map answers natural-language queries such as “the red mug on the left shelf”.

OW2

Non-stationary world

Not a time-invariant map: objects move, furniture is rearranged, seasons change, and the map is maintained across sessions.

OW3

Calibrated semantics

Not point-estimate labels: every map element carries a calibrated belief over what it is, not only over where it is.

Timeline of SLAM from 2011 to 2026 in four eras: classical geometric, closed-set semantic, neural and Gaussian, and open-world, with three waves of foundation models above.
Fig. 2: evolution of SLAM from classical to open-world. Each lane is one era; above, the three waves of foundation models (CLIP, SAM, LLMs).

02 · Four pillars

Four pillars that constrain one another

The four pillars as a diamond: P1 representations, P2 foundation models, P3 uncertainty and P4 lifelong adaptation, with arrows for how each constrains the others.
Fig. 3: the pillars are pairwise coupled. Solid arrows: evidence cited in the survey; dashed: not yet shown in SLAM.

03 · The 81 systems

Explore the taxonomy

Every system appears once, under the pillar it primarily advances (Fig. 4). Filter by pillar or group, or search by name or title. Click a system for its reference.

04 · Cross-pillar analysis

Specialists, not generalists

21 representative systems scored 0–3 on each pillar (0: not addressed · 1: superficially · 2: a primary contribution · 3: state of the art at publication). The survey's target is at least 2 on every pillar, 8 in total.

05 · 17 claims

Explicit, falsifiable claims

Each pillar chapter ends with a critical assessment stated as claims (Table 6). Filter by pillar; claims that feed an open problem link to it.

    06 · Research agenda

    Seven open problems, each with a success test

    07 · Reference architecture

    One system, five modules, four pillars

    Tracking and the map back-end (P1) run at frame and keyframe rate; semantic grounding (P2) produces open-vocabulary embeddings; an uncertainty engine (P3) keeps per-element posteriors; a lifelong manager (P4) detects change and updates submaps; an LLM interface answers compositional queries.

    Architecture: RGB-D input to a tracking front-end and a map back-end of Gaussian or neural-point submaps, a semantic grounding module with CLIP, SAM and DINOv2, an uncertainty engine with Dirichlet BKI or EDL updates, a lifelong manager, and an LLM interface over the open-world map.
    Fig. 11: conceptual architecture for open-world semantic SLAM.
    Structure of the survey: foundations, four pillars, synthesis and outlook.
    Fig. 1: how the survey is organized.

    08 · Cite

    Citation

    @misc{canh2026openworld,
  title  = {Open-World Semantic {SLAM}: A Survey on Representations, Uncertainty, and Lifelong Adaptation},
  author = {Canh, Thanh Nguyen and Zhang, Haolan and HoangVan, Xiem and Chong, Nak Young},
  note   = {Preprint},
  year   = {2026}
}