PAIRS Lab

Preprint 2026 · under review

USGLoc

Uncertainty-aware semantic visual localization from 3D Gaussian splatting

Thanh Nguyen Canh1, Xiem HoangVan2, Nak Young Chong1,3

1Japan Advanced Institute of Science and Technology   2VNU University of Engineering and Technology   3Hanyang University

Code Video
A pose, and how much to trust it.

01 · Abstract

Where is the camera, and how sure are we?

A visual localizer should tell a robot where the camera is and how much to trust that answer, because the pose is fused into a Bayesian state estimator. Recent semantic 3D Gaussian Splatting localizers are accurate but return a single pose with no confidence, while uncertainty-aware mapping methods quantify map reliability but do not localize.

USGLoc is an evidential localizer built on one principle: the map's own uncertainty drives every stage and survives to the output. Each primitive of an evidential semantic Gaussian map carries a Dempster–Shafer belief over open-vocabulary semantics with an explicit vacuity mass, and the query is encoded into the same space. Vacuity gates 2D–3D semantic matching, which turns retrieval ambiguity into a pose mixture; it gates the correspondences of a render-and-match correction; and it weights the rendering residual per pixel. A tempered Laplace approximation turns each refined hypothesis into a covariance, giving a Gaussian-mixture posterior on SE(3) that condenses to one Gaussian for a filter.

Without an external pose prior, USGLoc localizes to 0.83 cm / 0.26° on average on 7Scenes and 0.62 cm / 0.26° on 12Scenes. The poses it trusts most are the most accurate in the comparison: 0.30 cm on its most-confident tenth of 7Scenes queries, against 0.46 cm for a strong scene-coordinate baseline ranked by a PnP covariance. The tempered covariance is close to nominal at the 68% level and over-confident in the tails.

USGLoc returns a pose and its covariance for a 7Scenes Fire query.
Fig. 1: USGLoc returns a pose and its covariance.

02 · Video

USGLoc in three minutes

03 · Method

Uncertainty from the map all the way to the pose

USGLoc overview: an evidential semantic Gaussian map is built offline; online, the query is segmented and encoded, vacuity-gated matching gives a pose mixture, a render-and-match correction and refinement follow, and a tempered Laplace approximation gives each hypothesis a covariance.
Fig. 2: USGLoc overview.
  1. 1

    Evidential semantic map

    Every Gaussian primitive stores geometry, a CLIP embedding and a Dempster–Shafer belief with an explicit vacuity mass ("I don't know").

  2. 2

    Query in the same space

    The query image is segmented (FastSAM) and encoded by the same evidential head, so map and query confidences share one scale.

  3. 3

    Gated matching → pose mixture

    Vacuity gates 2D–3D semantic matching; ambiguous retrievals become several pose hypotheses instead of one forced guess.

  4. 4

    Render-and-match correction

    Each hypothesis is corrected by matching the query to renders of the map, with vacuity gating the correspondences.

  5. 5

    Vacuity-weighted refinement

    The rendering residual is weighted per pixel by vacuity, so unreliable map regions pull the pose less.

  6. 6

    Calibrated covariance

    A tempered Laplace approximation gives each refined hypothesis a covariance: a Gaussian mixture on SE(3) that condenses to one Gaussian for a filter.

Stages on two Office queries: query, render at the retrieved pose and at the refined pose, vacuity, pixel weight, and a map view of the coarse, refined and true poses.
Fig. 6: stages on two Office queries (favorable examples, chosen by design).

04 · Accuracy

Localization on 7Scenes and 12Scenes

Median translation and rotation error under the SfM pseudo ground truth, and success rates. Pick a dataset, a scene and a metric.

The map rendered at the poses of ACE, Marepo and USGLoc on the same queries, with a free-camera view of the estimated and true poses.
Fig. 5: map rendered at the pose of ACE, Marepo and USGLoc on the same queries.

05 · Trust

The poses it trusts most are the most accurate

Selective localization on 700 7Scenes queries: keep only the most-confident fraction and report the median translation error of what is kept. Methods without a confidence cannot improve.

median translation error, cm · lower is better
Calibration on 7Scenes: empirical coverage against nominal level, and the distribution of squared Mahalanobis distances against chi-square with 6 degrees of freedom.
Fig. 3: calibration on 7Scenes. Close to nominal at 68%, over-confident in the tails (pooled 95% region covers 81%).
Uncertainty quality against ACE on the same 700 queries: sparsification curves and AUSE.
Fig. 4: uncertainty quality against ACE on the same 700 queries.

06 · Ablation

Which parts matter

Each row removes one component from the full model on earlier maps (Chess and Stairs, 100 queries each). Bars show the success rate at 5 cm / 5°.

success rate at 5 cm / 5°, % · higher is better

The render-and-match correction is essential: without it, success on Stairs drops from 58% to 6%. Runtime: about 12 s per query on an RTX 4070 Ti, 78% of it in the matching correction, so USGLoc is not yet real-time.

07 · Cite

Citation

@misc{canh2026usgloc,
  title  = {USGLoc: Uncertainty-Aware Semantic Visual Localization from 3D Gaussian Splatting},
  author = {Canh, Thanh Nguyen and HoangVan, Xiem and Chong, Nak Young},
  note   = {Preprint, under review},
  year   = {2026}
}