Tracking the movement of fluorescent single-molecule (SM) tracers has provided several new insights into the local structure and dynamics in complex environments such as soft materials and biological systems. However, SM tracking (SMT) remains unreliable at molecular length scales, as the localization error (LE) of SM trajectories (∼30–50 nm) is considerably larger than the size of molecular tracers (∼1–2 nm). Thus, instances of tracer (im)mobility in heterogeneous media, which provide indicators for underlying anomalous-transport mechanisms, remain obscured within the realms of SMT. Since the translation of passive tracers in an isotropic media is associated with fast dipolar rotation, we propose that authentic pauses within the LE can be revealed by probing the hindrance of SM reorientational dynamics. Here, we demonstrate how polarization-resolved SMT (PR-SMT) can provide emission anisotropy at each super-localized position, thereby revealing the tumbling propensity of SMs during random walks. For rhodamine 6G tracers undergoing heterogeneous transport in a hydrated polyvinylpyrrolidone (PVP) network, analysis of PR-SMT trajectories enabled us to discern instances of genuine immobility and localized motion within the LE. Our investigations on 100 SMs in (plasticized) PVP films reveal a wide distribution of dwell times and pause frequencies, demonstrating that most probes intermittently experience complete translational and rotational immobilization. This indicates that tracers serendipitously encounter compact, rigid polymer cavities during transport, implying the existence of nanoscale glass-like domains sparsely distributed in a predominantly deep-rubbery polymer network far above the glass transition.