WayTrace: A Map-to-Language-to-Trace Hierarchy for Embodied Urban Navigation

WayTrace turns map directions into language commands, image-space traces and robot motion for long-range urban navigation.

Abstract

Web services provide walking directions for nearly every city, but following them reliably over long routes remains challenging for autonomous robots. Point-goal navigation policies rely heavily on accurate GNSS, while most vision-language navigation models struggle to execute even short instructions outdoors. We present WayTrace, a hierarchical system that connects web-map directions to robot motion through language. An off-the-shelf orchestrator vision-language model reads the turn-by-turn directions, a satellite map and recent camera images, and issues one atomic language command at a time, such as ‘cross the road to the left’. An onboard trace predictor, trained on over one million automatically labelled samples from street-level images, draws the path for this command in the image, and a reinforcement learning policy tracks it. In long-range simulated missions, WayTrace remains robust to GNSS noise and rarely leaves crosswalks or sidewalks, while point-goal baselines degrade sharply with about ten times more violations. Deployed on a wheeled-legged robot, WayTrace completes real-world missions of up to 1.37 km at about 1.1 m/s through a busy city centre, rural trails and a university campus, handling crosswalks, staircases and construction sites. Project page: https://way-trace.github.io

Publication
arXiv preprint arXiv:2611.05129
Pascal Roth
Pascal Roth
Ph.D. in Robot Learning & Software Engineer

Research on learning-based navigation and autonomy for legged robots, with focus on perceptive planning and sim-to-real transfer.