What is NavDP Research Sandbox?

Background & Motivation

NavDP (Navigation Diffusion Policy) is a breakthrough research paper by Wenzhe Cai et al. that demonstrates how a diffusion-based model trained entirely in simulation can achieve robust, real-world mapless navigation. The key insight is using privileged information during training (e.g., ground-truth depth, full scene geometry) that is unavailable at inference time, dramatically narrowing the sim-to-real gap.

This fork is an academic research sandbox that extends the official evaluation framework in three primary directions:

• Hardware Expansion: Integrating the LeKiwi 3-wheeled omni-directional robot beyond the default Clearpath Dingo differential-drive base
• Visualization Research: Adding Bird's Eye View and Third-Person perspective cameras for richer qualitative evaluation
• Tooling: Building reusable diagnostic scripts that automate the robot on-boarding pipeline (USD fixing → collision injection → physics validation)

Navigation Tasks

The benchmark supports three classes of navigation challenges, each with separate evaluation scripts and goal-conditioning inputs:

Episode Format

Start/goal pairs for each task are stored as .npy files inside the scene directories. Both PointGoal and ImageGoal episodes are pre-generated per scene so that evaluations are reproducible across methods:

assets/scenes/cluttered_easy/easy_0/
├── cluttered-0.usd/                     # Scene geometry
├── imagegoal_start_goal_pairs.npy       # ImageGoal episodes
└── pointgoal_start_goal_pairs.npy       # PointGoal episodes

Scene Categories

The benchmark supports four scene categories from InternScene-N1, spanning a spectrum from synthetic clutter to photorealistic indoor environments:

Visualization System

One of the key contributions of this fork is the multi-perspective visualization pipeline implemented in utils_tasks/visualization_utils.py. The VisualizationManager class provides:

Bird's Eye View (BEV)

• Renders a top-down occupancy grid centered on the robot
• Tracks historical positions (configurable history_size) in gray
• Current obstacle points in red
• Planned trajectory overlaid with a robot rectangle (white outline, oriented by yaw)
• Anti-aliased via OpenCV LINE_AA and Gaussian blur post-processing

Multi-Trajectory Value Heatmap

• Renders all candidate trajectories proposed by the diffusion policy
• Colors each trajectory by its value score using a blue→green→red gradient
• Fixed normalization range [-1.2, 0.2] for consistent comparison across episodes
• Composited below the RGB+BEV strip in the final output video frame

Output Format

Evaluation runs write video frames via imageio to evaluation_outputs/videos/. Raw metrics (success rate, path length, episode duration) are logged as CSV to evaluation_outputs/benchmark_runs/.

Robot Comparison

Repository Structure