Exploring Diffusion Policies for omni-directional robotics navigation. Built on NavDP · Runs on Isaac Sim 4.2 + Isaac Lab 1.2
Three primary areas of extension beyond the official NavDP framework
Full integration of the LeKiwi 3-wheeled omni-directional robot into Isaac Lab. Replacing the default Dingo differential-drive base with a more complex kinematic model required solving ghost robots, physics explosions, and friction modeling — all documented in the Integration Log.
→
Added Bird's Eye View (BEV) and Third-Person View camera perspectives
for richer evaluation analysis. The VisualizationManager renders trajectory overlays,
occupancy grids, and multi-trajectory value heat-maps in real time.
A dedicated research_tools/ directory houses automated scripts for USD fixing,
collision injection, physics validation, and joint inspection — dramatically reducing robot
on-boarding time for new hardware.
Three evaluation modes from the official NavDP benchmark, now runnable on both Dingo and LeKiwi
The robot explores freely with no explicit target. Tests general spatial reasoning and obstacle avoidance under the diffusion policy.
python eval_nogoal_wheeled.py --robot lekiwi --port 8888
Navigate to a given (x, y) coordinate in the world frame. Evaluates metric localization and trajectory planning accuracy.
python eval_pointgoal_wheeled.py --robot lekiwi --scene_dir [PATH]
Navigate to a location depicted in a reference image. Tests visual goal-conditioning and semantic understanding of the policy.
python eval_imagegoal_wheeled.py --robot lekiwi --port 8888
The evaluation framework is cleanly separated into three independent layers
Four navigation approaches integrated as plug-and-play HTTP servers
The primary method. End-to-end diffusion policy for mapless navigation trained entirely in simulation with privileged information guidance.
baselines/navdp/navdp_server.py
baselines/navdp/policy_network.py
Language-guided planner with depth estimation via Depth Anything V2. Extended with a real-world LeKiwi host client for sim-to-real transfer.
baselines/logoplanner/logoplanner_server.py
baselines/logoplanner/lekiwi_logoplanner_host.py
Goal-conditioned diffusion model for topological navigation with sub-goal prediction. Uses YAML-based configuration for easy parameter sweeps.
baselines/nomad/nomad_server.py
baselines/nomad/configs/nomad.yaml
Visual Navigation Transformer. A foundation model for mobile robotics navigation using a transformer backbone for visual goal conditioning.
baselines/vint/vint_server.py
baselines/vint/vint_network.py
Get up and running in minutes
git clone https://github.com/xoTEMPESTox/NavDP
cd NavDPconda create -n isaaclab python=3.10
conda activate isaaclab
pip install isaacsim==4.2.0.2 isaacsim-extscache-physics==4.2.0.2 \
isaacsim-extscache-kit==4.2.0.2 isaacsim-extscache-kit-sdk==4.2.0.2 \
--extra-index-url https://pypi.nvidia.comcd baselines/navdp/
pip install -r requirements.txt
python navdp_server.py --port 8888 --checkpoint /path/to/checkpoint.pth# PointGoal with Dingo (default)
python eval_pointgoal_wheeled.py --port 8888 --scene_dir ./assets/scenes/cluttered_easy
# PointGoal with LeKiwi
python eval_pointgoal_wheeled.py --port 8888 --robot lekiwi --scene_dir ./assets/scenes/cluttered_easyKey engineering discoveries documented during the LeKiwi integration
The robot spawned but refused to move — caused by a fixed root_joint in the USD that treats it as a static prop, not a mobile agent. Fix: fix_lekiwi_usd.py deactivates the joint and applies ArticulationRootAPI.
Wheels spinning at 20 rad/s, robot not moving an inch. The collision meshes were missing — the physics engine saw an empty ghost. Fix: add_wheel_collisions.py procedurally injects cylinder/box geometry.
Simulation flooded with NaN errors. Self-collisions between overlapping shapes + high actuator gains caused infinite force vectors. Fix: Disabled self-collisions, used torch.linalg.pinv instead of torch.inverse.
Robot dragging like a weight — cylinder collision on the caster wheel resisted lateral sliding. Fix: Front wheels → Cylinder (traction). Back wheel → Sphere (zero friction, free sliding).