Sombit Dey

I am a Second-Year PhD student at the INSAIT in Sofia, Bulgaria, where I work on Robotics and Robot perception. My PhD supervisors are is Luc Van Gool. and Dr. Danda Dani Paudel

I have an MS in Robotics ( Computer Vision) from ETH-Zurich, where I was completed my Master thesis on 3D Visual Grounding at CVL Lab. I have a BTech in Electronics( CS-Minor) from IIT Kharagpur, where I worked in AGV lab with Prof. Debashish Chakravarty

Email  /  GitHub  /  Google Scholar  /  LinkedIn

Asynchronous Robot Policies

We run the large vision-language model and the small action network of a robot policy as two separate processes. The robot reacts to every new camera frame without waiting for the large model, and the large model only needs to update now and then.

AR-VLA: Autoregressive Action Expert for Vision-Language-Action Models

We replace the chunk-based action head of a VLA with a standalone autoregressive action expert that keeps a long-lived memory of its own actions while attending to vision-language features that refresh asynchronously. AR-VLA produces smoother trajectories, remembers task state that is no longer visible, and matches or beats chunk-based VLAs in SIMPLER and on a real WidowX robot.

SPEAR-1: Scaling Beyond Robot Demonstrations via 3D Understanding

We build SPEAR-VLM, a 3D-aware vision-language model that infers object coordinates from a single image, and use it to train SPEAR-1, a robotic foundation model with grounded 3D perception and language-conditioned control. SPEAR-1 trains on ~45M frames from 24 Open X-Embodiment datasets and matches or surpasses π0-FAST and π0.5 while using 20× fewer robot demonstrations. Model weights are openly released.

ReVLA: Reverting Visual Domain Limitation of Robotic Foundation Models

We study the visual generalization capabilities of three existing robotic foundation models, and propose a corresponding evaluation framework.

Fine-Grained Spatial and Verbal Losses for 3D Visual Grounding

We introduce two novel losses for 3D visual grounding: a visual-level offset loss on regressed vector offsets from each instance to the ground-truth referred instance and a language-related span loss on predictions for the word-level span of the referred instance in the description

Learning whom to trust in navigation: dynamically switching between classical and neural planning

We introduce two novel losses for 3D visual grounding: a visual-level offset loss on regressed vector offsets from each instance to the ground-truth referred instance and a language-related span loss on predictions for the word-level span of the referred instance in the description