A MEMS-based Foveating LIDAR to enable Real-time Adaptive Depth Sensing

Francesco Pittaluga

NEC Labs America

Zaid Tasneem

Univ. of Florida

Justin Folden

Univ. of Florida

Brevin Tilmon

Univ. of Florida

Ayan Chakrabarti

Wash U. in St Louis

Sanjeev Koppal

Univ. of Florida

Adaptive LIDAR Design -- We have designed a flexible MEMS mirror-modulated scanning LIDAR. We co-locate this directionally controllable LIDAR with a color camera, allowing for deep depth completion of the sparse LIDAR measurements. We show a ray diagram of our design (left) and a picture of the hardware setup (right) above. The long optical path in the hardware setup is simply an artifact of having a single circuit board for both the LIDAR receiver and transmitter.


Most active depth sensors sample their visual field using a fixed pattern, decided by accuracy, speed and cost trade-offs, rather than scene content. However, a number of recent works have demonstrated that adapting measurement patterns to scene content can offer significantly better trade-offs. We propose a hardware LIDAR design that allows flexible real-time measurements according to dynamically specified measurement patterns. Our flexible depth sensor design consists of a controllable scanning LIDAR that can foveate, or increase resolution in regions of interest, and that can fully leverage the power of adaptive depth sensing. We describe our optical setup and calibration, which enables fast sparse depth measurements using a scanning MEMS (micro-electro mechanical) mirror. We validate the efficacy of our prototype LIDAR design by testing on over 75 static and dynamic scenes spanning a range of environments. We also show CNN-based depth-map completion from measurements obtained by our sensor. Our experiments show that our sensor can realize adaptive depth sensing systems.


  title={A MEMS-based Foveating LIDAR to enable Real-time Adaptive Depth Sensing},
  author={Pittaluga, Francesco and Tasneem, Zaid and Folden, Justin and Tilmon, Brevin and Chakrabarti, Ayan and Koppal, Sanjeev J},
  journal={arXiv preprint arXiv:2003.09545},