3D point cloud and depth map reconstruction with liquid lenses

Abstract

Liquid lenses are variable focus components actuated by specific external control with high repetitive accuracy. If the focusing distance is varied over a range large enough, each object is captured once in focus by the image sensor. Taking advantage of these circumstances and the known optical properties of the liquid lens, an all-in-focus image of the environment can be created, also called as image with Extended Depth of Field (EDoF). Furthermore, it is possible to create a 3D (more precisely a 2.5D) representation of the entire view by using only one camera resp. image sensor.

In this white paper we present a technological approach and implementation that demonstrates the described principle by using Corning® Varioptic® Lenses. The required image and video processing, which includes the algorithmic core, is realized by a GPU implementation within an Embedded System, the NVIDIA® Jetson Nano™. In addition, performance data like image processing rate, spatial resolution, and 3D representation acquisition time are given

Introduction to liquid lenses 

Introduction to liquid lenses Inspired by the functionality of a human eye, liquid lenses offer manufacturers and OEMs improved speed and reliability over mechanical solutions. The human eye can adjust its focus to the environment at incredibly fast speeds; similarly, Corning’s liquid lenses emulate the eyes’ fluid and adaptable characteristics to create a rapid response to variable circumstances. This process is made possible by a technology called electrowetting, which uses an electrical signal to manipulate a liquid solution into a workable lens.

Traditional mechanical solutions are limited in their ability to deliver sharp images continuously and reliably. Corning® Varioptic® Lenses (Figure 1) offer innovative solutions to complex optical challenges. Varioptic Lenses enable fast focus and microfocus without moving parts. Traditional camera systems require moving parts which could begin to wear down and fail over the lifetime of the device. In comparison, liquid lenses function without the use of mechanically moving parts, eliminating much of the maintenance typically associated with vision systems.

Methods for 3D Imaging 

Over the past decade, there has been a growing interest for the capture, processing, and imaging of 3D information. The applications for this imaging modality are wide, ranging from information technology to life sciences or even more recently entertainment. 3D information refers to the additional depth cues that are added to a standard 2D image, captured by an imaging system. Many techniques have been developed to estimate depth using either dedicated software and/or hardware. The most commons are the following:

- Stereovision: The principle is to capture a 3D scene under different viewing angles to estimate the depth of objects using either multiple images acquisition or binocular systems.

- Time of Flight or Phase Shift: A dedicated sensor module sends and receives a laser impulse or a modulated signal, the analysis of which determines the depth of objects depending on the received reflection of the light signal.

- Structured lighting: A structured pattern is projected and deformed by objects in the scene depending on their depths, which allows a 3D reconstruction with a monocular system.

- Depth from focus/defocus: Image processing and contrast analysis gives information on the depth of objects using a variable focus system.

This last technique of depth from focus/defocus benefits from a simple hardware with a monocular system (one camera only), no specific lightning or external sensor and relies on the vari-focus properties of the imaging optical system. Liquid lenses have here a key role to play since their working principle results in an optical power variation depending on the voltage applied to their electrodes. The integration of liquid lenses into imaging devices, such as camera modules, together with some image processing (contrast analysis) create an optical system capable of performing 3D reconstruction using the depth from focus modality.