## Embedded Phase Shifting: Robust Phase Shifting With Embedded Signals

by Baniel Moreno, Kilho Son, and Gabriel Taubin
Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015
pp. 2301-2309
doi: 10.1109/CVPR.2015.7298843

Dowload the PAPER from this web site. Download the PAPER from the CVF Open Access repository. Dowload the POSTER from this web site.

This project was sponsored by the National Science Foundation.

# Brief Description

Embedded Phase Shifting (Embedded PS) is a structured light phase shifting method for 3D scanning presented in CVPR 2015 at Boston. Embedded PS was created at LEMS in Brown University by Daniel Moreno, Kilho Son, and Gabriel Taubin. The main contributions of this paper are:
• Projection patterns contain only high spatial frequency sinusoidals, which makes the method robust to global illumination effects and defocus of the illumination source.
• Embedded low frequency phases, not present in the projection patterns, are synthezised during decoding and used for accurate and fast unwrapping of the high frequency phases.
• Multiple measurements, one from each projected high frequency, allow to create more accurate and less noisy 3D models.
• A set of mathematical relations applicable to any periodic signal group is presented which may allow to create `embedded' versions of other SL methods.
If you use our work as part of your research please cite our paper like this:

@InProceedings{Moreno_2015_CVPR,
author = {Moreno, Daniel and Son, Kilho and Taubin, Gabriel},
title = {Embedded Phase Shifting: Robust Phase Shifting With Embedded Signals},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2015}
}

Models generated by Embedded PS from 9 images with no post processing such as filtering or mesh reconstruction. Fine structures below 1mm depth in a 180mm object are correctly measured. Total decoding time with a single thread Matlab implementation takes about 1.2s, image resolution is 1600x1200.

# Summary

### Objective

To create a Phase Shifting (PS) method for 3D scanning robust to global illumination effects and with low decoding computational cost.

### Related work

• Multiple PS [1] projects patterns of multiple frequencies and uses Temporal Phase Unwrapping [2], which is accurate and computationally efficient, but it is not robust to global illumination effects.

Frequencies from low to high are projected.

• Micro PS [3] is robust to global illumination effects, because it uses only high frequency patterns, but decoding requires search in a LUT which is inefficient and produces frequent errors.

High frequencies only are projected.

### Embedded PS

• High frequency patterns robust to global illumination
• Embedded low frequencies permit efficient and accurate phase unwrapping
• Every projected frequency generates a depth measurement, multiple measurements combined produce more accurate results.

Only high frequencies are projected, but low frequencies are recovered during decoding.

### Coding method

Let be $$\{T_1,\ldots, T_M\}$$ real numbers greater than 1.  We define $$M$$ embedded frequencies $$F_m$$ as:

$$F_m = \frac{1}{T_1}\cdots\frac{1}{T_m}.$$

We define $$M$$ pattern frequencies $$f_m$$ as:

$$\begin{bmatrix} f_1\\ f_2\\ f_3\\ \vdots\\ f_M \end{bmatrix} = \begin{bmatrix} 1 &0 &0 & &0\\ 1 &1 &0 & \ldots &0\\ 1 &0 &1 & &0\\ & \vdots& &\ddots&0 \\ 1 & 0&0&\ldots &1 \end{bmatrix} \begin{bmatrix} F_1\\ F_2\\ F_3\\ \vdots\\ F_M \end{bmatrix}.$$

Create a sinusoidal pattern for each of the $$M$$ pattern frequencies and project 2 or 3 shifts of each frequency.

Example: if $$T_1=16$$, $$T_2=8$$, and $$T_3=8$$; then

• embedded frequencies are $$F_1=1/16$$, $$F_2=1/128$$, and $$F3=1/1024$$
• pattern frequencies are $$f_1=1/16$$, $$f_2=1/14.22$$, and $$f_3=1/15.75$$

### Decoding method

Use image intensities to recover the phase vector:

$$\mathbf{u}=[o,c_1,s_1,\ldots,c_M,s_M]^T,\ \ c_m=a_m\cos(\omega_m\phi_m),\ s_m=a_m\sin(\omega_m\phi_m).$$

From the phase vector recover pattern frequency phases:

$$\phi_m= \tan^{-1}\dfrac{s_m}{c_m}$$

Extract the embedded frequencies:

$$\left\{\begin{array}{l} \Phi_1 \equiv \phi_1\\ \Phi_m=\phi_m-\phi_1 \ \ \ \text{for}\ m>1. \end{array}\right.$$

Use Temporal Phase Unwrapping [2] to unwrap the embedded frequency phases $$\Phi_m$$ and pattern frequency phases $$\phi_m$$.  Each pattern frequency phase provides a separate depth measurement.

### Comparison of PS unwrapping methods

Profile of a metal plane
• Micro PS produces frequent unwrapping errors, some of them can be corrected with a median filter.
• Multiple PS and Embedded PS show now errors.

### Surface Properties

(LEFT) Lambertian surface. (RIGHT) Specular concave surface.
• Lambertian Surface: in the absence of global illumination effects Embedded PS and Multiple PS performs similarly.
• Specular Concave Surface: Embedded PS and Micro PS performs well in the presence of global illumination effects.

### Multiple Measurements

3D models from same image data using only a single measurement (blue) and multiple measurements (red). The latter one has less noise and it is more accurate.

Single vs. Multiple measurements.

### 3D Models Comparison

Figure columns: Scene picture, Embedded PS, Micro PS, and Multiple PS

Rectangular Bowl: concave object with specular surface, exhibit strong interreflections.

Tree Leaf: fine structures with small details.

Orange: semi translucent skin results in subsurface scattering.

Spatula: polished metal anisotropic surface.

### Computation Complexity

Embedded PS has low coputational complexity, similar to Multiple PS, contrary to Micro PS which executes a search for each image pixel to decode.

Plot of the decoding algorithm computation time vs. the number of captured images.

### Conclusion

Embedded PS is robust to global illumination effects, it is fast and accurate, and produces multiple measurements which are used to generate 3D models with very low noise.

### References

1. T. Pribanić, H. Dźapo, and J. Salvi. Efficient and low-cost 3D structured light system based on a modified number-theoretic approach. EURASIP, pages 1–11, 2010.
2. H. O. Saldner and J. M. Huntley. Temporal phase unwrapping: application to surface profiling of discontinuous objects. Applied optics, 36(13):2770–2775, 1997.
3. M. Gupta and S. Nayar. Micro Phase Shifting. In CVPR, pages 1–8, Jun 2012.