Site Network: Personal | Taubin Group LEMS | Brown

Welcome

Welcome to my homepage. I am a fourth-year graduate student in Electrical Engineering at Brown. My primary academic interests are computer vision, graphics, and machine learning, with applications to consumer electronics, entertainment, and robotics. Thanks for taking the time to visit.

Read more...

Modeling and Synthesis of Aperture Effects in Cameras (June 20, 2008)

My recent work on coded aperture imaging was presented at the International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (CAe 2008). This paper focuses on the capture, analysis, and synthesis of vignetting and depth-of-field effects in conventional cameras. In order to simultaneously estimate vignetting-correction tables and spatially-varying blur kernels, we propose a simple calibration pattern consisting of a two-dimensional array of point light sources. In addition, this paper introduces the Bokeh Brush: a novel, post-capture method for full-resolution control of the shape of out-of-focus highlights. Typical results are summarized in the animations below. More information, including a copy of the paper and the presentation slides, is available here.


Surround Structured Lighting for Full Object Scanning (August 21, 2007)

I recently presented my paper Surround Structured Lighting for Full Object Scanning at the 6th International Conference on 3-D Digital Imaging and Modeling (3DIM 2007) in Montréal. This paper introduces a new system for acquiring complete 3D surface models using a single structured light projector, a pair of planar mirrors, and one or more synchronized cameras. Using an orthographic projector composed of a Fresnel lens and a DLP projector, we display a single Gray code sequence to encode all the illumination planes within the scanning volume. More information is available here.


Structured Light for 3D Scanning (May 15, 2007)

My research during the last semester has focused on the topic of 3D reconstruction using controlled illumination. As the head TA for CS 220, I created an assignment in which the students implemented Bouguet's desktop 3D scanner using only a webcam, a halogen lamp, and a stick. The results of my reference implementation are presented here. To support additional class projects, I also implemented a conventional 3D scanning system using structured light. A given object was scanned by illuminating with a temporally-multiplexed Gray code sequence. By observing the projected patterns from one or more viewpoints, each visible point was reconstructed in 3D by ray-plane triangulation. A complete Matlab projector-camera calibration toolbox was developed to estimate the relative projector-camera position/pose and intrinsic parameters. For more information, see my class presentation here. Finally, my own project on Surround Structured Lighting was recently accepted for publication and oral presentation at 3DIM 2007.


Spring Course Schedule (January 21, 2007)

This term I will be taking a variety of courses in Computer Science and Electrical Engineering, however my primary focus will be on expanding my research. See the webpages below for detailed course descriptions. In addition, I am currently working with the Department of Engineering to secure funding for a computer vision seminar series. For more information, see the Computer Vision Reading Group website.

Downloads: course schedule

Exemplar-based Image Inpainting (December 19, 2006)

I recently completed my final project for EN 256: Computer Vision. Building on my prior Image Quilting implementation, I created a complete system for automatically inpainting large image regions. More specifically, I implemented Exemplar-based Image Inpainting by Criminisi, Pérez and Toyama. As shown in the figure below, this approach involves copying small image fragments to complete a user-specified masked region. For the "Hollywood" example, we find that this system reliably removes a large region of the original image while preserving the underlying background textures. In general, this approach has broad applications extending from photographic restoration to removing wires from special effects sequences. For more information, see my final report and the accompanying movies (video1, video2).

Texture Synthesis using Image Quilting (November 27, 2006)

For my final project in EN 256: Computer Vision, I am currently studying texture synthesis algorithms and their application to problems in computer vision and graphics. If we view a given image as a sample from a local stationary random process, then the goal of texture synthesis is to generate additional images which appear (to a human observer) to be produced from the same underlying distribution. Applications include non-periodic texture mapping for graphics, as well as inpainting for photographic restoration. For reasons discussed in my project proposal, I decided to implement Efros and Freeman's Image Quilting algorithm – a particularly simple, yet effective, patch-based solution. As shown in the figure below, the algorithm begins by selecting a random seed block from the sample texture. Afterwards, additional blocks are assigned (in raster order) such that the overlapping regions are as similar as possible. For more information regarding this technique, please consult my recent progress report. Source code will be made available in late December.

Unsupervised Learning of Handwriting Styles (November 10, 2006)

Recently, Bernoulli mixture models (BMM) have been used in a wide variety of applications including document understanding, image analysis, and financial modeling. I recently implemented the EM algorithm for multivariate BMM estimation. As an example, a 12 component model was trained on the first five digits in the MNIST handwriting dataset (see the 10,000 training samples here). The per-pixel values of the Bernoulli parameters, for the first 20 steps of the EM algorithm, are animated below. Note that the BMM automatically identified the five digit classes, as well as the dominant styles for writing each digit. Additionally, the mixture components can be used for text recognition; for instance, an optimal Bayes classifier built using the BMM obtains a test error rate on the order of 3% – comparable to the results reported by LeCun and Cortes here. For additional information, see the following report.


Bilateral Filtering and Image Abstraction (September 2, 2006)

I have recently implemented a form of automatic image abstraction inspired by the SIGGRAPH paper by Winnemöller et al. My source code for bilateral filtering and image/video abstraction is now available on Matlab Central. Results are presented in the following album and movies (video1, video2, video3, video4).


Fall Course Schedule (September 1, 2006)

This term I am taking a variety of courses in the Engineering, Computer Science, and Applied Math. departments. Please see the following websites for course descriptions:

Downloads: course schedule

SIGGRAPH and France (August 5, 2006)

I recently attended SIGGRAPH 2006 to present my poster on Multi-Flash 3D Photography. There were a great variety of papers involving Computational Photography this year, including: Photosynth, Coded Exposure Photography, and Multiview Radial Catadioptric Imaging for Scene Capture.

 

Prior to attending SIGGRAPH, I was in France for a month working as a summer intern at INRIA Rhône-Alpes in the PERCEPTION team. (Note that the artistic rendering shown at the left was generated automatically by the NPR system presented by ETRI in the Guerilla Studio.)

Publications Update (June 7, 2006)

The Research and Courses/Projects pages have been updated. Three papers have been accepted for presentation at the Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT 2006). My SIGGRAPH poster has also been accepted. In addition, a variety of homework solutions, course projects, and other tutorials have been made available for download.

Matlab Central Contributions (January 15, 2006)

In my research I often implement a variety of basic image processing algorithms in Matlab. I've decided to start posting these programs to the Matlab Central File Exchange. Please see my author page for more information.

Update (January 5, 2006)

This term I will be focusing my research on visual sensor networks (VSN). In addition, I will continue my efforts on catadioptric mirror arrays, especially for 3D reconstruction. Coursework will include:

Downloads: course schedule

 

Last Updated: June 27, 2008