Workshop

Machine Learning meets Computational Photography

NIPS 2011, Granada

Machine Learning meets Computational Photography

NIPS 2011 Workshop, Granada

Sponsered by:
PASCAL_logo


Program

Saturday, December 17th


Morning Session
7.30 - 7.32 Opening remarks
7.32 - 8.12 Invited talk: From Image Restoration to Compressive Sampling in Computational Photography. A Bayesian Perspective. <pdf>
Rafael Molina
8.12 - 8.52 Invited talk: Old and New algorithm for Blind Deconvolution <pdf>
Yair Weiss

8.52 - 9.10

Coffee break


9.10 - 9.50 Invited talk: The Light Field Camera: Extended Depth of Field, Aliasing and Superresolution <pdf>
Paolo Favaro
9.50 - 10.30 Invited talk: Efficient Regression for Computational Photography: From Color Management to Omnidirectional Superresolution <pdf>
Maya Gupta



Afternoon Session
16.00 - 16.40 Invited talk: Real-time Image Enhancement Using Edge-Optimized a-trous Wavelets <ppt>
Hendrik Lensch
16.40 - 17.20 Invited talk: Superresolution imaging - from equations to mobile applications <ppt>
Filip Šroubek

17.20 - 17.40

Coffee break


17.40 - 18.20 Invited talk: Modeling the Digital Camera Pipeline: From RAW to sRGB and Back <ppt>
Michael Brown


18.20 - 19.00 Invited talk: TBA - 2D and 3D Sparse Geometric Decomposition <pdf>
Jean-Luc Starck
19.00 - 19.02 Closing remarks




Abstracts

Invited talk: From Image Restoration to Compressive Sampling in Computational Photography. A Bayesian Perspective.
Rafael Molina, Universidad de Granada
In numerous applications where acquired images are degraded and where either improving the quality of the imaging system or reproducing the scene conditions in order to acquire another image is not an option, computational approaches provide a powerful means for the recovery of lost information. Image recovery is the process of estimating the information lost due to the acquisition or processing system and obtaining images with high quality, additional information, and/or resolution from a set of degraded images. Three specific areas of image recovery are today of high interest. The first one is image restoration, blind deconvolution, and super-resolution, with application, for instance, on surveillance, remote sensing, medical and nano-imaging applications, and improving the quality of photographs taken by hand-held cameras. The second area is compressive sensing (CS). CS reformulates the traditional sensing processes as a combination of acquisition and compression, and traditional decoding is replaced by recovery algorithms that exploit the underlying structure of the data. Finally, the emerging area of computational photography has provided effective solutions to a number of photographic problems, and also resulted in novel methods for acquiring and processing images. Image recovery is related to many problems in computational photography and, consequently, its algorithms are efficiently utilized in computational photography tasks. In addition, image recovery research is currently being utilized for designing new imaging hardware. In this talk, we will provide a brief overview of Bayesian modelling and inference methods for image recovery and the very related of compressive sensing, and computational photography.

Invited talk: Old and New algorithm for Blind Deconvolution
Yair Weiss, Hebrew University of Jerusalem
I will discuss blind deconvolution algorithms that have been successfully used in the field of communications for several decades and how they can be adapted to the problem of blind deconvolution of images. This yields algorithms that can be rigorously shown to recover the correct blur kernel under certain conditions. I will also discuss the relationship between these algorithms and some recent heuristic algorithms for blind image deconvolution.

Invited talk: The Light Field Camera: Extended Depth of Field, Aliasing and Superresolution
Paolo Favaro, Heriot-Watt University and University of Edinburgh
Portable light field cameras have demonstrated capabilities beyond conventional cameras. In a single snapshot, they enable digital image refocusing, i.e., the ability to change the camera focus after taking the snapshot, and 3D reconstruction. We show that they also achieve a larger depth of field than conventional cameras while maintaining the ability to reconstruct detail at high resolution. More interestingly, we show that their depth of field is essentially inverted compared to regular cameras. Crucial to the success of the light field camera is the way it samples the light field, trading off spatial vs. angular resolution, and how aliasing affects the light field. We present a novel algorithm that estimates a full resolution sharp image and a full resolution depth map from a single input light field image. The algorithm is formulated in a variational framework and it is based on novel image priors designed for light field images. We demonstrate the algorithm on synthetic and real images captured with our own light field camera, and show that it can outperform other computational camera systems.

Invited talk: Efficient Regression for Computational Photography: from Color Management to Omnidirectional Superresolution
Maya Gupta, University of Washington
Many computational photography applications can be framed as low-dimensional regression problems that require fast evaluation of test samples for rendering. In such cases, storing samples on a grid or lattice that can be quickly interpolated is often a practical approach. We show how to optimally solve for such a lattice given non-lattice data points. The resulting lattice regression is fast and accurate. We demonstrate its usefulness for two applications: color management, and superresolution of omnidirectional images.

Invited talk: Real-time Image Enhancement Using Edge-Optimized a-trous Wavelets Hendrik Lensch, University of Tübingen
Edge-avoiding à-trous wavelets (EAAW) offer an elegant and simple way for advanced real-time image smoothing and contrast enhancement. Based on the bilateral filter, edge-avoiding wavelets perform multi-resolution analysis. By optimizing the edge weights to match the edge shape at every scale contrast enhancement can be obtained without the typically encountered artifacts of halos or gradient reversals. We demonstrate examples of real-time aging, as well as smooth reconstruction of noisy Monte Carlo simulations. In addition, an edge-avoiding wavelet framework is presented to emphasize monocular depth cues in 2D images captured with a stereo camera. We demonstrate that these enhanced depth cues can aid the human in 3D search tasks on traditional monitors without degenerating the image.

Invited talk: Superresolution imaging - from equations to mobile applications
Filip Šroubek, Institute of Information Theory and Automation of the ASCR
In the last five years we have witnessed a rapid improvement of methods that perform image restoration, such as, denoising, deconvolution and superresolution. We will provide a brief mathematical background to superresolution as an optimization problem and summarize our contribution. Specifically, we will talk about robustness to misregistration, an extension to space-variant cases and a fast converging method of augmented Lagrangian suitable for constrained optimization problems. We will also give an overview of our past and ongoing commercial applications in which superresolution plays a key role.

Invited talk: Modeling the Digital Camera Pipeline: From RAW to sRGB and Back
Michael Brown, National University of Singapore
This talk presents a study of the in-camera imaging process through an extensive analysis of more than 10,000 images from over 30 cameras. The goal is to investigate if output image values (i.e. sRGB) can be transformed to physically meaningful values, and if so, when and how this can be done. From our analysis, we show that the conventional radiometric model fits well for image pixels with low color saturation but begins to degrade as color saturation level increases. This is due to a color mapping process which includes gamut mapping in the in-camera processing that cannot be modeled with conventional methods. To address this issue we introduce a new imaging model for radiometric calibration together with an effective calibration scheme that allows us to compensate for the nonlinear color correction to convert non-linear sRGB images to CCD RAW responses.

Invited talk: TBA - 2D and 3D Sparse Geometric Decomposition
Jean-Luc Starck, CEA Saclay, Paris
We present several 2D multiscale geometric transforms such as ridgelet and curvelet. We show how they can be extended to the third dimension and how these new decompositions can be used for applications such as denoising or inpainting.

Description

In recent years, computational photography (CP) has emerged as a new field that has put forward a new understanding and thinking of how to image and display our environment. Besides addressing classical imaging problems such as deblurring or denoising by exploiting new insights and methodology in machine learning as well as computer and human vision, CP goes way beyond traditional image processing and photography.

By developing new imaging systems through innovative hardware design, CP not only aims at improving existing imaging techniques but also aims at the development of new ways of perceiving and capturing our surroundings. However, CP is not only about to redefine "everyday" photography but also aims at applications in scientific imaging, such as microscopy, biomedical imaging, and astronomical imaging, and can thus be expected to have a significant impact in many research areas.

After the great success of last year's workshop on CP at NIPS, this workshop proposal tries to accommodate the strong interest in a follow-up workshop expressed by many workshop participants last year. The objectives of this workshop are: (i) to give an introduction to CP, present current approaches and report about the latest developments in this fast-progressing field, (ii) spot and discuss current limitations and present open problems of CP to the NIPS community, and (iii) to encourage scientific exchange and foster interaction between researchers from machine learning, neuro science and CP to advance the state of the art in CP.

The tight interplay between both hardware and software renders CP an exciting field of research for the whole NIPS community, which could contribute in various ways to its advancement, be it by enabling new imaging devices that are possible due to the latest machine learning methods or by new camera and processing designs that are inspired by our neurological understanding of natural visual systems.

Thus the target group of participants are researchers from the whole NIPS community (machine learning and neuro science) and researchers working on CP and related fields.


Looking forward to it!
Michael Hirsch, Stefan Harmeling, Rob Fergus and Peyman Milanfar.




Organizers

michael

Michael Hirsch studied physics and mathematics at the University of Erlangen and at Imperial College London. He received a Diploma in theoretical physics in 2007, before joining the Department of Empirical Inference of Prof. Dr. Bernhard Schölkopf at the Max Planck Institute for Intelligent Systems (formerly MPI for Biological Cybernetics). Since 2011 he works as a post-doctoral researcher at the interplay of machine learning and cosmology at University College London. His research interests cover a wide range of signal and image processing problems in scientific imaging as well as computational Photography.


stefan

Stefan Harmeling is a Senior Research Scientist at the Max Planck Institute for Intelligent Systems (formerly MPI for Biological Cybernetics) in Prof Bernhard Schölkopf's department of Empirical Inference. His interests include machine learning, image processing, probabilistic and causal inference, and general computer science.

Dr Harmeling studied mathematics and logic at the University of Münster (Dipl Math 1998) and computer science with an emphasis on artificial intelligence at Stanford University (MSc 2000). During his doctoral studies he was a member of Prof Klaus-Robert Müller's research group at the Fraunhofer Institute FIRST (Dr rer nat, 2004). Thereafter he was a Marie Curie Fellow at the University of Edinburgh from 2005 to 2007, before joining the Max Planck Institute for Intelligent Systems.


rob

Rob Fergus is an Assistant Professor of Computer Science at the Courant Institute of Mathematical Sciences, New York University. He received a Masters in Electrical Engineering with Prof. Pietro Perona at Caltech, before completing a PhD with Prof. Andrew Zisserman at the University of Oxford in 2005. Before coming to NYU, he spent two years as a post-doc in the Computer Science and Artificial Intelligence Lab (CSAIL) at MIT, working with Prof. William Freeman. He has received several awards including a CVPR best paper prize (2003) and a Sloan Fellowship (2011).


peyman

Peyman Milanfar is Professor of Electrical Engineering at the University of California, Santa Cruz. He received a B.S. degree in Electrical Engineering/Mathematics from the University of California, Berkeley, and a Ph.D. degree in Electrical Engineering from the Massachusetts Institute of Technology. Prior to coming to UCSC, he was at SRI (formerly Stanford Research Institute) and a Consulting Professor of computer science at Stanford. In 2005 he founded MotionDSP Inc., which has brought state-of-art video enhancement technology to consumer and forensic markets. His technical interests are in statistical signal, image and video processing, and computational vision. He is a Fellow of the IEEE.