Machine Learning meets Computational Photography
NIPS 2011 Workshop, Granada
Sponsered by:
Program
Saturday, December 17th
Abstracts
Invited talk: From Image Restoration to Compressive Sampling
in Computational Photography. A Bayesian Perspective.
Rafael Molina, Universidad de Granada
In numerous applications where acquired images are
degraded and where either improving the quality of the imaging
system or reproducing the scene conditions in order to acquire
another image is not an option, computational approaches provide
a powerful means for the recovery of lost information. Image
recovery is the process of estimating the information lost due to
the acquisition or processing system and obtaining images with
high quality, additional information, and/or resolution from a
set of degraded images.
Three specific areas of image recovery are today of high
interest. The first one is image restoration, blind
deconvolution, and super-resolution, with application, for
instance, on surveillance, remote sensing, medical and
nano-imaging applications, and improving the quality of
photographs taken by hand-held cameras. The second area is
compressive sensing (CS). CS reformulates the traditional sensing
processes as a combination of acquisition and compression, and
traditional decoding is replaced by recovery algorithms that
exploit the underlying structure of the data. Finally, the
emerging area of computational photography has provided effective
solutions to a number of photographic problems, and also resulted
in novel methods for acquiring and processing images. Image
recovery is related to many problems in computational photography
and, consequently, its algorithms are efficiently utilized in
computational photography tasks. In addition, image recovery
research is currently being utilized for designing new imaging
hardware.
In this talk, we will provide a brief overview of Bayesian
modelling and inference methods for image recovery and the very
related of compressive sensing, and computational photography.
Invited talk: Old and New algorithm for Blind Deconvolution
Yair Weiss, Hebrew University of Jerusalem
I will discuss blind deconvolution algorithms that
have been successfully used in the field of communications for
several decades and how they can be adapted to the problem of
blind deconvolution of images. This yields algorithms that can be
rigorously shown to recover the correct blur kernel under certain
conditions. I will also discuss the relationship between these
algorithms and some recent heuristic algorithms for blind image
deconvolution.
Invited talk: The Light Field Camera: Extended
Depth of Field, Aliasing and Superresolution
Paolo Favaro, Heriot-Watt University and University of Edinburgh
Portable light field cameras have demonstrated
capabilities beyond conventional cameras. In a single snapshot,
they enable digital image refocusing, i.e., the ability to change
the camera focus after taking the snapshot, and 3D
reconstruction. We show that they also achieve a larger depth of
field than conventional cameras while maintaining the ability to
reconstruct detail at high resolution. More interestingly, we
show that their depth of field is essentially inverted compared
to regular cameras.
Crucial to the success of the light field camera is the way it
samples the light field, trading off spatial vs. angular
resolution, and how aliasing affects the light field. We present
a novel algorithm that estimates a full resolution sharp image
and a full resolution depth map from a single input light field
image. The algorithm is formulated in a variational framework and
it is based on novel image priors designed for light field
images. We demonstrate the algorithm on synthetic and real images
captured with our own light field camera, and show that it can
outperform other computational camera systems.
Invited talk: Efficient Regression for Computational Photography: from Color
Management to Omnidirectional Superresolution
Maya Gupta, University of Washington
Many computational photography applications can be
framed as low-dimensional regression problems that require fast
evaluation of test samples for rendering. In such cases, storing
samples on a grid or lattice that can be quickly interpolated is
often a practical approach. We show how to optimally solve for
such a lattice given non-lattice data points. The resulting
lattice regression is fast and accurate. We demonstrate its
usefulness for two applications: color management, and
superresolution of omnidirectional images.
Invited talk: Real-time Image Enhancement Using Edge-Optimized
a-trous Wavelets Hendrik Lensch, University of Tübingen
Edge-avoiding à-trous wavelets (EAAW) offer an elegant and simple
way for advanced real-time image smoothing and contrast
enhancement. Based on the bilateral filter, edge-avoiding wavelets
perform multi-resolution analysis. By optimizing the edge weights
to match the edge shape at every scale contrast enhancement can be
obtained without the typically encountered artifacts of halos or
gradient reversals. We demonstrate examples of real-time aging, as
well as smooth reconstruction of noisy Monte Carlo simulations. In
addition, an edge-avoiding wavelet framework is presented to
emphasize monocular depth cues in 2D images captured with a stereo
camera. We demonstrate that these enhanced depth cues can aid the
human in 3D search tasks on traditional monitors without
degenerating the image.
Invited talk: Superresolution imaging - from equations to mobile applications
Filip Šroubek, Institute of Information Theory and Automation of the ASCR
In the last five years we have witnessed a rapid
improvement of methods that perform image restoration, such as,
denoising, deconvolution and superresolution. We will provide a
brief mathematical background to superresolution as an optimization
problem and summarize our contribution. Specifically, we will talk
about robustness to misregistration, an extension to space-variant
cases and a fast converging method of augmented Lagrangian suitable
for constrained optimization problems. We will also give an
overview of our past and ongoing commercial applications in which
superresolution plays a key role.
Invited talk: Modeling the Digital Camera
Pipeline: From RAW to sRGB and Back
Michael Brown, National University of Singapore
This talk presents a study of the in-camera imaging
process through an extensive analysis of more than 10,000 images
from over 30 cameras. The goal is to investigate if output image
values (i.e. sRGB) can be transformed to physically meaningful
values, and if so, when and how this can be done. From our
analysis, we show that the conventional radiometric model fits
well for image pixels with low color saturation but begins to
degrade as color saturation level increases. This is due to a
color mapping process which includes gamut mapping in the
in-camera processing that cannot be modeled with conventional
methods. To address this issue we introduce a new imaging model
for radiometric calibration together with an effective
calibration scheme that allows us to compensate for the nonlinear
color correction to convert non-linear sRGB images to CCD RAW
responses.
Invited talk: TBA - 2D and 3D Sparse
Geometric Decomposition
Jean-Luc Starck, CEA Saclay, Paris
We present several 2D multiscale geometric transforms such
as ridgelet and curvelet. We show how they can be extended to the
third dimension and how these new decompositions can be used for
applications such as denoising or inpainting.
Description
In recent years, computational photography (CP) has emerged as a new field that has put forward a new understanding and thinking of how to image and display our environment. Besides addressing classical imaging problems such as deblurring or denoising by exploiting new insights and methodology in machine learning as well as computer and human vision, CP goes way beyond traditional image processing and photography.
By developing new imaging systems through innovative hardware design, CP not only aims at improving existing imaging techniques but also aims at the development of new ways of perceiving and capturing our surroundings. However, CP is not only about to redefine "everyday" photography but also aims at applications in scientific imaging, such as microscopy, biomedical imaging, and astronomical imaging, and can thus be expected to have a significant impact in many research areas.
After the great success of last year's workshop on CP at NIPS, this workshop proposal tries to accommodate the strong interest in a follow-up workshop expressed by many workshop participants last year. The objectives of this workshop are: (i) to give an introduction to CP, present current approaches and report about the latest developments in this fast-progressing field, (ii) spot and discuss current limitations and present open problems of CP to the NIPS community, and (iii) to encourage scientific exchange and foster interaction between researchers from machine learning, neuro science and CP to advance the state of the art in CP.
The tight interplay between both hardware and software renders CP an exciting field of research for the whole NIPS community, which could contribute in various ways to its advancement, be it by enabling new imaging devices that are possible due to the latest machine learning methods or by new camera and processing designs that are inspired by our neurological understanding of natural visual systems.
Thus the target group of participants are researchers from the whole NIPS community (machine learning and neuro science) and researchers working on CP and related fields.
Looking forward to it!
Michael Hirsch, Stefan Harmeling, Rob Fergus and Peyman Milanfar.
Michael
Hirsch studied physics and mathematics at the University of
Erlangen and at Imperial College London. He received a Diploma in
theoretical physics in 2007, before joining the Department of
Empirical Inference of Prof. Dr. Bernhard Schölkopf at the Max
Planck Institute for Intelligent Systems (formerly MPI for Biological
Cybernetics). Since 2011 he works as a post-doctoral researcher at the
interplay of machine learning and cosmology at University College
London. His research interests cover a wide range of signal and image
processing problems in scientific imaging as well as computational
Photography.
Stefan Harmeling is a Senior Research Scientist at the Max Planck Institute for Intelligent Systems (formerly MPI for Biological Cybernetics) in Prof Bernhard Schölkopf's department of Empirical Inference. His interests include machine learning, image processing, probabilistic and causal inference, and general computer science.
Dr Harmeling studied mathematics and logic at the University of Münster (Dipl Math 1998) and computer science with an emphasis on artificial intelligence at Stanford University (MSc 2000). During his doctoral studies he was a member of Prof Klaus-Robert Müller's research group at the Fraunhofer Institute FIRST (Dr rer nat, 2004). Thereafter he was a Marie Curie Fellow at the University of Edinburgh from 2005 to 2007, before joining the Max Planck Institute for Intelligent Systems.
Rob Fergus is an Assistant Professor of Computer Science at the Courant Institute of Mathematical Sciences, New York University. He received a Masters in Electrical Engineering with Prof. Pietro Perona at Caltech, before completing a PhD with Prof. Andrew Zisserman at the University of Oxford in 2005. Before coming to NYU, he spent two years as a post-doc in the Computer Science and Artificial Intelligence Lab (CSAIL) at MIT, working with Prof. William Freeman. He has received several awards including a CVPR best paper prize (2003) and a Sloan Fellowship (2011).
Peyman Milanfar is Professor of Electrical Engineering at the University of California, Santa Cruz. He received a B.S. degree in Electrical Engineering/Mathematics from the University of California, Berkeley, and a Ph.D. degree in Electrical Engineering from the Massachusetts Institute of Technology. Prior to coming to UCSC, he was at SRI (formerly Stanford Research Institute) and a Consulting Professor of computer science at Stanford. In 2005 he founded MotionDSP Inc., which has brought state-of-art video enhancement technology to consumer and forensic markets. His technical interests are in statistical signal, image and video processing, and computational vision. He is a Fellow of the IEEE.