The burst shot has become a mainstay of smartphone photography. When you’re trying to capture someone playing a sport, pets playing, or a portrait of a family with kids, burst mode can be a lifesaver. It’s unfortunate, though, that so many burst shots end up being blurry and unusable. Luckily, a pair of researchers at MIT are working on a way to fix that problem.
In the paper “Burst Image Deblurring Using Permutation Invariant Convolutional Neural Networks,” published at ECCV 2018, Miika Aittila and Frédo Durand describe a way to reduce the blur in a burst shot using a neural network.
“Motion blur and noise remain a significant problem in photography despite advances in light efficiency of digital imaging devices,” the research paper explains. “Mobile phone cameras are particularly suspect to handshake and noise due to the small optics and the typical unsupported free-hand shooting position.”
This is particularly problematic when trying to capture the action, like with a burst shot. To fix this, the team decided to treat a series of burst mode images differently. Normally, the first and last shot play a disproportionately important role in selecting the most important image of the set (and cleaning up those images). Instead, their method treats the burst images as an unordered set, treating each shot with equal importance. They then use a neural network to clean up the images.
https://twitter.com/trustswz/status/1039468169772113920
They trained this network with a wide variety of synthetic data of common imaging defects such as noise and camera shake. Once treated with this algorithm, a blurry burst shot is transformed into “a sharp and noise-free image,” according to the paper. The method can even extract “accurate detail that is not discernible from any of the individual frames in isolation”—so even if every single photo in the burst sequence were blurry, it would be able to accurately sharpen it to the image it should have captured.
Aittila and Durand’s findings could help influence how our phones handle blurry photos and burst shots in the future, making it increasingly difficult for even the worst photographers to take a bad photo.
H/T Twitter
