This Algorithm Magically Removes Reflections from Images

A group of very smart people has figured out how to remove reflections and obstructions from photographs using an algorithm, and the results are pretty amazing.

Anyone who’s a photographer (which in 2015 is basically everyone) has dealt with the annoyance of taking pictures through windows, through fences, or of something under glass. There are a couple of tricks to minimize reflections or obstructions, but nothing achieves results like this.

The researchers’ algorithm requires a short, relatively static video taken of the subject, then uses edge detection to separate foreground (reflection) from background (desired subject) in a few frames. Not only can the researchers’ process recreate an almost crystal-clear, unobstructed image of the intended subject, they can also create a full image from the reflection.

The researchers’ presentation includes several examples of the algorithm at work, including an example where it effectively removes raindrops on a window—which is more difficult because the raindrops are transparent objects. It can also remove artifacts from panorama images.

The work was presented at SIGGRAPH 2015, a conference on “Computer Graphics and Interactive Techniques.” It’s hard to imagine all the applications this work could have, but I can definitely see it being used by crime labs and security agencies. Zoom in and enhance.