This Lensless Camera Never Needs Focusing, Has Already Taken Cat Pics

If you get annoyed when your pictures of your cat Mopsy end up out of focus, imagine how frustrating out of focus images are in situation that are actually important, like surveillance. Researchers at Bell Labs have a potential solution: a lensless architecture for taking pictures that are never out of focus.

The new camera is based on the concept of compressive sensing, which, as the authors write in a paper available on the arXiv preprint server, makes it “possible to represent an image by using fewer measurements than the number of pixels.” In other words, it develops an image while tossing out superfluous data.

The architecture described by Gang Huang et al. is surprisingly simple: a single pixel sensor that can record three colors of light is arrayed behind an aperture assembly, potential made by an LCD, that can create a matrix of apertures of varying opacity. By using this matrix with multiple apertures to direct light to the sensor, multiple measurements of light data can be conducted at once. And because the sensor images what passes through the flat sensor, and not what’s been focused on it through a lens, images physically can’t be out of focus. (Although, as we’ll see, at this stage they aren’t very sharp.)

 As the authors write:

Each row of the sensing matrix defines a pattern for the elements of the aperture assembly, and the number of columns in a sensing matrix is equal to the number of total elements in the aperture assembly. Each value in a row of the sensing matrix is used to define the transmittance of an element of the aperture assembly. A row of the sensing matrix therefore completely defines a pattern for the aperture assembly, and it allows the sensor to make one measurement for the given pattern of the aperture assembly. The number of rows of the sensing matrix is the number of measurements, which is usually much smaller than the number of aperture elements in the aperture assembly (the number of pixels).

Whether sensing visible or infrared light, the architecture is able to compress the amount of measurements needed to create an image by recording a set of pictures from the varying aperture assembly and combining that data into one set image. Additionally, a second sensor can be added, which creates more truly detailed images by allowing for different perspectives to be added to the mix.

To test things out, the team created an experimental rig with their LCD aperture assembly and a dual-sensor rig. The LCD panel was designed to operate at a resolution of 302×217, with each square programmed to be either black (opaque) or white (transparent), which then gives binary data for image reconstruction.

Thus, you can kind of think of this camera as reverse of a normal digital camera: Rather than having thousands of pixels in the sensor, this camera splits up resolution before the sensor, which then records whatever is allowed through.

The problem with the system is that it’s slow. Rather than take millions of measurements of color depth at once, the camera’s two sensors scan rows of the LCD matrix until all have been measured.

But here’s where it pays off: the resolution of 302×217 means the camera has a total resolution of 65534 pixels. Yet not all of them actually have to be scanned. In testing, a visible image of a soccer ball was able to be constructed with only 12.5 percent of the image plane scanned. For something more complex, like a sleeping cat (naturally), a visible image was developed with only 25 percent of the image scanned.

Yes, that’s a sleeping cat.

It’s early days yet, but as noted at the MIT Technology Review, the camera architecture described was super cheap to create and removes a lot of problems and compromises inherent to cameras with lenses. Yes, the resolution isn’t great, and it’s a slow process, but the ability to develop images that are fully in focus (don’t confuse being in focus with being sharp or of high resolution, as the image above shows) and with extremely small file sizes is a big development in imaging.

That’s doubly true in applications for which clarity, cost, and small file sizes (and thus large recording capacity) are paramount. What might that be? How about small, low cost surveillance tools that can be tossed up on a warehouse wall to make sure boxes aren’t disappearing? The authors note it could also be used to record multiple images to find the speed of moving objects. Plus, the tech is naturally going to evolve and produce better images, and hell, maybe the lack of lens constraints will make it great for gadgets. Consistently in-focus selfies? Count me in.

@derektmead