Summary
Metalenz has acquired PxE Holographic Imaging, the Israeli startup behind an optical layer called the HoloCoder that the Boston company says can deliver up to 2.5 times more light to a standard image sensor, WIRED reported on 11 November in an exclusive. Details of the deal were not disclosed. The HoloCoder is a microscopic structure that sits directly on top of the image sensor. Rather than absorbing the wavelengths it does not want, it diffracts incoming light, bending the waves toward the pixels that should record them. The 2.5 times figure is Metalenz’s claim, and no independent test of the technology has been published. TechRepublic, which covered the acquisition on 13 November, wrote that “Metalenz claims HoloCoder can deliver up to 2.5 times more light to a standard image sensor,” and said real-world smartphone tests are still needed. What the color filter throws away The part being replaced is the Bayer filter, a red, green and blue layer that sits above the sensor and adds color to images. It was created in 1974 and has changed little since. To record a red pixel, the filter blocks the blue and green light arriving at that spot. According to Metalenz, conventional Bayer filters absorb roughly 60 percent of incoming light before it reaches the sensor. The filter array is one of the layers covered in this beginner’s guide to the smartphone camera. Phone makers have leaned on computational photography to make up the difference. Google’s approach is one example: Pixel Night Sight takes up to 15 short exposures, aligns them to a fraction of a pixel and averages out the noise to build a usable picture in near darkness. Rob Devlin, Metalenz chief executive and cofounder, told WIRED that software cannot recover what the optics discard. “Apple, Samsung, Google—everyone is fighting to improve the visible camera every year, right?” Devlin said. “A lot of what they’re doing is investing hundreds of millions of dollars into software to try to somehow improve it, but the downside of that is that once information is lost when you’re absorbing all of that light, you can’t really buy it back, even if you’re throwing all of the software into the world at it.” Metalenz’s own description of the PxE technology puts the gain in signal-to-noise terms. The company’s site says PxE’s holographic encoder “replaces the color filter array, delivering simultaneous color and depth with 2.5x the SNR of Bayer-based cameras.” That wording is also the company’s, and no third-party measurement has been published. Depth from a single sensor The second thing the HoloCoder does is capture 3D depth information at the same time as color, according to WIRED and TechRepublic. Most smartphones work out the distance between camera and subject using a secondary camera, a dedicated depth sensor or AI guesswork, which is how portrait mode decides how much depth to show. Merging light sensing and depth sensing into one layer would remove some of those components, saving space on the back of a handset and cutting the processing power needed to line up data from separate sensors. Metalenz was founded in 2016 by Devlin and Federico Capasso, built on research from the Capasso Lab at Harvard, and emerged from stealth in 2021 as the first company to commercialize metalenses, flat optics patterned with nanostructures smaller than a wavelength of light. Through a partnership with STMicroelectronics, metalenses shipped in consumer devices for the first time in 2022. More than 300 million of the company’s metasurfaces are in use today across consumer, automotive and industrial markets. The HoloCoder is not one of them. It works as a diffractive optic rather than a true metasurface, although Devlin said Metalenz has already replicated the technology in the lab using its own metasurface optics. He expects it to fit into the supply chain, image pipeline and machine learning work the company has already built, and to “completely disrupt” the visible sensing market. Devlin described the long-term goal as a single layer that reads everything light carries. “It’s really then bringing all of that information together through a single approach,” he said. “We would do a metasurface design where now from a single image sensor, you’re getting all of the information that is in light: You’re getting polarization, you’re getting color without filtering, and you’re getting this depth channel. It’s like the one sensor to rule them all.” Unproven, and not in any phone For now the two technologies stay apart. Devlin said they will be separate products for different markets, with a possible merger into one Metalenz system in the long term. The company’s other work includes Polar ID, a face authentication system for Android built with Qualcomm that reads polarization data and is expected on consumer devices at the end of 2027. That date applies to Polar ID, not the HoloCoder. On the HoloCoder, Devlin told WIRED he expects it could be used in consumer electronics as early as late 2027 or early 2028. No manufacturer has been named as a customer, and no phone uses the layer. Metalenz is not the only company working on the problem. Arka Majumdar, who researches low-energy opto-electronics at the University of Washington, told WIRED that Samsung and a startup called Eyeo have been researching their own alternatives to Bayer filters, known as color routers. “It is a real problem in the sense that, in the color filter, you lose a lot of light,” Majumdar said. He agreed that dropping the filter can meaningfully improve smartphone photography in low light. He was less convinced by the pitch for robotics and self-driving cars, which Devlin has called transformational categories for the technology. Vehicles have plenty of space to mount stereo cameras or lidar, Majumdar said, and their bottleneck is the computational power needed for split-second decisions rather than low light or image clarity. “In fact, arguably, if you get better data, it’s actually worse,” he said.