Qualcomm’s next flagship processor is embracing the same photo format technology championed by Apple, a move that will let you squeeze more pictures into your next high-end Android smartphone and maybe advance beyond ordinary snapshots.
Last year, Apple, as part of an effort to improve digital photography, giving it an advantage in the area for more than a year. HEIF’s most obvious selling point is that photos are half the size of JPEGs of the same quality, which means you can fit more on your phone and use less network data to share shots with your friends.
But HEIF offers other advancements, too — support for depth maps that record how far away parts of a scene are from the camera, for example, or a way to house the quick videos called live photos. Now those advantages will come to at least the high end of the Android market, helping casual snapshooters and serious photographers get more out of their smartphones.
“When Qualcomm puts it in hardware, everyone is going to tap into it,” said P.J. Jacobowitz, Qualcomm’s senior marketing manager for camera and computer vision.
The support for HEIF comes at a time when we’re snapping more photos than ever — baby pictures, Facebook selfies, Instagram food shots. The result: many of us are running out of space on our phones, not to mention eating through our monthly network data plans sooner. But HEIF offers at least a partial answer to those problems.
HEIF rises from obscurity
HEIF was an obscure technology compared to the universally embraced JPEG format. But that changed last year when Apple, which supports a specific variant called HEIC, designed to hardware and software to utilize the technology.
Google’s Android Pie, aka version 9.0, added the ability to view HEIC files earlier this year, and Windows 10 followed suit soon after. But hardware support in Qualcomm processors will encourage phone makers to actually let us take photos with the technology.
“This will be the first Snapdragon processor that has dedicated hardware for the HEIF format,” Jacobowitz said. “You’ll be able to capture HEIF photos at really high speed and really low power by burning it into the hardware.”
The HEIF support is part of Qualcomm’s general push to advance image processing in its chips and to promote computer vision in phones. Thealso can detect, identify and track objects in a scene; judge depth so portrait mode can be applied to 4K video at 60 frames per second; electronically stabilize video to compensate for shaky hands; and monitor a scene so VR and AR devices know how they’re moving around, Jacobowitz said.
The Snapdragon 855 will appear in phones from Samsung and others, Qualcomm announced Tuesday at a Snapdragon debut event in Maui, Hawaii.
What’s so great about HEIF?
Shrinking file sizes is a great selling point, especially when phone makers charge so much money for higher-end models with more storage space and carriers often impose data caps.
But HEIF has other abilities that go beyond JPEG:
- It can accommodate groups of photos, including the short videos called live photos; groups of single frames you might combine into a single high-dynamic range (HDR) shot; and bursts of photos for action shots like people diving into lakes or kids hitting baseballs. (Apple offers some of these features, but not through HEIF’s capabilities.)
- It can add “depth map” data so you know how far away from your phone camera each pixel in the photo is located. That’s useful for creating portrait photos with blurred backgrounds or studio-lighting headshots with completely black backgrounds. It’s also useful for improving editing, and indeed Adobe Lightroom now can selectively edit portions of an HEIC photo depending on depth map data.
- It can store a package of photos taken at different focal lengths. Smartphones increasingly have two, three and even four main cameras, and LG Electronics has patented a 16-camera design. One shutter release could capture multiple photos in one HEIF file, and you could share the portions of the scene you liked.
- It can store raw photos — the unprocessed sensor data out of which JPEG or HEIC images are created as camera electronics lock in decisions about exposure, toning, white balance, noise reduction and other matters. Many advanced photographers prefer raw photos for their flexibility, but HEIF could let you store raw and processed images along with depth maps and other information in a single file.
- Further blurring the boundaries between still images and video, HEIF files can incorporate audio and can house the partly moving, partly still images called cinemagraphs. Indeed, at its Snapdragon event, Qualcomm has a demonstration of AI-created cinemagraphs stored as HEIF, Jacobowitz said.
The new features are a broad collection of what’s possible with modern smartphone cameras. Although HEIF provides a foundation, though, phone makers and camera apps for those advanced features. They don’t come automatically just because a phone supports HEIF.
HEIF? HEIC? Huh?
HEIF grew out of video compression technology called HEVC, or High Efficiency Video Coding. Video compression is somewhat different from photo compression, but they both shrink file sizes by throwing away data artfully so we human viewers notice as little image quality reduction as possible.
HEIF actually is a container for an image that can be compressed with different compression technologies. If you’re using HEVC’s technology, the photo format is called HEIC, which is why you’ll see iPhone photos with .heic as the filename extension.
Indeed, HEIF is flexible enough to also accommodate one HEIC rival, AVIF, which Google, Netflix and Mozilla are developing out of a video technology called AV1 from the Alliance for Open Media consortium. Another potential challenger is a modernized version of 25-year-old JPEG called JPEG XL.
Those competitors aren’t nearly as mature as HEIF and HEIC today. But they have a big potential advantage: they’ll likely be free of patent royalties. In comparison, HEVC and by extension HEIC are mired in patent licensing complications for hardware and software makers, and even if camera and phone makers agree to pay up, browser makers have a powerful aversion to patent barriers on the web.
“The patent situation makes it impossible for Mozilla to deploy HEIC in an open-source browser like Firefox,” said Nathan Egge, a Mozilla senior research engineer. “AVIF offers better compression than HEIC and is royalty-free. We believe AVIF has a much more promising future on the web.”
And the World Wide Web Consortium (W3C), a key web technology development group, opposes any web standards that are restricted by patent licensing requirements. So don’t count on an ability to view HEIC photos in a browser anytime soon.
Is JPEG doomed?
No, JPEG’s usefulness and universal support ensures it won’t disappear. But HEIF and HEIC, at least for now, looks like the strongest contender to provide a widely adopted alternative.
“We’ll continue to support JPEG. You’ll always be able to view it and have an option to capture JPEG,” Jacobowitz said.
But in his view, HEIF is on the way to being as pervasive as JPEG is today.
“To me it feels inevitable,” Jacobowitz said of HEIF’s rise. “JPEG can’t handle all the computer vision data that’s going to be essential to skyrocket image quality. Now that [HEIF] is in our hardware, it’s going to be easier for everybody to tap into it.”
CNET’s Holiday Gift Guide: The place to find the best tech gifts for 2018.
Cambridge Analytica: Everything you need to know about Facebook’s data mining scandal.