The most dazzling magic trick in iOS 27, coming later this year, is a new photo-editing mode that’s like something out of Blade Runner. With a glimmer, a still shot becomes shiftable. You can turn the angle, and the picture becomes 3D. You can change the shot a bit, and the environment repaints around it.
As a VR/AR device wearer for years, and someone who spends a lot of time in Apple Vision Pro, I nodded when I saw the reveal at Apple’s WWDC 2026 developer event. I can see the threads. This feature is flexing 3D tools that Apple’s already been playing with for a while now, expanding them outward in fascinating ways. And like with that 3D-tilted spatially reframed shot, which I’m now trying on my own photos on iOS 27’s developer beta, I want to see what else is around the bend.
It’s about more than just editing photos, I think. A lot more.
A step towards scannable worlds
The toolset Apple’s using to make these 3D effects happen, called Gaussian splatting, is exactly the same AI model that makes my body become a 3D Persona avatar on Vision Pro headsets, and what makes Apple Maps now show remarkably clearer 3D maps. It’s a technology I’ve seen emerging for years now, that already turns 3D objects and environments (and even videos) into shockingly convincing walkthrough worlds.
Apps on Vision Pro already showcase Gaussian splats that cover entire city blocks captured and converted into incredible worlds. I presented a Polys Award for some of the best of them this winter, in fact. (You can explore the award-winning Pfarrkirche Kefermarkt splat, screenshot below, to see a beautiful example.) Meta’s Quest headsets can scan your space and create 3D walkable replicas using the tech, too, to stunning effect.
To me, it represents the very future of where photography and videos are headed, and the way we can represent our memories. But for Apple, that tech hasn’t emerged into its own camera app… yet.
Apple is slow-playing Gaussian splatting in its own tech, introducing it in thoughtful and various ways. This year at WWDC, Apple showcased it in Spatial Reframing on iOS, plus a new panorama photo conversion tool on the Vision Pro that turns previously shot panoramas into wraparound 3D environments you can then do your work in, and revamped 3D Maps views that haven’t made an immersive move into any Vision Pro mode, although Google’s 3D Maps were what wowed me the most on Samsung’s Galaxy XR headset, released last year.
Pieces forming around the edges
I talked to Apple’s Vision team last year about Personas on the Vision Pro, when Apple first discussed its use of Gaussian splatting as the secret sauce in making these impressive facial captures happen. Apple’s using the same Gaussian splatting models in the new iOS 27/VisionOS 27 features this time around, but to new effect.
Spatial reframing, to me, is very much like a more interactive version of the spatial scenes that photos could be transformed into last year, and seen in impressive head-tilting 3D on the Vision Pro (or by tilting your phone back and forth). It also has similar limits, in that you can’t actually go that deep into a photo, or truly turn everything around. It’s a gradual range of angles, created by AI generating what could be there behind someone a bit, or just out of range.
The freedom to re-pin a new angle and make that permanent, though, is new, and that level of control gives a bit more of that feeling I get when I step into full-3D Gaussian splat captures.
Of course, bigger scans need more photos to knit together. That could be Apple’s next trick, since developers are already doing it elsewhere.
I’d love a way for multiple photos of a place to be knit by Apple into a whole scene, or recreate a space I can walk in. The panorama conversion tool in VisionOS 27 now, which I haven’t tried yet, almost gets there. It wraps a photo around and makes it that surrounding experience, minus the walking.
I think it’s coming. But Apple needs immersive glasses or headsets that are actually affordable first, and Apple doesn’t even have any smart glasses expected until next year — those models might not even have displays onboard to start. But the $3,500 Vision Pro headsets could easily do full-room scans in ways that Apple could handle with its own apps. And iPhones could evolve its camera tools too.
Eventually, it’s going to be the future of what immersive video and photos truly mean. And right now, we’re getting little peeks of it all in plain sight.
Read the full article here