Smartphone Photos Are Getting Faker. Uh-Oh?

Smartphone cameras became extremely powerful over the last five years. Their leap in quality was largely driven by advancements in computational photography, a technology that uses algorithms, artificial intelligence and sensors to produce sharp, lifelike pictures. Now we all can shoot stunning images that rival the work of professionals.

So what’s next? I hate to say it: faker photos.

Google, which has long been an industry leader in smartphone photography, will on Thursday start shipping the Pixel 8, a $700 handset with a suite of A.I.-powered photo-editing tools. The phone software does much more than adjust the sharpness and brightness of a photo — it uses A.I. to generate imagery or to remove elements to give you exactly the photo you want.

Imagine, for instance, a photo in which a person’s shoulder is cut off. With Google’s software, you can now tap the Magic Editor button and scoot that person over in the frame. From there, the software will use A.I. to produce the rest of that person’s shoulder.

Or consider a picture you shot of a friend in front of a historical monument, but the background is crowded with other tourists. Using the same editing tool, you can select the photo bombers and hit the Erase button. In seconds, the strangers will vanish — and Google’s software will automatically generate imagery to fill in the background.

Google has integrated these new A.I. editing tools into Google Photos, its free photo album app for Android devices and iPhones, which has more than one billion users. The company said the Pixel 8 was the first device with the A.I. editor, which means the same tools could soon arrive for other devices.

Google’s A.I. photo editor is part of a wave of generative A.I., which became popular in the last year after the release of the ChatGPT chatbot, which produces text in response to prompts. Image-based generative A.I. tools like DALL-E, Midjourney and Adobe Firefly also let people create pictures by simply typing in a prompt, such as “a cat sleeping on a windowsill.”

Yet the Pixel 8 is a turning point. It is the first mainstream phone to bake generative A.I. directly into the photo creation process at no extra cost, pushing smartphone photography into an era when people will increasingly have to question whether what they see in their images is real — including photos from loved ones.

(Apple’s iPhone camera can add some artificial effects, such as a “stage light” that brightens a subject and blacks out the background, but it stops short of generating fake imagery.)

“This is a really big moment that’s going to change a lot of things about imagery,” said Ren Ng, a computer science professor at the University of California, Berkeley, who teaches courses on computational photography. “As we go boldly forth into this future, a photo is no longer a visual fact.”

To test whether this is a good thing, I shot and edited dozens of photos with the Pixel 8. I was impressed, creeped out and skeptical that I would want to keep generating fake photos. Here’s what I found.

Picture Imperfect

Continuing my tradition when testing many smartphone cameras, I used the Pixel 8 to snap photos of my dogs — Max, a corgi, and Mochi, a brown Labrador — and then applied the A.I.

The results were hit and miss.

In one photo of Max sitting on a large rock, I wanted to remove a citation form from a police officer for letting my dogs run off leash without a permit in an off-leash dog park. (Who has ever heard of such a thing?) In the Google Photos app, I tapped the Magic Editor button and traced an outline around the piece of paper.

The software did a remarkable job. It replaced the maddening piece of bureaucracy with the rock slab and some pine needles.

In another photo, where Mochi is standing near Max and the right side of her butt is cut off in the frame, I tried scooting her to the left. The Pixel 8 did OK moving her, but the right side of Mochi’s computer-generated behind was blurry and her left paw was cut off.

Then came the most jarring result. In a photo of a pizza restaurant where Mochi’s face was cut off in the frame, I tried moving her over to test if the A.I. could generate the rest of her head. I didn’t expect the software to perfectly reproduce her grizzled mug, but the A.I. produced something nightmarish, a half-demi-god hellhound with a pair of hooves sprouting from her legs.

Google includes a Regenerate button for when you are unhappy with the results, which I tried. But it yielded equally off-putting results each time.

In the same photo, I tried highlighting and deleting the strangers in the background. This worked well but felt unsettling, like watching the “snap” scene in “Avengers: Infinity War,” when half the universe’s population disappeared.

It is early days, and Google expects people to run into imperfections. “This feature is in early stages and won’t always get it right,” the company said in a statement. “We’re looking for feedback to continually improve our models.”

To use or not to use

Here’s my feedback: I don’t think these A.I. editing tools should be featured so prominently in the photos app of a flagship smartphone, especially in their imperfect state.

And even when the technology matures, there are broader questions — such as the ethical issues of artificial images — to consider and navigate.

Editing photos for clarity and brightness improves an image without altering its substance. But artificially adding elements to a photo crosses a threshold, rendering an image a fake. Using these A.I. tools to produce and share photos could contribute to the spread of fake media online when misinformation is already rampant and it’s hard to know what to trust.

Dr. Ng, the computer science professor, said it was up to us to decide how to use generative photo technology responsibly, especially now that it has arrived on smartphones. He has set his own limits.

“Anything that touches authenticity to me, as a photographer, would be very problematic,” he said.

As for myself, I would use these A.I. photo tools to remove visual distractions, like the photo bomber ruining an otherwise great picture, from photos shared among family. But even then, I would use these tools sparingly, and I would not publish the fakery online.