Google’s AI ‘Reimagine’ tool enabled us to incorporate wrecks, disasters, and corpses into our photos

It turns out that a rabbit sporting an AI-generated top hat was just the beginning.

This year, Google has joined the ranks of phone companies introducing AI photo editing tools, following Samsung’s somewhat unsettling yet intriguing sketch-to-image feature and Apple’s more restrained Image Playground, set to debut this fall. The Pixel 9’s new offering, called “Reimagine,” represents a significant leap from last year’s Magic Editor tools. While Magic Editor allowed users to select and remove elements or change the sky to a sunset, Reimagine takes things much further—breaking new ground in photo manipulation.

Reimagine enables users to select any nonhuman object or part of a scene and then generate new content by typing in a text prompt. The results can be remarkably realistic, with lighting, shadows, and perspective often blending seamlessly with the original photo. This tool allows for creative additions, such as wildflowers or rainbows, but it also raises more unsettling possibilities.

In testing Reimagine with a few colleagues using their Pixel 9 and 9 Pro review units, we explored its limits and discovered that it can produce some disturbing results. By carefully crafting our prompts, we managed to generate convincingly eerie images, such as a body covered with a blood-stained sheet. The tool’s ability to create such realistic and potentially unsettling images raises important questions about the ethical use of AI in photo editing.

During our week of testing, we managed to add car wrecks, smoking bombs in public places, sheets that seemed to cover bloody corpses, and drug paraphernalia to images. This raises serious concerns, especially since this functionality is integrated into a consumer phone that anyone, including my dad, could easily purchase from Verizon.

In response to our findings, Google spokesperson Alex Moriconi provided the following statement:

"Pixel Studio and Magic Editor are designed to inspire creativity with text-to-image generation and advanced photo editing on Pixel 9 devices. Our Generative AI tools aim to respect the user's intent, which means they might create content that some users could find offensive if prompted. However, we have established clear policies and Terms of Service regarding the types of content allowed. We implement safeguards to prevent misuse and are dedicated to continuously improving these protections. While some prompts may challenge these safeguards, we are committed to refining them to prevent abuse."

Admittedly, our attempts to bypass the filters were a clear breach of these policies, akin to misusing self-checkout systems to mislabel products. Moreover, individuals with malicious intent are unlikely to be deterred by Google's terms and conditions. The most troubling aspect is the gap between our ability to create problematic content and the tools available to detect it. As technology advances, our methods for identifying and managing such content have not kept pace.

When using Reimagine to edit images, there’s no obvious indication that the content is AI-generated—there's only a tag in the metadata. While this tagging system, known as SynthID, is robust for fully synthetic images created with Pixel Studio, it’s not applied to images edited with Magic Editor. This means that images edited with Magic Editor could easily be stripped of their metadata through a simple screenshot.

Historically, tampering with images has been possible, but never has it been this straightforward to create realistic alterations. Previously, adding a convincing car crash to a photo required expertise, understanding Photoshop layers, and access to expensive software. Now, with a few text prompts and a Pixel phone, anyone can achieve similar results in moments.

The ease with which these manipulated images can be created and shared is unprecedented. We tested this by uploading a “Reimagined” image to an Instagram story, which Meta did not automatically flag as AI-generated. Given that users can now produce and disseminate misleading photos effortlessly, it’s essential to approach online images with heightened skepticism, especially if they seem too good (or too bad) to be true.

While it’s hoped that users will apply Google’s AI tools responsibly, adding creative touches like wildflowers and rainbows, the reality is that such powerful technology can also be used to deceive. Hence, being cautious about the authenticity of photos you encounter online is more important than ever.