"Firefly 3.0 is changing the art of photography… and it shouldn't scare you" says Adobe's Zeke Koch
Generative AI has quickly emerged as one of the most transformative and controversial technologies of our era. For photographers and photo editors, the implications are profound. Will it just make our jobs easier? Or will it replace them altogether?
At the center of this unfolding revolution / disruption / total calamity (delete as appropriate) is Adobe's Firefly, a groundbreaking generative AI model now baked directly into Creative Cloud apps such as Photoshop.
You might have ignored Firefly up until now, assuming that it's a temporary phenomenon—a mere blip or novelty that will soon be forgotten about. But if that's the case, then Zeke Koch, VP of product management at Firefly, is here to tell you (warmly and diplomatically) that you're very wrong. As far as Adobe's concerned, Firefly is the future, not just of Photoshop and Lightroom but all of its creative software.
To give us more details, Zeke sat down with us to discuss how Firefly is transforming the creative process and why photographers shouldn't fear this disruptive new tech. Even though all the images that accompany this article were generated entirely by AI, with no actual photographer needed.
What's new in Firefly 3.0
Koch, a product veteran who has worked on Adobe Stock and Photoshop, explains that the AI model has been trained on a vast dataset of images and concepts, allowing it to understand and generate a wide variety of visual elements.
In plain language, that means the software has trawled millions of photos in the Adobe Stock library, and learned how to automatically craft new images that will look similarly "real" and professional.
"We're trying to teach the model to understand reality, so it can really go to photographic reality," he explains. "We're trying to teach it to understand how a camera would understand the world. And then to better give you the kind of image you're trying to generate, we've introduced the Reference Image feature. That means you don't have to rely on words in your prompts, you can upload an image to show the AI the kind of thing you're looking for."
If you've tried Firefly in the last year and found it unimpressive, it worth noting that this week, Firefly has been updated from model 2.0 to model 3.0. And it's quite the game changer.
This update delivers, in the words of the release notice: "stunning advancements in photorealistic quality, styling capabilities detail and accuracy along with greater variety [and] significant advancements in speed of generation [that] make the ideation and creation process more productive and efficient".
These are not just flowery words. Koch explains that there are real, verifiable software enhancements behind all the hyperbolic adjectives.
"There are a couple of different measures of the complexity that the model understands objectively," he begins. "And so one is what we call the latent space. It's effectively how many concepts you have. Those are more tied to the words in the prompt than they are to the picture. But essentially if you type something like 'bear with guitar' how many different variants are there of the bear guitar and what's in the background and all those different things.
"So that represents all the concepts that you can explain in the network. And so we've made the latent space twice as big as it was before. And then the other thing is the imagery part: the part that tries to render what we think you want. That's about three times as big as it used to be. And then the third thing is called attention. It's basically, when we take your prompt and try to convert it into an image, how much does the model look at all the inputs that you give it? How much attention does it paid all those things? So we've also about tripled that."
What this means in practice is that Firefly delivers higher-quality image generations, with better details and variety. Koch showed me a number of pictures produced by both 2.0 and 3.0 versions of Firefly. And, once he zoomed into the details, it was clear that the clarify of images produced by the latest version is palpably higher, as well as being more representative of real photography.
Officially titled Firefly Image 3 Foundation Model (beta), the new model is available for use in Photoshop (beta) and on firefly.adobe.com.
How it can help photographers
The short version is that this is impressive tech indeed, with its powerful AI capabilities, not to mention increasingly seamless integration with Adobe Creative Cloud. But what's it all actually for? Kock is clear on this one: Adobe believes Firefly is poised to become an essential tool for photographers, image editors and indeed creatives of all stripes.
"Generative AI doesn't replace creativity: it turbocharges and extends the creator's abilities," he stresses. "The creative spark still comes from you as the photographer. The AI amplifies your efforts so you can create, ideate and iterate much faster."
In other words, Koch sees Firefly as a "supercharged editorial assistant" handling tedious tasks like background removal and object insertion, while the human being continues to drive the overarching artistic vision.
"We're not just doing text to image generation from scratch," he points out. "That's been on the Firefly web app from the beginning. But the magic in Photoshop is using generative AI to augment and accelerate your existing creative process on real photos you care about."
Specifically, he points to new tools like Generative Fill which allows replacing backgrounds, adding objects, or tweaking visual elements in one click instead of tedious editing. "We're trying to make the model understand photographic reality as closely as possible, so it can do the rote editing tasks for you, while you focus on your creative vision."
By providing a starting point and a range of options, Firefly can help photographers explore new ideas and push the boundaries of their craft. Ultimately, Koch believes that Firefly and other generative AI tools will become an integral part of the creative workflow, just as digital cameras and editing software have transformed photography over the past few decades.
The ethics of it all
Whether or not you buy the argument that AI won't replace our jobs, there's also the separate issue of how AI is learning from our existing work by studying it online. To many people's minds, that's simply another form of piracy.
On this issue, Koch is keen to stress the ethical nature of Firefly, based on the fact that it doesn't scrape content people haven't agreed to be scraped. "We're creating models with deep expertise in areas like imagery, video and design," Koch explains. "And we're building these commercially-safe models only using data we have proper rights and licenses for, like Adobe Stock."
As an added bonus, Adobe embeds content credentials for transparency on whether something was computer-generated. "As this technology proliferates, understanding where content comes from and if it was AI-generated is crucial for maintaining trust," he adds.
The future of Firefly
A more functional aspect of Firefly that sets it apart is its integration with other Adobe Creative Cloud apps beyond Photoshop. "There's text to vector in Adobe Illustrator. There's text effects in Express, there's generative images in Adobe Express," Koch explains. "There's a little bit in Adobe Stock. So there are a bunch of different systems that can work together seamlessly."
Lightroom is noticeably absent on the list for now, but as Terry White teased on stage at Adobe Max London it's not been forgotten: generative AI capabilities will be coming to Lightroom on desktop and mobile soon.
In the future, such integrations will allow for a seamless workflow, where photographers can use Firefly to generate initial concepts or fill in gaps, and then refine and polish them using their favorite Adobe tools. "Having AI in Photoshop allows you to quickly remove the things in a picture that don't meet your needs," Koch says. "Or you can start with a blank canvas, get something started, for your inspiration."
