DALL-E 3 with ChatGPT could spell the end of text prompts
In the early days of diffusion-based text-to-image AI art generators (we're talking like all of 12 months ago), DALL-E 2 was king. There was a moment when OpenAI's tool was the one people were clamouring to get access to, capable of producing more photo-realistic results than Stable Diffusion and Midjourney.
In just a few months, the story changed. Discord-based Midjourney, in particular, has made rapid advances in both the tools it offers and in the reliability and versatility of its results. DALL-E seemed to have stalled, losing its crown as the best AI art generator as OpenAI focused on its AI chatbot ChatGPT. But the company has just announced its latest update, DALL-E 3, and ChatGPT is a big part of it.
When text-based AI image generators began to emerge around a year ago, there were fears it had set the clock ticking on many jobs in the creative sector. At the same time, a new profession in making appeared. Since the results produced by models like DALL-E 2, Midjourney and Stable Diffusion could depend on subtle variations in how text prompts are written, we saw people calling themselves 'prompt engineers' or 'prompt hackers' offering to sell the wording for prompts that could generate specific results. But such 'jobs' may not last long.
OpenAI says that in DALL-E 3, you won't even have to write your own prompts any more. The upcoming version of the text-to-image generator incorporates integration with the company's AI chatbot ChatGPT. Those who want more control will still be able to use their own prompts, but those who are feeling lazy or don't know what language the model will understand can ask ChatGPT to do it for them.
ChatGPT is much better at recognising natural language. The demonstration provided by OpenAI (see the video above) suggests that the addition of the chatbot will make using DALL-E 3 more conversational, with the user able to use natural language to ask for what they want. It seems the integration will work both ways too, so it will be possible to generate an image and then askfor ChatGPT to write text about it, for example to write a story about a character that DALL-E 3 generated.
Of course, it remains to be seen how good ChatGPT will be at interpreting what the user wants, and I suspect that a certain level of detail will still be needed. DALL-E 3 still won't be able to mind read (see our guide to how to use DALL-E 2 for details on writing your own prompts).
OpenAI says the ChatGPT integration will allow more people to create AI art since they will no longer need to know how to phrase an effective prompt. Other changes promised for DALL-E 3 include a more accurate interpretation of context, resulting in a closer adherence to the prompt provided. The overall quality of images also appears to be much improved based on the few examples that OpenAI has provided comparing DALL-E 3 vs DALL-E 2.
It's also added more safety options resulting from work with external red teamers. These will include a block on using the names of public figures or living artists in prompts. OpenAI says it will also allow artists to opt out of future versions of the model by submitting their images to request their removal using an online form. That's at least a gesture towards addressing the questionable ethics of scraping online imagery without the owners consent but it sounds like it could be a laborious process for anyone who has a lot of images.
The first version of DALL-E was released in January 2021. DALL-E 2, which was made available to all in September last year, introduced a diffusion model that provided a major improvement in the quality of images generated.
OpenAI says that DALL-E 3 will see a staggered release. It will be released to ChatGPT Plus and ChatGPT Enterprise users in October and to research labs and its API service later in the autumn. It has not provided a date for when access will be opened to the public in general. To learn more about how to use the existing version of the model, see our roundup of the best AI art tutorials.
