How Google’s Imagen AI Can Turn Any Text Into Stunning Images

Have you ever wondered what a dragon fruit wearing a karate belt in the snow would look like? Or how about a corgi dog riding a bike in Times Square? Well, now you can find out with Google’s new text-to-image AI, called Imagen.

Imagen is a powerful AI system that can generate realistic images and art from any text description. It uses a large language model, pretrained on text-only corpora, to encode the input text into embeddings. Then, it uses a diffusion model, trained on image data, to decode the embeddings into high-resolution images.

Imagen can produce images that are photorealistic or an artistic rendering, depending on the style specified in the text. It can also combine concepts, attributes, and styles that are not commonly seen together, such as a transparent sculpture of a duck made out of glass or a Rembrandt painting of a raccoon.

Imagen is not only a fun and creative tool, but also a breakthrough in AI research. It demonstrates an unprecedented degree of photorealism and a deep level of language understanding. It also shows that generic large language models, such as T5-XXL, are surprisingly effective at encoding text for image synthesis.

Google Research, Brain Team, the creators of Imagen, published their paper on May 24, 2023. They also launched a website where anyone can try Imagen for themselves. The website features some examples of Imagen’s capabilities, such as outpainting, inpainting, and variations.

Outpainting is the ability to extend an image beyond its original boundaries. For example, given an image of a house and a text prompt of “a lake behind the house”, Imagen can generate an image of the house with a lake in the background.

Inpainting is the ability to fill in missing parts of an image. For example, given an image of a face with sunglasses and a text prompt of “remove the sunglasses”, Imagen can generate an image of the face without sunglasses.

Variations is the ability to generate multiple images from the same text prompt. For example, given a text prompt of “a photo of a Persian cat wearing a cowboy hat and red shirt playing a guitar on a beach”, Imagen can generate different images of the cat with different poses, expressions, and backgrounds.

Imagen is not the first text-to-image AI system, but it is the most advanced one so far. Previous systems, such as DALL-E 2 by OpenAI and Latent Diffusion Models by Facebook AI Research, also used language models and diffusion models to generate images from text. However, Imagen outperforms them in terms of fidelity and alignment.

Fidelity is the measure of how realistic and detailed the generated images are. Alignment is the measure of how well the generated images match the text description. Google Research, Brain Team evaluated Imagen on the COCO dataset, which contains images and captions of everyday scenes. They found that Imagen achieved a new state-of-the-art FID score of 7.27 on the COCO dataset, without ever training on COCO. FID (Fréchet Inception Distance) is a metric that compares the feature distributions of real and generated images using an Inception network. A lower FID score means higher fidelity.

They also conducted human evaluations to compare Imagen with other methods. They asked human raters to rate the quality and alignment of samples from Imagen, DALL-E 2, Latent Diffusion Models, and VQ-GAN+CLIP on various text prompts. They found that human raters preferred Imagen over other methods in side-by-side comparisons.

To assess text-to-image models in greater depth, Google Research, Brain Team also introduced DrawBench, a comprehensive and challenging benchmark for text-to-image models. DrawBench consists of four datasets: COCO, CUB-200-2011, WikiArt, and Conceptual Captions. Each dataset covers different domains and difficulties of text-to-image generation.

COCO contains images and captions of everyday scenes. CUB-200-2011 contains images and descriptions of 200 bird species. WikiArt contains images and metadata of artworks from various artists and styles. Conceptual Captions contains web images and captions that are more abstract and diverse than COCO.

Google Research, Brain Team evaluated Imagen on DrawBench and compared it with other methods. They found that Imagen achieved the best performance on all four datasets in terms of FID score and human preference.

Imagen is an impressive achievement in AI research that opens up new possibilities for creative expression and visual communication. It also helps us understand how advanced AI systems see and understand our world, which is critical for creating AI that benefits humanity.

However, Imagen is not perfect and has some limitations and risks. For example, Imagen may generate images that are inaccurate, incomplete, or inappropriate. It may also generate images that are harmful to someone physically, emotionally, or financially. Therefore, Google Research, Brain Team has implemented some safety mitigations and content policies to prevent harmful generations and curb misuse.

They have also adopted a phased deployment approach based on learning from real-world use. They started by previewing Imagen to a limited number of trusted users. As they learned more about the technology’s capabilities and limitations, and gained confidence in their safety systems, they slowly added more users and made Imagen available in beta in July 2023.

Google Research, Brain Team hopes that Imagen will empower people to express themselves creatively and inspire new applications and innovations. They also invite feedback and suggestions from the community to improve Imagen and make it more useful and responsible.

If you are interested in trying Imagen for yourself, you can visit their website¹ and enter any text prompt you can imagine. You can also follow them on Instagram to see more examples of Imagen’s creations.

I hope you enjoyed reading this blog article. If you have any questions or comments, please let me know.

Visit Imagen