{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Improve image quality with deterministic generation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A common way to improve the quality of generated images is with *deterministic batch generation*, generate a batch of images and select one image to improve with a more detailed prompt in a second round of inference. The key is to pass a list of [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html#generator)'s to the pipeline for batched image generation, and tie each `Generator` to a seed so you can reuse it for an image.\n", "\n", "Let's use [`runwayml/stable-diffusion-v1-5`](https://huggingface.co/docs/diffusers/main/en/using-diffusers/runwayml/stable-diffusion-v1-5) for example, and generate several versions of the following prompt:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "prompt = \"Labrador in the style of Vermeer\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Instantiate a pipeline with [DiffusionPipeline.from_pretrained()](https://huggingface.co/docs/diffusers/main/en/api/pipelines/overview#diffusers.DiffusionPipeline.from_pretrained) and place it on a GPU (if available):" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from diffusers import DiffusionPipeline\n", "\n", "pipe = DiffusionPipeline.from_pretrained(\"runwayml/stable-diffusion-v1-5\", torch_dtype=torch.float16)\n", "pipe = pipe.to(\"cuda\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, define four different `Generator`'s and assign each `Generator` a seed (`0` to `3`) so you can reuse a `Generator` later for a specific image:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import torch\n", "\n", "generator = [torch.Generator(device=\"cuda\").manual_seed(i) for i in range(4)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Generate the images and have a look:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "images = pipe(prompt, generator=generator, num_images_per_prompt=4).images\n", "images" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![img](https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/reusabe_seeds.jpg)\n", "\n", "In this example, you'll improve upon the first image - but in reality, you can use any image you want (even the image with double sets of eyes!). The first image used the `Generator` with seed `0`, so you'll reuse that `Generator` for the second round of inference. To improve the quality of the image, add some additional text to the prompt:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "prompt = [prompt + t for t in [\", highly realistic\", \", artsy\", \", trending\", \", colorful\"]]\n", "generator = [torch.Generator(device=\"cuda\").manual_seed(0) for i in range(4)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create four generators with seed `0`, and generate another batch of images, all of which should look like the first image from the previous round!" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "images = pipe(prompt, generator=generator).images\n", "images" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![img](https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/reusabe_seeds_2.jpg)" ] } ], "metadata": {}, "nbformat": 4, "nbformat_minor": 4 }