Skip to content
Merged
27 changes: 25 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ pipe = StableDiffusionPipeline.from_pretrained(
torch_dtype=torch.float16 if device != 'cpu' else None
).to(device)

# optional: reduce memory requirement with a speed trade off
pipe.enable_attention_slicing()

# pass pipeline to the explainer class
explainer = StableDiffusionPipelineExplainer(pipe)

Expand All @@ -58,8 +61,11 @@ output.image

![](assets/corgi_eiffel_tower.png)

You can also check all the images that the diffusion process generated at the end of each step.
![](assets/image_slider.gif)
You can also check all the images that the diffusion process generated at the end of each step:
```python
output.all_images_during_generation.show()
```
![](assets/image_slider_cropped.gif)
Comment thread
TomPham97 marked this conversation as resolved.

To analyse how a token in the input `prompt` influenced the generation, you can study the token attribution scores:
```python
Expand Down Expand Up @@ -126,6 +132,23 @@ The token attributions are now computed only for the area specified in the image
('background', 23.05)]
```

If you are having GPU memory problems, try reducing `n_last_diffusion_steps_to_consider_for_attributions`, `height`, `width` and/or `num_inference_steps`.
`
output = explainer(
prompt,
num_inference_steps=15,
generator=generator,
height=448,
width=448,
n_last_diffusion_steps_to_consider_for_attributions=5
)
`
You can completely deactivate token/pixel attributions computation by passing `n_last_diffusion_steps_to_consider_for_attributions=0`.

Gradient checkpointing also reduces GPU usage, but makes computations a bit slower:
`
explainer = StableDiffusionPipelineExplainer(pipe, gradient_checkpointing=True)
`
Check other functionalities and more implementation examples in [here](https://github.com/JoaoLages/diffusers-interpret/blob/main/notebooks/).

## Future Development
Expand Down
Binary file removed assets/image_slider.gif
Binary file not shown.
Binary file added assets/image_slider_cropped.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 21 additions & 0 deletions src/diffusers_interpret/generated_images.py
Original file line number Diff line number Diff line change
Expand Up @@ -144,3 +144,24 @@ def show(self, width: Union[str, int] = "100%", height: Union[str, int] = "400px
self.image_slider_iframe.width = width
self.image_slider_iframe.height = height
display.update(self.image_slider_iframe)

def gif(self, file_name: str = "diffusion_process.gif", duration: int = 400, show: bool = True) -> None:

if len(self.images) == 0:
raise Exception("`self.images` is an empty list, can't show any images")
if isinstance(self.images[0], list):
raise NotImplementedError("GeneratedImages.gif is not supported "
"when `self.images` is a list of lists of images")
'''
Generate and display a GIF from the denoising process
'''
self[0].save(file_name,
save_all = True,
append_images = self[1:],

optimize = False,
duration = duration,
loop = 0)
if show:
d.Image(file_name)