Proposal: Diffusion Python Library

Proposals Round 6
#1

The currently available python code for stable diffusion and its various samplers can be overwhelming. To facilitate faster integration for developers, a straight forward and simple to use api (python library) is needed.

Additionally, there are features that the native stable diffusion implementation does not have, but are great to have. We propose a library that both simplifies and adds valuable extensions. Such as masking and custom samplers.

pip install diffusionkit

then

@dataclass
class DiffuseParams:
	prompt: str
	width: int = 512
	height: int = 512
	ddim_steps: int = 50
	sampler_name: str = 'lms'
	cfg_scale: float = 5.0
	denoising_strength: float = 0.75
	seed: int = 0
	count: int = 1

diffuse(params: DiffuseParams, image: Image = None, mask: Image = None)
enhance(...)
...

Your funding will allow us to expand on this, implement more features, and write solid documentation.

1 Like
#2

This can be useful! Got a few questions.

  1. Is this based in HF’s diffuser library
  2. What are some other features you guys have in the plan
#3

This is a reimplementation based on a pull of the compvis repository while it was stil MIT licensed, rather than openrail licensed.

Right now I am trying to get this photoshop plugin shipped, but I have considered processing videos / images batches, and returning a video files / image batches, and was intending to use it to test “remastering” / reimaging a trailer of the 80’s anime “legend of the galactic heroes”

#4

  1. It’s currently based on CompVis’ implementation of Stable Diffusion, and HF’s implementation of Clip, and crowsonkb’s sampler suite. These can become swappable components further down the road, however.

  2. Masking, image enhancement (RealESR, GFP, etc), cross attention