Lucidly | Generating Perspectives with Stable-Diffusion

Project Statement
One statement: Engineering a Stable-Diffusion model which takes in a text-prompt finds its coherent representations in its latent space and generating videos/GIFs on different machine representation (encoding).

[Results from last model] (Lucidly – Google Drive ).

My last iterations on this project aimed at generating multiple interpretations off a text prompt and engineering animations between them smoothly compiled as GIFs.
Findings was, human text is often mis-represented in machine’s encoding, therefore, methods like contrastive generation layer needs to be constructed to create a one-many mapping of [human_text] → machine_encodings [0 to n], a walk from 0 → n makes prompts come alive in perspectives. The second half is generating animations like KPMC2 by Katherine Crownson and Stability-AI crew.
After all the machine interrogation, the soundness of generated animations and graphics can only be validate by humans.
Therefore, Algovera like community stands a light where feedback can be crucial.

Research and code helper functions on how to generate coherent text prompts and meaningful visual stories with animation with Lucidly.

Amit Singh (metamyth)
R&D, Paperplane Technology
Researcher, Active Inference Lab

Discord: metamyth #8558
USDC Wallet Address: 0x0159af752e0220ed3eef439bef36f982cc0a6fbf

1 Like