Instructions to use hanlincs/Ctrl-Adapter with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use hanlincs/Ctrl-Adapter with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("hanlincs/Ctrl-Adapter", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
how does it use in comfyui?
how does it use in comfyui?
Hi, thanks for your interest in our work!
To be honest, I'm not very familiar with comfyUI. I'm a little bit tight for time recently, and it might take me some time to figure it out. May I ask do you know who/where I could get some guidance/help to set this up?
Thanks!
I'm not an expert, but here you can find an example of how to create a custom node in comfyui: https://github.com/comfyanonymous/ComfyUI/blob/master/custom_nodes/example_node.py.example
There are also some videos on youtube.
You can also look into some similar project, like this: https://github.com/cubiq/ComfyUI_IPAdapter_plus.
thanks for your pointer!
I'll work on it later!
Would be great! ;-)
The thing that I personally found most difficult to understand is the system that comfyui uses to apply patches to the model. Here is an example, if you need it.
https://github.com/cubiq/ComfyUI_IPAdapter_plus/blob/d76bdbd2b59cb4164adb662359dc59b5aa213548/IPAdapterPlus.py#L144
wow, indeed looks pretty complicated lol