tgohblio/InstantID-MultiControlNet

InstantID with multiple controlnets and SDXL-Lightning

20

stars

67

commits

Python

primary language

May 5, 2024

updated

README

InstantID Cog Model

Overview

This repository contains the implementation of InstantID as a Cog model.

Using Cog allows any users with a GPU to run the model locally easily, without the hassle of downloading weights, installing libraries, or managing CUDA versions. Everything just works.

Development

To push your own fork of InstantID to Replicate, follow the Model Pushing Guide.

Basic Usage

To make predictions using the model, execute the following command from the root of this project:

Note:
default SDXL model: AlbedoBase XL V2
default scheduler: 4-step sdxl-lighting for fast inference

cog predict \
-i face_image_path=@examples/halle-berry.jpeg \
-i prompt="woman as elven princess, with blue sheen dress" \
-i negative_prompt="nsfw" \
-i adapter_strength_ratio=0.8 \
-i identitynet_strength_ratio=0.8 \
-i safety_checker=True

Input

Sample Input Image

Output

Sample Output Image

To change the denoising steps, use argument:

-i lightning_steps="2step" (or "8step")

To use a custom scheduler, pose controlnet and a different base SDXL model:

Example:
cog predict \
-i face_image_path=@examples/halle-berry.jpeg \
-i pose_image_path=@examples/poses/ballet-pose.jpg \
-i prompt="photo of a ballerina on stage" \
-i model="Juggernaut XL V8" \
-i adapter_strength_ratio=0.8 \
-i identitynet_strength_ratio=0.8 \
-i pose=True \
-i pose_strength=0.4 \
-i enable_fast_mode=False \
-i scheduler="DPMSolverMultistepScheduler-Karras" \
-i num_steps=30 \
-i guidance_scale=4 \
-i safety_checker=True

Input Parameters

The following table provides details about each input parameter for the predict function:

ParameterDescriptionDefault ValueRange
face_image_pathInput imageA path to the input image filePath string
pose_image_pathInput imageA path to the reference pose image filePath string
promptInput prompt"a person"String
negative_promptInput Negative Prompt"ugly, low quality, deformed face"String
modelSDXL image model choices"AlbedoBase XL V2"String
enable_fast_modeenable SDXL-Lightning LoRATrueBoolean
lightning_stepsselect SDXL-Lightning denoising steps"4step"String
schedulerscheduler algorithm choices"DPMSolverMultistepScheduler"String
adapter_strength_ratioScale for IP adapter0.80.0 - 1.0
identitynet_strength_ratioScale for ControlNet conditioning0.80.0 - 1.0
poseselect ControlNet pose modelFalseBoolean
pose_strengthScale for pose conditioning0.50.0 - 1.5
cannyselect ControlNet canny edge modelFalseBoolean
canny_strengthScale for canny edge conditioning0.50.0 - 1.5
depth_mapselect ControlNet depth modelFalseBoolean
depth_strengthScale for depth map conditioning0.50.0 - 1.5
num_stepsNumber of denoising steps251 - 50
guidance_scaleScale for classifier-free guidance71 - 10
seedRNG seed number0 (= random seed)0 - int MAX
safety_checkerEnable or disable NSFW filterTrueBoolean

Contributors

tgohblio

47 commits

reedemus

20 commits

tgohblio/InstantID-MultiControlNet

InstantID with multiple controlnets and SDXL-Lightning

20

stars

67

commits

Python

primary language

May 5, 2024

updated

README

InstantID Cog Model

Overview

This repository contains the implementation of InstantID as a Cog model.

Using Cog allows any users with a GPU to run the model locally easily, without the hassle of downloading weights, installing libraries, or managing CUDA versions. Everything just works.

Development

To push your own fork of InstantID to Replicate, follow the Model Pushing Guide.

Basic Usage

To make predictions using the model, execute the following command from the root of this project:

Note:
default SDXL model: AlbedoBase XL V2
default scheduler: 4-step sdxl-lighting for fast inference

cog predict \
-i face_image_path=@examples/halle-berry.jpeg \
-i prompt="woman as elven princess, with blue sheen dress" \
-i negative_prompt="nsfw" \
-i adapter_strength_ratio=0.8 \
-i identitynet_strength_ratio=0.8 \
-i safety_checker=True

Input

Sample Input Image

Output

Sample Output Image

To change the denoising steps, use argument:

-i lightning_steps="2step" (or "8step")

To use a custom scheduler, pose controlnet and a different base SDXL model:

Example:
cog predict \
-i face_image_path=@examples/halle-berry.jpeg \
-i pose_image_path=@examples/poses/ballet-pose.jpg \
-i prompt="photo of a ballerina on stage" \
-i model="Juggernaut XL V8" \
-i adapter_strength_ratio=0.8 \
-i identitynet_strength_ratio=0.8 \
-i pose=True \
-i pose_strength=0.4 \
-i enable_fast_mode=False \
-i scheduler="DPMSolverMultistepScheduler-Karras" \
-i num_steps=30 \
-i guidance_scale=4 \
-i safety_checker=True

Input Parameters

The following table provides details about each input parameter for the predict function:

ParameterDescriptionDefault ValueRange
face_image_pathInput imageA path to the input image filePath string
pose_image_pathInput imageA path to the reference pose image filePath string
promptInput prompt"a person"String
negative_promptInput Negative Prompt"ugly, low quality, deformed face"String
modelSDXL image model choices"AlbedoBase XL V2"String
enable_fast_modeenable SDXL-Lightning LoRATrueBoolean
lightning_stepsselect SDXL-Lightning denoising steps"4step"String
schedulerscheduler algorithm choices"DPMSolverMultistepScheduler"String
adapter_strength_ratioScale for IP adapter0.80.0 - 1.0
identitynet_strength_ratioScale for ControlNet conditioning0.80.0 - 1.0
poseselect ControlNet pose modelFalseBoolean
pose_strengthScale for pose conditioning0.50.0 - 1.5
cannyselect ControlNet canny edge modelFalseBoolean
canny_strengthScale for canny edge conditioning0.50.0 - 1.5
depth_mapselect ControlNet depth modelFalseBoolean
depth_strengthScale for depth map conditioning0.50.0 - 1.5
num_stepsNumber of denoising steps251 - 50
guidance_scaleScale for classifier-free guidance71 - 10
seedRNG seed number0 (= random seed)0 - int MAX
safety_checkerEnable or disable NSFW filterTrueBoolean

Contributors

tgohblio

47 commits

reedemus

20 commits

Languages

Python

99.9%