ToME + ControlNet is supported and how much is the speed ratio?

[bigmover]

Have you modified the implementation of controlnet to acheive ToME + ControlNet？

[JunnYu]

you can test speed use this code

# install develop paddlemix
pip install git+https://github.com/PaddlePaddle/PaddleMIX.git
# install paddle
pip install paddlepaddle-gpu==0.0.0.post117 -f https://www.paddlepaddle.org.cn/whl/linux/gpu/develop.html

import paddle
from ppdiffusers import ControlNetModel, StableDiffusionControlNetPipeline
from ppdiffusers.utils import load_image

controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny")
pipe = StableDiffusionControlNetPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5", safety_checker=None, controlnet=controlnet, paddle_dtype=paddle.float16
)

# Apply ToMe with a 50% merging ratio
pipe.unet.apply_tome(ratio=0.5) 
pipe.controlnet.apply_tome(ratio=0.5) 

generator = paddle.Generator().manual_seed(0)
prompt = "bird"
image = load_image(
    "https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/sd_controlnet/bird_canny.png"
)

image = pipe(prompt, image, generator=generator).images[0]

image.save("bird.png")