Set-of-Masks for Multiple Motion Guidance

[HariWu1995]

Hi,

Thanks for a great repo. When I read your paper, I see that the model can generate multiple motions from multiple masks. In the demo code, I see that you combine all motions into 1 mask and 1 prompt.

Is it the official setting or can we input the model with multiple masks and prompts to have a clearer description? I mean something like Set-of-Mark Prompting.

[zhenghao977]

In our advanced iteration, we implement a multi-step approach for animation composition. Notably, it can feed an entire video as the reference condition, wherein this video can originate from the same reference frame yet animate distinct content elements. We will release this branch soon.