swiglu_op _Stack_or_None RunTimeError

[yup3antarctica]

❓ Questions and Help

Hi

I tried to call xformers's Swiglufusedpacked op on my codes and get the following error during training:

By searching on web with keywords "UnbindBackward is a view and is being modified inplace", I found in general the inplace operation issue doesn't apply here.

Here's the wrapper module and how I called it during model forward:

Thank you so much!

Allen