Layoutlmv3 LayoutLMv3ForSequenceClassification NaN loss

[Rithsek99]

Hello,

I'm trainining Layoutlmv3 LayoutLMv3ForSequenceClassification and facing issue where loss goes to NaN on very first iteration. I've tried to increase batch size and gradient accumulation with lower lrate but it's still goes to NaN. I'm training with pytorch lightning.

My source code is pretty much this https://www.mlexpert.io/machine-learning/tutorials/document-classification-with-layoutlmv3 (layoutlmv3-large) with little tweak.

@HYPJUDY

thanks