Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Huggingface demos use an older version (4.6.1) of transformers, which has a bug in the implementation of t5 #1655

Closed
sam-writer opened this issue Dec 1, 2021 · 3 comments · May be fixed by #1680
Closed

Huggingface demos use an older version (4.6.1) of transformers, which has a bug in the implementation of t5 #1655

sam-writer opened this issue Dec 1, 2021 · 3 comments · May be fixed by #1680
Labels
Network: Transformers Samples triaged Issue has been triaged by maintainers

Comments

@sam-writer
Copy link

The Huggingface/demos requirements file specifies transformers==4.6.1

It is my understanding from this PR in transformers that in 4.6.1 and before, T5's decoder used the encoder's attention mask when decoding and that the changes introduced in that PR (which show up in all versions of transformers>4.6.1) change the signature of T5 so that the decoder can have its own attention masking.

(Note: I didn't use the normal issue template because this isn't really a bug in TRT, but it seems worth being aware of.)
@ttyio
Copy link
Collaborator

ttyio commented Dec 14, 2021 •
edited

Thanks @sam-writer , I have create internal issue to track this.

@ttyio ttyio added Release: 8.x Samples triaged Issue has been triaged by maintainers labels Dec 14, 2021
@sam-writer sam-writer mentioned this issue Dec 22, 2021
Draft
sam-writer pushed a commit to sam-writer/TensorRT that referenced this issue Dec 22, 2021
WIP: add support for byt5 and mt5, closes NVIDIA#1655
d2fa90b 
Signed-off-by: sam-writer <sam.havens@writer.com>
@nvpohanh
Copy link
Collaborator

@ttyio Is this still an issue?

@nvpohanh
Copy link
Collaborator

nvpohanh commented Jul 1, 2022

Closing due to >14 days without activity. Please feel free to reopen if the issue still exists. Thanks

@nvpohanh nvpohanh closed this as completed Jul 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Network: Transformers Samples triaged Issue has been triaged by maintainers
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

WIP: add support for byt5 and mt5 sam-writer/TensorRT
3 participants
@ttyio @sam-writer @nvpohanh