Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LinearInt8 layer for inference of int8-quantized LLMs and Arm intrinsics #5007

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Open

LinearInt8 layer for inference of int8-quantized LLMs and Arm intrinsics #5007

wants to merge 14 commits into from

Conversation

lrw04
Copy link
Contributor

@lrw04 lrw04 commented Sep 5, 2023 •
edited

Please provide feedback on how the patch could be improved.

@codecov-commenter
Copy link

codecov-commenter commented Sep 5, 2023 •
edited

Codecov Report

Merging #5007 (68b4b2d) into master (bcfec1d) will decrease coverage by 5.20%.
The diff coverage is 89.47%.

@@             Coverage Diff             @@
##           master    #5007       +/-   ##
===========================================
- Coverage   94.72%   89.52%    -5.20%     
===========================================
  Files         772      303      -469     
  Lines      228777    89191   -139586     
===========================================
- Hits       216705    79848   -136857     
+ Misses      12072     9343     -2729
Files Changed Coverage Δ
src/layer/linearint8.cpp 89.47% <89.47%> (ø)

... and 650 files with indirect coverage changes

@lrw04
Copy link
Contributor Author

lrw04 commented Sep 7, 2023

oh no, it didn't work

@lrw04
Copy link
Contributor Author

lrw04 commented Sep 7, 2023

Closing following @nihui's instructions.

@lrw04 lrw04 closed this Sep 7, 2023
@lrw04 lrw04 reopened this Sep 7, 2023
@lrw04 lrw04 force-pushed the patch-linearint8 branch 2 times, most recently from ae6ab89 to 307e635 Compare September 9, 2023 02:34
@lrw04 lrw04 changed the title Add LinearInt8 layer for inference of int8-quantized LLMs LinearInt8 layer for inference of int8-quantized LLMs and Arm intrinsics Sep 16, 2023
@nihui nihui closed this Oct 11, 2023
@nihui nihui reopened this Oct 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
arm core doc layer test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@lrw04 @codecov-commenter @nihui