v1.2.0 #467
Replies: 3 comments 6 replies
-
|
Congrats on another awesome release! I'm super impressed by this project @ggerganov (and contributors). Two thoughts:
I am happy to provide more info to you about the podcast, if you're curious. Here works or you can email me jerod@changelog.com if you'd like to discuss privately. 💚 |
Beta Was this translation helpful? Give feedback.
-
|
Exciting! Our normal recording day is Wednesday. How about March 15th at 11am our time/7pm your time? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, this works for me! |
Beta Was this translation helpful? Give feedback.
-
|
Please post the link once live @jerodsanto ! |
Beta Was this translation helpful? Give feedback.
-
|
Will do @strangelearning! @ggerganov you should have a calendar invite in your inbox. Also check out our guest guide and holler back any questions you have. |
Beta Was this translation helpful? Give feedback.
-
|
Here's the link to the episode: The Changelog 532: Bringing Whisper and LLaMA to the masses – Listen on Changelog.com |
Beta Was this translation helpful? Give feedback.
-
|
Giving more visibility to this - there has been a bug for a while that can be triggered if using the language auto-detect feature: It's now fixed on latest |
Beta Was this translation helpful? Give feedback.
-
|
I've not had any issues with it so far, and absolutely loved playing around with the model. I am packaging it up to be available via tea, so the following commands will work for any user who has tea: $ whisper.main
$ whisper.stream
$ whisper.commandI'll let you know when it's available! |
Beta Was this translation helpful? Give feedback.
-
Overview
In this release we significantly reduce the memory usage during inference by introducing "scratch" buffers to
ggml.The new memory requirements per model are as follows:
It's a simple idea that instead of creating a new memory buffer for each new tensor in the computation, we reuse the memory of old tensors that are no longer needed. The implementation is in PR #431. It's not very clean - I think there is some better way to do this, but for now it will work.
Additionally, there might be some inference speed improvements on Apple Silicon in the Decoder part of the transformer. I haven't done proper benchmarks, but seems there is about ~30% performance boost. The results are identical to
v1.1.1.What's Changed
Core
ggml/whisperwhisper: PPC64 big-endian support by @fitzsim in PPC64 big-endian support #398whisper: condition sampled timestamp tokens to be monotonically increasing by @ggerganov in Condition sampled timestamp tokens to be monotonically increasing #425wasm: fix typo in helper.js by @bhbs in wasm : fix typo in helper.js #459ggml/whisper: reduce memory usage during inference by @ggerganov in Reduce memory usage during Whisper inference #431Bindings
ci: run workflows on pull requests + bindings depend on .h by @ggerganov in Run workflows on pull requests + bindings depend on .h #446go: added wrappers to reset and print timings by @glaslos in go: added wrappers to reset and print timings #436go: add WhisperLangAutoDetect method to go binding by @RobinXL in add WhisperLangAutoDetect method to go binding #451go: add wrapper for system info by @glaslos in go: add wrapper for system info #456go: support "auto" as an option when set language by @polarmoon in Go binding: support "auto" as an option when set language #462Examples
whisper.wasm: add labels for easier radio selection by @kokes in Add labels for easier radio selection #435livestream.sh: run main with model arg instead of default by @EricTendian in livestream.sh : run main with model arg instead of default #453main: CSV format export trimmed spaces fix by @alex-bacart in CSV format export trimmed spaces fix #444addon.node: using whisper as a Node.js addon by @chenqianhe in addon: implement node addon call whisper through cpp #443New Contributors
Full Changelog: v1.1.1...v1.2.0
Highlights
I'll use these release notes to write some random thoughts about the project - sort of a short blog post.
I'm really happy with how
whisper.cppturned out to be so far. There is a very positive reception in the ML community - most people seem to be excited by the simplicity of the implementation and the fact that it is quite self-contained. I receive a lot of questions about the project and about various ideas that it can be applied to. I really enjoy it and I try to respond to everyone!I also find it very satisfying that there are so many contributions already happening by so many people. To me this illustrates the power of open-source collaboration. The contributions not only improve the functionality and the quality of the code, but also help to generate various new ideas and approaches to explore.
Another interesting thing is that the project keeps on giving. Every time I start to think that now is a good time to put it in the background for a while and focus on other stuff, some new cool idea pops up and I can't help but start working on it. Having this custom implementation allows me to interact with the model on a lower level which opens some interesting ways to explore it.
So far the development has been focused on improving the performance, expanding the platform coverage and having robust decoding strategies with a variety of examples. During this time, there have been several ideas that accumulated over-time which I find interesting to explore (diarization, token-level timestamps, improved timestamp accuracy, etc). I think I'll try to focus more on these in the future and see if I can achieve something interesting.
Windows port of
whisper.cpputilising vendor-agnostic GPGPU based on DirectCompute by @Const-mehttps://github.com/Const-me/Whisper
whisper.cppWhispers of A.I.’s Modular Future
This discussion was created from the release v1.2.0.
Beta Was this translation helpful? Give feedback.
All reactions