Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue
N/A
Change
In order to ensure that Ollama model is ready to go at startup, I have introduced an optional preload boolean keyword argument (default set to False) to the Ollama LLM, allowing the model to be preloaded into memory during the initialization phase. This feature includes a preload_model method to trigger preloading at a later time if desired. The preloading process leverages the Ollama FAQ's recommended approach of calling generate with an empty prompt (see how-can-i-pre-load-a-model-to-get-faster-response-times).
Additionally, the keep_alive parameter is now usable to be passed and tell for how long the model stays in memory.
General checklist