LLM-model-switcher #32
Conversation
| @@ -83,6 +83,8 @@ def __init__( | |||
| max_new_tokens=300, | |||
| ) | |||
| self.llm = OpenAI(model="gpt-3.5-turbo-instruct") | |||
There was a problem hiding this comment.
Lets have llama3 8B as default model
There was a problem hiding this comment.
And then for line 80: multimodal model default should be set to Llava or even llava-gradient using ollama with option to change to gpt-4-preview
| available_models = tuple(model["name"] for model in models_info["models"]) | ||
|
|
||
| if "llama3:8b" in available_models: | ||
| selected_model = st.selectbox( |
There was a problem hiding this comment.
Does this link back to self.llm?
There was a problem hiding this comment.
available models is grabbing from ollama installed models on current device
so no, not yet
Good stuff here, let me give it a go |
|
All proposed functionalities are included in PR #36 and in release v0.1.2 |
Got the UI and the function + ollama to switch models
Stuck on writing the actual function:
def setLLM(self, llm): print("Setting LLM...", llm)But will try again tomorrow
Sending PR if anyone knows how to go about this