Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: configurable model name, description and vision capability #2140

Merged
merged 21 commits into from
May 23, 2024

Conversation

cheahjs
Copy link
Contributor

@cheahjs cheahjs commented May 9, 2024

Pull Request Checklist

  • Target branch: Pull requests should target the dev branch.
  • Description: Briefly describe the changes in this pull request.
  • Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
  • Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
  • Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
  • Testing: Have you written and run sufficient tests for the changes?
  • Code Review: Have you self-reviewed your code and addressed any coding standard issues?

Description

Adds the ability to set a custom display name, description, and the vision capability of a model via the web UI, which is then persisted into the database.

image image image image image

A new field custom_info is injected into each model list API. An alternative implementation would be to override the model's name field, but would require additional mapping inside of the proxy endpoints.

If a model is marked as not vision capable,

  1. Image uploads are disabled.
  2. Any existing images that are in the current input are removed - but not any images that are already in the chat
  3. If multiple models are selected, and not all of them are vision capable:
    1. A warning icon is added to image input warning about this
    2. An error toast is shown if a user still sends the prompt.

I've tested locally with Ollama and OpenAI endpoints, have not tested LiteLLM endpoints yet.

Changelog

Added

  • Model Customization: Configure custom display names, descriptions, and vision capabilities for models.
  • Image Input Validation: Models marked as not vision-capable will reject or warn about image uploads.

@cheahjs cheahjs marked this pull request as draft May 9, 2024 12:51
@justinh-rahb
Copy link
Collaborator

rogan

LEGENDARY.

@tjbck
Copy link
Contributor

tjbck commented May 9, 2024

About time we make config.json persistent as well, I'll investigate this.

@tjbck
Copy link
Contributor

tjbck commented May 9, 2024

Also, it would be perfect if we could rename the fields like such:

{
  "id": "llama3:latest",
  "name": "Meta: Llama3 8B (Instruct, Q3, K, S)",
  "description": "Llama3 is a large language model trained on a diverse range of internet text.\nIt is capable of generating human-like text and answering a wide range of questions.",
  "vision_capable": false
},

@cheahjs
Copy link
Contributor Author

cheahjs commented May 9, 2024

Done the field rename, and also added settings UI for updating the model info.

@Yanyutin753
Copy link
Contributor

It is so unbelievable. I've been waiting for a long time! @cheahjs i admire you so much!!!🤩🤩🤩

@arsaboo
Copy link
Contributor

arsaboo commented May 9, 2024

This is epic. I am assuming we can extend this (in a separate PR) to configure per-model context window, output tokens, etc.

@cheahjs cheahjs marked this pull request as ready for review May 12, 2024 03:58
@tjbck
Copy link
Contributor

tjbck commented May 13, 2024

On second thought, we might want to save the model configs to our database to cover #665 as well. I'll keep this open in case I change my mind. I appreciate the effort as always 🙌

@jannikstdl jannikstdl mentioned this pull request May 19, 2024
@tjbck tjbck changed the base branch from dev to main May 22, 2024 23:11
@tjbck tjbck changed the base branch from main to dev May 22, 2024 23:11
@tjbck tjbck changed the base branch from dev to dev-models May 23, 2024 00:03
@tjbck tjbck merged commit f34fd3f into open-webui:dev-models May 23, 2024
3 of 5 checks passed
@cheahjs cheahjs deleted the feat/model-config branch May 27, 2024 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants