Replies: 5 comments 3 replies
-
Yes. LiteLLM: models |
Beta Was this translation helpful? Give feedback.
-
This is the configuration that I used to set up open-webui with llama3-70b running on Groq's servers. I've added three files: config.yamlmodel_list:
- model_name: 'llama3-70b'
litellm_params:
model: 'groq/llama3-70b-8192'
api_key: 'os.environ/GROQ_API_KEY' docker-compose.yamlversion: '3.7'
services:
services:
open-webui:
container_name: 'open-webui'
image: 'ghcr.io/open-webui/open-webui:main'
volumes:
- './open-webui:/app/backend/data:rw'
- './config.yaml:/app/backend/data/litellm/config.yaml:ro'
ports:
- '3003:8080'
environment:
DEFAULT_MODELS: 'llama3-70b'
env_file:
- 'GROQ_API_KEY.env'
extra_hosts:
- 'host.docker.internal:host-gateway'
restart: 'unless-stopped' GROQ_API_KEY.envGROQ_API_KEY=<YOUR API KEY> After running this with $ docker compose up I had a working web UI with llama3 running at http://127.0.0.1:3003. If you're saving this in Git, then don't forget to add environment:
ENABLE_SIGNUP: 'False' if you put this configuration on a public server. |
Beta Was this translation helpful? Give feedback.
-
have a look in this video the issue is easily explained in this . https://youtu.be/qOxsIlpIMt4 |
Beta Was this translation helpful? Give feedback.
-
Adding the Groq API
You will see all LLMs available on Groq |
Beta Was this translation helpful? Give feedback.
-
did you watch this video ? https://youtu.be/qOxsIlpIMt4
…On Fri, 24 May 2024 at 15:31, el_even ***@***.***> wrote:
So, I'm following this tutorial, in the video, but open webui does not
save my changes. Then, when I refresh the page, its blank (I know for a
fact that the default OPEN AI URL is removed and as the groq url and api
key are not changed, the OPEN AI URL is void). Any idea why (open webui is
not saving my changes) ? I have also tried to set the OPEN AI URL directly
in the docker env variables but I get the same result (blank page).
—
Reply to this email directly, view it on GitHub
<#974 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/BAQEM5CUNVJPKSVQ4XO6S2TZD4FYPAVCNFSM6AAAAABEBBR6K2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TKNBVGM3DG>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
@tjbck @justinh-rahb have y'all seen this?
Is there a way to integrate Groq API into OWUI?
https://wow.groq.com/
Beta Was this translation helpful? Give feedback.
All reactions