You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current ghcr.io/bentoml/openllm:latest image (sha256:1860863091163a8e8cb1225c99d6e1b0735c11871e14e8d8424a22a5ad6742fa) shows an error:
ValueError: The checkpoint you are trying to load has a model type of `cohere`, which Transformers does not recognize. This may be due to a problem with the checkpoint or an outdated version of Transformers.
when doing this:
docker run --rm --gpus all -p 3000:3000 -it ghcr.io/bentoml/openllm start CohereForAI/c4ai-command-r-v01 --backend vllm
also when installing openllm[vllm] it brings 0.2.7 version of vLLM
I think this should be the same prompting system, there is also CohereForAI/c4ai-command-r-plus available and it would be nice to be able to run it too.
Feature request
Would be nice to have ability to run Command-R (
CohereForAI/c4ai-command-r-v01
) using OpenLLMMotivation
No response
Other
vLLM backend already supports Command-R in v0.4.0: vllm-project/vllm#3330 (comment)
The text was updated successfully, but these errors were encountered: