New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Instructor with llmlite doenst support vision for Claude 3 Haiku and Sonnet models #617
Comments
It appears the issue you're encountering with the Claude 3 Haiku and Sonnet models in the Instructor repository for vision tasks is due to an assertion failure, suggesting that the "Instructor" object does not support multiple tool calls. The error recommends using "list[Model]" instead, indicating a problem with how the tool calls are structured in your code. To address this, ensure your implementation aligns with the repository's guidelines for handling vision tasks, particularly with models like Claude 3 Haiku and Sonnet. The repository's documentation and examples, such as If the documentation does not explicitly cover the Claude 3 Haiku and Sonnet models for vision tasks, this might indicate a gap in the current repository support or documentation. Considering there's an ongoing discussion in the repository's issues section regarding this matter, it would be beneficial to monitor that for any updates or solutions shared by the community or maintainers. For a more immediate resolution, providing the exact error message or exception details could help in diagnosing the issue more accurately. Additionally, sharing more about your implementation approach with the Claude 3 models could clarify the problem and facilitate targeted assistance.
|
Workaround for now is to use two Anthropic's instances, one is patched with instructor another without. |
Thank you. Is this something that can be solved easily? Or is an known issue that you persist? |
What Model are you using?
Describe the bug
Instructor with llmlite doesn't support vision for Claude 3 Haiku and Sonnet models
Information about the models is here:
https://docs.litellm.ai/docs/providers/anthropic
#claude-3-opus-20240229 - ok
#claude-3-haiku-20240307 - not ok
#claude-3-sonnet-20240229 - not ok
To Reproduce
Expected behavior
Should return a response, instead raises the exception on the screenshot
Screenshots
The text was updated successfully, but these errors were encountered: