Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Embeddings model from website model explorer can't be instantiated in Python bindings #2289

Open
AtR1an opened this issue Apr 30, 2024 · 0 comments
Labels
bindings gpt4all-binding issues bug-unconfirmed

Comments

@AtR1an
Copy link

AtR1an commented Apr 30, 2024

Bug Report

Instantiating the embeddings model all-MiniLM-L6-v2-f16.gguf (https://gpt4all.io/models/gguf/all-MiniLM-L6-v2-f16.gguf) from the model explorer on https://gpt4all.io/ fails with Unable to instantiate model: Unsupported model architecture: bert in the 2.6.0 version of the Python bindings.

Example Code

import gpt4all
gpt4all.Embed4All(model_name="all-MiniLM-L6-v2-f16.gguf", model_path=<download directory>)

This results in

  File "<stdin>", line 1, in <module>
  File "...\Lib\site-packages\gpt4all\gpt4all.py", line 58, in __init__
    self.gpt4all = GPT4All(model_name, n_threads=n_threads, device=device, **kwargs)
  File "...\Lib\site-packages\gpt4all\gpt4all.py", line 206, in __init__
    self.model = LLModel(self.config["path"], n_ctx, ngl)
  File "...\Lib\site-packages\gpt4all\_pyllmodel.py", line 218, in __init__
    raise RuntimeError(f"Unable to instantiate model: {'null' if s is None else s.decode()}")
RuntimeError: Unable to instantiate model: Unsupported model architecture: bert

Steps to Reproduce

  1. Install gpt4all version 2.6.0 from pip: pip install gpt4all==2.6.0
  2. Execute example code

Expected Behavior

The model available via the model explorer on the website also works with the Python bindings.

Your Environment

  • Bindings version (e.g. "Version" from pip show gpt4all): Python 2.6.0
  • Operating System: Windows (also reproduced on Ubuntu)
  • Chat model used (if applicable): all-MiniLM-L6-v2-f16.gguf
@AtR1an AtR1an added bindings gpt4all-binding issues bug-unconfirmed labels Apr 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bindings gpt4all-binding issues bug-unconfirmed
Projects
None yet
Development

No branches or pull requests

1 participant