-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OpenVINO A770 segfaults after some number of tokens #2219
Comments
It appears that your model loading process encountered an issue while trying to run on the CPU. The error message indicates a segmentation fault, which is a type of error that typically results from an attempt to access memory that doesn't belong to the process. Possible reasons for this error could be hardware issues, conflicts between different processes for the CPU's resources, or incompatibilities between the software and the CPU architecture. To troubleshoot the issue, you might want to try the following steps:
If you need further assistance, provide more details about your system' |
Hi @richiejp thanks again for your time. This seems an upstream (openvino) error. All the above just to say that sadly I have no clue because is an error related to the underlying operating system plugin that is transparent to LocalAI. May I ask you to test directly openvino inference outside of LocalAI? Another check is to be sure to have the latest ARC driver from Intel |
OK, I'll see if I can get it running outside LocalAI. As for drivers, I had some issues installing the out-of-tree driver, so that could take a while otherwise I have to wait for a kernel update from Ubuntu/Dell. |
I'd like to understand if it's LocalAI specific or not, so that in case we can open an issue upstream. |
Yup, it also core dumped: |
Thank you, it's segfaulting at the same instruction address I'm sorry for not being of more help 😞 but I don't have the resources to investigate more. |
C.C @fakezeta
LocalAI version:
quay.io/go-skynet/local-ai@sha256:4e4e427433285b056f32bfaa313ec0e75aeacb5b5c8c273953f9d2242fb55a60
This is still the version without the AUTO GPU changes. I'll try updating when I get chance.
Environment, CPU architecture, OS, and Version:
Same as #2208, but using just the Arc dGPU
Describe the bug
libopenvino_intel_gpu_plugin.so segfaults during inference. It seems to be when the number of tokens produced is above some amount because it tends to fail in the same place, but sometimes it succeeds as well. I don't know how many tokens are being produced or if it is related to the context size.
To Reproduce
Ask it to summarize the output of for e.g.
lscpu
or explain 50 lines of a Makefile.Expected behavior
Not to segfault.
Logs
From the kernel log
LocalAI log after a previous crash, hence why it is restarting the process:
Additional context
Similar requests succeed on iGPU.
The text was updated successfully, but these errors were encountered: