Skip to content

Commit

Permalink
bindings/go: also look into stubs when CUBLAS is enabled
Browse files Browse the repository at this point in the history
Signed-off-by: mudler <mudler@localai.io>
  • Loading branch information
mudler committed Mar 18, 2024
1 parent 725350d commit af1b301
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 1 deletion.
2 changes: 1 addition & 1 deletion Makefile
Expand Up @@ -224,7 +224,7 @@ ifdef WHISPER_CUBLAS

CFLAGS += -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I$(CUDA_PATH)/targets/$(UNAME_M)-linux/include
CXXFLAGS += -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I$(CUDA_PATH)/targets/$(UNAME_M)-linux/include
LDFLAGS += -lcuda -lcublas -lculibos -lcudart -lcublasLt -lpthread -ldl -lrt -L/usr/local/cuda/lib64 -L/opt/cuda/lib64 -L$(CUDA_PATH)/targets/$(UNAME_M)-linux/lib -L/usr/lib/wsl/lib
LDFLAGS += -lcuda -lcublas -lculibos -lcudart -lcublasLt -lpthread -ldl -lrt -L/usr/local/cuda/lib64 -L/opt/cuda/lib64 -L$(CUDA_PATH)/targets/$(UNAME_M)-linux/lib -L$(CUDA_PATH)/targets/$(UNAME_M)-linux/lib -L/usr/lib/wsl/lib
WHISPER_OBJ += ggml-cuda.o
NVCC = nvcc
NVCCFLAGS = --forward-unknown-to-host-compiler -arch=$(CUDA_ARCH_FLAG)
Expand Down
14 changes: 14 additions & 0 deletions bindings/go/README.md
Expand Up @@ -94,6 +94,20 @@ Getting help:

* Follow the discussion for the go bindings [here](https://github.com/ggerganov/whisper.cpp/discussions/312)

## Notes

### CUDA

If you are using the binding with cuda and having linker issues, you may need to set the `CGO_LDFLAGS` environment variable to include the cuda libraries. For example:

```bash
CGO_LDFLAGS="-L/usr/local/cuda-11.7/targets/x86_64-linux/lib/stubs/ -L/usr/local/cuda/lib64/ -lcuda -lcublas -lcudart -lculibos -lcublasLt"
```

Note that the exact paths and libraries may vary depending on your system and CUDA installation, notice some symbols are present in the `stubs` directory and others in the `lib64` directory.

See also https://github.com/ggerganov/whisper.cpp/issues/1553 for more information.

## License

The license for the Go bindings is the same as the license for the rest of the whisper.cpp project, which is the MIT License. See the `LICENSE` file for more details.
Expand Down

0 comments on commit af1b301

Please sign in to comment.