You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For the v1.1, we have not yet completed the multiple cards inference, and thus can only support one GPU. We are working on supporting sequence parallelism in inference.
For the v1.1, we have not yet completed the multiple cards inference, and thus can only support one GPU. We are working on supporting sequence parallelism in inference.
运行命令
CUDA_VISIBLE_DEVICES=0,1 python gradio/app.py
推理时报显存不够
但是显存显示
`
nvidia-smi
Tue May 7 10:29:12 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4090 Off | 00000000:3B:00.0 Off | Off |
| 44% 29C P8 18W / 450W | 24183MiB / 24564MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA GeForce RTX 4090 Off | 00000000:AF:00.0 Off | Off |
| 44% 28C P8 22W / 450W | 14MiB / 24564MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 3547 G /usr/lib/xorg/Xorg 4MiB |
| 0 N/A N/A 72296 C python 24166MiB |
| 1 N/A N/A 3547 G /usr/lib/xorg/Xorg 4MiB |
+-----------------------------------------------------------------------------------------+
`
有类似的情况吗
The text was updated successfully, but these errors were encountered: