Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Model/Framework] error in run the script run_inference_on_triton.py #1388

Open
sepzjh opened this issue May 6, 2024 · 0 comments
Open

[Model/Framework] error in run the script run_inference_on_triton.py #1388

sepzjh opened this issue May 6, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@sepzjh
Copy link

sepzjh commented May 6, 2024

Related to Model/Framework(s)
Triton
Describe the bug
A clear and concise description of what the bug is.
when i run the scipt:
python3 triton/run_inference_on_triton.py
--server-url localhost:8001
--model-name ${MODEL_NAME}
--model-version 1
--output-dir ${SHARED_DIR}/accuracy_dump

--precision ${PRECISION}
--dataloader triton/dataloader.py
--data-dir ${DATASETS_DIR}/imagenet
--batch-size ${MAX_BATCH_SIZE}
--dump-labels
error:
2024-05-06 08:36:19,172 INFO dataloader Creating PIL dataloader on data_dir=/data/DeepLearningExamples/PyTorch/Classification/ConvNets/workspace/datasets_dir/imagenet #images=50000 image_size=(224, 224) batch_size=16
Exception in thread Thread-1:
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/opt/conda/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "triton/run_inference_on_triton.py", line 165, in req_loop
infer_input.set_data_from_numpy(data)
File "/opt/conda/lib/python3.8/site-packages/tritonclient/grpc/init.py", line 1441, in set_data_from_numpy
self._input.parameters.pop('shared_memory_region', None)
TypeError: pop() takes from 1 to 2 positional arguments but 3 were given

To Reproduce

follow the Guide:
DeepLearningExamples/PyTorch/Classification/ConvNets/triton/resnet50
/README.md Quick Start Guide

model_store

  • resnet50
    • config.pbtxt
    • 1
      • model.onnx

Expected behavior
no error in running the scirpt and have accuracy_metrics.csv

Environment
Please provide at least:

  • Container version: nvcr.io/nvidia/tritonserver:21.02-py3 and resnet50:latest
  • GPUs in the system: 1x NVIDIA A800 80GB PCIe):
  • CUDA driver version :525.85.12
  • triton_version:2.7.0
@sepzjh sepzjh added the bug Something isn't working label May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant