- Inference for object detection from a video or image input source, with support for multiple switchable frameworks to manage the inference process, and optional GStreamer integration for video capture.
- CMake (3.22.1)
- OpenCV (4.7.0) (apt install libopencv-dev)
- spdlog (1:1.9.2+ds-0.2) (apt-get install libspdlog-dev)
- C++ compiler with C++17 support (i.e. GCC 8.0 and later)
- GStreamer (1.20.3)
- CUDA (if you want to use GPU, CUDA 12 is supported for LibTorch and TensorRT, I used CUDA 11.8 for onnx-rt)
- ONNX Runtime (1.15.1 gpu package)
- LibTorch (2.0.1-cu118)
- TensorRT (8.6.1.6)
- OpenVino (2023.2)
- If you need a specific inference backend, set DEFAULT_BACKEND in CMakeLists with the appropriate option (i.e. ONNX_RUNTIME, LIBTORCH, TENSORRT, LIBTENSORFLOW, OPENCV_DNN, OPENVINO) or set it using cmake from the command line. If no inference backend is specified, the OpenCV-DNN module will be used by default.
- Models with dynamic axis are currently not supported(at least not all)
- Windows build not supported.
mkdir build
cd build
cmake -DDEFAULT_BACKEND=chosen_backend -DCMAKE_BUILD_TYPE=Release ..
cmake --build .
To enable GStreamer support, you can add -DUSE_GSTREAMER=ON when running cmake, like this:
mkdir build
cd build
cmake -DDEFAULT_BACKEND=chosen_backend -DUSE_GSTREAMER=ON -DCMAKE_BUILD_TYPE=Release ..
cmake --build .
This will set the USE_GSTREAMER option to "ON" during the CMake configuration process, enabling GStreamer support in your project.
Remember to replace chosen_backend with your actual backend selection.
./object-detection-inference \
--type=<model type> \
--source="rtsp://cameraip:port/somelivefeed" (or --source="path/to/video.format") (or --source="path/to/image.format") \
--labels=</path/to/labels/file> \
--weights=<path/to/model/weights> [--config=</path/to/model/config>] [--min_confidence=<confidence value>].
./object-detection-inference --help
Running inference with yolov8s and the TensorRT backend:
build setting for cmake DEFAULT_BACKEND=TENSORRT, then run
./object-detection-inference \
--type=yolov8 \
--weights=/path/to/weights/your_yolov8s.engine \
--source=/path/to/video.mp4 \
--labels=/path/to/labels.names
Run the inference with rtdetr-l and the Onnx runtime backend:
build setting for cmake DEFAULT_BACKEND=ONNX_RUNTIME, then run
./object-detection-inference \
--type=rtdetr \
--weights=/path/to/weights/your_rtdetr-l.onnx \
--source=/path/to/video.mp4 \
--labels=/path/to/labels.names [--use-gpu]
- Inside the project, in the Dockerfiles folder, there will be a dockerfile for each inference backend (currently onnxruntime, libtorch, tensorrt)
docker build --rm -t object-detection-inference:<backend_tag> -f Dockerfiles/Dockerfile.backend .
This command will create a docker image based on the provided docker file.
Replace the wildcards with your desired options and paths:
docker run --rm -v<path_host_data_folder>:/app/data -v<path_host_weights_folder>:/weights -v<path_host_labels_folder>:/labels object-detection-inference:<backend_tag> --type=<model_type> --weights=<weight_according_your_backend> --source=/app/data/<image_or_video> --labels=/labels/<labels_file>.
- The following table provides information about available object recognition models and supported framework backends: Link to Table Page
- The following page provides information on how to export supported object recognition models: Link to Export Page
- YoloV5
- YoloV6
- YoloV7
- YoloV8
- YoloV9
- YoloNas
- RT-DETR
- RT-DETR (Ultralytics implementation)
- Reimplement Libtensorflow backend
- Run inside a docker container
- Add Windows building support
- Add tests
- Any feedback is greatly appreciated, if you have any suggestions, bug reports or questions don't hesitate to open an issue.