ultralytics 8.2.16 DDP pretrained argument fix

[comlhj1114]

This PR solves an issue (#11769), which is related to loading a pretrained model in multi-gpu training.

I solved the problem by using the "pretrained" argument and modifying setup_model function in BaseTrainer to load a pretrained weight.

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Enhancements and bug fixes in the Ultralytics framework, bringing it to version 8.2.16.

📊 Key Changes

• 📖 Tutorial notebook (tutorial.ipynb) cleaned up to remove error messages related to cuDNN, cuFFT, and cuBLAS registration failures.
• 🆙 Framework version bump in __init__.py from 8.2.15 to 8.2.16.
• 🔨 In trainer.py, improved model setup logic by refining the handling of configuration files and pretrained weights, ensuring that the correct setup path is chosen based on the input.

🎯 Purpose & Impact

• 🚀 Tutorial Clarity: Removing error messages from the tutorial notebook ensures a smoother learning experience for new users, making the educational content clearer and more accessible.
• ✨ Framework Updates: The version increment signifies improvements and possibly new features or bug fixes, contributing to the overall maturity and stability of the Ultralytics framework.
• 🛠 Model Setup Enhancements: Changes to the model setup improve the framework's flexibility and ease of use. Users can now expect a more straightforward and error-resistant process when loading models with predefined weights or configurations, facilitating both development and deployment workflows.

[github-actions]

All Contributors have signed the CLA. ✅
_{Posted by the CLA Assistant Lite bot.}

[comlhj1114]

I have read the CLA Document and I sign the CLA

[github-actions]

👋 Hello @comlhj1114, thank you for submitting an Ultralytics YOLOv8 🚀 PR! To allow your work to be integrated as seamlessly as possible, we advise you to:

• ✅ Verify your PR is up-to-date with ultralytics/ultralytics main branch. If your PR is behind you can update your code by clicking the 'Update branch' button or by running git pull and git merge main locally.
• ✅ Verify all YOLOv8 Continuous Integration (CI) checks are passing.
• ✅ Update YOLOv8 Docs for any new or updated features.
• ✅ Reduce changes to the absolute minimum required for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." — Bruce Lee

See our Contributing Guide for details and let us know if you have any questions!

[codecov]

Codecov Report

Attention: Patch coverage is 80.00000% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 70.60%. Comparing base (590002f) to head (16fcda2).

❗ Current head 16fcda2 differs from pull request most recent head 894e160. Consider uploading reports for the commit 894e160 to get more accurate results

Files	Patch %	Lines
ultralytics/engine/trainer.py	80.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #11787      +/-   ##
==========================================
- Coverage   74.77%   70.60%   -4.18%     
==========================================
  Files         122      122              
  Lines       15634    15632       -2     
==========================================
- Hits        11691    11037     -654     
- Misses       3943     4595     +652     

Flag	Coverage Δ
Benchmarks	35.50% <0.00%> (-0.05%)	⬇️
GPU	37.28% <60.00%> (-5.64%)	⬇️
Tests	66.75% <80.00%> (-3.70%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[comlhj1114]

I have read the CLA Document and I sign the CLA

[glenn-jocher]

@Laughing-q can you review this proposed fix? Is it necessary? Thanks!

[Laughing-q]

@glenn-jocher yes I tested and confirmed pretrained arg does not work when using DDP mode.
@comlhj1114 Hi, I updated the PR a little bit and tested pretrained arg with DDP mode locally, everything works to me! Can you test the PR again to see if it still works to you? Thanks!
BTW here's my test command which aims to load yolov8n.pt by pretrained=yolov8n.pt for a quick verification:

yolo detect train data=coco128.yaml model=yolov8n.yaml pretrained=yolov8n.pt device=0,1

and it got exactly the same results as directly loading model by model=yolov8n.pt:

yolo detect train data=coco128.yaml model=yolov8n.pt

which means yolov8n.pt is loaded successfully in the first command.

[johnnynunez]

I've checked and it's true. There are differences.

[Laughing-q]

@johnnynunez is the one with dark blue that trained with pretrained? if so then I think the updates are working!

[johnnynunez]

@johnnynunez is the one with dark blue that trained with pretrained? if so then I think the updates are working!

Both are pretrained. But Blue is applying the changes of PR

[Laughing-q]

@johnnynunez got it! Thanks! Then it means the PR is working!
@glenn-jocher I think the PR is ready! @johnnynunez tested it as well for double check!

[glenn-jocher]

@Laughing-q @johnnynunez awesome, thanks guys!

[glenn-jocher]

@comlhj1114 PR merged! Thank you for your contributions. Please let us know if you spot any other areas we could improve :)