Object Counting - Ultralytics YOLOv8 Docs

[giscus[bot]]

Object Counting - Ultralytics YOLOv8 Docs

Object Counting Using Ultralytics YOLOv8

https://docs.ultralytics.com/guides/object-counting/

[kemalkilicaslan]

this even after downloading ultralytics as "pip install ultralytics" ModuleNotFoundError: No module named 'ultralytics.solutions' I get the error can you help? thanks

[kemalkilicaslan]

thanks.

[RizwanMunawar]

@kemalkilicaslan We have updated the modules with new options and arguments!

[kemalkilicaslan]

Thanks @RizwanMunawar, thanks for your efforts. I will review it as soon as possible. I closely examine your work, very successful, congratulations to you and the @ultralytics team.

[nicoffein13]

there is an error in my code line 21 how to solved it? i cant

[pderrenger]

@nicoffein13 could you please provide the specific error message you're receiving and the relevant portion of your code around line 21? This will help me understand the issue better and provide you with a more accurate solution.

[yeongnamtan]

How do you save the output as a mp4 file ?

[RizwanMunawar]

@yeongnamtan You can utilize the provided code to save the output while performing Object Counting with Ultralytics YOLOv8.

from ultralytics import YOLO
from ultralytics.solutions import object_counter
import cv2

model = YOLO("yolov8n.pt")
cap = cv2.VideoCapture("path/to/video.mp4")

# Video Writer
fps, fourcc = int(cap.get(5)), cv2.VideoWriter_fourcc(*'mp4v')
video_writer = cv2.VideoWriter("ultralytics_object_counting.avi", fourcc,
                               fps, [int(cap.get(3)), int(cap.get(4))])

counter = object_counter.ObjectCounter()  # Init Object Counter
region_points = [(20, 400), (1080, 404), (1080, 360), (20, 360)]
counter.set_args(view_img=True, reg_pts=region_points,
                 classes_names=model.names, draw_tracks=True)

while cap.isOpened():
    success, frame = cap.read()
    if not success:
        exit(0)
    tracks = model.track(frame, persist=True, show=False)
    counter.start_counting(frame, tracks)
    video_writer.write(frame)

[yeongnamtan]

Thanks @RizwanMunawar

[HugeBra1n]

I tried recording a processed video file, but the output was always empty.
managed to solve this problem by changing the values in line 11
fps, [int(cap.get(3)), int(cap.get(4))])

[RizwanMunawar]

@HugeBra1n Thanks for the update, yes you are right. Anyway, this new update will be released today or tomorrow. :)

[RizwanMunawar]

@yeongnamtan @HugeBra1n We have updated the modules with new options and arguments!

[yeongnamtan]

How do I select a particular object class (eg. person) to count ?

[RizwanMunawar]

@yeongnamtan The classes argument is not supported yet! It will be available in the next release.

Thanks
Ultralytics Team!

[yeongnamtan]

Thanks @RizwanMunawar

[RizwanMunawar]

@yeongnamtan We have updated the modules with new options and arguments! You can now select the specific classes by following Ultralytics Object Counting Docs Page.

[yeongnamtan]

how do you set the parameters of the tracker ? eg thresh level for bytesort

[RizwanMunawar]

@yeongnamtan The tracking configuration is not supported yet! It will be available in the next release.

Thanks
Ultralytics Team!

[yeongnamtan]

Thanks @RizwanMunawar

[RizwanMunawar]

@yeongnamtan We have updated the modules with new options and arguments! You can now set the tracker configuration by following the Arguments model.track available on the Ultralytics Object Counting Docs Page.

[yeongnamtan]

Thanks @RizwanMunawar

[yeongnamtan]

what does INCOUNT and OUTCOUNT mean ? I have fish swimming in one direction from right to left but I still see figures in both INCOUNT and OUTCOUNT ?

[RizwanMunawar]

The default values for incounts and outcounts will be in effect. If the fish are moving in a specific direction, you will observe that either the incount or outcount value is 0.

[yeongnamtan]

what does INCOUNT and OUTCOUNT mean ?

Is there a way for me to upload an image to show you ?

[RizwanMunawar]

@yeongnamtan

InCount: Fish that pass the region from right to left or bottom to top
OutCount: Fish that pass the region from left to right or top to bottom

Thanks

[yeongnamtan]

Somehow the InCount and OutCount has values even though the fish was swimming all in one direction.
See video result:
https://drive.google.com/file/d/1I8RZEYmGXmAkB801SQTNI_RoSVhNcp6K/view?usp=sharing

[yeongnamtan]

This doesn't look right.
I thought Object Counter will simply count all objects crossing the region irrespective of what shape the region is.

[RizwanMunawar]

@yeongnamtan, consider updating the region coordinates. The provided values [(20, 400), (1080, 404), (1080, 360), (20, 360)] are the default settings. Feel free to adjust them based on your specific use case to achieve improved results.

Note: The default counting region coordinates might not work correctly. Generic region counting irrespective of shape is not added and it's in our TODO list.

Thanks
Ultralytics Team!

[yeongnamtan]

@RizwanMunawar thank you

[netpbcl]

@@yeongnamtan
Automatic Region Point box solution !!
Replace section with:

in the section : # Define region points

line_position = 50
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

y_line = int(line_position * height / 100)
pt1 = (0, y_line)
pt2 = (width, y_line )

pt11 = (width, y_line -100)
pt22 = (0, y_line -100)

region_points = [pt1,pt2,pt11,pt22]

[glenn-jocher]

Hi @netpbcl! Yes, adjusting the region_points to better align with the specific area where the objects are crossing can help improve the accuracy of the in/out counts. Your suggested modification to dynamically set the region based on the video dimensions is a great approach, especially for adapting to different video inputs. This ensures that the region is always appropriately scaled and positioned. If you need further customization, consider adjusting the line_position percentage to better capture the desired area for counting. Keep experimenting with these parameters to find the best setup for your specific scenario! 🚀

[kemalkilicaslan]

import cv2
from ultralytics import YOLO
from ultralytics.solutions import object_counter

# Load the YOLO model
model = YOLO("yolov8m.pt")

# Path to the video file
video_file = "video.mp4"

# Open the video file
cap = cv2.VideoCapture(video_file)

# Check if the video file is opened successfully
if not cap.isOpened():
    print("Error: Couldn't open the video file")
    exit(1)

# Define the region of interest (ROI)
roi_start_point = (820, 800)
roi_end_point = (1090, 1000)
region_points = [(820, 900), (1090, 800), (1090, 800), (820, 900)]

# Initialize video writer for output
video_writer = cv2.VideoWriter("object_counting.mp4",
                               cv2.VideoWriter_fourcc(*'mp4v'),
                               int(cap.get(5)),
                               (int(cap.get(3)), int(cap.get(4))))

# Initialize the object counter
counter = object_counter.ObjectCounter()
counter.set_args(view_img=True,
                 reg_pts=region_points,
                 classes_names=model.names,
                 draw_tracks=True)

while True:
    # Read a frame from the video
    success, im0 = cap.read()

    # Check if the frame is read successfully
    if not success:
        print("Finished reading the video")
        break

    # Perform object tracking with YOLO
    tracks = model.track(im0, persist=True, show=False)

    # Start counting objects in the frame
    im0 = counter.start_counting(im0, tracks)

    # Draw ROI rectangle on the frame
    cv2.rectangle(im0, roi_start_point, roi_end_point, (0, 255, 0), 2)
    
    # Check if the frame is not None before processing
    if im0 is not None:
        roi = im0[roi_start_point[1]:roi_end_point[1], roi_start_point[0]:roi_end_point[0]]

        gray_roi = cv2.cvtColor(roi, cv2.COLOR_BGR2GRAY)
        edges = cv2.Canny(gray_roi, 50, 150)
        contours, _ = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

        # Count objects within the defined area
        for contour in contours:
            area = cv2.contourArea(contour)
            if area > 100:
                if roi_end_point[1] < contour[0][0][1] < roi_start_point[1]:
                    counter.count(contour)
                    print("object count number:", counter.get_counts())

        # Draw line on the frame
        cv2.line(im0, (820, region_points[0][1]), (1090, region_points[1][1]), (0, 0, 255), 2)
        cv2.imshow("Video", im0)

    # Exit loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release video capture and destroy windows
cap.release()
cv2.destroyAllWindows()

This python code does not perform counting! Will you add modules like line_position in the set_args class? can you help me?

@RizwanMunawar

[RizwanMunawar]

@kemalkilicaslan The Line module is already available. Documentation for this is coming soon. To convert a Region to a line, set coordinates according to the mentioned rule:

• For Line: Region Coord 1 = Coord 4, Coord 2 = Coord 3.

The complete code for your use case is mentioned below.

from ultralytics import YOLO
from ultralytics.solutions import object_counter
import cv2

model = YOLO("yolov8n.pt")
cap = cv2.VideoCapture("path/to/video/file.mp4")
assert cap.isOpened(), "Error reading video file"

counter = object_counter.ObjectCounter()  # Init Object Counter
region_points = [(820, 900), (1090, 800), (1090, 810), (820, 910)]
counter.set_args(view_img=True,
                 reg_pts=region_points,
                 classes_names=model.names,
                 draw_tracks=True)

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        exit(0)
    tracks = model.track(im0, persist=True, show=False)
    im0 = counter.start_counting(im0, tracks)

Hope this helps! Let me know if you have any additional questions.

Thanks and Regards
Ultralytics Team!

[netpbcl]

@kemalkilicaslan, The problem with counting is that the in_count or out_count variables will increase or decrease only if the object crosses the region or line, on the right side or the left side of the area. The function used is centroid.x, this works calculates if the object is on the left or right side

[7anand7s]

How do I access in_count and out_count values? Thanks.

[7anand7s]

Simply counter.in_counts ?

[RizwanMunawar]

@7anand7s Absolutely, without a doubt!!!

[Azer2401]

Hi, can I implemented it in android? The use case is to count palm fruit in palm plantation and hard to find signal. Previously I have succeeded in creating an object detection model using yolov8s and exporting it to onnx then to ncnn

[RizwanMunawar]

Object counting support for Android is currently unavailable, but we have an implementation plan for this in 2024.

Regards, Ultralytics Team!

[meerte]

Hi, even though I have cloned ultralytics repository and download setup.py file, I couldn't import ultralytics.solutions. When I write ' from ultralytics.solutions' it says there is no module named solutions,however other modules, assets, cfg, data, engine, hub, models, nn, trackers and utils are shown.
Thanks.

[RizwanMunawar]

@Meete! Attempt to uninstalling the ultralytics package and subsequently reinstall it. There's no necessity to execute the setup.py file.

• pip uninstall ultralytics
• pip install ultralytics # Make sure to install the latest version

Now you can import ultralytics solutions i.e

from ultralytics.solutions import object_counter, ai_gym, heatmap

Hope this helps! Let me know if you have any additional questions.

Thanks and Regards
Ultralytics Team!

[UgurcanCirak]

@glenn-jocher and @RizwanMunawar.
First of all, thank you for your help. Can I ask one more question ? How can I print the incount and outcount sections in the object counter section and the number of engines and helmets detected in that area? What path should I follow?

[pderrenger]

Yes, @Atharva-Malode, real-time object counting with YOLOv8 is possible. You can use a live video stream from a camera as the input source for the model. The ObjectCounter class from the ultralytics.solutions module can be used to count objects as they appear in the video feed.

Here's a basic example of how you might set this up:

from ultralytics import YOLO
from ultralytics.solutions import object_counter
import cv2

# Load the model
model = YOLO('yolov8n.pt')

# Initialize video stream (0 for the first webcam)
cap = cv2.VideoCapture(0)

# Initialize Object Counter with desired settings
counter = object_counter.ObjectCounter()
counter.set_args(view_img=True, draw_tracks=True)

# Process video stream
while cap.isOpened():
    success, frame = cap.read()
    if not success:
        break

    # Run tracking and counting
    results = model.track(frame, persist=True)
    frame = counter.start_counting(frame, results)

    # Display the frame with counts
    cv2.imshow('Real-time Object Counting', frame)

    # Break loop with 'q'
    if cv2.waitKey(1) == ord('q'):
        break

# Release resources
cap.release()
cv2.destroyAllWindows()

This script will display the real-time video feed with object counts overlaid. Press 'q' to exit the loop.

For specific class counts, you would need to modify the ObjectCounter class or create a custom solution that filters the tracked objects by class and maintains separate counts for each class.

Thanks,
Ultralytics Team!

[dgodus]

when you open new news about Every class counting

[pderrenger]

@dgodus hey there! 😊 We're currently working on enhancing our object counting features, including counting specific classes that cross a defined line. This update will let you get counts like "In Counts: 4 motorcycle, 2 car, 3 truck", exactly as you're looking for!

For real-time footage, our YOLOv8 models can indeed do object counting. Here's a simple way to display specific class counts on your video stream using cv2.putText():

import cv2
from ultralytics import YOLO

# Load the model
model = YOLO('yolov8n.pt')

# Initialize video stream (replace 0 with your video source)
cap = cv2.VideoCapture(0)

while True:
    ret, frame = cap.read()
    if not ret:
        break

    # Run prediction (replace with model.track() for real-time tracking)
    results = model.predict(frame)

    # Example: Get and display counts for 'person' and 'car' classes
    person_count = len([x for x in results.xyxy[0] if model.names[int(x[5])] == 'person'])
    car_count = len([x for x in results.xyxy[0] if model.names[int(x[5])] == 'car'])

    # Display counts on frame
    cv2.putText(frame, f'Person: {person_count} Car: {car_count}', (50, 50), 
                cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)

    cv2.imshow('frame', frame)
    if cv2.waitKey(1) == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Please keep an eye on our updates for the feature that offers detailed class counts across lines. It's on our roadmap, and we appreciate your patience and excitement! 🚀 If you have more questions, feel free to ask.

[zixindh]

Is it possible to export the in-count and out-count and timestamp to a csv file?

[pderrenger]

Absolutely, exporting the in-counts, out-counts, and timestamps to a CSV file can be very useful for monitoring and analysis. Here's a concise way to do it using Python:

import csv
import datetime
from ultralytics import YOLO
from ultralytics.solutions import object_counter
import cv2

# Initialize model and counter
model = YOLO("yolov8n.pt")
counter = object_counter.ObjectCounter()
# Setup your video source
cap = cv2.VideoCapture("path/to/video.mp4")

# CSV file setup
with open('object_counts.csv', 'w', newline='') as file:
    writer = csv.writer(file)
    writer.writerow(["Timestamp", "In Count", "Out Count"])

    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break

        # Track objects
        results = model.track(frame, persist=True)
        # Use counter to count objects
        frame = counter.start_counting(frame, results)
        
        # Write counts and timestamp to CSV
        timestamp = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
        writer.writerow([timestamp, counter.in_counts, counter.out_counts])  # Assuming counter object has these attributes

        # Optional: Display the frame
        cv2.imshow('frame', frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):  # Press 'q' to exit
            break

cap.release()
cv2.destroyAllWindows()

This example captures object counting data per frame and writes it into a CSV file with a timestamp. Adjust it to your needs, and make sure your ObjectCounter object appropriately tracks in_counts and out_counts.

🌟 Keep experimenting, and feel free to reach out if you have more questions!

[Anandukc]

how can i get image of the frame whenever a person is going in or out.Like i want the image of frame when person is entering ,saved in a folder and leaving in another folder. @RizwanMunawar

[RizwanMunawar]

@Anandukc, From my comprehension, you aim to capture and store images of objects as they cross the counting line or region. Presently, this feature is not supported. However, we plan to incorporate this functionality in the coming days and will notify you!

We would suggest you keep a close watch on https://docs.ultralytics.com/ for updates.

Thanks,
Ultralytics Team!

[Anandukc]

ya i want images of person who enters and leaves in separate folders. Is there any way, like you are taking in and out count right so when the in count or out count is 1 can we take image of frame and each time it get incremented we take images, is this possible .if so can you help me how to do it. @RizwanMunawar

[RizwanMunawar]

@Anandukc, as of now, this feature is not supported. However, we have a plan to incorporate it in the upcoming days. We appreciate your patience.

Thanks,
Ultralytics Team!

[Anandukc]

ok. please do update when you incorporate this feature. Thankyou for your response.

[pderrenger]

@Anandukc absolutely, we'll keep you updated on our progress. Keep an eye on our documentation for the latest features and enhancements. If you have any more questions or need further assistance in the future, don't hesitate to reach out.

Thank you for your interest in Ultralytics YOLOv8! 🚀

[pathu999]

Issue: Current implementation uses object tracking, but I want object counting without tracking.
Requirement 1: Update object count when an object enters the frame, with tracking its movement.
Requirement 2: Support counting multiple types of objects separately.
Please guide on modifying the code to meet the above requirements.
Thank you!

[RizwanMunawar]

@pathu999, counting objects without object tracking is possible, but it may result in low accuracy as direct object detection might not capture each object in every frame. Tracking helps the model persistently detect objects.

To count objects without tracking, replace model.track with model.predict and eliminate all tracking modules in object_counting.py. After making these changes, execute python setup.py install from the ultralytics repository.

I hope this helps! Let me know if you have any additional questions.

Thanks and Regards
Ultralytics Team!

[yeongnamtan]

Can we add more than 1 lines for object counting ?

[RizwanMunawar]

@yeongnamtan, currently there is no support for multiple lines in object counting.

Regards,
Ultralytics Team

[yeongnamtan]

thank you @RizwanMunawar

[TUCJVXCB]

hello, as you said, one object pass the region from left to right or top to bottom will be considered as output, can I restrict it to only count as output if it's moving from top to bottom

[RizwanMunawar]

@TUCJVXCB, indeed, you can opt for the line counter module instead of relying on the region. This module specifically counts objects from top to bottom by the line direction.

Object Counting using Line Code: https://docs.ultralytics.com/guides/object-counting/#__tabbed_1_2

Thanks
Ultralytics Team!

[kewchi]

when i use region i will get an incount and an outcount, however, i would like it to count the number of detections in the region (eg. 3 detections in one frame, 1 detection in next frame) like as shown in the Conveyor Belt Packets Counting Using Ultralytics YOLOv8 gif shown above. Could i have some advice regarding this?

[RizwanMunawar]

For your particular use case, consider employing our 'object counting in region' module. You can find comprehensive details about it in the following link: https://docs.ultralytics.com/guides/region-counting/

Best regards,
Ultralytics Team

[kewchi]

I am unable to run the codes in the given link in colab, may I know if there is a certain way to code it in colab? thanks!

[pderrenger]

@kewchi certainly! To run the object counting code in Google Colab, you'll need to ensure that you have installed the necessary packages and have the correct setup. Here's a step-by-step guide:

1. Install Ultralytics YOLOv8:
   Install the Ultralytics YOLOv8 package using pip. You can do this by running the following command in a Colab cell:

   !pip install ultralytics

2. Load the Model:
   Load the YOLOv8 model using the YOLO class from the ultralytics package.

3. Set Up Video Capture:
   Use cv2.VideoCapture() to load your video file. If you're using a video from your local system, you'll need to upload it to Colab first.

4. Run Object Counting:
   Use the object_counter module from ultralytics.solutions to count objects within a defined region.

Here's a sample code snippet that you can use as a template in Colab:

# Install the ultralytics package
!pip install ultralytics

# Import necessary libraries
from ultralytics import YOLO
from ultralytics.solutions import object_counter
import cv2
from google.colab.patches import cv2_imshow

# Load the model
model = YOLO('yolov8n.pt')

# Define your video source (upload your video to Colab first)
video_path = '/content/your_video.mp4'

# Set up video capture
cap = cv2.VideoCapture(video_path)
assert cap.isOpened(), "Error reading video file"

# Define region points (customize this according to your video)
region_points = [(20, 400), (1080, 404), (1080, 360), (20, 360)]

# Initialize Object Counter
counter = object_counter.ObjectCounter()
counter.set_args(view_img=True, reg_pts=region_points, classes_names=model.names, draw_tracks=True)

# Process video frames
while cap.isOpened():
    success, frame = cap.read()
    if not success:
        break
    tracks = model.track(frame, persist=True, show=False)
    frame = counter.start_counting(frame, tracks)
    cv2_imshow(frame)  # Use cv2_imshow instead of cv2.imshow in Colab

# Release resources
cap.release()
cv2.destroyAllWindows()

Remember to replace '/content/your_video.mp4' with the path to your uploaded video file. You can upload files to Colab using the file explorer in the sidebar or by using the following cell command:

from google.colab import files
uploaded = files.upload()

Run each cell sequentially, and you should be able to see the object counting results displayed directly within Colab.

[jizijun666666]

I have established a zone at the bottom of the frame and implemented object counting to track interactions between fish and soil. Currently, the counting is based solely on the center of the detecting box crossing the zone. I need assistance in modifying the counting mechanism to include instances where the fish's tail or mouth interacts with the soil. Can you provide guidance on adjusting the counting logic to account for these specific interactions

[jizijun666666]

Thank you
((float((box[0] + box[2]) / 2), float(box[3]+(box[1]-box[3])/3)))Can I modify this code to extract keypoints' coordinates？ Where are the coordinates of keypoints stored in YOLO Pose？

[RizwanMunawar]

The keypoints are seperate from bounding boxes. It will be stored in kpts or keypoints variables.

[jizijun666666]

def extract_and_process_tracks(self, tracks):
    boxes = tracks[0].boxes.xyxy.cpu()
    clss = tracks[0].boxes.cls.cpu().tolist()
    track_ids = tracks[0].boxes.id.int().cpu().tolist()
    keypoint = tracks[0].keypoints.data

    self.annotator = Annotator(self.im0, self.tf, self.names)
    self.annotator.draw_region(reg_pts=self.reg_pts, color=(0, 255, 0))

    for box, track_id, cls, keypoint in zip(boxes, track_ids, clss, keypoint):
        self.annotator.box_label(box, label=self.names[cls], color=colors(int(cls), True))  # Draw bounding box

        # Draw Tracks
        x, y = int(keypoint[0].item()), int(keypoint[1].item())
        track_line = self.track_history[track_id]
        track_line.append((x, y))
        track_line.pop(0) if len(track_line) > 30 else None

This code doesn't work. Please fix it for me.
Thank you so much

[jizijun666666]

I only have one keypoint, the head keypoint

[glenn-jocher]

@jizijun666666 it seems like you're trying to extract the head keypoint from the tracks object and draw it on the image. If you only have one keypoint (the head keypoint), you should ensure that the keypoints attribute of the tracks object is being accessed correctly and contains the data you expect.

Here's a revised version of your code snippet that should work if tracks[0].keypoints.data contains the keypoints data in the expected format:

def extract_and_process_tracks(self, tracks):
    boxes = tracks[0].boxes.xyxy.cpu()
    clss = tracks[0].boxes.cls.cpu().tolist()
    track_ids = tracks[0].boxes.id.int().cpu().tolist()
    keypoints = tracks[0].keypoints.data.cpu()

    self.annotator = Annotator(self.im0, self.tf, self.names)
    self.annotator.draw_region(reg_pts=self.reg_pts, color=(0, 255, 0))

    for box, track_id, cls, keypt in zip(boxes, track_ids, clss, keypoints):
        self.annotator.box_label(box, label=self.names[cls], color=colors(int(cls), True))  # Draw bounding box

        # Draw Tracks
        if keypt is not None and len(keypt) > 0:
            # Assuming the head keypoint is the first one in the list
            head_keypoint = keypt[0]
            x, y = int(head_keypoint[0].item()), int(head_keypoint[1].item())
            track_line = self.track_history[track_id]
            track_line.append((x, y))
            if len(track_line) > 30:
                track_line.pop(0)
            # Draw the track line
            for i in range(1, len(track_line)):
                if track_line[i - 1] is None or track_line[i] is None:
                    continue
                cv2.line(self.im0, track_line[i - 1], track_line[i], (0, 255, 0), 2)

Please note the following:

• The keypoints attribute is expected to be a list of keypoints for each detection, with each keypoint represented as a (x, y) tuple.
• The head_keypoint is assumed to be the first keypoint in this list.
• The track_history dictionary is used to store the historical positions of each track ID.
• The Annotator class is used to draw the bounding boxes and region on the image.
• The colors function is used to get a unique color for each class ID.

Make sure that the tracks[0].keypoints.data is in the correct format and contains the head keypoints as expected. If the format is different, you'll need to adjust the code accordingly.

[Sahil123445]

Hello,

I am trying to use this to count the number of vehicles by class whenever they cross the line. Rather than giving in_count and out_count is there a way to achieve this to either classify the vehicles by class and save it into a csv file or similar to how the in and out count display to show them on the screen as it counts?

Thank you

[RizwanMunawar]

@Sahil123445 data storage feature is not supported in object counting module, we will add this feature in coming days.

Thanks
Ultralytics Team!

[ShreyasChhetri]

Hi, Im getting an error ModuleNotFoundError: No module named 'ultralytics.solutions' when I'm running my file any help here please

[RizwanMunawar]

You can modify the code based on your requirements available at: https://github.com/ultralytics/ultralytics/blob/main/ultralytics/solutions/object_counter.py

[ShreyasChhetri]

Hi , no I'm not asking to change the code I just want to print the ID number with the remaining boxes object like xyxy coordinates class names etc.

[ShreyasChhetri]

Is there any method for doing that ?

[RizwanMunawar]

@ShreyasChhetri you will need to change the code for your specific implementation. Thanks

[ShreyasChhetri]

from ultralytics import YOLO
from ultralytics.solutions import object_counter
import cv2

model = YOLO("../ultralytics/ultralytics/yolo/v8/runs/weights/best_car.pt")
cap = cv2.VideoCapture("./test.mp4")
assert cap.isOpened(), "Error reading video file"
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

Define line points

line_points = [(5, 10), (2000, 800)]

# Video writer

video_writer = cv2.VideoWriter("object_counting_output.avi",
cv2.VideoWriter_fourcc(*'mp4v'),
fps,
(w, h))

Init Object Counter

counter = object_counter.ObjectCounter()
counter.set_args(view_img=True,
reg_pts=line_points,
classes_names=model.names,
draw_tracks=True,region_thickness=1,track_color=(255,255,255))

while cap.isOpened():
success, im0 = cap.read()
if not success:
print("Video frame is empty or video processing has been successfully completed.")
break
tracks = model.track(im0, persist=True, show=False,tracker = "bytetrack.yaml",classes = [0,1],verbose=True)

im0 = counter.start_counting(im0, tracks)
in_counts = counter.in_counts
out_counts = counter.out_counts
print(f"In_Counts - {in_counts} and Out_Counts - {out_counts}")
video_writer.write(im0)

cap.release()
video_writer.release()
cv2.destroyAllWindows()

this is my code how can I print out Tracking ID and xyxy coordinates as well after the video is saved and also how can I change the line point vertically to the left and start from there please help
@RizwanMunawar

[saurabhsensai]

@RizwanMunawar can you please provide any estimate about how much time needed team for implement multiclass counting. Or are there any alternatives for that.

[RizwanMunawar]

@saurabhsensai We are actively working on this feature, It will be released in the next week or two.

Thanks
Ultralytics Team!

[nicoffein13]

i can install "ultralytics.solutions" on vscode. please help

[RizwanMunawar]

On VsCode, make sure you have the Python environment configured. Otherwise, the installation will not work.

Thanks
Ultralytics Team!

[borissedlak]

hey! I have the issue that as soon as I import ultralytics, the cv2.imshow does not work anymore, in the sense of that it just blocks without anything happen. Under the hood, the check_imshow cannot be passed, though it is also not failed.

[borissedlak]

found out that python av was responsible (#3005), thanks!

[RizwanMunawar]

Great Thanks

[Moomal22]

How can we only detect, track and count persons and bicycles using yolov8?

[RizwanMunawar]

@Moomal22 you can use class argument. i.e. class number for a person is 0 and for a bicycle, it's 1. the sample code is mentioned below.

from ultralytics import YOLO
from ultralytics.solutions import object_counter
import cv2

model = YOLO("yolov8n.pt")
cap = cv2.VideoCapture("path/to/video/file.mp4")
assert cap.isOpened(), "Error reading video file"
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

line_points = [(20, 400), (1080, 400)]  # line or region points
classes_to_count = [0, 1]  # person and car classes for count

# Video writer
video_writer = cv2.VideoWriter("object_counting_output.avi",
                       cv2.VideoWriter_fourcc(*'mp4v'),
                       fps,
                       (w, h))

# Init Object Counter
counter = object_counter.ObjectCounter()
counter.set_args(view_img=True,
                 reg_pts=line_points,
                 classes_names=model.names,
                 draw_tracks=True)

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break
    tracks = model.track(im0, persist=True, show=False,
                         classes=classes_to_count)

    im0 = counter.start_counting(im0, tracks)
    video_writer.write(im0)

cap.release()
video_writer.release()
cv2.destroyAllWindows()

I hope this helps! Feel free to let me know if you have any additional questions.

Thanks and Regards
Ultralytics Team!

[Moomal22]

Thank you Rizwan. This worked for me. If at the end I want to add the total persons and bicycles crossing the region of interest then I'll just add the In count and Out count. how do I access those through im0?

…

On Tue, Jan 23, 2024 at 2:36 AM Muhammad Rizwan Munawar < ***@***.***> wrote: @Moomal22 <https://github.com/Moomal22> you can use class argument. i.e. class number for a person is 0 and for a bicycle, it's 1. the sample code is mentioned below. from ultralytics import YOLOfrom ultralytics.solutions import object_counterimport cv2 model = YOLO("yolov8n.pt")cap = cv2.VideoCapture("path/to/video/file.mp4")assert cap.isOpened(), "Error reading video file"w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS)) line_points = [(20, 400), (1080, 400)] # line or region pointsclasses_to_count = [0, 1] # person and car classes for count # Video writervideo_writer = cv2.VideoWriter("object_counting_output.avi", cv2.VideoWriter_fourcc(*'mp4v'), fps, (w, h)) # Init Object Countercounter = object_counter.ObjectCounter()counter.set_args(view_img=True, reg_pts=line_points, classes_names=model.names, draw_tracks=True) while cap.isOpened(): success, im0 = cap.read() if not success: print("Video frame is empty or video processing has been successfully completed.") break tracks = model.track(im0, persist=True, show=False, classes=classes_to_count) im0 = counter.start_counting(im0, tracks) video_writer.write(im0) cap.release()video_writer.release()cv2.destroyAllWindows() I hope this helps! Feel free to let me know if you have any additional questions. Thanks and Regards Ultralytics Team! — Reply to this email directly, view it on GitHub <#6738 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BFQ5L6L6JXA7475U3VK4IXDYP5ZCPAVCNFSM6AAAAABAECARIOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DEMJXGY4TQ> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[pderrenger]

Hey there! 👋

Glad to hear Rizwan's solution worked out for you! To access the total count of persons and bicycles crossing your region of interest, you can use the counter object that's part of the object_counter module. Here's a quick example of how you might retrieve the counts:

from ultralytics import YOLO
from ultralytics.solutions import object_counter

# Initialize the model and object counter as before
# ...

# After processing the video
total_persons = counter.total_in[0] + counter.total_out[0]  # Assuming 0 is the class index for persons
total_bicycles = counter.total_in[1] + counter.total_out[1]  # Assuming 1 is the class index for bicycles

print(f"Total persons: {total_persons}")
print(f"Total bicycles: {total_bicycles}")

This will give you the cumulative counts after your video processing loop. If you need any more help, feel free to ask!

[zixindh]

@Moomal22 are you trying to get the total of unique count?

[tadiwanashechinoz]

from ultralytics import YOLO
import cv2
import math
import time
import mysql.connector
from mysql.connector import Error
from ultralytics.solutions import object_counter

def video_detection(path_x):
video_capture = path_x

cap=cv2.VideoCapture(video_capture)
frame_width=int(cap.get(3))
frame_height=int(cap.get(4))
out=cv2.VideoWriter('output.avi', cv2.VideoWriter_fourcc('M', 'J', 'P','G'), 10, (frame_width, frame_height))

model=YOLO("best.pt")
classNames = ['fire ', 'human being ', 'animals ', 'other objects ', 'vehicle ', 'buildings ']

human_being = 0
animals=0
other_objects =0
vehicle = 0
buildings = 0
fire = 0 
while True:
    success, img = cap.read()
    results=model(img,stream=True)
    #new_results=model.track(img, persist=True)
    for r in results:
        boxes=r.boxes
        for box in boxes:
            x1,y1,x2,y2=box.xyxy[0]
            x1,y1,x2,y2=int(x1), int(y1), int(x2), int(y2)
            #print(x1,y1,x2,y2)
            cv2.rectangle(img, (x1,y1), (x2,y2), (255,0,255),3)
            conf=math.ceil((box.conf[0]*100))/100
            cls=int(box.cls[0])
            class_name=classNames[cls]
            label=f'{class_name}{conf}'
            t_size = cv2.getTextSize(label, 0, fontScale=1, thickness=2)[0]
            print(t_size)
            #current_frame_count=len(new_results[0].boxes)
            
            c2 = x1 + t_size[0], y1 - t_size[1] - 3
            cv2.rectangle(img, (x1,y1), c2, [255,0,255], -1, cv2.LINE_AA)  # filled
            cv2.putText(img, label, (x1,y1-2),0, 1,[255,255,255], thickness=2,lineType=cv2.LINE_AA)
            
            # calculate  objects detected 
            #human being 
            if class_name =="human being ":
                human_being = human_being + 1 
                print("human_being")
            if class_name =="animals ":
                animals = animals + 1 
                print(animals)
            if class_name =="other objects ":
                other_objects = other_objects + 1 
                print(other_objects)
            if class_name =="vehicle ":
                vehicle= vehicle + 1 
                print(vehicle)
            if class_name =="fire ":
                fire = fire + 1 
                print(fire)
            if class_name =="buildings ":
                buildings = buildings + 1 
                print(buildings)
            try:
                connection =mysql.connector.connect(host='localhost', database='drone_project',user='root', password='')
                cursor=connection.cursor()
                mysql_select_query ="""INSERT INTO objects_data (human_being, animals, other_objects, fire, vehicle ,buildings) VALUES (%s,%s,%s,%s,%s,%s) """ 
                record =(human_being, animals, other_objects, fire, vehicle ,buildings)
                cursor.execute(mysql_select_query, record)
                connection.commit()
                print("VALUES ADDED INTO THE DATABASE ")

            except mysql.connector.Error as error:
                print("Error", error)
            
            finally:
                if connection.is_connected():
                    cursor.close()
                    connection.close()
                    print("mysql connected is closed ")            
                
    yield img
    out.write(img)
    cv2.imshow("image", img)
    #cv2.imshow('object counting ', success)
    if cv2.waitKey(1) & 0xFF==ord('1'):
        break
out.release()

cv2.destroyAllWindows()

#Hy guys i am using the code above to run my yolov8 model , but i a falling to implement

1. frame rate count
2. object detected count using id implement for example if its human being it must add all of them without mixing with animals

[pderrenger]

@tadiwanashechinoz hey there! 👋 It looks like you're doing a great job with object detection and counting using Ultralytics YOLOv8. For your specific needs on implementing frame rate count and ensuring a correct count of detected object classes without mixing, here's a brief guide:

1. Frame Rate Count: To calculate the frame rate, you can use the time module to measure the time it takes to process a frame and then calculate the rate as follows:

import time

start_time = time.time()
# Your detection code here
elapsed_time = time.time() - start_time
frame_rate = 1 / elapsed_time
print(f"Frame Rate: {frame_rate} frames per second")

2. Object Counting: Your approach is quite close! To avoid mixing different classes when counting, it's good to keep a separate tally for each class like you've done. If you're using model.track() to persist IDs across frames, make sure to update the counters inside the loop only when a new ID appears:

# Example simplified snippet to adjust based on your implementation
detected_ids = set()
for track in tracks:
    if track.id not in detected_ids:
        detected_ids.add(track.id)
        # Update your counters based on track.cls here

For frame rate and making your object counts more accurate without mixing different classes, these adjustments should prove helpful! 🚀

If you're still encountering issues or have more specific requirements, don't hesitate to ask. Happy coding!

[tadiwanashechinoz]

thanks , implementation of the FPS done , now working on counting the objects

[tadiwanashechinoz]

i am facing problems when implementing the count function, i tried to use track the object but its not maintanning the frame, anyone to help with full code that can work with mine to impement counting each object

[pderrenger]

Hey! It sounds like you're doing some really interesting work with object counting! 🚀 If you're experiencing difficulty with tracking objects across frames, ensuring each detected object is counted correctly can indeed be a challenge.

Based on the Ultralytics YOLOv8 docs you've consulted, I see you're interested in keeping precise counts of each type of object without mixing them up. Here’s a simplified approach to implement this using the ObjectCounter from Ultralytics solutions:

1. Initialize your model and object counter correctly, which you're already doing, so great job there! Just make sure the persist=True flag is used when calling model.track() to maintain the continuity of object tracks across frames.

2. Class Filtering: You can specify which classes to count by setting the classes parameter in the model.track() call, this way you won't mix animals with humans, etc. Here's a quick example:

tracks = model.track(im0, persist=True, classes=[0])  # Just an example class index

3. Unique Counting: For counting distinct objects without double counting across frames, maintain a set of track IDs you've already encountered and counted:

seen_ids = set()
# Inside your loop
for track in tracks:
    if track.id not in seen_ids:
        seen_ids.add(track.id)
        # Increment your object count here based on the class

4. Object Count: If the ObjectCounter solution doesn't directly meet your need, considering the tracking IDs is crucial as illustrated above. Ensure you’re updating counts based on the presence of new IDs mainly to avoid recounting already counted objects.

If these steps still don't completely solve your problem, consider diving deeper into the track object returned by model.track() to better understand how to utilize the IDs and class info it provides per track.

Hope this nudges you in the right direction! Keep innovating! ✨ If there's more we can help with, feel free to drop another comment.

[tadiwanashechinoz]

from ultralytics import YOLO
import cv2
import math
import time
import mysql.connector
from mysql.connector import Error
from ultralytics.solutions import object_counter

def video_detection(path_x):
video_capture = path_x

cap=cv2.VideoCapture(video_capture)
frame_width=int(cap.get(3))
frame_height=int(cap.get(4))
out=cv2.VideoWriter('output.avi', cv2.VideoWriter_fourcc('M', 'J', 'P','G'), 10, (frame_width, frame_height))

model=YOLO("best.pt")
classNames = ['fire ', 'human being ', 'animals ', 'other objects ', 'vehicle ', 'buildings ']

human_being = 0
animals=0
other_objects =0
vehicle = 0
buildings = 0
fire = 0 
while True:
    success, img = cap.read()
    results=model(img,stream=True)
    #new_results=model.track(img, persist=True)
    for r in results:
        boxes=r.boxes
        for box in boxes:
            x1,y1,x2,y2=box.xyxy[0]
            x1,y1,x2,y2=int(x1), int(y1), int(x2), int(y2)
            #print(x1,y1,x2,y2)
            cv2.rectangle(img, (x1,y1), (x2,y2), (255,0,255),3)
            conf=math.ceil((box.conf[0]*100))/100
            cls=int(box.cls[0])
            class_name=classNames[cls]
            label=f'{class_name}{conf}'
            t_size = cv2.getTextSize(label, 0, fontScale=1, thickness=2)[0]
            print(t_size)
            #current_frame_count=len(new_results[0].boxes)
            
            c2 = x1 + t_size[0], y1 - t_size[1] - 3
            cv2.rectangle(img, (x1,y1), c2, [255,0,255], -1, cv2.LINE_AA)  # filled
            cv2.putText(img, label, (x1,y1-2),0, 1,[255,255,255], thickness=2,lineType=cv2.LINE_AA)
            
            # calculate  objects detected 
            #human being 
            if class_name =="human being ":
                human_being = human_being + 1 
                print("human_being")
            if class_name =="animals ":
                animals = animals + 1 
                print(animals)
            if class_name =="other objects ":
                other_objects = other_objects + 1 
                print(other_objects)
            if class_name =="vehicle ":
                vehicle= vehicle + 1 
                print(vehicle)
            if class_name =="fire ":
                fire = fire + 1 
                print(fire)
            if class_name =="buildings ":
                buildings = buildings + 1 
                print(buildings)
            try:
                connection =mysql.connector.connect(host='localhost', database='drone_project',user='root', password='')
                cursor=connection.cursor()
                mysql_select_query ="""INSERT INTO objects_data (human_being, animals, other_objects, fire, vehicle ,buildings) VALUES (%s,%s,%s,%s,%s,%s) """ 
                record =(human_being, animals, other_objects, fire, vehicle ,buildings)
                cursor.execute(mysql_select_query, record)
                connection.commit()
                print("VALUES ADDED INTO THE DATABASE ")

            except mysql.connector.Error as error:
                print("Error", error)
            
            finally:
                if connection.is_connected():
                    cursor.close()
                    connection.close()
                    print("mysql connected is closed ")            
                
    yield img
    out.write(img)
    cv2.imshow("image", img)
    #cv2.imshow('object counting ', success)
    if cv2.waitKey(1) & 0xFF==ord('1'):
        break
out.release()

cv2.destroyAllWindows()

#Hy guys i am using the code above to run my yolov8 model , but i a falling to implement

1. frame rate count
2. object detected count using id implement for example if its human being it must add all of them without mixing with animals

[tadiwanashechinoz]

i really need your help

[glenn-jocher]

Hey there! 👋 It looks like you're diving into Object Counting with Ultralytics YOLOv8 – that's fantastic! From your query, I gather you need help with two specific aspects:

1. Frame rate count: To measure and display the frame rate of your video processing in real-time, you can utilize a simple FPS (frames per second) calculation. Here's how you might integrate it:

start_time = time.time()
frame_count = 0

while True:
    # Your existing video processing loop...
    frame_count += 1
    elapsed_time = time.time() - start_time
    fps = frame_count / elapsed_time
    print(f"FPS: {fps}")

This snippet calculates the FPS by dividing the total number of frames processed by the elapsed time since the start of the loop.

2. Object counting without mixing categories: To accurately count distinct object types (such as separating humans from animals), you're already on the right track by categorizing detections. To improve on this, make sure to reset your counting variables (human_being, animals, etc.) to 0 at the beginning of each frame processing loop. This ensures you're counting objects per frame, not accumulating counts across the entire video. If you need to keep a running total, consider using separate accumulators outside the loop.

Also, your existing implementation with the database insertions inside the processing loop might significantly slow down your processing, especially if connection times are non-negligible. Consider batching these inserts or updating these counts periodically rather than on a per-frame basis for efficiency.

Regarding the model.track() for getting unique object IDs, YOLOv8's tracking functionality already does a good job keeping track of objects across frames. Ensure you're using persist=True to maintain object identities across frames.

If you have any further questions or hit any roadblocks, feel free to ask. Happy coding! 🚀

[tadiwanashechinoz]

thanks , implementation of the FPS done , now working on counting the objects

[glenn-jocher]

Hey there! 😄 It's great to hear that you've successfully implemented the FPS calculation and are now working on the object counting part. For counting distinct objects like humans and animals without mixing them up, you're already on a good path.

Here's a short tip to ensure you get accurate counts per object category:

• Instead of incrementing counts within your main loop, consider using a dictionary to store counts for each class during each frame. Then, you could either update the total count after processing each frame or use these per-frame counts directly, depending on your application needs.

# Example snippet for object counting
object_counts = {'human_being': 0, 'animals': 0, 'other_objects': 0, 'vehicle': 0, 'buildings': 0, 'fire': 0}
for class_name in classNames:
    object_counts[class_name.strip()] = sum([1 for box in boxes if classNames[int(box.cls[0])].strip() == class_name.strip()])
    
# Example: Print counts
for object_type, count in object_counts.items():
    print(f"{object_type}: {count}")

# Your database update logic here

This approach lets you maintain clear counts for each category. 📊 Remember to reset object_counts at the start of processing each frame if you're using this for real-time counting.

If you encounter any further questions or need more assistance, the Ultralytics community is here to help. Keep up the great work!

[tadiwanashechinoz]

0: 480x640 1 human being, 261.1ms
Speed: 2.0ms preprocess, 261.1ms inference, 2.2ms postprocess per image at shape (1, 3, 480, 640)

those are outpu when running a yolov8 model, is there a way to extarct them

[pderrenger]

Yes, there's a way to extract those outputs. You can utilize the results object returned by the model.track() method for custom processing, including the extraction of preprocessing, inference, and post-processing times, and object counts. Here's a brief example:

from ultralytics import YOLO

# Initialize the model
model = YOLO('yolov8n.pt')

# Load your image or video frame
frame = ... # Add your code to load a frame here

# Perform tracking
results = model.track(frame)

# Access and print specific metrics
print(f"Preprocess time: {results.speed['preprocess']} ms")
print(f"Inference time: {results.speed['inference']} ms")
print(f"Postprocess time: {results.speed['postprocess']} ms")

# Count objects
object_counts = len(results.boxes)
print(f"Detected {object_counts} objects")

This code snippet demonstrates accessing the preprocessing, inference, and post-processing times, alongside counting the detected objects. Adjust the parameters and methods as needed for your specific scenario. 🚀

[kalwinskidawid]

Hi,
I have a couple of questions regarding detection:

1. Is it possible to save the output from the rtsp stream (not .mp4 just H264 real time from the camera)? It should close/save the video file when it does not detect objects crossing the line after 5 sec.
2. if I remove video_writer from my count project - will it speed up?
3. Is it possible to use mode.predict to count objects crossing a line?
4. If the whole dataset is learned, e.g. 640px, and the rtsp video is 1280x720 then each frame from the rtsp is scaled to 640px and only then is it processed? If the dataset is the same resolution will the processing be faster?
5. What could be the reason that the Nvidia P2000 card is used 40% (about 1.5GB/4GB) when processing mp4 ? Shouldn't it be using max - about 81% ? I see that the 25fps video is not being processed at the same speed, just slower, and it looks like the graphics card is not running at full performance.

[kalwinskidawid]

I have a one more question

counter = object_counter.ObjectCounter()
counter.set_args(view_img=False,
                 reg_pts=region_points,
                 classes_names=model.names,
                 draw_tracks=True,
                 line_thickness=2)

while cap.isOpened():

    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break
    tracks = model.track(im0, persist=True, show=False, stream=True)
    for r in tracks: 
        im0 = counter.start_counting(im0, r)

If I use stream, the tracks will be generator objects - so I have to use a for loop. However, object_coutner throws an error

im0 = counter.start_counting(im0, track)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path>\ultralytics\solutions\object_counter.py", line 275, in start_counting
    self.extract_and_process_tracks(tracks)  # draw region even if no objects
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path>s\ultralytics\solutions\object_counter.py", line 172, in extract_and_process_tracks
    if tracks[0].boxes.id is not None:
       ~~~~~~^^^
  File "<path>\ultralytics\engine\results.py", line 126, in __getitem__
    return self._apply("__getitem__", idx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path>\ultralytics\engine\results.py", line 163, in _apply
    setattr(r, k, getattr(v, fn)(*args, **kwargs))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path>\ultralytics\engine\results.py", line 63, in __getitem__
    return self.__class__(self.data[idx], self.orig_shape)
                          ~~~~~~~~~^^^^^
IndexError: index 0 is out of bounds for dimension 0 with size 0
Exception ignored in: <generator object BasePredictor.stream_inference at 0x000001CF609531C0>
Traceback (most recent call last):
  File "<path>\torch\utils\_contextlib.py", line 44, in generator_context
  File "<path>\torch\autograd\grad_mode.py", line 275, in clone
  File "<path>\torch\autograd\grad_mode.py", line 256, in __init__
AttributeError: 'NoneType' object has no attribute '_jit_internal'

How to fix it?

[glenn-jocher]

Hey there! Thanks for reaching out with your questions. Let's quickly address them. 😊

1. Best Input Resolution for Performance: It's often a balance between speed and accuracy. Commonly, an input size of 640x640 is used for a good compromise. For more fine-grained details, increase the resolution, bearing in mind the computational cost.

2. Code Example for Object Counting: Absolutely! Here's a quick snippet to get you started with object counting. This uses the object_counter solution from Ultralytics:

from ultralytics import YOLO
from ultralytics.solutions import object_counter
import cv2

model = YOLO("yolov8n.pt")
cap = cv2.VideoCapture("path/to/video.mp4")
assert cap.isOpened(), "Error reading video file"

counter = object_counter.ObjectCounter()

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break
    tracks = model.track(frame, persist=True)
    frame = counter.start_counting(frame, tracks)
    # Process frame (e.g., display or save)

cap.release()

Don't forget to adjust the region_points or line_points according to your use case. 🚀

3. Identifying Bottlenecks: A great way to check for bottlenecks is by using tools like htop for CPU monitoring and nvidia-smi for GPU. They'll help you understand resource utilization. Profiling your code with Python's cProfile or a similar tool can also provide insights into which part of your code is consuming the most time.

The utilization and performance of hardware resources like GPUs often depend on a complex interplay of factors including memory bandwidth, computational complexity of the tasks, and efficiency of the underlying implementations. Performance might not always align with GPU memory consumption as subtle nuances in the computation or I/O operations (like reading from disk) can be the actual limitations.

Hope this helps! Dive into the code, experiment, and feel free to reach out if you've got more questions. Happy coding! 🌟

[Githubprojectsacc]

Hi!@glenn-jocher I have some doubt regarding the hyperparameter tuning for YOLOv8 model.I have just posted my question in the discussion page.can u pls verify that

[kalwinskidawid]

@glenn-jocher Thanks, but I would like to know if it is possible to use the stream=True parameter in model.track with object_counter.start_counting - you gave a basic example from docs.
The second issue is that if I pass live from RTPS as input, it stops processing after a short while because cap.read() returns false - I've already tried doing continue instead of break in if not ret but that doesn't help.

[glenn-jocher]

Hey @kalwinskidawid! Thanks for reaching out with your questions. Let's address them:

1. Using stream=True with object_counter.start_counting: Currently, the object_counter.start_counting is designed to process individual frames rather than a stream of frames directly. When using stream=True, you indeed get a generator, which object_counter.start_counting isn't set up to handle directly. You would need to modify your loop to handle frames one at a time as they come from the generator.

2. RTSP Stream Stops Processing: If cap.read() returns False, it typically indicates no more frames are available or there's an issue with the stream. Using continue instead of break won't help if the stream itself is interrupted or has stopped sending data. Ensure your RTSP source is stable and continuously streaming. Sometimes network issues or source-side buffering settings can cause interruptions.

For your coding issue with the stream=True, it seems like there's a misunderstanding in handling the generator. Here's a corrected approach:

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Waiting for video frame...")
        continue  # Skip processing if frame isn't ready

    tracks_generator = model.track(im0, persist=True, show=False, stream=True)
    for tracks in tracks_generator:
        if tracks.boxes:
            im0 = counter.start_counting(im0, tracks)
            video_writer.write(im0)
        else:
            print("No objects detected in the current frame.")

This setup ensures that you handle frames as they become available and only attempt to count objects when detections are present.

Hope this helps! Let me know if there's anything else you need. 😊

[Githubprojectsacc]

Hi,@glenn-jocher If I select a set of hyperparameters like activation function,momentum,learning rate,batch size,optimizer,num_layers for tuning the model.After selecting the best hyperparameters ,I need to store it in a seperate yaml file like best_hyperparamter.yaml to pass into the training the model using cfg attribute to the model.train method.But If I passed it I get an error for some hyperparameter like activation function,num_layers. So,I saw some comments regarding the activation function for the model they described to put into the yolov8n.yaml file configuration by setting for example: activation: relu and then we need to load the modeifiedyolov8n.yaml file for further training.But can we able to pass without that method and also how should I set the num_layers hyperparameter for the model training.

[glenn-jocher]

Hi there! 🙌 It seems you're exploring advanced customization for your YOLOv8 model training by adjusting hyperparameters like the activation function, momentum, learning rate, etc., and facing some issues when trying to store and use them. Let's address your concerns:

1. Storing Hyperparameters: It's a good practice to save your best-performing hyperparameters in a yaml file, such as best_hyperparameters.yaml. For training-related parameters like learning rate, momentum, batch size, and optimizer, this approach should work seamlessly when you pass the file using the cfg attribute in model.train().

2. Activation Function & num_layers: For modifications like activation functions and num_layers, which directly relate to the model architecture, modifying the model definition file (yolov8n.yaml for example) is indeed the correct approach. This is because these parameters dictate how the model is built, unlike other hyperparameters that influence the training process itself.

   To modify the activation function across the model, you'd add or modify the activation key in your model definition yaml file. For num_layers, the adjustment depends on what exactly you're aiming to change—usually, changing the depth of the model requires adding or removing layers in this file.

3. Passing Model Architecture Hyperparameters: Currently, there isn't a method to pass architecture-related parameters (like num_layers and custom activation functions) directly through the training command or a separate config file that exclusively deals with training hyperparameters. The architecture modifications need to be done in the model's yaml config.

Here's a quick example of how you might adjust the activation function in a model definition file:

# Example model definition in yolov8n.yaml
nc: 80  # number of classes
depth_multiple: 0.33  # model depth
width_multiple: 0.50  # layer channel width

# Backbone
backbone:
  # [from, number, module, args]
  [[-1, 1, Focus, [64, 3]],  # 0-P0/4
   ...]

# Changing activation function
activation: relu

Remember, after modifying the model's yaml file, you should point to this modified file when training:

yolo detect train data=coco128.yaml model=modified_yolov8n.yaml epochs=100 imgsz=640

Hope this clears up the issues! If you have further questions or run into more complex scenarios, feel free to ask. Happy modeling! 🚀

[Githubprojectsacc]

Hey@glenn-jocher I had performed hyperparameter tuning and got the best hyperparameters for example,
Best hyperparameters: {'lr0': 0.08298087567118992, 'lrf': 0.9581907264917989, 'batch_size': 16, 'momentum': 0.8038698076704441, 'activation': 'tanh', 'optimizer': 'Adam', 'weight_decay': 0.00021037546468872538, 'warmup_epochs': 3.6269052385459415, 'warmup_momentum': 0.00392050995297643}

Best evaluation metric: 0.1017250973486669.

Although the metrics are same to some batches in the trial but not all,some are different but why some trials proceed with same metric values. For just trying out I took small epochs values and I tried for 30 trials ......

Then I stored the values in best_hyperparameter(bh) file and I also took the default hyperparmeter in yolov8 configuration and stored it as default.yaml.(df)Then I performed individual training for some epochs and I took the evaluation of both sets.
default set values :
lr0: 0.01 lrf: 0.01 momentum: 0.937 weight_decay: 0.0005 warmup_epochs: 3.0 warmup_momentum: 0.8 warmup_bias_lr: 0.1

But I have noticed in the (bh) set the evaluation is somewhat lower than the default. But while tuning I saw the metric value of that set (bh)hyperparameters is good Best evaluation metric: 0.1017250973486669 ( similar or greater than default ).But performing individually applying those hyperparameters the metric value changed it showed less than Best evaluation metric: 0.1017250973486669.But I need to prove (bh)this set is the best hyperparameter than the default. Is there any step missing here for perfect hyperparameter tuning or I need to perform any steps further. Can u pls provide me with some code examples to clear it out correctly.

I have another doubt while performing the training the training the entire configuration gets overrided with nc=2 the class number that I gave in the yaml file. Also the convolution layer parameters like kernel,padding got changed from its default.
from n params module arguments
0 -1 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
1 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
2 -1 1 7360 ultralytics.nn.modules.block.C2f [32, 32, 1, True]
3 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
4 -1 2 49664 ultralytics.nn.modules.block.C2f [64, 64, 2, True]
5 -1 1 73984 ultralytics.nn.modules.conv.Conv [64, 128, 3, 2]
6 -1 2 197632 ultralytics.nn.modules.block.C2f [128, 128, 2, True]
7 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
8 -1 1 460288 ultralytics.nn.modules.block.C2f [256, 256, 1, True]
9 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
10 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
11 [-1, 6] 1 0 ultralytics.nn.modules.conv.Concat [1]
12 -1 1 148224 ultralytics.nn.modules.block.C2f [384, 128, 1]
13 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
14 [-1, 4] 1 0 ultralytics.nn.modules.conv.Concat [1]
15 -1 1 37248 ultralytics.nn.modules.block.C2f [192, 64, 1]
16 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
17 [-1, 12] 1 0 ultralytics.nn.modules.conv.Concat [1]
18 -1 1 123648 ultralytics.nn.modules.block.C2f [192, 128, 1]
19 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
20 [-1, 9] 1 0 ultralytics.nn.modules.conv.Concat [1]
21 -1 1 493056 ultralytics.nn.modules.block.C2f [384, 256, 1]
22 [15, 18, 21] 1 751702 ultralytics.nn.modules.head.Detect [2, [64, 128, 256]]
Model summary: 225 layers, 3011238 parameters, 3011222 gradients, 8.2 GFLOPs

I need to modify the default yolov8 model to get good performance model.Can I utilise this confiuration to change my yolov8n.yaml file configuration for modifying the model. But I dont know how to correctly modify the values.
default :
backbone:

[from, repeats, module, args]

• [-1, 1, Conv, [64, 3, 2]] # 0-P1/2
• [-1, 1, Conv, [128, 3, 2]] # 1-P2/4
• [-1, 3, C2f, [128, True]]
• [-1, 1, Conv, [256, 3, 2]] # 3-P3/8
• [-1, 6, C2f, [256, True]]
• [-1, 1, Conv, [512, 3, 2]] # 5-P4/16
• [-1, 6, C2f, [512, True]]
• [-1, 1, Conv, [1024, 3, 2]] # 7-P5/32
• [-1, 3, C2f, [1024, True]]
• [-1, 1, SPPF, [1024, 5]] # 9

YOLOv8.0n head

head:

• [-1, 1, nn.Upsample, [None, 2, "nearest"]]

• [[-1, 6], 1, Concat, [1]] # cat backbone P4

• [-1, 3, C2f, [512]] # 12

• [-1, 1, nn.Upsample, [None, 2, "nearest"]]

• [[-1, 4], 1, Concat, [1]] # cat backbone P3

• [-1, 3, C2f, [256]] # 15 (P3/8-small)

• [-1, 1, Conv, [256, 3, 2]]

• [[-1, 12], 1, Concat, [1]] # cat head P4

• [-1, 3, C2f, [512]] # 18 (P4/16-medium)

• [-1, 1, Conv, [512, 3, 2]]

• [[-1, 9], 1, Concat, [1]] # cat head P5

• [-1, 3, C2f, [1024]] # 21 (P5/32-large)

• [[15, 18, 21], 1, Detect, [nc]] # Detect(P3, P4, P5)

[glenn-jocher]

Hey there! Thanks for diving deep into hyperparameter tuning with YOLOv8 and sharing your results 🚀. It seems like you've encountered a bit of a puzzle when comparing your tuned hyperparameters (bh set) with the defaults.

Regarding the observed differences in evaluation metrics, remember that multiple factors influence the model's performance, like dataset diversity, initialization states, and even minor changes in training conditions. Even with the same hyperparameters, results can slightly vary between runs due to these factors.

For hyperparameters like activation functions and num_layers, these are indeed structural changes to the model architecture rather than training configurations. To modify these, you should adjust your model definition file (yolov8n.yaml) like so:

backbone:
  # Example: Change activation to tanh for all layers
  [[-1, 1, Focus, [64, 3]],  # 0-P0/4
   ...
   activation: tanh]

Unfortunately, direct CLI or config file adjustments for structural hyperparameters aren’t supported—you’ll need to manually adjust the model config file.

As for num_layers, there's no direct hyperparameter since the model's depth and complexity are determined by the defined layers and blocks within the yaml config. You could theoretically "add" layers by expanding the model definition, but it requires a good understanding of the architecture.

Lastly, your observed discrepancy in performance could be due to overfitting, evaluation on a different subset, or slight stochastic variations in training. I recommend cross-validating with different sets of data or applying regularization techniques to see if the new hyperparameters consistently outperform the defaults across varied conditions.

For custom modifications causing architecture alterations like changing convolution layer parameters (kernel, padding), these are specified within the YAML file:

[[-1, 1, Conv, [64, 3, 2, 2]]]  # Example syntax: [input_layer_index, count, Module, [filters, kernel_size, stride, padding]]

Adjusting these parameters directly impacts how the model processes input images.

Key takeaway: While hyperparameter tuning is powerful, aligning model architecture tweaks with training hyperparameters necessitates thoughtful adjustments and testing. Keep exploring and refining to achieve the best model performance. 🌟

Hope this helps! Keep up the great work, and feel free to ask if anything else comes up.

[kalwinskidawid]

Hey,
I'm using ultralystic on the GPU, but looking at the CPU preview it's 1 or 2 threads running at 100% all the time - which parts of ultralystic are running on the GPU and which on the CPU? How to check why GPU ulitization never hits 100%? It oscillates around 60%, PCI-E bandwitch at 10%, while GPU memory utilization is at 20-30%, could the problem here be the 128bit memory-bus?

Generally without saving that to file I could get around 6-7 fps more. The live preview took another few fps.

[kalwinskidawid]

I may have found the main problem model.track does not support mulithreading so... CPU is a bottleneck.

[glenn-jocher]

Hi,

Thanks for pointing this out. Multithreading isn't natively supported in the model.track function, which can indeed lead to the CPU becoming a bottleneck if it's managing multiple heavy tasks like video decoding or handling high FPS inputs.

A potential workaround to enhance performance could be to implement your threading logic outside of the model tracking, managing the video capture and preprocessing in separate threads while feeding the prepared frames to the model in the main thread. However, care must be taken to synchronize the data appropriately to avoid conflicts.

Here is a simplified example of how you might manage frames with threading, assuming you have a way to safely share frames between threads:

import cv2
from threading import Thread
from queue import Queue
from ultralytics import YOLO

def capture_frames(source, frame_queue):
    cap = cv2.VideoCapture(source)
    while cap.isOpened():
        ret, frame = cap.read()
        if ret:
            frame_queue.put(frame)
        else:
            break
    cap.release()

def process_frames(frame_queue, model):
    while True:
        if not frame_queue.empty():
            frame = frame_queue.get()
            results = model.track(frame, persist=True)
            # Process results...

model = YOLO('yolov8n.pt')
frame_queue = Queue()

# Start the frame capture thread
thread = Thread(target=capture_frames, args=('path/to/video.mp4', frame_queue))
thread.start()

# Start processing frames
process_frames(frame_queue, model)

This setup can help offload some of the video I/O operations to another thread, potentially reducing the load on the main thread running the model track operations. This could help minimize CPU bottlenecks.

Regarding GPU utilization not hitting 100%, this can be normal especially if the CPU is bottlenecked, or if the data feeding rate into the GPU is the limiting factor rather than the computation itself. The memory bus width could be a factor if the model is very large or the batch sizes are big, but with the numbers you've noted, it more likely indicates that the GPU is waiting on data or other operations to complete.

As always, profiling tools for both CPU and GPU can give more insights into the exact bottlenecks.