Ultralytics
tasks/segment/ #8734
giscus
bot
Replies: 37 comments 85 replies
-
|
I have a question about segmentation dataset and really wish you can answer, my dataset contains lots of information including bbox and segmentations, however it has wrong category_id and all categories have been marked as the same category, does this has bad influence on finetuing? If so, how can I fix that? |
Beta Was this translation helpful? Give feedback.
-
|
@jrryzh hey there! 👋 Absolutely, having incorrect To fix this, you'll need to correct the For instance, if you're working with a JSON format similar to COCO, you could use Python to update the category IDs: import json
# Load your dataset
with open('your_dataset.json') as f:
data = json.load(f)
# Assuming all categories are incorrectly marked as category_id 1 and you want to update them to correct IDs
correct_mapping = {1: 'new_correct_id'} # Update this with your correct category IDs
# Update category IDs
for ann in data['annotations']:
if ann['category_id'] in correct_mapping:
ann['category_id'] = correct_mapping[ann['category_id']]
# Save the corrected dataset
with open('your_corrected_dataset.json', 'w') as f:
json.dump(data, f)Remember to replace If you're working with segmentation, our Instance Segmentation guide might be helpful. It covers how to use our segmentation models, like Hope this helps! Let us know if you have any more questions. 😊 |
Beta Was this translation helpful? Give feedback.
-
|
How can I run my code on GPU ? From the processor the recognition is freezing |
Beta Was this translation helpful? Give feedback.
-
|
@Boaruzhanchik hey there! 😊 To run your code on a GPU, you typically just need to ensure your model is set to use the GPU. With Ultralytics YOLO, your model should automatically utilize a CUDA-compatible GPU if available. Here's a quick check to make sure: from ultralytics import YOLO
# Make sure 'device' is set to 'cuda' if you want to use GPU (default if CUDA is available)
model = YOLO('yolov8n.pt', device='cuda')This small addition directs the model to use the GPU. Make sure you have a CUDA-compatible Nvidia GPU and have installed the necessary CUDA and CuDNN libraries. If recognition is freezing, it's usually due to the heavy computation not being efficiently processed by the CPU. Running it on the GPU should significantly speed things up! 🚀 If CUDA is correctly set up and you're still encountering issues, make sure to check your GPU drivers and CUDA installation. Keep me posted on how it goes or if you run into any further questions! |
Beta Was this translation helpful? Give feedback.
-
|
Hi, i'm install pythorch and have problem when program detect object Traceback (most recent call last): CPU: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\cpu\nms_kernel.cpp:112 [kernel] |
Beta Was this translation helpful? Give feedback.
-
|
Sorry for the inconvenience, everything was solved, thank you ! |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 😊 Glad to hear everything got sorted out. If you ever run into another hiccup or have any questions, please don't hesitate to reach out. Wishing you continued success with your projects! If instance segmentation or anything else piques your interest, our Ultralytics Docs are there to help guide you along. Cheers! 🎉 |
Beta Was this translation helpful? Give feedback.
-
|
I am building a streamlit app on YOLOv8 models, |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 😊 For displaying the result or the annotated image/video in your Streamlit app using YOLOv8 models, you can follow this approach: After running the prediction with a YOLOv8 model, you get a from ultralytics import YOLO
import streamlit as st
from PIL import Image
import numpy as np
# Load a YOLO model
model = YOLO('yolov8n.pt') # Or your custom model
# Run prediction
results = model('path/to/image.jpg') # Replace with your source
# Convert results image to PIL Image for Streamlit
annotated_img = Image.fromarray(results[0].plot()[..., ::-1]) # Convert BGR to RGB
# Display the image in Streamlit
st.image(annotated_img, caption='Annotated Image')Replace Hope this helps you integrate YOLOv8 with your Streamlit app smoothly! 🚀 Let me know if you have any more questions. |
Beta Was this translation helpful? Give feedback.
-
|
has the ability to use image masks instead of contour points as labels ever been implemented ? |
Beta Was this translation helpful? Give feedback.
-
|
Hello! 👋 Yes, Ultralytics YOLO models have the ability to perform instance segmentation, which can utilize image masks to segment individual objects in an image. This is particularly useful for datasets where contour points might not adequately represent the object due to internal gaps or complex shapes. For instance segmentation, you can use models with the Here's a quick example on how to predict with a pretrained segmentation model: from ultralytics import YOLO
# Load a segmentation model
model = YOLO('yolov8n-seg.pt')
# Predict with the model
results = model('https://ultralytics.com/images/bus.jpg')This will give you both the bounding boxes and segmentation masks for detected objects, handling cases where simple contours are not sufficient. 🎭 Let us know if you need further assistance or examples! |
Beta Was this translation helpful? Give feedback.
-
|
Hi there! I am using the pre-trained segmentation model by Yolov8. I have an image that contains several objects, and I only want to receive the mask of any person present in the frame, while ignoring all the other masks. My requirements:
|
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 For your requirement to only get the person mask from an image using the YOLOv8 pre-trained segmentation model, and ignoring other detections, you can do something like this: from ultralytics import YOLO
# Load the YOLOv8 segmentation model
model = YOLO('yolov8n-seg.pt')
# Perform prediction
results = model('path/to/your/image.jpg')
# Filter for 'person' class masks only (class_id for 'person' is usually 0)
person_masks = results.masks[results.boxes.cls == 0]
# Loop through each person mask to process/visualize as needed
for mask in person_masks:
# Here, 'mask' represents individual person masks
# You can proceed to draw contours or other operations as required
# Save or visualize person_masks as needed, maintaining original image dimensionsThis snippet is a starting point. The By utilizing this approach, you'll be able to:
Hope this helps! Let me know if you need further assistance. 😊 |
Beta Was this translation helpful? Give feedback.
-
|
results.masks gives an attribute error. It was fixed with results[0].masks. I have the tensor containing the mask data of the person through person_masks.data. Just help me save the masked image now, that only contains mask of the detected person. (There is always going to be one person in my usecase) |
Beta Was this translation helpful? Give feedback.
-
|
Hey! 👋 Great to hear you've successfully isolated the person mask using YOLOv8 segmentation! To save the mask image with the same spatial dimensions as the original, here is a quick way: import cv2
import torch
# Assuming `person_masks.data` is your mask tensor
# Convert mask to a binary image (0 or 255), change the shape to match the original image height and width
mask_image = person_masks.data.squeeze().cpu().numpy() * 255 # Adjust the multiplication factor as necessary
mask_image = mask_image.astype(np.uint8)
# Save the mask image
cv2.imwrite('path/to/save/person_mask.jpg', mask_image)This code snippet takes your Let me know if this helps or if you have any further questions! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
I have a question, how do you visualize the segmentation mask with the detected label? it seems it's not discussed in the documentation, it just shows here how to predict using it but not visualizing it with an image. I even checked the list of results but it seems its just returning segmentation masks, not the classes that it predicts, with that said my question is can you visualize the segmentation mask with a label only because of what i see in the tutorial it has a bounding box too which i think that's how you put the label there because i think the segmentation masks are not directly related in the class ID, my goal is to detect object with segmentation and put label to it without bounding box. looking forward to your answer thank you |
Beta Was this translation helpful? Give feedback.
-
|
Hello there! 👋 To visualize the segmentation mask with its corresponding label (without bounding boxes), you can use the from ultralytics import YOLO
import matplotlib.pyplot as plt
# Load a pretrained segmentation model
model = YOLO('yolov8n-seg.pt')
# Predict on an image and return a Results object
results = model('https://ultralytics.com/images/bus.jpg')
# Visualize segmentation with labels (without bounding boxes)
fig = plt.figure(figsize=(10, 10))
plt.imshow(results.render()[0]) # `render()` method overlays masks & class labels onto the image
plt.axis('off')
plt.show()This generates an output where each segmentation mask is color-coded, and the class labels are overlaid directly on the masks. The Unfortunately, our default method integrates both segmentation masks and bounding boxes for comprehensive detection visualization. If you specifically want only the masks with labels and without the bounding boxes in your plots, the code tweak above should help. Hope this answers your question! 😊 If you have any further queries or need more insights, feel free to reach out. |
Beta Was this translation helpful? Give feedback.
-
|
I have a question. Can anyone suggest me how can I save the predicted bounding box and segmented mask in the format of : |
Beta Was this translation helpful? Give feedback.
-
|
@utokyo-sm96 hey there! 😊 To save the predicted bounding box and segmentation mask in your specified format, you can iterate over the detected objects and their corresponding masks to build the desired string for each detection. Below is a Python code snippet using the Ultralytics YOLO model to demonstrate how you could achieve this: from ultralytics import YOLO
# Load your YOLOv8 segment model
model = YOLO('yolov8n-seg.pt')
# Run a prediction
results = model('path/to/your/image.jpg')
# Iterate through results to format and save your detections and masks
for i, det in enumerate(results[0].boxes.xywh):
class_id = int(det[4])
bbox_center_x, bbox_center_y, bbox_width, bbox_height = det[:4]
segment = results[0].masks.xyn[i] # Get mask points for this detection
# Build the desired format string
output_str = f"{class_id} {bbox_center_x} {bbox_center_y} {bbox_width} {bbox_height}"
for x, y in zip(segment[0::2], segment[1::2]): # Iterate over points in mask
output_str += f" {x} {y}"
print(output_str) # Or write to a fileThis code will print out the details for each detection in your specified format. Note that Hope this helps! Let me know if you need further assistance. |
Beta Was this translation helpful? Give feedback.
-
|
When I try to predict some images using the best weights of this trained model (!python3 segment/predict.py --save-crop --save-txt) I expect to get predicted segment coordinates in the saved txt files as: |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 It looks like you're encountering an issue where you're expecting segmentation polygon coordinates in the For including segmentation mask coordinates in your output, you'd need to programmatically access the segmentation masks from the model's output and convert them into your desired format. Here's a quick example on how you might approach it: from ultralytics import YOLO
import numpy as np
model = YOLO('yolov8n-seg.pt') # Load your segment model
results = model('path/to/image.jpg') # Perform inference
for i, (box, mask) in enumerate(zip(results[0].boxes, results[0].masks)):
class_id = box[4].int()
bbox = box[:4] # bbox coordinates (center_x, center_y, width, height)
polygon = mask.to_polygons()[0] # Assuming one polygon per mask for simplification
# Convert polygon to your desired format here
# Example: polygon might need flattening and conversion to (x, y) pairs
flat_polygon = np.array(polygon).flatten()
output_line = f"{class_id} {bbox[0]} {bbox[1]} {bbox[2]} {bbox[3]} " + ' '.join(map(str, flat_polygon))
# Now write output_line to your .txt fileThis is just a starter; you might need to adjust the mask processing part depending on your exact needs. Remember, the actual implementation details may vary as this example assumes direct access to methods for converting masks to polygons which might not exist as is and you may need to implement or adapt them based on the format of the masks. Don't hesitate to reach out again if you need more help! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
` ` if len(det): ` Masks Shape: torch.Size([3, 640, 640]) can you help me out where am i wrong. |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 It looks like your script is running into an issue where it's not properly identifying the segmentation masks, as indicated by "Number of non-zero values in class mask: 0" and "Found 0 polygons." This typically happens if the predicted masks are not properly generated or if there's an issue in the conversion process from the mask tensor to the segmentation polygons. A couple of things you might want to check:
If the masks are indeed being generated but not captured correctly, you might need to adjust the process of converting masks to polygons. Remember, masks are typically provided as binary arrays (or tensors), where you need to find contours and then approximate those contours to polygons. Here's a simplified approach to debugging and trying to fix your issue:
Also, for your specific case of wanting to write the segmentation polygons alongside bounding box details, ensure your Lastly, remember that each segmentation mask corresponds to one detected object, so ensure your processing loop correctly matches each mask with its corresponding detection. If after checking these aspects you still face issues, consider sharing a minimal but complete snippet of how you're processing the image through the model and attempting to save the results. It might offer more clues on where things might be going astray. Hope this helps you to debug the situation! Keep experimenting, and don't hesitate to reach out for more guidance. 😊 |
Beta Was this translation helpful? Give feedback.
-
|
I have an important question, i want going to train yolov8 model for instance segmentation but i need to save all the segmented mask on my disk after training the model. how i can get that |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! Saving segmented masks to disk with Ultralytics YOLO is doable, and arranging them by class labels adds a nice level of organization for further training or analysis. 🧐 Here's a quick guide to help you out:
from ultralytics import YOLO
# Load your trained segmentation model
model = YOLO('path/to/yolov8n-seg.pt')
# Predict on an image, replace with your image path or URL
results = model('path/to/image.jpg')
# Save results, this includes saving masks
results.save()
Here's a very basic starter: import os
import numpy as np
# Assuming 'results' from the prediction step
for det in results.xyxy[0]: # Iterate through detections
class_id = int(det[-2]) # Class ID for detected object
class_name = results.names[class_id] # Class name
mask = results.masks[class_id].numpy() # Convert mask to numpy array
# Create directory for the class if it doesn't exist
os.makedirs(class_name, exist_ok=True)
# Filename for the mask
mask_path = os.path.join(class_name, f"{det[5]}.png") # det[5] is the unique ID, you might adjust this
# Save mask, you may need to adjust the saving depending on mask format
np.save(mask_path, mask)Note: You'll need to adjust the looping and saving part based on your actual needs and mask data structure. Remember, getting into the specifics might require tweaking according to your dataset, and the exact implementation might vary. If you're dealing with a custom dataset or more complex requirements, you might need to dive deeper into handling the output data. Happy coding! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
Query: |
Beta Was this translation helpful? Give feedback.
-
|
@utokyo-sm96 hello! For saving instance segmentation predictions, including masks, Here's a quick example on how you might go about this for saving segmentation masks as images, but you can adapt the logic to save in your preferred format: from ultralytics import YOLO
import numpy as np
from PIL import Image
# Load a model
model = YOLO('yolov8n-seg.pt') # Assuming you're using a segmentation model
# Predict with the model
results = model('path/to/image.jpg')
# Save masks
for i, mask in enumerate(results[0].masks):
img = Image.fromarray((mask.numpy() * 255).astype(np.uint8))
img.save(f'mask_{i}.png')In this example, each object's mask is saved as a separate Let us know if you need further assistance! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
hi i am trying to get each mask in a separate photo and i used this code why an i getting this error |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! It looks like you're encountering an error because you're trying to directly operate on the You're on the right track with wanting to convert each mask to an image format! However, you need to access the numpy representation of each mask by iterating over the masks within the from ultralytics import YOLO
from PIL import Image
# Assuming you're using a YOLOv8 segmentation model
model = YOLO('yolov8n-seg.pt')
results = model('path/to/your/image.jpg')
# Iterate over each detected object's mask
for i, mask in enumerate(results[0].masks.numpy()):
# Convert mask to an image
mask_img = Image.fromarray((mask * 255).astype(np.uint8))
# Save each mask as an image file
mask_img.save(f'mask_{i}.png')In this adjusted approach, Hope this helps! Let me know if you have further questions. 😄 |
Beta Was this translation helpful? Give feedback.
-
|
Hello, |
Beta Was this translation helpful? Give feedback.
-
|
Hello 😊! To get the mIoU (mean Intersection over Union) for each epoch end during training/validation with our segmentation models (e.g., from ultralytics import YOLO
# Load your segmentation model
model = YOLO('yolov8n-seg.pt') # using an official segmentation model for example
# Perform validation (after training or directly if the model is pre-trained)
metrics = model.val() # Assumption: model and dataset are correctly configured
# Access mIoU (mean IoU for segmentation)
miou = metrics.seg.map # This is your mIoU value
print(f'mIoU: {miou}')This will give you the mIoU for segmentation masks at the end of each validation phase, which is typically run after each training epoch. Hope this helps! If you have further questions or need more assistance, feel free to ask. 🌟 |
Beta Was this translation helpful? Give feedback.
-
|
Hi there! I'd like to share with you a project I've recently worked on. Together with a colleague, we've created a repository that serves as a tool with a SAHI-like inference but specifically tailored for instance segmentation tasks. Our repository allows for segmenting small objects in images by combining mask predictions from various overlapping patches. We support both YOLOv8-seg and FastSAM. Additionally, we have a variant for object detection tasks, and the key distinction from SAHI is the support for all the current models from the Ultralytics team: YOLOv9, YOLOv8, RTDTR, and others. I'm a huge fan of Ultralytics, so I'd be thrilled to assist you if you're interested in our project. I'm confident that for many people, the task of finding a large number of segments would be beneficial, especially when using standard Ultralytics models. Here's the link to the project: YOLO-Patch-Based-Inference. Honestly, I wasn't sure where to share information about the project, so I decided to start with the YOLO-seg support section, as it seemed like the most obvious choice. I would greatly appreciate your feedback on the potential usefulness of our project for you. Thank you! Example: |
Beta Was this translation helpful? Give feedback.
-
|
Hello there! Wow, your project sounds absolutely fascinating! 🚀 The idea of tailoring SAHI-like inference for instance segmentation, and supporting an impressive range of Ultralytics models, including YOLOv8-seg and FastSAM, is a commendable effort. It's particularly exciting that you're extending support to the latest models like YOLOv9, RTDTR, and so forth. Your repository is a brilliant testament to the passion and expertise in the community. I'm thrilled to see your innovative use of our models, and I'm confident that your project could significantly benefit those needing refined segmentation, especially for small objects. Your approach to combining mask predictions from overlapping patches is a smart solution to a common challenge in instance segmentation. I've taken a look at your YOLO-Patch-Based-Inference repo, and it’s clearly well-thought-out. The instance segmentation community will surely find tremendous value in the tools you've developed. Sharing it in the YOLO-seg support section was a great choice! If there's anything more we can do to support your project or if you have further questions or insights, feel free to reach out. Feedback like yours is invaluable—it helps us continue to innovate and support projects that enhance the capabilities of Ultralytics models. Once again, thank you for your enthusiasm for Ultralytics and for sharing your project with us. Keep up the fantastic work! 🌟 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much for your feedback. |
Beta Was this translation helpful? Give feedback.
-
|
Hello! Thank you so much for reaching out and offering to collaborate on documenting your project for Ultralytics documentation. Your initiative aligns perfectly with our goal of enriching the community with valuable resources and tools. 😊 Given the potential impact and the innovative approach of your SAHI-like tool for instance segmentation, featuring support for Ultralytics models including YOLOv8-seg, FastSAM, and more, it sounds like something that would greatly interest and benefit our users. Documenting your solution in a manner similar to our SAHI guide could indeed foster wider adoption and assist others in their instance segmentation endeavors. Let's connect to discuss how we can work together on creating this documentation. Please feel free to reach out to us via our repository's "Discussions" section or directly via email with a brief outline or proposal for the documentation content. This will help us kickstart the process and determine how best to feature your work within our documentation ecosystem. Again, thank you for your enthusiasm and looking forward to collaborating with you! 🌟 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much. I wrote a proposal in the "Discussions" section - https://github.com/orgs/ultralytics/discussions/9381 |
Beta Was this translation helpful? Give feedback.
-
|
@Koldim2001 that's fantastic! 😊 Your proposal in the "Discussions" section here is a great step forward, and I appreciate your willingness to contribute and help enhance the Ultralytics documentation. Your project's unique approach to instance segmentation, especially for small objects and full support for our model range, sounds like a valuable resource for our users. Collaborating on similar documentation to what we've done with SAHI would not only benefit our community but also raise awareness of your innovative solution. Let's definitely explore this further. Please expect a follow-up in the "Discussions" section as we discuss next steps and how we can integrate your project into our ecosystem effectively. Thank you for reaching out and looking forward to working together! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
YOLO V8 has pre-trained models that are performed on the COCO dataset. However, I am working on a project for image segmentation where the classes are not available in the COCO dataset. In this case, do I need to train a model from scratch? About the dataset: Looking for an answer. Thankyou! |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 😊 It's great that you've already annotated your custom dataset in YOLOv8 format, especially for such an impactful project involving property damage assessment from satellite and aerial imagery. Given your custom classes are outside the COCO dataset, you're on the right track thinking about training a model specifically for your use case. The good news is, you don't need to start completely from scratch! You can leverage one of our pre-trained segmentation models like Here's a quick example of how you might kick off your training process: from ultralytics import YOLO
# Initialize your model for training
model = YOLO('yolov8n-seg.pt') # Load a pre-trained segmentation model
# Fine-tune the model on your custom dataset
results = model.train(data='your_custom_dataset.yaml', epochs=100, imgsz=640)Ensure your dataset YAML file is correctly set up with paths to your training and validation data, classes, etc. This method will help your model learn the specific classes and damage levels you're interested in! Best of luck with your project, and don't hesitate to reach out if you have more questions. Happy detecting! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Thanks a lot again. |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 Since your custom dataset annotations are ready and in the proper YOLOv8 format, you're all set to start fine-tuning one of our pre-trained segmentation models, such as from ultralytics import YOLO
# Load the pre-trained model
model = YOLO('yolov8n-seg.pt')
# Fine-tune the model on your custom dataset
results = model.train(data='your_custom_dataset.yaml', epochs=100, imgsz=640)Make sure your |
Beta Was this translation helpful? Give feedback.
-
|
Is Panoptic Segmentation supported by Yolov8 or in any version of Yolo? |
Beta Was this translation helpful? Give feedback.
-
|
Hello! At the moment, YOLOv8 specifically supports Instance Segmentation with models like For panoptic segmentation, which combines both semantic segmentation (segmenting all pixels in an image) and instance segmentation (distinguishing between individual object instances), we're always exploring new features and capabilities to incorporate into our models. Your interest is definitely noted! 🚀 If you're specifically looking to work with panoptic segmentation, I'd encourage staying tuned to our updates. Also, feel free to contribute or suggest improvements; we welcome collaboration from the community! 🤝 Here's a quick example of how you can work with our segmentation models: from ultralytics import YOLO
# Load a pretrained segment model
model = YOLO('yolov8n-seg.pt')
# Segment objects in an image
results = model('https://ultralytics.com/images/bus.jpg')
# Visualize results
results.show()Let us know if there's anything else we can help you with! |
Beta Was this translation helpful? Give feedback.
-
|
I have trained custom yolov8 model for Instance segmentation on video. Now I have to make video inference for the same please guide me. And I have to deploy it give me guidance on these. |
Beta Was this translation helpful? Give feedback.
-
|
@mayurkatre18 hey there! It sounds like you've trained a custom YOLOv8 model for instance segmentation on video, which is great 😊. For video inference and deployment, here's a quick guide to get you started: Video InferenceTo make video inferences with your trained model, you can use the from ultralytics import YOLO
# Load your trained model
model = YOLO('path/to/your/custom-model.pt')
# Predict on a video
results = model('path/to/your/video/file.mp4')
# Iterate over results if needed
for result in results:
# Process your results here
result.show() # Display the frame with predictionsThis example assumes you have a video file. If you’re processing real-time video streams, you’ll read frames from the stream and pass them to DeploymentDepending on where you want to deploy your model, you might need to export it to a different format that suits the deployment environment. Supported export formats include ONNX, TensorRT, CoreML, etc. Here’s how to export your model to ONNX, which is widely supported: # Assuming you’ve loaded your model as shown above
model.export(format='onnx')This command will generate an ONNX file that you can then move to your production environment. For detailed guidance on deployment platforms, it greatly depends on your target environment (e.g., web, mobile, edge devices). Make sure the environment supports the format you export your model to. I hope this helps get you started with video inference and deployment! If you have any more questions or need further assistance, feel free to ask. 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hi, I have a question. I have trained a custom model based on a pre trained one to segment the shirts from cyclists. The problem however is that it recognizes 2 slightly different shirt on 1 image. My idea was to only use the one with the highest confidence score, but this object appears to be none (results.probs or results.keypoint are both none objects). Is it possible to get the confidence score or do I have to apply a different methode? |
Beta Was this translation helpful? Give feedback.
-
|
@pietervhij hey there! 👋 With only 100 training images, there's indeed a risk of overfitting when training over a large number of epochs like 100, especially for complex tasks like instance segmentation. Here are a couple of tips to mitigate that:
Training on a smaller Keep innovating! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the response, i have a little followup question. You say Ultralytics supports augmentations directly in the training pipeline. What do you mean by this? Is there a parameter that could implement this? Or is it you should adjust your dataset beforehand? |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 🤗 Great follow-up question. Ultralytics YOLO integrates data augmentations directly into the training pipeline, meaning you don't have to modify your dataset beforehand. These augmentations are applied on-the-fly during training to enhance the diversity of your dataset and improve model generalization. When you train your model using the No need to set a specific parameter for basic augmentations; they're handled for you by default! However, you can customize augmentations to some extent using the training configuration file ( If you've missed it, don't worry. It's a fairly common question, and the details are indeed more implicit within the documentation. Always feel free to ask for clarifications! 😄 Happy training! |
Beta Was this translation helpful? Give feedback.
-
|
So if I understand correctly, when i train my model with following code, the ultralytics YOLO is already implementing augmentations? from ultralytics import YOLO model = YOLO("yolov8n-seg.pt") # load a pretrained model (recommended for training) freeze_layers = 20 results = model.train(data="shirt.yaml", epochs=50) # train the model The yaml file looks as following: train: "train/images" roboflow: |
Beta Was this translation helpful? Give feedback.
-
|
Yes, you've got it right! 😊 When you train your model with the Ultralytics YOLO framework, data augmentations are indeed automatically applied during training. This helps enhance the diversity of your dataset, which is especially valuable for small datasets or to improve model generalization. Here's a quick look at your training setup: from ultralytics import YOLO
model = YOLO("yolov8n-seg.pt") # Pretrained model loaded
# ... your freezing logic ...
results = model.train(data="shirt.yaml", epochs=50) # Augmentations are applied here!In this setup, augmentations such as random scaling, cropping, and flipping are employed. This is seamlessly handled by the Ultralytics framework, so there's no need for manual augmentation of your dataset before training. Your training configuration through Keep up the great work on your project! If you have more questions or need further assistance, feel free to ask. |
Beta Was this translation helpful? Give feedback.
-
|
when I run |
Beta Was this translation helpful? Give feedback.
-
|
@chenchen-boop hi there! 👋 The values you're seeing in the saved For getting each corner of a rectangle or trapezoid, you'll be interested in the def convert_to_corners(img_width, img_height, center_x, center_y, width, height):
x1 = int((center_x - width / 2) * img_width)
x2 = int((center_x + width / 2) * img_width)
y1 = int((center_y - height / 2) * img_height)
y2 = int((center_y + height / 2) * img_height)
return x1, y1, x2, y2
# Example usage
img_width, img_height = 640, 640 # Adjust these to your actual image dimensions
center_x, center_y, width, height = 0.70408, 0.421875, 0.421875, 0.707081 # Normalize values from your .txt
corners = convert_to_corners(img_width, img_height, center_x, center_y, width, height)
print(corners) # prints (x1, y1, x2, y2) of the bounding box in pixel coordinatesMake sure to replace the |
Beta Was this translation helpful? Give feedback.
-
|
But my my model is for Instance Segmentation, what did u means width height ? |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 😊 It seems like you're diving into the details of instance segmentation with YOLOv8, and you've got some data in a text file that you're curious about. Based on the continuous data you provided, your guess is pretty spot on. For instance segmentation tasks, the format indeed goes beyond the simple class ID and bounding box. The format For converting these polygon points into a more rectangular representation (if needed), you would typically calculate the axis-aligned bounding box that encompasses all the points or compute the minimum bounding rectangle that covers all these points for a tighter fit. However, keep in mind that these transformations might not fully capture the nuanced shape of more complex objects indicated by the segmentation points. Feel free to ask if you have more questions or need further clarification! Happy coding! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you,I will try it.😁 |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 😁 Happy to hear you'll give it a try! You're right, for instance segmentation models, the format If you need to extract or visualize these points, the coordinates in your To visualize or work with these points in Python, consider using libraries like Here’s a snippet to get you started with plotting using import matplotlib.pyplot as plt
import numpy as np
# Assuming 'data' is your list of classId, posX1, posY1, posX2, posY2, ...
data = [2, 0.423438, 0.70408, 0.421875, 0.707081, ... ] # Your data here
classId = data[0]
points = np.array(data[1:]).reshape(-1, 2) # Reshaping the points into a 2D array
plt.figure()
plt.plot(points[:, 0], points[:, 1], 'r-') # Plot the points as a line
plt.fill(points[:, 0], points[:, 1], 'r', alpha=0.2) # Fill the polygon
plt.title(f"Segmentation of class {classId}")
plt.show()Remember, each Keep exploring and happy coding! If you have more questions or need further assistance, feel free to ask! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hey~ After training my dataset with YOLOv8, I found jagged edges on the mask in the results. Do you have any improvement methods? |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 😊 It seems like you're experiencing jagged edges on your segmentation masks. A common method to smooth out these edges is to employ morphological operations like erosion or dilation, which can be performed using OpenCV or similar libraries. Here's a quick example using OpenCV in Python: import cv2
import numpy as np
# Assuming 'mask' is your binary segmentation mask
kernel = np.ones((5,5),np.uint8)
mask_smoothed = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)You can adjust the kernel size and experiment with different morphological operations ( Adjusting your training parameters or increasing the resolution can also help improve mask quality. However, if you're looking for finer control over segmentation outputs, considering a model like Let the YOLO community know how it goes, and best of luck! |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! |
Beta Was this translation helpful? Give feedback.
-
|
Hey Simon, Great to hear about your interest in working with segmentation using YOLO! ✨ For converting YOLO format segmentation annotations to more universally accepted formats like COCO, you might want to explore the Unfortunately, direct conversion tools for YOLO formatted segmentation annotations may not be widely available, primarily due to how unique the YOLO's new segmentation format might be. However, script customization or a bit of manual coding could bridge the gap. Consider parsing the YOLO annotations to extract segmentation data and then reformatting it according to the COCO schema. Here's a basic Python snippet as a starting point for a custom script: import json
# Assuming 'yolo_annots' is a list of your YOLO annotations
# and 'coco_annots_structure' is a dict structured like a COCO annotations file
coco_annots_structure['annotations'] = []
for idx, yolo_annot in enumerate(yolo_annots):
coco_annot = {
"id": idx,
"image_id": yolo_annot['image_id'],
"category_id": yolo_annot['class_id'],
"segmentation": yolo_annot['segmentation_points'], # Format as per COCO
"area": yolo_annot['area'], # Calculate if not provided
"bbox": yolo_annot['bbox'], # Convert from YOLO to COCO format if necessary
"iscrowd": 0 or 1,
}
coco_annots_structure['annotations'].append(coco_annot)
with open('coco_formatted_annots.json', 'w') as f:
json.dump(coco_annots_structure, f)This script needs to be customized based on your dataset's structure and the exact nature of your annotations. The key takeaway is mapping between the formats and ensuring that the segmentation points are correctly converted into the COCO format. Unfortunately, the specifics of the conversion will largely depend on the details of your YOLO segmentation annotations. If your project heavily relies on this conversion, developing a bespoke script might be the best way forward. Best of luck with your segmentation projects, and don't hesitate to reach out if you have more questions! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hi! Is TTA available for the segmentation task? TypeError Traceback (most recent call last) File ~/.local/lib/python3.10/site-packages/ultralytics/engine/model.py:452, in Model.predict(self, source, stream, predictor, **kwargs) File ~/.local/lib/python3.10/site-packages/ultralytics/engine/predictor.py:168, in BasePredictor.call(self, source, model, stream, *args, **kwargs) File ~/.local/lib/python3.10/site-packages/torch/utils/_contextlib.py:35, in _wrap_generator..generator_context(*args, **kwargs) File ~/.local/lib/python3.10/site-packages/ultralytics/engine/predictor.py:255, in BasePredictor.stream_inference(self, source, model, *args, **kwargs) File ~/.local/lib/python3.10/site-packages/ultralytics/models/yolo/segment/predict.py:54, in SegmentationPredictor.postprocess(self, preds, img, orig_imgs) TypeError: 'NoneType' object is not subscriptable |
Beta Was this translation helpful? Give feedback.
-
|
@kruzaeva hello! Test-Time Augmentation (TTA) is not currently supported for the segmentation tasks using For segmentation tasks using YOLOv8 models such as from ultralytics import YOLO
# Load a segmentation model
model = YOLO('yolov8n-seg.pt') # Make sure to use a segmentation model
# Predict with the model
results = model('path/to/your/image.jpg') # Path to your image
results.show() # To display the resultsThis approach ensures that the segmentation model runs without the TTA feature, allowing you to perform instance segmentation effectively. If you have further questions or need more assistance, feel free to ask! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
Thanks a lot! Any plans on implementing TTA and OBB it in the near future for instance segmentation? |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 🌞 Thanks for reaching out with your question. Currently, Test-Time Augmentation (TTA) and Oriented Bounding Boxes (OBB) are not implemented for instance segmentation tasks in YOLOv8. Our focus has been on refining and optimizing segmentation models like That being said, we're always looking to improve and expand the capabilities of our models based on community feedback! While I can't provide a specific timeline, I suggest keeping an eye on our updates for any future enhancements in this area. In the meantime, if you're looking to dive into instance segmentation with YOLOv8, our pretrained Segment models, such as For any further questions or ideas, feel free to drop us a message. Happy coding! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
I have a training problem. After improving the yolov8 network, the training speed is normal, but the accuracy and loss calculation becomes very slow after the training, which takes about half an hour, and normally it may take one minute. It is worth noting that I concat the feature map of the shallow network with the feature map of the deep network. I am doing the instance splitting task on coco2017 dataset. The following is the yaml file of the network structure and the record during the training process: [from, repeats, module, args]
YOLOv8.0n headhead:
- [[15, 18, 21], 1, Detect, [nc]] # Detect(P3, P4, P5)
|
Beta Was this translation helpful? Give feedback.
-
|
@LeaderOfFuture hey! It looks like you're dealing with a bottleneck during the loss calculation and accuracy evaluation phase of your training, especially after introducing modifications for instance segmentation on the COCO2017 dataset. When concatenating feature maps from different network depths, it's critical to ensure that the increase in computational complexity doesn't disproportionately affect evaluation phases. A potential quick check you can do is to ensure that any tensor operations, especially concatenations, are as efficient as possible and that unnecessary operations are minimized or eliminated. For instance, carefully manage the dimensionality and size of the feature maps you're concatenating to avoid excessive computational load. Regarding your specific setup with the feature map concatenation and the network configuration you've provided, here's a lightweight suggestion focused on optimizing your custom layers (like # Assuming these are your custom layers with increased computational demand
# - [-1, 1, ASPP_improve, [1024]] # 9
# - [-1, 1, CBAM, [1024, 7]]
# Consider revising their internal operations for efficiency,
# possibly by reducing the redundancy in operations or optimizing the implementationOne approach is profiling your model to identify exactly where the bottlenecks occur during training. Tools like PyTorch's Lastly, modifying the network architecture for tasks like instance segmentation often requires fine-tuning and optimization post-modification to maintain a balance between complexity and performance. Ensure your amendments align well with the computational resources available and the objectives of your instance segmentation task. If you find these steps don't alleviate the slowdown, consider sharing more details on the implementation of your custom modifications for further insights. 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
This is code of ASPP_improve and CBAM. class ASPP_improve(nn.Module): =======================================================================
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks for sharing the details of your implementation with ASPP_improve and CBAM modules! It looks like your architecture modifications are quite compute-intensive, particularly due to the use of atrous convolutions and feature concatenations across multiple levels of the network. To potentially improve the training efficiency:
You might also look into profiling tools available in PyTorch ( Keep in mind that such deep modifications may sometimes require trade-offs between accuracy and inference speed. If you have any more questions or require further assistance, feel free to ask. Keep up the great work! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Can you use the track function with the segment models? |
Beta Was this translation helpful? Give feedback.
-
|
@pietervhij currently, the If tracking individual segmented objects is a requirement, you might need to implement a custom tracking layer on top of the segmentation results. This could involve assigning and updating identities based on the segmentation masks' overlaps or other continuity criteria between frames. For more insights or if you'd like an example of how this can be approached, please let us know! |
Beta Was this translation helpful? Give feedback.
-
|
Hello there, i am trying to segment some trees in an image and would like to use the mask of these seperate trees to have a cut out image of each of these trees individually. |
Beta Was this translation helpful? Give feedback.
-
|
@pietervhij hello! 👋 It looks like you're almost there with your code to extract individual tree images using segmentation masks. To save each mask as an individual image, you'll need to iterate through each mask and apply it to the original image. Here's a revised approach to ensure all tree masks are processed: results = model(frame)
if results is not None:
for result in results:
if result.masks is not None:
for i, mask in enumerate(result.masks.data):
# Apply the mask to the image to extract the tree
tree_image = frame * mask[..., None] # mask needs to be 3D to apply
# Save each tree image
cv2.imwrite(f'tree_{i}.jpg', tree_image) # Ensure each tree has a unique filenameThis code snippet generates a |
Beta Was this translation helpful? Give feedback.
-
|
hi for the segmentation model can I specify which class I want detected? |
Beta Was this translation helpful? Give feedback.
-
|
Hi there! 👋 For the segmentation models, specifically selecting which classes to detect isn't supported directly via an argument in the model's prediction method. However, you can filter the classes after you perform the predictions. Here’s a small example using Python to demonstrate how you can filter specific class detections after the model has made its predictions: from ultralytics import YOLO
# Load the segmentation model
model = YOLO('yolov8n-seg.pt')
# Run predictions
results = model('path/to/image.jpg')
# Filter results for a specific class, e.g., class ID for 'person' might be 0
filtered_results = [x for x in results if x.cls == 0]
# Now you can work with 'filtered_results' containing only the classes you specifiedThis example assumes you know the class ID you wish to filter for. Let me know if you need further guidance! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
i have pretrained detection model, how to use a pretrained detection model for segmentation training and inference. thank you in advance. |
Beta Was this translation helpful? Give feedback.
-
|
Hello! If you want to train a detection model on a dataset with mask segment annotations and get segmentation masks as output, you'll need to use a segmentation model instead of a detection model. Here’s how you can do it: Firstly, ensure you're using a segmentation model, such as from ultralytics import YOLO
# Load a pretrained segmentation model
model = YOLO('yolov8n-seg.pt')
# Train the model on your segmentation dataset
results = model.train(data='your_dataset.yaml', epochs=100, imgsz=640)For inference, to get segmentation masks, you can use the same model: # Predict with the model
results = model('path_to_your_image.jpg')
masks = results.masks # This will contain the segmentation masksMake sure your dataset is properly formatted and the model used is specifically for segmentation tasks, as regular detection models will not output masks. Refer to the Ultralytics documentation on training segmentation models for detailed instructions and guidelines. If you need further help, feel free to ask. Good luck with your model training and inference! 😊🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hello, I am building models for brain tumor segmentation from MRI images on the BRATS dataset. I have a custom subset of dataset as my training images and it is saved as a numpy array currently. The array is of shape (128, 128, 4) and its values are 0 or 1 representing whether the pixel is of the corresponding class (I have 4 classes). I want to test YOLO models for segmentation on that dataset but it requires data in a specific format that defines a contour by pixels. I do not have much time as this is part of a research project due a few weeks and thus I do not want to use a tool like Roboflow to draw contours for all my data. Is there an automated solution for transforming my binary segmentation masks to a format YOLO models can use? |
Beta Was this translation helpful? Give feedback.
-
|
Hello! For converting your binary segmentation masks into a format compatible with YOLO models, you can use a script to generate the required contours from your masks. Here’s a basic example using Python with OpenCV to help you get started: import numpy as np
import cv2
# Example mask array
mask = np.random.randint(0, 2, (128, 128, 4), dtype=np.uint8)
# Convert mask to contours
contours_list = []
for i in range(mask.shape[2]):
contours, _ = cv2.findContours(mask[:, :, i], cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours_list.append(contours)
# Example: Access contours for the first class
first_class_contours = contours_list[0]
for contour in first_class_contours:
# Here you can format and save each contour as needed for YOLO
print(contour)This script will help you extract contours for each class from your masks. You'll need to format these contours according to the YOLO dataset format, typically involving saving each contour's bounding box and class information in a text file corresponding to each image. If you need further customization or assistance, feel free to ask. Good luck with your project! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
How to use the segmentation function of YOLOv8 to segment a panoramic image and a target image simultaneously, and then use feature point matching to perform feature point matching on the segmented object in both images. Match feature points only between segmented objects in two images |
Beta Was this translation helpful? Give feedback.
-
|
To perform instance segmentation on both a panoramic image and a target image simultaneously using YOLOv8, and then match feature points between segmented objects, you can follow these steps:
This workflow allows you to segment objects in both images, extract and match features only within those segments, ensuring that the feature matching is constrained to the segmented objects. |
Beta Was this translation helpful? Give feedback.
-
|
Please tell me the correct format for calling the modified yolov8 seg. yaml |
Beta Was this translation helpful? Give feedback.
-
|
@Daniel-lucifer to call a modified Pythonfrom ultralytics import YOLO
# Load and train a modified segmentation model
model = YOLO('path/to/your_modified-yolov8-seg.yaml')
results = model.train(data='your_dataset.yaml', epochs=100, imgsz=640)CLIyolo segment train data=your_dataset.yaml model=path/to/your_modified-yolov8-seg.yaml epochs=100 imgsz=640Make sure the path to your YAML file is correct and that the YAML file is properly configured for segmentation tasks. If you have further questions or need more detailed assistance, feel free to ask! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
How to use examples to segment and identify objects such as rings. |
Beta Was this translation helpful? Give feedback.
-
|
@shiliyu123 to segment and identify objects like rings using YOLOv8, you can utilize the pretrained segmentation models such as from ultralytics import YOLO
# Load the pretrained segmentation model
model = YOLO('yolov8n-seg.pt')
# Predict with the model on an image
results = model('path/to/your/image.jpg')
# Visualize the segmentation results
results.show()This will generate segmentation masks for each detected object in the image, including rings if they are present and similar to the objects the model was trained on (COCO dataset). For custom objects like specific types of rings, you might consider fine-tuning the model on a custom dataset that includes these objects. For more detailed guidance, please refer to our segmentation documentation. 🌟 |
Beta Was this translation helpful? Give feedback.
-
|
There will be a background area inside the ring. How to deal with such a label? |
Beta Was this translation helpful? Give feedback.
-
|
Hello! For handling the background area inside the ring in your segmentation task, you can treat it as a separate class in your dataset. When labeling your data, create a distinct mask for the ring and another mask for the interior background. This way, the model can learn to differentiate between the ring and its internal area. Here's a quick example of how you might set up your training data for this scenario: # Example of dataset annotation for segmentation
- image: path/to/image.jpg
labels:
- class: ring
segmentation: [[x1, y1, x2, y2, ..., xn, yn]] # Coordinates for the ring's outer boundary
- class: ring_background
segmentation: [[x1, y1, x2, y2, ..., xn, yn]] # Coordinates for the ring's inner boundaryBy training your model with such annotations, it should be able to segment both the ring and its background effectively. For more detailed guidance on setting up your segmentation model, please refer to our segmentation documentation. Happy modeling! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
What is the best opensource annotation tool which can install locally for image segmentation for YOLO, |
Beta Was this translation helpful? Give feedback.
-
|
@aditcodiant for local image segmentation annotation compatible with YOLO format, you might want to try LabelMe or MakeSense.ai. Both tools offer polygonal segmentation and can be configured to export data in a format suitable for YOLO. After annotating your images, you can convert these annotations to YOLO format using the JSON2YOLO tool provided by Ultralytics, which is designed to handle such conversions efficiently. For more details on handling segmentation tasks with YOLO, you can refer to our guide on Instance Segmentation. This should provide you with further insights into preparing your data and using it with YOLOv8 models. |
Beta Was this translation helpful? Give feedback.
-
|
@glenn-jocher thanks for the reply |
Beta Was this translation helpful? Give feedback.
-
|
You're welcome! 😊 For instance segmentation, YOLOv8 Segment models are a great choice. They use the Trainfrom ultralytics import YOLO
# Load a pretrained model
model = YOLO("yolov8n-seg.pt")
# Train the model
results = model.train(data="coco8-seg.yaml", epochs=100, imgsz=640)Predictfrom ultralytics import YOLO
# Load a pretrained model
model = YOLO("yolov8n-seg.pt")
# Predict with the model
results = model("https://ultralytics.com/images/bus.jpg")For more detailed information, you can refer to the Instance Segmentation page. If you have any more questions, feel free to ask! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
@glenn-jocher |
Beta Was this translation helpful? Give feedback.
-
|
How to extract feature points on segmented objects |
Beta Was this translation helpful? Give feedback.
-
|
@Daniel-lucifer to extract feature points from segmented objects using a YOLOv8 segmentation model, you can follow these steps:
This approach allows you to identify the shapes and feature points of each segmented object. For more detailed guidance on segmentation tasks, please refer to the Ultralytics segmentation documentation. |
Beta Was this translation helpful? Give feedback.
-
|
hi, I trained a custom YOLO segmentation model, I want to get midpoint of each mask, how can I do that? |
Beta Was this translation helpful? Give feedback.
-
|
@MoAbbasid hello! To find the midpoint of each mask from your custom YOLO segmentation model, you can calculate the centroid of each mask. Here's a simple way to do it using Python: import numpy as np
# Assuming 'masks' is a list of mask arrays from your model's output
for mask in masks:
# Find indices where mask is True
y_indices, x_indices = np.where(mask)
# Calculate centroid
x_center = np.mean(x_indices)
y_center = np.mean(y_indices)
print(f"Centroid: ({x_center}, {y_center})")This code snippet will print the centroid coordinates for each mask, assuming that your masks are binary arrays where the object's pixels are For more detailed guidance on working with segmentation models, check out the Ultralytics segmentation documentation. 📘 |
Beta Was this translation helpful? Give feedback.
and others
-
tasks/segment/
Learn how to use instance segmentation models with Ultralytics YOLO. Instructions on training, validation, image prediction, and model export.
https://docs.ultralytics.com/tasks/segment/
Beta Was this translation helpful? Give feedback.
All reactions