Skip to content

BBAM: Bounding Box Attribution Map for Weakly Supervised Semantic and Instance Segmentation (CVPR 2021)

License

Notifications You must be signed in to change notification settings

jbeomlee93/BBAM

Repository files navigation

BBAM: Bounding Box Attribution Map for Weakly Supervised Semantic and Instance Segmentation

The implementation of BBAM: Bounding Box Attribution Map for Weakly Supervised Semantic and Instance Segmentation, Jihun Yi, Chaehun Shin, and Sungroh Yoon, CVPR 2021. [paper]

Input image and predicted box for 'banana' (red box) BBAM over iterations
a b

Installation

We kindly refer to INSTALLATION from maskrcnn-benchmark.

  • In our case, the build is successful only when the CUDA version of the system matches the version of the cuda-toolkit of the conda environment.
  • When you encounter a 'gcc' failed error, this can help.
  • This repository is tested on Ubuntu 18.04, with Python 3.6, PyTorch 1.4, pydensecrf, scipy, imageio, and opencv-python, joblib, and scikit-image.

Examples

Step 1. Prepare Dataset

  • Download PASCAL VOC 2012 benchmark: Download

Step 2. Prepare pre-trained object detector

Step 3. Obtain BBAMs for PASCAL VOC validation images

python tools/BBAM/BBAM_FRCNN.py --gpu 0 --img_idx 30 --visualize True

img_idx denotes image index of PASCAL VOC validation images to be explained, ranging from 0 to 1448 (inclusive).

Step 4. Obtain pseudo ground-truth masks for PASCAL VOC training images

  • Obtain BBAMs for training images
python tools/BBAM/BBAM_training_images_multi.py
  • Create COCO style annotation files for training Mask R-CNN
python tools/BBAM/make_annotation/make_cocoann_topCRF_parallel.py
  • (Optional) Refine pseudo ground-truth masks with pre-computed MCG proposals. You can download pre-computed MCG proposals at here
python tools/BBAM/make_annotation/BBAM_plus_mcg.py
  • Create semantic segmentation annotations for training DeepLab-v2
python tools/BBAM/make_annotation/create_semantic_labels.py

Step 5. Train an instance segmentation network.

CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 tools/train_net.py --config-file configs/pascal_voc/BBAM_VOC_aug_FPN_R101_MaskRCNN.yaml
  • Evaluation
CUDA_VISIBLE_DEVICES=0 python tools/test_net.py --config-file "configs/pascal_voc/BBAM_VOC_aug_FPN_R101_MaskRCNN.yaml" --ckpt BBAM_Mask_RCNN_logs_mcg85/model_final.pth TEST.IMS_PER_BATCH 1
  • (Optional) Evaluation with CRFs
python tools/BBAM/CRF_for_prediction.py

Step 6. Train a semantic segmentation network.

Reference

About

BBAM: Bounding Box Attribution Map for Weakly Supervised Semantic and Instance Segmentation (CVPR 2021)

Topics

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published