Skip to content
/ SSL Public

[NIPS 2016] Learning Structured Sparsity in Deep Neural Networks

License

Notifications You must be signed in to change notification settings

ZJCV/SSL

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Language: 🇺🇸 🇨🇳

«SSL» re-implements the paper Learning Structured Sparsity in Deep Neural Networks

In addition to the different pruning positions mentioned in the paper (filter_wise/channel_wise/filter_and_channel_wise/depth_wise), the warehouse also tried different weight functions (group_lasso/mean_abs/mean/sum_abs/sum).

See

More training statistics can see:

Table of Contents

Background

Based on Group Lasso, SSL can achieve Filter/Channel/Filter Shape/Depth pruning.

Installation

$ pip install -r requirements.txt

Usage

First, you need set env for PYTHONPATH and CUDA_VISIBLE_DEVICES

$ export PYTHONPATH=<project root path>
$ export CUDA_VISIBLE_DEVICES=0

Then, begin train-prune-finetuning

  • For train
$ python tools/train.py -cfg=configs/vggnet/vgg16_bn_cifar100_224_e100_sgd_mslr_ssl_filter_wise_1e_5.yaml
  • For prune
$ python tools/prune/prune_vggnet.py
  • For fine-tuning
$ python tools/train.py -cfg=configs/vggnet/refine_mean_abs_0_2_vgg16_bn_cifar100_224_e100_sgd_mslr_ssl_filter_wise_1e_5.yaml

Finally, set the fine-tuning model path in the PRELOADED option of the configuration file

$ python tools/test.py -cfg=configs/vggnet/refine_mean_abs_0_2_vgg16_bn_cifar100_224_e100_sgd_mslr_ssl_filter_wise_1e_5.yaml

Maintainers

  • zhujian - Initial work - zjykzj

Thanks

@misc{wen2016learning,
      title={Learning Structured Sparsity in Deep Neural Networks}, 
      author={Wei Wen and Chunpeng Wu and Yandan Wang and Yiran Chen and Hai Li},
      year={2016},
      eprint={1608.03665},
      archivePrefix={arXiv},
      primaryClass={cs.NE}
}

Contributing

Anyone's participation is welcome! Open an issue or submit PRs.

Small note:

License

Apache License 2.0 © 2021 zjykzj