Continual Learning Benchmark for KWS

We apply the continual learning as the training framework to improve the performance of continuous keyword spotting in the edge/mobile devices.

Here is a list of continual learning methods available for KWS,

• Elastic Weight Consolidation (EWC) [view]
• Online Elastic Weight Consolidation (Online EWC) [view]
• Synaptic Intelligence (SI) [view]
• Gradient Episodic Memory (GEM) [view]
• Advanced Gradient Episodic Memory (A-GEM) [view]
• Naive Rehearsal (NR) [view]
• Meta-Experience Replay (MER) [view]
• Progressive Neural Networks (PNN) [view]
• Temporal Progressive Net (PCL-KWS) [view]

Quick Start

Setup Environment

You need to create the running environment by Anaconda,

conda env create -f environment.yml
conda active kws

If you don't have Anaconda, you can install (Linux version) by

cd <ROOT>
bash conda_install.sh

Download Dataset

We use the Google Speech Commands Dataset (GSC) as the training data. By running the script, you can download the training data:

cd <ROOT>/dataset
bash download_gsc.sh

Note: to run the download_gsc.sh in macOS, please run brew install coreutils to install realpath first.

Start Training

We use Neptune to log the training process, please setup the logging configuration in each training script.

    ...
    # initialize Neptune
    neptune.init(<your workspace name>, api_token=<your token>)
    neptune.create_experiment(name=<your experiment name>, params=PARAMETERS)
    ...

Or you can just run the script without --log.

After the logging configuration, you are all set to run the training script by

python train.py

If you want to reproduce all the experiment results, you can simply run

bash ./run_all.sh

Which contains the hyperparameters of all the different CL methods.

Citation

If the work did some helps on your research/project, please cite our paper in ICASSP 2022. Thank you!

@inproceedings{huang2022progressive,
  title={Progressive Continual Learning for Spoken Keyword Spotting},
  author={Huang, Yizheng and Hou, Nana and Chen, Nancy F},
  booktitle={ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={7552--7556},
  year={2022},
  organization={IEEE}
}

License

The project is available as open source under the terms of the MIT License.