awesome_music_generation.md

Awesome music generation

• Awesome music generation
  • Papers
  • Misc
  • Datasets
  • Products
  • Other
  • Extra reference

Papers

• DITTO-2: Distilled Diffusion Inference-Time T-Optimization for Music Generation, arXiv, 2405.20289, arxiv, pdf, cication: -1

  Zachary Novack, Julian McAuley, Taylor Berg-Kirkpatrick, Nicholas Bryan · (ditto-music.github)

• Instruct-MusicGen: Unlocking Text-to-Music Editing for Music Language Models via Instruction Tuning, arXiv, 2405.18386, arxiv, pdf, cication: -1

  Yixiao Zhang, Yukara Ikemiya, Woosung Choi, Naoki Murata, Marco A. Martínez-Ramírez, Liwei Lin, Gus Xia, Wei-Hsiang Liao, Yuki Mitsufuji, Simon Dixon

  · (instruct-musicgen - ldzhangyx)

• Naturalistic Music Decoding from EEG Data via Latent Diffusion Models, arXiv, 2405.09062, arxiv, pdf, cication: -1

  Emilian Postolache, Natalia Polouliakh, Hiroaki Kitano, Akima Connelly, Emanuele Rodolà, Taketo Akama

• Music Consistency Models, arXiv, 2404.13358, arxiv, pdf, cication: -1

  Zhengcong Fei, Mingyuan Fan, Junshi Huang

• Long-form music generation with latent diffusion, arXiv, 2404.10301, arxiv, pdf, cication: -1

  Zach Evans, Julian D. Parker, CJ Carr, Zack Zukowski, Josiah Taylor, Jordi Pons

• Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompaniment, arXiv, 2404.09313, arxiv, pdf, cication: -1

  Hong Zhiqing, Huang Rongjie, Cheng Xize, Wang Yongqi, Li Ruiqi, You Fuming, Zhao Zhou, Zhang Zhimeng · (text2songmelodist.github)

• MuPT: A Generative Symbolic Music Pretrained Transformer, arXiv, 2404.06393, arxiv, pdf, cication: -1

  Xingwei Qu, Yuelin Bai, Yinghao Ma, Ziya Zhou, Ka Man Lo, Jiaheng Liu, Ruibin Yuan, Lejun Min, Xueling Liu, Tianyu Zhang · (map-mupt.github)

• Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt, arXiv, 2403.11780, arxiv, pdf, cication: -1

  Yongqi Wang, Ruofan Hu, Rongjie Huang, Zhiqing Hong, Ruiqi Li, Wenrui Liu, Fuming You, Tao Jin, Zhou Zhao

  · (prompt-singer.github)

• MusicHiFi: Fast High-Fidelity Stereo Vocoding, arXiv, 2403.10493, arxiv, pdf, cication: -1

  Ge Zhu, Juan-Pablo Caceres, Zhiyao Duan, Nicholas J. Bryan

  · (musichifi.github)

• musiclang_predict - MusicLang

  AI Prediction api of the MusicLang package · (huggingface)

• ChatMusician: Understanding and Generating Music Intrinsically with LLM, arXiv, 2402.16153, arxiv, pdf, cication: -1

  Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou

• MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models, arXiv, 2402.06178, arxiv, pdf, cication: -1

  Yixiao Zhang, Yukara Ikemiya, Gus Xia, Naoki Murata, Marco Martínez, Wei-Hsiang Liao, Yuki Mitsufuji, Simon Dixon · (wry-neighbor-173.notion)

• MusicRL: Aligning Music Generation to Human Preferences, arXiv, 2402.04229, arxiv, pdf, cication: -1

  Geoffrey Cideron, Sertan Girgin, Mauro Verzetti, Damien Vincent, Matej Kastelic, Zalán Borsos, Brian McWilliams, Victor Ungureanu, Olivier Bachem, Olivier Pietquin · (google-research.github)

• DITTO: Diffusion Inference-Time T-Optimization for Music Generation, arXiv, 2401.12179, arxiv, pdf, cication: -1

  Zachary Novack, Julian McAuley, Taylor Berg-Kirkpatrick, Nicholas J. Bryan · (ditto-music.github)

• Masked Audio Generation using a Single Non-Autoregressive Transformer, arXiv, 2401.04577, arxiv, pdf, cication: -1

  Alon Ziv, Itai Gat, Gael Le Lan, Tal Remez, Felix Kreuk, Alexandre Défossez, Jade Copet, Gabriel Synnaeve, Yossi Adi · (pages.cs.huji.ac) · (audiocraft - facebookresearch) · (MAGNeT-colab - camenduru) · (huggingface)

• StemGen: A music generation model that listens, arXiv, 2312.08723, arxiv, pdf, cication: 1

  Julian D. Parker, Janne Spijkervet, Katerina Kosta, Furkan Yesiler, Boris Kuznetsov, Ju-Chiang Wang, Matt Avent, Jitong Chen, Duc Le

• M$^{2}$UGen: Multi-modal Music Understanding and Generation with the Power of Large Language Models, arXiv, 2311.11255, arxiv, pdf, cication: -1

  Atin Sakkeer Hussain, Shansong Liu, Chenshuo Sun, Ying Shan · (M2UGen - shansongliu) · (crypto-code.github)

• Mustango: Toward Controllable Text-to-Music Generation, arXiv, 2311.08355, arxiv, pdf, cication: -1

  Jan Melechovsky, Zixun Guo, Deepanway Ghosal, Navonil Majumder, Dorien Herremans, Soujanya Poria · (mustango - AMAAI-Lab) · (huggingface)

• Music ControlNet: Multiple Time-varying Controls for Music Generation, arXiv, 2311.07069, arxiv, pdf, cication: 1

  Shih-Lun Wu, Chris Donahue, Shinji Watanabe, Nicholas J. Bryan · (MusicControlNet.github)

• Controllable Music Production with Diffusion Models and Guidance Gradients, arXiv, 2311.00613, arxiv, pdf, cication: 1

  Mark Levy, Bruno Di Giorgi, Floris Weers, Angelos Katharopoulos, Tom Nickson

• Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative Editing, arXiv, 2310.12404, arxiv, pdf, cication: 1

  Yixiao Zhang, Akira Maezawa, Gus Xia, Kazuhiko Yamamoto, Simon Dixon

• MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Models, arXiv, 2310.11954, arxiv, pdf, cication: -1

  Dingyao Yu, Kaitao Song, Peiling Lu, Tianyu He, Xu Tan, Wei Ye, Shikun Zhang, Jiang Bian

• UniAudio: An Audio Foundation Model Toward Universal Audio Generation, arXiv, 2310.00704, arxiv, pdf, cication: 5

  Dongchao Yang, Jinchuan Tian, Xu Tan, Rongjie Huang, Songxiang Liu, Xuankai Chang, Jiatong Shi, Sheng Zhao, Jiang Bian, Xixin Wu · (UniAudio - yangdongchao) · (dongchaoyang)

• AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining, arXiv, 2308.05734, arxiv, pdf, cication: 7

  Haohe Liu, Qiao Tian, Yi Yuan, Xubo Liu, Xinhao Mei, Qiuqiang Kong, Yuping Wang, Wenwu Wang, Yuxuan Wang, Mark D. Plumbley · (AudioLDM2 - haoheliu) · (audioldm.github) · (huggingface)

• JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models, arXiv, 2308.04729, arxiv, pdf, cication: 6

  Peike Li, Boyu Chen, Yao Yao, Yikai Wang, Allen Wang, Alex Wang · (futureverse)

• From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion, arXiv, 2308.02560, arxiv, pdf, cication: 1

  Robin San Roman, Yossi Adi, Antoine Deleforge, Romain Serizel, Gabriel Synnaeve, Alexandre Défossez · (audiocraft_plus - GrandaddyShmax) · (arxiv) · (audiocraft - facebookresearch) · (huggingface) · (ai.honu) · (huggingface) · (huggingface)

• EmoGen: Eliminating Subjective Bias in Emotional Music Generation, arXiv, 2307.01229, arxiv, pdf, cication: -1

  Chenfei Kang, Peiling Lu, Botao Yu, Xu Tan, Wei Ye, Shikun Zhang, Jiang Bian · (ai-muzic.github) · (muzic - microsoft)

• VampNet: Music Generation via Masked Acoustic Token Modeling, arXiv, 2307.04686, arxiv, pdf, cication: 13

  Hugo Flores Garcia, Prem Seetharaman, Rithesh Kumar, Bryan Pardo · (vampnet - hugofloresgarcia)

• Anticipatory Music Transformer, arXiv, 2306.08620, arxiv, pdf, cication: 5

  John Thickstun, David Hall, Chris Donahue, Percy Liang · (anticipation - jthickstun)

• Efficient Neural Music Generation, arXiv, 2305.15719, arxiv, pdf, cication: 10

  Max W. Y. Lam, Qiao Tian, Tang Li, Zongyu Yin, Siyuan Feng, Ming Tu, Yuliang Ji, Rui Xia, Mingbo Ma, Xuchen Song · (efficient-melody.github)

• MusicLM: Generating Music From Text, arXiv, 2301.11325, arxiv, pdf, cication: 219

  Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi · (google-research.github) · (aitestkitchen.withgoogle)

• jukebox-diffusion - jmoso13

  · (betterprogramming)

• A Review of Intelligent Music Generation Systems, arXiv, 2211.09124, arxiv, pdf, cication: 4

  Lei Wang, Ziyi Zhao, Hanwei Liu, Junwei Pang, Yi Qin, Qidi Wu

Misc

• musicgen-songstarter-demo - artificialguybr 🤗

• rwkv-music - mrfakename 🤗

• Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model, arXiv, 2311.00968, arxiv, pdf, cication: -1

  Jaeyong Kang, Soujanya Poria, Dorien Herremans

• Controllable Music Production with Diffusion Models and Guidance Gradients, arXiv, 2311.00613, arxiv, pdf, cication: 1

  Mark Levy, Bruno Di Giorgi, Floris Weers, Angelos Katharopoulos, Tom Nickson

• Fast Diffusion GAN Model for Symbolic Music Generation Controlled by Emotions, arXiv, 2310.14040, arxiv, pdf, cication: -1

  Jincheng Zhang, György Fazekas, Charalampos Saitis

• riffusion - riffusion

  Stable diffusion for real-time music generation

• 🎵 The MusicBox - a fffiloni Collection

• ERNIE-Music: Text-to-Waveform Music Generation with Diffusion Models, arXiv, 2302.04456, arxiv, pdf, cication: 5

  Pengfei Zhu, Chao Pang, Yekun Chai, Lei Li, Shuohuan Wang, Yu Sun, Hao Tian, Hua Wu

• A Survey of AI Music Generation Tools and Models, arXiv, 2308.12982, arxiv, pdf, cication: 1

  Yueyue Zhu, Jared Baca, Banafsheh Rekabdar, Reza Rawassizadeh

• Moûsai: Text-to-Music Generation with Long-Context Latent Diffusion, arXiv, 2301.11757, arxiv, pdf, cication: 42

  Flavio Schneider, Ojasv Kamal, Zhijing Jin, Bernhard Schölkopf

Datasets

• The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation, arXiv, 2311.10057, arxiv, pdf, cication: -1

  Ilaria Manco, Benno Weck, SeungHeon Doh, Minz Won, Yixiao Zhang, Dmitry Bogdanov, Yusong Wu, Ke Chen, Philip Tovstogan, Emmanouil Benetos · (song-describer-dataset - mulab-mir) · (huggingface)

• MusicBench - amaai-lab 🤗

Products

• X

• Udio | Make your music

• Introducing Stable Audio 2.0

  · (mp.weixin.qq)

• Introducing v3 – Suno

  · (mp.weixin.qq)

• MusicFX

• AIVA, the AI Music Generation Assistant

• Musicfy AI

• Transforming the future of music creation - Google DeepMind

• 网易天音 - 一站式AI音乐创作工具 - 官网

• 音乐ChatGPT时刻来临！「天工SkyMusic」音乐大模型今日启动邀测

Other

• Site Unreachable
• Stable-Diffusion - FurkanGozukara

Extra reference

• awesome-deep-learning-music - ybayle

  List of articles related to deep learning applied to music