Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fine-tuning voice-cloning capability of metavoice #137

Open
abhijeethp opened this issue Apr 27, 2024 · 2 comments
Open

Fine-tuning voice-cloning capability of metavoice #137

abhijeethp opened this issue Apr 27, 2024 · 2 comments

Comments

@abhijeethp
Copy link

Hey Team,
Can anyone help me understand the following regarding the metavoice model fine-tuning process?
https://github.com/metavoiceio/metavoice-src/tree/main?tab=readme-ov-file#finetuning

  • For fine-tuning the mode what is the minimum and maximum audio length I can use that is allowed by the system?
  • The fine-tuning script takes only 2 files as input -- a speech (audio) file and it's transcription. How is this possible? is the SiSNR calculated against the same audio?
  • I want fine-tune the voice cloning aspect of metavoice if possible. Is there anything extra I need to implement to do this?
@abhijeethp abhijeethp changed the title Fine-tuning the cloning of metavoice Fine-tuning voice-cloning capability of metavoice Apr 27, 2024
@Arman12345677

This comment was marked as spam.

@lucapericlp
Copy link
Contributor

Hey @abhijeethp, sorry for only getting to this now, we've seen people finetuning using chunks of 5-10s audio in their training datasets (but it's not a hard range). We're not calculating SiSNR as part of finetuning - are you asking whether using the same audio is appropriate?

Re finetuning the voice cloning, you should be all good if you follow the finetuning guide with a solid dataset & play around with the hyperparameters and then use a good reference clip upon inference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants