[BUG]: ns2_dataset.py does not have this two part, phones and num_frames, which must be need in ns2_trainer.py

[a897456]

Amphion/models/tts/naturalspeech2/ns2_dataset.py

Line 121 in 5cb75d8

self.utt2phone[utt] = utt_info["phones"]

Amphion/models/tts/naturalspeech2/ns2_dataset.py

Line 131 in 5cb75d8

self.utt2len[utt] = utt_info["num_frames"]

Amphion/models/tts/naturalspeech2/ns2_trainer.py

Line 269 in 5cb75d8

train_dataset.num_frame_indices,

These two elements are not integrated into train.json which will be used in ns2_trainer.py

[shreeshailgan]

I am also facing the same problem. You can work around this problem temporarily:

Amphion/models/tts/naturalspeech2/ns2_dataset.py

Line 121 in 5cb75d8

self.utt2phone[utt] = utt_info["phones"]

You can replace the above line with

with open(os.path.join(self.phone_dir, uid + ".phone"), "r") as f:
    self.utt2phone[utt] = f.read().strip()

while setting

self.phone_dir = os.path.join(processed_data_dir, 'phones')

in the __init__ of NS2Dataset

You can just comment out the parts containing frame counts because that is only being used to perform dynamic batching. Also, set "use_dynamic_batchsize": false in exp_config.json

[HeCheng0625]

Hi, you need to generate the phone sequence and record the number of frames of samples.

[shreeshailgan]

does number of frames mean the number of phones in the phone sequence?

[HarryHe11]

does number of frames mean the number of phones in the phone sequence?

Hi @shreeshailgan , according to the NS2 paper, "As shown in Figure 2, our neural audio codec consists of an audio encoder, a residual vector-quantizer (RVQ), and an audio decoder: 1) The audio encoder consists of several convolutional blocks with a total downsampling rate of 200 for 16KHz audio, i.e., each frame corresponds to a 12.5ms speech segment." You could refer to https://arxiv.org/pdf/2304.09116.pdf for more details.