Skip to content

Error occurred when changing the feature extraction method to wavlm. #39

Description

@greatnoble

@yeyupiaoling

Thank you so much for sharing your research findings.

The feature extraction method based on Fbank was trained normally and I was able to confirm results similar to the results you shared.

<cam++.yml>

  1. use_hf_model: True
  2. feature_method: 'microsoft/wavlm-base-plus'

However, when changing the feature extraction method to wavlm, there are cases where training is suddenly stopped.

I would like to ask for advice on the error.

`2025-08-11 02:04:38.377 | INFO | macls.optimizer:build_optimizer:16 - 成功创建优化方法:Adam,参数为:{'lr': 0.001, 'weight_decay': 1e-05}

2025-08-11 02:04:38.377 | INFO | macls.optimizer:build_lr_scheduler:31 - 成功创建学习率衰减:WarmupCosineSchedulerLR,参数为:{'min_lr': 1e-05, 'max_lr': 0.001, 'warmup_epoch': 5, 'fix_epoch': 60, 'step_per_epoch': 245}

2025-08-11 02:04:38.377 | INFO | macls.trainer:train:335 - 训练数据:7858
/root/miniconda3/lib/python3.12/site-packages/torch/nn/functional.py:5849: UserWarning: Support for mismatched key_padding_mask and attn_mask is deprecated. Use same type for both instead.

warnings.warn(
2025-08-11 02:04:42.653 | INFO | macls.trainer:__train_epoch:278 - Train epoch: [1/60], batch: [1/245], loss: 2.78939, accuracy: 0.09375, learning rate: 0.00001000, speed: 1.87 data/sec, eta: 17:27:25

2025-08-11 02:05:16.676 | INFO | macls.trainer:__train_epoch:278 - Train epoch: [1/60], batch: [11/245], loss: 3.13291, accuracy: 0.26562, learning rate: 0.00001808, speed: 2.35 data/sec, eta: 13:52:54

2025-08-11 02:05:50.317 | INFO | macls.trainer:__train_epoch:278 - Train epoch: [1/60], batch: [21/245], loss: 2.62199, accuracy: 0.30000, learning rate: 0.00002616, speed: 2.38 data/sec, eta: 13:43:00

2025-08-11 02:06:26.822 | INFO | macls.trainer:__train_epoch:278 - Train epoch: [1/60], batch: [31/245], loss: 2.13521, accuracy: 0.32500, learning rate: 0.00003424, speed: 2.19 data/sec, eta: 14:52:28

2025-08-11 02:07:00.074 | INFO | macls.trainer:__train_epoch:278 - Train epoch: [1/60], batch: [41/245], loss: 1.89144, accuracy: 0.39062, learning rate: 0.00004233, speed: 2.41 data/sec, eta: 13:32:23

Traceback (most recent call last):
File "/mnt/nas5/docker/AudioClassification-Pytorch/train.py", line 27, in
trainer.train(save_model_path=args.save_model_path,
File "/mnt/nas5/docker/AudioClassification-Pytorch/macls/trainer.py", line 353, in train
self.__train_epoch(epoch_id=epoch_id, local_rank=local_rank, writer=writer, nranks=nranks)
File "/mnt/nas5/docker/AudioClassification-Pytorch/macls/trainer.py", line 231, in __train_epoch
for batch_id, (features, label, input_len) in enumerate(self.train_loader):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 701, in next
data = self._next_data()
^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 757, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
~~~~~~~~~~~~^^^^^
File "/mnt/nas5/docker/AudioClassification-Pytorch/macls/data_utils/reader.py", line 100, in getitem
feature = self.spec_augment(feature.cpu().numpy())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/yeaudio/augmentation.py", line 291, in call
x = self.time_mask(x)
^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/yeaudio/augmentation.py", line 273, in time_mask
length = random.randint(1, mask_time_len)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/random.py", line 336, in randint
return self.randrange(a, b+1)
^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/random.py", line 319, in randrange
raise ValueError(f"empty range in randrange({start}, {stop})")

ValueError: empty range in randrange(1, 1)`

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions