Deep Models for Low-Resourced Speech Recognition: Livvi-Karelian Case
Submitted to Mathematics. Special Issue Recent Advances in Neural Networks and Application.
Status: Published.
Kipyatkova I., Kagirov I. Deep Models for Low-Resourced Speech Recognition: Livvi-Karelian Case // Mathematics. 2023, vol. 11(18), ID 3814.
Authors: Irina Kipyatkova, Ildar Kagirov
Abstract: Recently, there has been a growth in the number of studies addressing the automatic processing of low-resource languages. The lack of speech and text data significantly hinders the development of speech technologies for such languages. This paper introduces an automatic speech recognition system for Livvi-Karelian. Acoustic models based on artificial neural networks with time delays and hidden Markov models were trained using a limited speech dataset of 3.5 hours. To augment the data, pitch and speech rate perturbation, SpecAugment, and their combinations were employed. Language models based on 3-grams and neural networks were trained using written texts and transcripts. The achieved word error rate metric of 22.80% is comparable to other low-resource languages. To the best of our knowledge this is the first speech recognition system for Livvi-Karelian. The results obtained can be of a certain significance for development of automatic speech recognition systems not only for Livvi-Karelian, but also for other low-resource languages, including the fields of speech recognition and machine translation systems. Future work includes experiments with Karelian data using techniques like transfer learning and DNN language models.
Keywords: low-resource languages; automatic speech recognition; audio data augmentation; time delay neural network; hidden Markov models; long short-term memory.
The Karelian ASR system training and testing script for Kaldi: run_tdnnf.sh
The Karelian ASR system training and testing script with spectral augmentation for Kaldi: run_tdnnf_SpecAugment.sh
Examples of test set with reference transcriptions and recognition results (“Ref” means reference transcription; “Hyp” means recognition hypothesis; C is a number of correctly recognized words; S, I, D are the number of substituted, inserted, and deleted words respectively):
Ref: vot konzu kondiedu näit midä pidäy ruadua
Hyp: vot konzu kondiedu näit midä pidäy ruadua
C=7; S=0; I=0; D=0
Ref: da nygöi kaikile kalmužimale levittih landišat
Hyp: da nygöi kaikile kalmužimale levitti ellendykset
C=4; S=2; I=0; D=0
Ref: noril'skas oli ruadoi sie sit že zavodas vot
Hyp: no rikas oli ruadoi sie sit zavodas vot
C=6; S=1; I=1; D=1