Data collection

The MuPTA corpus was recorded in an office condition and is designed for multi-modal human’s personality traits assessment. Three video recording devices with different angles and speaker distances were used to collect multimedia data. It is necessary to train a robust video model with videos from multiple perspectives. Each device has its own microphone, as the two devices are equidistant from the speaker, one built-in microphone was replaced with a lavalier microphone and placed it closer to the speaker. This approach adds some variability to the speech signal: each microphone produces a different speech signal depending on its own characteristics (sensitivity, directivity, noise level, etc.).

The corpus recording involved 30 individuals, each performing three different tasks. These tasks included:

1. provided brief information about themselves;
2. described the actions presented in the two pictures;
3. read aloud several sentences from a prepared script. The script consisted of a list of 40 sentences taken from the phonetically balanced text "It was a quiet, gray evening" [Stepanova S. B. Phonetic properties of Russian speech: realization and transcription: Ph. L., 1988]. The chosen text was carefully selected to examine speech patterns and variations among native Russian speakers with distinct phonetic features. This selection allows for the compilation of a comprehensive speech profile for each speaker.

Data annotation

The MuPTA corpus is annotated based on the Big Five model, which includes the following traits: Openness to experience, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. Each speaker (informant) participated in a self-evaluating questionnaire consisting of 60 questions [Soto CJ, John OP. The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power // J Pers Soc Psychol. 2017. Vol. 113(1). pp. 117-143. doi: 10.1037/pspp0000096]. It is a widely used standard questionnaire for the Big Five traits assessment. Adapted versions of these questionnaires for the Russian language are available [Shchebetenko, S.A. The Best Man in the World: Attitudes Toward Personality Traits. Psychology // Journal of Higher School of Economics. 2014. Vol. 11(3). pp. 129-148]. Each question in the questionnaire is scored on a Likert scale ranging from 1 to 5. All scores are then normalized within the range of [0, 1]. Additionally, each informant provided personal information including their, gender, marital status, education, and occupation. Prior to data collection, all informants were asked to complete an informed consent form.

Advantages of the corpus

The MuPTA corpus differs from other existing corpora assembled for the personality traits assessment that it contains audio-visual recordings of 30 native Russian speakers, evenly distributed by gender and age, and includes both spontaneous and read speech.

Access to the corpus:

This corpus is available to the public. Permission to use, but not to reproduce or distribute our corpus is granted to all researchers, provided that the following steps are properly followed:

• Send an email to Elena Ryumina (ryumina_ev@mail.ru) to get a link to download this corpus and a password to access the files of this corpus. Your email MUST be sent from a valid university account and MUST contain the following text:

  1. Subject: Application to download the MuPTA corpus          
  2. Name: <your first and last name>
  3. Affiliation: <University where you work>
  4. Department: <your department>
  5. Position: <your job title>
  6. Email: <must be the email at the above mentioned institution>
  
  I have read and agree to the terms and conditions specified in the MuPTA corpus webpage. 
  This corpus will only be used for research purposes. 
  I will not make any part of this corpus available to a third party. 
  I'll not sell any part of this corpus or make any profit from its use.
  

• If you are going to use the data mentioned above, you MUST cite the paper below:

  Ryumina E., Ryumin D., Markitantov M., Kaya H., Karpov A. Multimodal Personality Traits Assessment (MuPTA) Corpus: The Impact of Spontaneous and Read Speech // In Proc. of INTERSPEECH. 2023. pp. 4049-4053.

  or:

  @inproceedings{mupta_corpus,
    title={Multimodal Personality Traits Assessment ({MuPTA}) Corpus: The Impact of Spontaneous and Read Speech},
    author={Elena Ryumina and Dmitry Ryumin and Maxim Markitantov and Heysem Kaya and Alexey Karpov},
    booktitle={Proc. of INTERSPEECH},
    pages={4049--4053},
    year={2023},
    doi={10.21437/Interspeech.2023-1686}
  }